[opensuse-factory] Does NFS allow different encodings to be used on server/client side?
Hello. While I use iso-8859-1 in my systems, my friend uses utf-8. This is actually no big issue when using rsync (as it can convert filenames on the fly), but now I wanted to use KODI on my FireTV stick. KODI appears to use only UTF8 when using its built-in nfs-client, or it has a bug and ignores the configured character set. Anyway, Kodi does not show German Umlauts therefore. Can I configure the kernel-nfs on my server to do something similar to rsync by using "--iconv=iso-8859-1,utf-8"? So my server-side encoding is translated to UTF8 only for this FireTV stick? I had a look into the man pages, but I could not find anything about that. I also googled and figured out that some nfs servers allow that, others don't, ... Would be great if this is possible on openSUSE. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Op vrijdag 18 januari 2019 08:52:33 CET schreef Rainer Hantsch:
Hello. While I use iso-8859-1 in my systems, my friend uses utf-8.
This is actually no big issue when using rsync (as it can convert filenames on the fly), but now I wanted to use KODI on my FireTV stick. KODI appears to use only UTF8 when using its built-in nfs-client, or it has a bug and ignores the configured character set.
Anyway, Kodi does not show German Umlauts therefore.
Can I configure the kernel-nfs on my server to do something similar to rsync by using "--iconv=iso-8859-1,utf-8"? So my server-side encoding is translated to UTF8 only for this FireTV stick?
I had a look into the man pages, but I could not find anything about that. I also googled and figured out that some nfs servers allow that, others don't, ...
Would be great if this is possible on openSUSE. Please let's move this to opensuse-support@ or opensuse@ , this is for development discussions, yours is a support question.
-- Gertjan Lettink a.k.a. Knurpht openSUSE Board Member openSUSE Forums Team -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hello, On Jan 18 08:52 Rainer Hantsch wrote (excerpt):
While I use iso-8859-1 in my systems, my friend uses utf-8. ... convert filenames on the fly ...
why converting only the file name but not the file content? Wouldn't such a half-baked conversion make things even worse because then the file name encoding would no longer match its content encoding which means in any locale either the file name or its content would be shown wrong unless it is ASCII which is the only encoding that "just works" (I already hear them screaming "nowadays all is UTF-8" ;-) For some more fun in encoding hell have a look at https://en.opensuse.org/SDB:Plain_Text_versus_Locale I know nothing about NFS details but I guess that for NFS a filename is a plain sequence of bytes. Kind Regards Johannes Meixner -- SUSE LINUX GmbH - GF: Felix Imendoerffer, Jane Smithard, Graham Norton - HRB 21284 (AG Nuernberg) -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hello Johannes. This is not "half baked". Converting the content of a file is often not possible. You cannot simply run everything (including the file content) through a translator, this will damage most of the files abd only work in pure text files. Am Freitag, 18. Jan 2019, 09:27:41 schrieb Johannes Meixner:
Hello,
On Jan 18 08:52 Rainer Hantsch wrote (excerpt):
While I use iso-8859-1 in my systems, my friend uses utf-8. ... convert filenames on the fly ...
why converting only the file name but not the file content?
Wouldn't such a half-baked conversion make things even worse because then the file name encoding would no longer match its content encoding which means in any locale either the file name or its content would be shown wrong unless it is ASCII which is the only encoding that "just works" (I already hear them screaming "nowadays all is UTF-8" ;-)
For some more fun in encoding hell have a look at https://en.opensuse.org/SDB:Plain_Text_versus_Locale
I know nothing about NFS details but I guess that for NFS a filename is a plain sequence of bytes.
Kind Regards Johannes Meixner
-- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Fri, Jan 18, 2019 at 9:40 AM Rainer Hantsch
Hello Johannes.
This is not "half baked". Converting the content of a file is often not possible. You cannot simply run everything (including the file content) through a translator, this will damage most of the files abd only work in pure text files.
And converting the content would not be wanted. Only the file and directory names are interesting at this level. We have a similar issue with CIFS and mixing ISO8859 (as some of the CIFS shares seem to be - I do not know of a reliable way to determine this when mounting it) and using UTF8 in the console. Or when copying a file from the local ext file system using UTF8 file names to a CIFS volume that has some other encoding. We just caution our users not to use extended characters in file and directory names. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 18/01/2019 09.40, Rainer Hantsch wrote:
Am Freitag, 18. Jan 2019, 09:27:41 schrieb Johannes Meixner:
Hello,
On Jan 18 08:52 Rainer Hantsch wrote (excerpt):
While I use iso-8859-1 in my systems, my friend uses utf-8. ... convert filenames on the fly ...
why converting only the file name but not the file content?
Wouldn't such a half-baked conversion make things even worse because then the file name encoding would no longer match its content encoding which means in any locale either the file name or its content would be shown wrong unless it is ASCII which is the only encoding that "just works" (I already hear them screaming "nowadays all is UTF-8" ;-)
Hello Johannes. This is not "half baked". Converting the content of a file is often not possible. You cannot simply run everything (including the file content) through a translator, this will damage most of the files abd only work in pure text files.
I guess Johannes was using irony ;-)
For some more fun in encoding hell have a look at https://en.opensuse.org/SDB:Plain_Text_versus_Locale
Interesting! Didn't know of this one.
I know nothing about NFS details but I guess that for NFS a filename is a plain sequence of bytes.
AFAIK, it is so for any filesystem. Best is to use software at the other end that uses the same encoding as the filesystem. Yes, it is a nuisance. Although Linux supports in fstab an entry for the encoding on fat or ntfs, I don't know how that works. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
Best is to use software at the other end that uses the same encoding as the filesystem. Yes, it is a nuisance.
If you have two NFS clients that expect different encodings and if the client that currently has problems with the encoding is read only, an idea might be to have a cron-job that hard links the files into a separate folder (hierarchy) using the target encoding and export this folder for the latter client. Disadvantage will be that new files added by the first client (or directly on the server) will take a while before they shows up on the second client. Joachim -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Fri, Jan 18, 2019 at 10:45 AM Carlos E. R.
I know nothing about NFS details but I guess that for NFS a filename is a plain sequence of bytes.
AFAIK, it is so for any filesystem.
Best is to use software at the other end that uses the same encoding as the filesystem. Yes, it is a nuisance.
Although Linux supports in fstab an entry for the encoding on fat or ntfs, I don't know how that works.
Although slightly off topic, just try specifying file names on the command line of a console program in Window's cmd.com. Locale stuff is so broken. You cannot reliably get locale-sensitive characters as you would like. Try forcing a file name on the command line to be UTF-8 in the program. It is inconsistent. It is a known issue that MS seem not to care about. They say you should only be making/using Windows GUI programs, where this will work as expected. In our software (Windows console apps), when looking for files that were specified on the command line, we would need to convert the name into some known encoding, and then try variations on the encoding until we find the file. We are still fighting with a predictable solution for console programs. It would be great if there was some way to find out how a mounted file system is encoded that works on all platforms... -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am Freitag, 18. Jan 2019, 11:46:22 schrieb Roger Oberholtzer:
On Fri, Jan 18, 2019 at 10:45 AM Carlos E. R.
wrote: Although slightly off topic, just try specifying file names on the command line of a console program in Window's cmd.com. Locale stuff is so broken. You cannot reliably get locale-sensitive characters as you would like. Try forcing a file name on the command line to be UTF-8 in the program. It is inconsistent. It is a known issue that MS seem not to care about. They say you should only be making/using Windows GUI programs, where this will work as expected. In our software (Windows console apps), when looking for files that were specified on the command line, we would need to convert the name into some known encoding, and then try variations on the encoding until we find the file. We are still fighting with a predictable solution for console programs. It would be great if there was some way to find out how a mounted file system is encoded that works on all platforms...
I am using Windows only in a window (of VirtualBOX) to do my banking things. All the rest is done on my workstation natively (openSUSE 13.2 x64) and on other workstations (up to Leap 15). Nowhere UTF8 is used at me, even samba is configured to translate the local iso-8859-1/15 into Windows codepage and/or cp850 (if I remember correctly). The only device using UTF8 is my FireTV stick. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Fri, Jan 18, 2019 at 12:06 PM Rainer Hantsch
I am using Windows only in a window (of VirtualBOX) to do my banking things. All the rest is done on my workstation natively (openSUSE 13.2 x64) and on other workstations (up to Leap 15). Nowhere UTF8 is used at me, even samba is configured to translate the local iso-8859-1/15 into Windows codepage and/or cp850 (if I remember correctly). The only device using UTF8 is my FireTV stick.
We do things like pack up directory trees of data on some data collection system running openSUSE. So all is UTF-8. These are then unpacked on a Windows machine. Usually on a Windows server drive mounted on the Windows machine. The encoding there is not guaranteed. Life in a networked environment, I guess. I do not think that WinZip or whatever program that they use to unpack the archive will do this file name translation when unpacking the files. To add to the confusion, some files' names are read from a data file that is UTF-8. If the software runs on openSUSE, the name can be used as is. All is UTF-8. When run on Windows, it gets less certain. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Am Freitag, 18. Jan 2019, 12:33:11 schrieb Roger Oberholtzer:
On Fri, Jan 18, 2019 at 12:06 PM Rainer Hantsch wrote: We do things like pack up directory trees of data on some data collection system running openSUSE. So all is UTF-8.
These are then unpacked on a Windows machine. Usually on a Windows server drive mounted on the Windows machine. The encoding there is not guaranteed. Life in a networked environment, I guess.
I do not think that WinZip or whatever program that they use to unpack the archive will do this file name translation when unpacking the files.
From my experience, unpacking a zip leaves the filenames as they are. Possibly it will be better if you try it from the other side (Linux) to unpack the files. Have you ever tried that?
To add to the confusion, some files' names are read from a data file that is UTF-8. If the software runs on openSUSE, the name can be used as is. All is UTF-8. When run on Windows, it gets less certain.
Usually I try to stay away from Windows as far as possible. Windows has too many "surprises" for my taste, and you get some new surprises with every particular update. ;) I would mount a Windows share and process data from a Linux system. But be warned... I tried to open+append+close files in a windows share from Linux). This worked during last 3 years at a customer, then suddenly not. I had to switch to a brute-force copy-delete method, creating a new file, copy first the old file into it, then also write the new data, and after finishing all appending, cloosing the new file and moving it in place of the old one. (Braindead, I know, but this works. On Linux I never had such effects.) But I think this is too off-topic to be continued here. We can email directly, if you like. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Fri, Jan 18, 2019 at 12:47 PM Rainer Hantsch
From my experience, unpacking a zip leaves the filenames as they are. Possibly it will be better if you try it from the other side (Linux) to unpack the files. Have you ever tried that?
Not all of our users have a Linux machine in their office. Sigh...
Usually I try to stay away from Windows as far as possible. Windows has too many "surprises" for my taste, and you get some new surprises with every particular update. ;)
I don't use it. But I do make software that is expected to run on it. (Yeah for MinGW and cross compiling on Linux - and all the Windows libraries in OBS!)
But I think this is too off-topic to be continued here. We can email directly, if you like.
Indeed. -- Roger Oberholtzer -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 18/01/2019 12.05, Rainer Hantsch wrote:
Nowhere UTF8 is used at me, even samba is configured to translate the local iso-8859-1/15 into Windows codepage and/or cp850 (if I remember correctly). The only device using UTF8 is my FireTV stick.
But why do you not use UTF? All Linux software is designed and tested for that. -- Cheers / Saludos, Carlos E. R. (from 15.0 x86_64 at Telcontar)
participants (6)
-
Carlos E. R.
-
Joachim Wagner
-
Johannes Meixner
-
Knurpht-openSUSE
-
Rainer Hantsch
-
Roger Oberholtzer