Mailinglist Archive: opensuse-bugs (8118 mails)

< Previous Next >
[Bug 249431] gettext should convert untranslated non-ASCII msgids into locale encoding
  • From: bugzilla_noreply@xxxxxxxxxx
  • Date: Tue, 27 Feb 2007 17:21:50 -0700 (MST)
  • Message-id: <20070228002150.404F825C887@xxxxxxxxxxxxxxxxxxxxxx>
https://bugzilla.novell.com/show_bug.cgi?id=249431


bruno@xxxxxxxxx changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |WONTFIX




------- Comment #4 from bruno@xxxxxxxxx 2007-02-27 17:21 MST -------
The function gettext() is now standardized by the LSB, see

http://www.linux-foundation.org/spec/booksets/LSB-Core-generic/LSB-Core-generic/baselib-gettext.html
therefore there is not much room for changing its behaviour.

But anyway, what does it mean to support non-ASCII msgids?

1) You must provide a way for handling the case that the user's locale
encoding cannot represent the non-ASCII characters. Let's take the
example from the BR you mentioned:
https://bugzilla.novell.com/show_bug.cgi?id=248859
(funny bug number, when it's about encodings, by the way :-))

gettext ("St Dévote Day")

The user certainly wishes to see "St Devote Day" then, i.e. the
transliteration of glibc and libiconv could do it. But in more
general cases, this does not work any more. So IMO the programmer
must jump in and provide the ASCII equivalent too:

gettext_na ("St Devote Day", "St Dévote Day")

2) The argument to gettext is used as a key into the hash table in the .mo
file. It's a hash table so that it's fast.

If you use "St Dévote Day" as gettext argument, you have
- to convert this string to the .mo file's text encoding first, using
iconv(), [you cannot reliably enforce that all .mo files are in
UTF-8 encoding],
- to specify globally the source encoding, for example, through a new
hypothetical function call
bind_textdomain_source_codeset ("ISO-8859-1");

If you use "St Devote Day" as gettext argument, you have none of these
two problems.

3) If gettext returns the msgid, i.e. if the message was not translated,
you have to convert it to the locale encoding yourself. Again, you
have to specify globally the source encoding, for example, through a
hypothetical function call
bind_textdomain_source_codeset ("ISO-8859-1");
Additionally, you don't want a memory leak here, when the same call
is made repeatedly. So you need to cache the result for later reuse.

You can get rid of the need for bind_textdomain_source_codeset if you
assume that the source code is in UTF-8. So what you end up with is a
user-defined function

/* Return the localization of a string whose original writing is not ASCII.
MSGID_UTF8 is the real string, written in UTF-8 with octal or hexadecimal
escape sequences. MSGID_ASCII is a fallback written only with ASCII
characters. */

const char *
gettext_utf8 (const char *msgid_ascii, const char *msgid_utf8)
{
/* See whether there is a translation. */
const char *translation = gettext (msgid_ascii);

if (translation == msgid_ascii)
{
/* Access a cache here. A little homework. */

/* locale_charset, c_strcasecmp, xstr_iconv are defined in gnulib. */
const char *locale_code = locale_charset ();
if (c_strcasecmp (locale_code, "UTF-8") == 0)
translation = msgid_utf8;
else
{
#if HAVE_ICONV
const char *converted = xstr_iconv (msgid_utf8, "UTF-8",
locale_code);
if (converted == NULL)
{
# if (__GLIBC__ == 2 && __GLIBC_MINOR__ >= 2) || __GLIBC__ > 2 \
|| _LIBICONV_VERSION >= 0x0105
size_t len = strlen (locale_code);
char *locale_code_translit = XNMALLOC (len + 10 + 1, char);
memcpy (locale_code_translit, locale_code, len);
memcpy (locale_code_translit + len, "//TRANSLIT", 10 + 1);
converted =
xstr_iconv (msgid_utf8, "UTF-8", locale_code_translit);
free (locale_code_translit);
# endif
if (converted == NULL)
{
/* Use msgid_ascii as a fallback. */
converted = msgid_ascii;
}
}
translation = converted;
}
/* Store the translation in the cache. Homework part 2. */
}

return translation;
}

Finally, you make this function available to xgettext by putting this into
po/Makevars:

XGETTEXT_OPTIONS = \
--keyword=gettext_utf8:1 --flag=gettext_utf8:1:pass-c-format


--
Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.

< Previous Next >
References