Feature changed by: jsuchome Feature #302955, revision 45 Title: Split translations out of installation system openSUSE-11.0: Candidate Priority Requester: Mandatory Projectmanager: Mandatory Requested by: coolo@novell.com Partner organization: openSUSE.org Description: The translations of yast are the biggest part of the installation image. These should not be part of the image, but should be pulled in from the product media as the user selects them. Discussion: #1: jsrain@novell.com (2007-11-09 13:18:17) Duncan, we will need a Linuxrc support for this. Can Steffen look into this feature? What comes to my mind is: Linuxrc needs to download and mount the language image for the language which user selected in bootloader menu (or English as fallback) Linuxrc needs to provide a support (eg. a script) which YaST can call when Language changes and downloads and "mounts" the new translation image It is questionable whether it should be YaST or Linuxrc who makes checks if the asked image is already downloaded, IMO it should be Linurc... #2: snwint@novell.com (2007-11-09 14:08:01) (reply to #1) I don't see linuxrc involved at all. YaST can just donwload he required files. Doesn't look that tricky to me. #3: jsuchome@novell.com (2007-11-09 14:29:36) (reply to #2) Maybe, but I do not know how. Lukas, don't we have some API in yast2- installation for that (copying target package from media to inst-sys, if I understand it correctly)? #4: snwint@novell.com (2007-11-09 14:32:40) (reply to #3) We've had exactly that discussion when we talked about loading the licenses. I think now is a good time to fix yast. I really don't see why other programs have to download various files for yast when yast could as well do it itself. And when this means doing the repo init earlier, then, well, do it earlier. #5: jsrain@novell.com (2007-11-09 15:02:22) (reply to #4) YaST can only fetch data from instlalation sources. Inst-sys can come from completly different location than installation sources do. YaST has no information where to get these images from. YaST would have to reimplement getting data from all kind of locations on its own (it cannot use libzypp, since at the location, there needn't be any valid source. That's why I'd like to reuse functionality which Linuxrc already has. #7: coolo@novell.com (2007-11-16 11:42:17) (reply to #5) Duncan, is it feasible to make the download functionality of libzypp seperate from valid sources. E.g. RepoManager::download(theurl_I_got_from_linuxrc, "/suse/translations/de.tar"); #10: dmacvicar@novell.com (2008-01-22 18:35:02) (reply to #7) Yes, MediaSetAccess or even MediaAccess can be used to get a file directly from an Url. The design is layered, MediaAccess sits on top of MediaManager(curl, iso, etc), MediaSetAccess on top, adding multiple media and media changing support, and RepoProvideFile on top, using the repos urls and trying different mirrors. There is also Fetcher which also sits on top of MediaSetAccess which provide queues and using already downloaded files. I think it is already used in YaST, YaST does transfer the content file from media independently from ZYpp. #6: locilka@novell.com (2007-11-09 16:54:01) (reply to #3) As already written by snwint, there is no way how to download anything from sources before they are initialized. So: no, there is no Installation API for that. As there can be an installation-(zypp)-source and a different inst-sys, Installation could either initialize the source as soon as possible but it still wouldn't know where to download inst-sys extensions. On the other hand, Linuxrc doesn't know which translations will be needed. So maybe inst-sys could provide some functionality (API) for downloading these extensions and merging them (downloaddir + adddir), YaST could then use that API. Just an idea... #8: michl@novell.com (2007-11-29 12:46:53) We should come down to tier1 languages here. #11: locilka@novell.com (2008-01-28 19:06:03) (reply to #9) According the forwarded mails, the Installation framework should call Get InstsysURL and RepoURL from /etc/install.inf (and combine them with a file/localization location to $url_to_download_file_from) /lbin/wget -v $url_to_download_file_from $path_to_save_file_to mkdir /mounts/localization mount -o loop $path_to_save_file_to /mounts/localization adddir /mounts/localization / Where (dir) and under which names will be the localization files (language images) stored? #12: locilka@novell.com (2008-01-29 10:39:24) (reply to #11) Snwint: does /lbin/wget check signatures of that file or does inst-sys provide some functionality I could use to check them? #14: snwint@novell.com (2008-01-29 11:06:11) (reply to #12) No it doesn't check them. But their SHA1sum will appear in /content. #13: snwint@novell.com (2008-01-29 11:03:05) (reply to #11) The language images are named root.XXX where XXX is XXX in yast2-trans- XXX. #15: locilka@novell.com (2008-02-04 07:25:10) (reply to #13) Stano's idea: For a better user experience, let's have all translations for the first dialog available from the very beggining (it's actually installation.mo ). In other words, installation textdomain should be still in the inst-sys-base, other text-domains in squashfs images. Download the language squashfs inst-sys extension after clicking on [Next] in the first dialog. That would make it visually faster. #16: locilka@novell.com (2008-02-04 07:58:54) (reply to #15) Ah, I've found that having installation.mo in the base inst-sys is currently the only way to avoid translation errors. YaST uses gettext for translating messages. The very first dialog using a gettext functions (e.g., initializing installation) is called even before the translation file is downloaded. In this case, gettext wouldn't find any translation. It would return untranslated message and also remember that there was no translation file available. After the installation adds the translation file, gettext would ignore it (This is known behavior since OES2 its Add-On integration into SLES10 SP1 workflow). #19: locilka@novell.com (2008-02-04 11:59:10) (reply to #16) Problem solved. According to snwint, the initial root.$LANG file is downloaded by Linuxrc, long time before YaST does anything. #17: locilka@novell.com (2008-02-04 08:06:35) (reply to #13) Where do we expect the root.$LANG files? On the installation repository or in the inst-sys? Consider this situation: RepoURL: nfs://install.suse.cz/11/?device=eth0 InstsysURL: nfs://miracle.suse.cz/11i/inst-sys/?device=eth0&aaa=bbb I assume that for de_DE URL nfs://miracle.suse.cz/11i/root.de_DE? device=eth0&aaa=bbb should be used but is that correct? Why not nfs: //install.suse.cz/11/boot/<arch>/root.de_DE?device=eth0 ? #18: snwint@novell.com (2008-02-04 14:22:41) (reply to #17) The files are taken from miracle because they should somehow match the other instsys files. #20: locilka@novell.com (2008-02-04 12:00:39) (reply to #13) Is there any possibility to list these (root.XX, root.xx_XX, ...) files? Because I haven't found any both systematic and valid solution. #21: snwint@novell.com (2008-02-04 18:07:19) (reply to #20) No idea; I'm using yast2-trans-allpacks when I build the instsys. #22: locilka@novell.com (2008-02-04 12:16:54) (reply to #21) This shows the list of $somehow supported translations ls -1 /usr/lib/YaST2/trans | sed 's/\.status//' however it is not ideal. #23: locilka@novell.com (2008-02-04 12:27:27) (reply to #22) A simple solution would be to introduce a new entry into the content file, e.g., SUPPLANGS . This variable would contain (space-separated) list of supported languages in YaST (all ${LANG}s for all root.$LANG files). Rudi, could you, please, add it? #24: snwint@novell.com (2008-02-05 10:07:12) (reply to #23) All root.xxx are already in content as they are sha1-summed. #25: locilka@novell.com (2008-02-05 04:34:37) (reply to #24) But their SHA1 is still not checked by the installation (YaST) when downloaded. It's in my TODO. #26: snwint@novell.com (2008-02-05 10:49:20) (reply to #25) Maybe. But it basically is the list you asked for in comment 23. Or not? #27: locilka@novell.com (2008-02-05 05:29:03) (reply to #26) It basically is the list I asked for for, but it doesn't contain the informative value that defines: hey! these languages are supported, use these files (...). For instance file, root.fonts , should be also signed and partly matches the pattern but fonts is not a language. And there might be several files later... So, I don't think this would work. #28: locilka@novell.com (2008-02-07 16:32:53) (reply to #25) I've found that SHA1 sum for checking the downloaded files cannot be computed :( Snwint: It seems that sha1sum is not part of the inst-sys. Linuxrc seems to check the SHA1 itself (sha1.c). What should I use? What would be the best solution? Integrate sha1sum into the inst-sys? #29: snwint@novell.com (2008-02-07 18:08:27) (reply to #28) Sure. I'll add it. #35: jsuchome@novell.com (2008-02-18 13:50:00) (reply to #23) What is the reason for SUPPLANGS when we already have LINGUAS? #36: locilka@novell.com (2008-02-18 14:12:17) (reply to #35) Because LINGUAS were in use already. And they currently DO NOT reflect the status of *all supported languages*. Anyway, the SUPPLANGS tag hasn't been added and installation uses a fallback that checks all trans-stats and takes the list of supported languages from there (which works well). /** * check if selected language has support on media (F301238) * show a warning when not */ define void check_languages_support (string selected_language) { string linguas = (string) SCR::Read (.content.LINGUAS); if (linguas == nil) { y2warning ("No LINGUAS tag defined in content file"); return; } y2milestone ("content LINGUAS %1", linguas); list <string> all_linguas = splitstring (linguas, " "); string language_short = splitstring (selected_language, "_") [0]:""; if (!contains (all_linguas, selected_language) && !contains (all_linguas, language_short)) { y2milestone ("Language %1 is not fully supported", selected_language); // popup message Popup::Message (_("Only minimal support for the selected language is included on the media. Add the Language Add-On CD as an additional repository to get better support for this language.")); } } #37: jsuchome@novell.com (2008-02-18 14:23:46) (reply to #36) I can read the code and I know that LINGUAS is used. Still, I think the meaning of SUPPLANGS is the same. Or it should be if it currently is not. #38: locilka@novell.com (2008-02-18 14:36:09) (reply to #37) I'm afraid that it is not correct. According to the soruce code, LINGUAS contain list of languages that are fully supported and if user selects language, that is NOT listed in LINGUAS he/she/it is warned. On the other hand SUPPLANGS (as it was requested) should have contained list of all languages supported in YaST (in other words: list of languages in a separate squashfs images that can be used as a plugin to the installation). That's why we can't reuse LINGUAS if we use them for this check. But, of course, as far as I know, using the LINGUAS should be replaced with getting the information from trans-stats files: less than XY% means -> language not fully supported. (Stano?) #39: jsuchome@novell.com (2008-02-18 14:52:12) (reply to #38) Well, if we want to know list of all languages supported in YaST, we do not need other variable in content file, we know this from YaST already: these are the languages offered in the language selection. And checking for insufficient languages support (now done against LINGUAS) and insufficient translation treshold (trans-stats) is a different thing: certain language could be only on Add-On (=not in LINGUAS), but it could be 100% translated and vice versa. #40: locilka@novell.com (2008-02-18 15:04:28) (reply to #39) Please, try to solve reasonable ammount of features and bugs at once. This feature is about translation split (out of the installation system). If some languages are additionally supported by some Add-On product, installation will not be able to add them on-the-fly (yast translations for inst-sys). Language Add-On might contain some additional packages but that's all. It will not have any effect on the inst-sys. If this is all about not using SUPPLANGS, please, change the client that tries to read it, to use the Language (or whatever) module. SUPPLANGS in not (and probably will not be) added. Installation currently uses trans-stats to find out all languages supported by the inst-sys. It can be changed to use Language module directly (if it provides the very same information). That's a little implementation detail. + #41: jsuchome@novell.com (2008-02-18 15:12:26) (reply to #40) + Sure. + I was not trying to solve anything, I asked why is it needed. Based on + your information I concluded that it is not. + I only wrote about Add-Ons to describe the distincition with LINGUAS + and trans-stats usage (= answer to as far as I know, using the LINGUAS + should be replaced with getting the information from trans-stats + files... ) #30: locilka@novell.com (2008-02-08 13:36:08) For jsuchome/country (possibly-buggy behavior reported by jsuchome): If user selects another language in the 'installation overview' (former 'proposal'), language selection needs to call this: if (Stage::initial()) { WFM::call ("integrate_translation_extension", [$["requested_language":language]]); } You should call it before Language::Set (language) is called. This is used only in inst-sys /Stage::initial()/, client integrate_translation_extension has been added to yast2-installation- 2.6.18 #31: jsuchome@novell.com (2008-02-08 13:52:32) (reply to #30) Actually, I would like to have this call (+all the checks when it is necessary of course) inside Language::Set call, so that no code that wants to change the language needs to care about this. #32: locilka@novell.com (2008-02-08 14:12:56) (reply to #31) Hmm, that sounds reasonable... We had better move the client from installation to country then... -- SUSE Feature Tool: http://partnerfate.suse.de/?rm=feature_show&id=302955