Hi Johannes, Dne 27.9.2013 11:49, Johannes Meixner napsal(a): [...]

...

- Remove the second installation stage (started after reboot), configure the system completely just in one stage, after finishing the installer the freshly installed system must be ready to use.

I think the "configure the system completely" is wrong here. ( I assume you would not like to have e.g. printer, scanner, and all kind of system services configuration in the installer ;-)

Yeah, bad wording, with "completely" and meant rather "do not touch it later", instead of "configure everything possible".

I assume what is meant is:

- Remove the second installation stage (started after reboot), do the basic system configuration in one stage, after finishing the installer the freshly installed system must be ready for basic usage.

Yes, that's it.

When those steps should be replaceable by 3rd party applications with reasonable effort for those who make the 3rd party applications those steps must work fully independent from each other.

For example the installer's configuration step must not somehow have unexpected/hidden/private dependencies to the previous installer's install step (except the obvious "dependency" that the install step had acually installed the software).

Yes, the idea is to have a "communication" file, very likely similar to what AutoYaST uses (XML profile).

I.e. the installer's configuration step must be implemented so that it can run on any system regardless how that system was installed.

The configuration should know in advance what it needs and should ask the installer to install it in the config step. This is already done in current Yast, e.g. bootloader can ask the software selection to install Grub (or whatever else is chosen for booting) which is configured later after installation

It must work to install the installer on an already installed system and run only the installer's configuration step there. This would overwrite the existing system's basic configuration by a new system's basic configuration that is the same as if the system would have been installed from scratch.

Um, well, there might be additional config files not touched by installer but still important for configuration. Example: Yast can configure repositories, install packages, patches... etc, but libzypp uses zypp.conf config file which is completely ignored by Yast, so whatever user does wrong there Yast is unable to fix it.

If it works this way, you get a powerful feature for free: The installer could be used as rescue/recovery tool when a [...]

Um, maybe as a side effect it could be used this way, but it would not be 100% reliable (as I mentioned before). We already had a repair tool in Yast but it was basically unmaintainable. So I'm skeptic about such "magic repair" functionality for free... [...]

Probably it helps to implement the installer's 3 basic steps as three fully separated programs [...]

Yes, it was our idea, but the current decision is to run it one process if possible (to avoid unnecessary save/load steps, hardware reprobing...) and run a separate program only when asked to.

To help you a bit, I volunteer for implementing proof of concept 3rd party applications for the install and configure steps.

Yeah, great! That would be nice to have an independent implementation which we could check against!

That file's format must be stable (both syntax and values must be stable - i.e. only fully backward compatible changes are allowed) so that those 3rd party applications can be implemented with reasonable (maintenance) effort.

Yes, I agree, the backward compatibility with AutoYast and stability is a must.

Perhaps you should make it more clear that "configure the installation" here and "configure the system" above are fully separated things. [...] If that file contains a complete configuration of the installation, the installation of the other system can run unattended in a full automated way or it can run in an interactive mode where each value can be modified by a user for the actual installation.

Yes, exactly.

If that file does not contain a complete configuration of the installation, the installation of the other system can run either unattended in a full automated way by using fallback values or it can run in an interactive mode where missing values are requested from a user.

We basically grouped the values to these groups: - mandatory (if missing the installer will interactively ask, otherwise use the predefined value) - optional (use the value, if missing use a default, don't ask) And there will be a special "do not touch" value (i.e. keep the current value, was/will be set by a 3rd party program) The "do not touch" is useful if you change the install step by your application (like your proposed wget/tar) then you want to keep some values untouched because they might be already configured in the tar file and they would get overwritten. [...]

One could migrate an existing openSUSE 13.2 system onto a different hardware platform or onto a virtual machine or one could migrate an existing openSUSE 13.2 system into an openSUSE 13.3 system on different hardware and so on.

Yes, there are more possibilities related to the config file. One idea was to add support to SUSE Studio to export such config file as every Studio appliance actually contains the system configuration. Then you could have an appliance (for direct deployment) and a config file which could be used as installation template which would allow custom changes to the installation (compared to the appliance). Or you could use the appliance in the install step instead of installing from RPMs (using your own install step implementation). Clear separation and the config file open a lot of new possibilities... Ladislav -- Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

Hello, I think I need to make it more clear: On Sep 27 15:49 Johannes Meixner wrote (excerpt):

What I meant with "fully separated programs" was "fully separated at development time" (i.e. fully separated program source files).

It won't help when there are separated program source files which are compiled into one big all-in-one installer where its 3 basic steps (collect data, install, apply config) communicate with each other via internal mechanisms. What I like to point out is that I think the YaST core team implementation of the installer's 3 basic steps should be of same design as a 3rd party would do it. I think the YaST core team should not behave as if it was a party of special kind (some kind of "first class citizen" and the rest of the world is only "second-class citizen"). I think this is needed to ensure that the YaST core team implementation of the installer's 3 basic steps is 100% compatible with 3rd party applications that could replace them. Assume there are many different 3rd parties who each provide their own implementations for the 3 basic steps: collect-data.party1 ... collect-data.partyN install.party1 ... install.partyN apply-config.party1 ... apply-config.partyN The YaST core team implementation should be just one of those parties. Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH -- Maxfeldstrasse 5 -- 90409 Nuernberg -- Germany HRB 16746 (AG Nuernberg) GF: Jeff Hawn, Jennifer Guild, Felix Imendoerffer -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

Dne 27.9.2013 15:49, Johannes Meixner napsal(a): [...]

For me a crucial part of the whole separation idea is that the system where the installation configuration is created (as a file) and the system where the actual installation happens (by using that file) can be two different systems.

Or do I misunderstand something here?

No, that's correct, they can be different systems, different tools... [...]

Running it this way can be only an attempt for a first recovery step. If that fails, the adminhe may try other recovery steps or he may have to re-install his messed up system from scratch.

Yes, but there is a danger of messing the system even more. I have never used any automatic repair tool (fortunately), just using a rescue CD and fixing problems manually was always sufficient for me. I'm little bit scared of "automagic" repairs... -- Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

On 09/27/2013 09:41 AM, Josef Reidinger wrote:

On Fri, 27 Sep 2013 09:37:50 -0400 Robert Schweikert <rjschwei@suse.com> wrote:

...

Hi,

On 09/26/2013 02:07 PM, Ladislav Slezak wrote:

...

Hi all,

The SUSE YaST team started developing some major YaST features for SUSE Linux Enterprise 12 (SLE12) installer and we would like to include these features in the next openSUSE-13.2 release.

The features are mainly about making the installer easier and to simplify the installation workflow.

We will not create a completely new installer from scratch (that would need too much time and manpower), we will reuse the existing functionality as much as possible, the current installation framework is quite flexible (e.g. the installation steps are defined in external control.xml file).

Here is the summary of the main goals:

- Remove the second installation stage (started after reboot), configure the system completely just in one stage, after finishing the installer the freshly installed system must be ready to use.

- It should be possible to NOT install the Yast installer itself into the target system to make it smaller (except when firstboot or some complex AutoYast setup is used, then the installer will be needed).

- Separate the installer into 3 basic steps (collect data, install, apply config), make the steps replaceable by 3rd party applications.

- It should be possible to use the installer just for configuring the installation and exporting the setup into a file (without performing the actual installation). The exported file could be then used to install other system(s). (Like AutoYast does, the new installer will contain some AutoYast functionality).

- It should be possible to alternatively deploy already existing appliance image (provided by user) instead of installing from RPMs.

- Simplify the installation workflow, auto-configure what's possible, advanced setup (NIS, LDAP, Kerberos...) should be done in installed system.

Hmm auto configuration is nice, but we should keep in mind that we probably should not take away the opportunity to manually configure things, such as the network for example. Today there is an autoconfiguration for DHCP, but the user can set it to manual configuration and set up the network as needed.

Later, Robert

Well, when we discussing it, we agreed that for network if DHCP is find, then use it and if it is not find, then allow user to configure it. Usual work-flow is that DHCP is set and it is just used, so do not bother user with network configuration if we can use what dhcp gives us.

But just because there is a DHCP server that does not necessarily imply that for a particular install the user wants to use it. Yes, one can argue that this is a corner case, but having a checkbox as it is today that allows the user to set "configure the network manually" is hardly "bothering the user", IMHO. Later, Robert -- Robert Schweikert MAY THE SOURCE BE WITH YOU SUSE-IBM Software Integration Center LINUX Tech Lead rjschwei@suse.com rschweik@ca.ibm.com 781-464-8147 -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

Dne 27.9.2013 16:04, Robert Schweikert napsal(a): [...]

But just because there is a DHCP server that does not necessarily imply that for a particular install the user wants to use it. Yes, one can argue that this is a corner case, but having a checkbox as it is today that allows the user to set "configure the network manually" is hardly "bothering the user", IMHO.

Of course, the proposed or automatically detected setting should be possible to change in the installation proposal. E.g. the current installer proposes the default runlevel based on desktop (package) selection, but you can click the link and change it as you need. The same should apply for network config. -- Ladislav Slezák Appliance department / YaST Developer Lihovarská 1060/12 190 00 Prague 9 / Czech Republic tel: +420 284 028 960 lslezak@suse.com SUSE -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

В Wed, 16 Oct 2013 16:58:02 +0200 Ladislav Slezak <lslezak@suse.cz> пишет:

Dne 26.9.2013 20:07, Ladislav Slezak napsal(a):

...

Hi all,

The SUSE YaST team started developing some major YaST features for SUSE Linux Enterprise 12 (SLE12) installer and we would like to include these features in the next openSUSE-13.2 release.

[...]

...

More details about the current proposal will be published later.

So here we go: https://github.com/yast/yast-installation/wiki/New-Installer

There are details, proposals, ideas and thoughts how to improve the YaST installer. Some features are optional and likely won't make it into SLE12/openSUSE-13.2 (like the installation server proposal) due to complexity and lack of developers.

If you have any questions, comments... feel free to ask here.

Is btrfs based layout on topic here or should I bring it up on factory? -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

Hello, On Oct 16 16:58 Ladislav Slezak wrote (excerpt):

So here we go: https://github.com/yast/yast-installation/wiki/New-Installer

https://github.com/yast/yast-installation/wiki/New-Installer%3A-SLE12-Instal... therein in section "3. Deployment (step 7 above) (§5)" reads: ------------------------------------------------------------------------- o Create filesystem, optionally untar pre-installed images, install RPMs. The images here are tar.xz images with pre-installed RPMs as are currently used in openSUSE. For SLE the image functionality will be probably not needed... ------------------------------------------------------------------------- I disagree that "for SLE the image functionality is not needed". In particular for SLE it is needed that after a system disaster the new installer can re-create the system on new (compatible) replacement hardware as it was before by using a saved config of the system and deploying one or more backup.tar.gz images that contain the files of the system. To support this the new installer has to do those steps: 1. Boot New Machine 2. Use given config (i.e. the saved system config) 3. Disk Activation (e.g. special stuff to make SAN accessible) 4. Storage/Partitioning 5. Create filesystems and mount them at the right mount points 6. Deploy the files (i.e. untar backup.tar.gz images) 7. Install bootloader 8. Reboot Ideally "Deploy the files" should support any mixture of installing RPMs and whatever kind of software packages plus untar backup.tar.gz images so that it would be possible to have only user data in backup.tar.gz but use RPMs for the basic system software and whatever kind of software packages for other software like third-party software. Is it planned that such system recovery/restore functionality will be supported by the new installer in particular for SLE? Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH -- Maxfeldstrasse 5 -- 90409 Nuernberg -- Germany HRB 16746 (AG Nuernberg) GF: Jeff Hawn, Jennifer Guild, Felix Imendoerffer

Hello, On Oct 17 11:59 Ladislav Slezak wrote (excerpt):

Dne 17.10.2013 10:40, Johannes Meixner napsal(a):

...

In particular for SLE it is needed that after a system disaster the new installer can re-create the system on new (compatible) replacement hardware as it was before by using a saved config of the system and deploying one or more backup.tar.gz images that contain the files of the system.

To support this the new installer has to do those steps: 1. Boot New Machine 2. Use given config (i.e. the saved system config) 3. Disk Activation (e.g. special stuff to make SAN accessible) 4. Storage/Partitioning 5. Create filesystems and mount them at the right mount points 6. Deploy the files (i.e. untar backup.tar.gz images)

So far we thought only about Studio images (raw disk images), but support for tar.gz is easy so it could be supported as well.

I would very much appreciate it if the new installer supports deployment via tar.gz.

...

7. Install bootloader 8. Reboot

Ideally "Deploy the files" should support any mixture of installing RPMs and whatever kind of software packages plus untar backup.tar.gz images so that it would be possible to have only user data in backup.tar.gz but use RPMs for the basic system software and whatever kind of software packages for other software like third-party software.

We did not think about this possibility, but it is an interesting idea to mix both approaches...

Initially the new installer may only support one type of data for deployment, i.e. deployment either by installing RPM packages or by installing an image or by untar of a tar.gz archive. For my particular use case "system recovery" it would be basically sufficient to untar a single tar.gz archive (which means the user must backup all in one single archive). But when the new installer supports each of those types of data for deployment, then - I think - it should be relatively easy to also support any mixture of those types. The precondition is that the deployment step works well separated from the other steps - i.e. that the deployment step can work basically on its own. Then it should be relatively easy to run several deployment steps one after the other. To do this the config file must support to define several deployment steps and the sequence when each step should run. For example (config in XML-like syntax): ---------------------------------------------------------------------- <deployments> <deployment> <deployment_sequence_number>10</deployment_sequence_number> <deployment_data_type>image</deployment_data_type> <image_uri>ftp://server/base_system_image.raw</image_uri> </deployment> <deployment> <deployment_sequence_number>11</deployment_sequence_number> <deployment_data_type>rpm</deployment_data_type> <repositories> <repository>cdrom</repository> <repository>http://server/...</repository> </repositories> <packages> <package>apache2-prefork</package> <package>apache2-doc</package> </packages> </deployment> <deployment> <deployment_sequence_number>20</deployment_sequence_number> <deployment_data_type>tgz</deployment_data_type> <tgz_uri>nfs://server/system_config_data.tar.gz</tgz_uri> </deployment> <deployment> <deployment_sequence_number>20</deployment_sequence_number> <deployment_data_type>tgz</deployment_data_type> <tgz_uri>https://server/user_data.tar.gz</tgz_uri> </deployment> <deployment> <deployment_sequence_number>90</deployment_sequence_number> <deployment_data_type>script</deployment_data_type> <script_uri>/home/admin/install_3rdparty_database.sh</script_uri> </deployment> <deployment> <deployment_sequence_number>91</deployment_sequence_number> <deployment_data_type>tgz</deployment_data_type> <tgz_uri>smb://server/database_data.tar.gz</tgz_uri> </deployment> </deployments> ---------------------------------------------------------------------- This would run the following deployment steps: 1. The basic system gets installed as an image from ftp://server/base_system_image.raw 2. On top of the basic system additional software like Apache gets installed as RPM packages. 3. Deployment steps with same sequence number run simultaneously where one installs system config data (e.g. the Apache config) and the other one installs user data (e.g. home directories) by untar of two tar.gz archives in parallel. 4. On top of that a third party database gets installed by running a script from the admin's home directory. 5. Finally the database data gets installed by untar of a tar.gz archive. As far as I understand your current proposal there is another major difference of my proposal compared to your proposal: In your current proposal one would need to replace the whole deployment step if one needs to do any (possibly tiny) special stuff by running a selfmade script. In contrast in my proposal running a selfmade script would be supported by the deployment module of the new installer. Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH -- Maxfeldstrasse 5 -- 90409 Nuernberg -- Germany HRB 16746 (AG Nuernberg) GF: Jeff Hawn, Jennifer Guild, Felix Imendoerffer -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org