Mailinglist Archive: zypp-devel (115 mails)
| < Previous | Next > |
Re: [zypp-devel] Extending pattern definition
- From: Lukas Ocilka <lukas.ocilka@xxxxxxx>
- Date: Mon, 02 Jul 2007 11:39:08 +0200
- Message-id: <4688C7BC.60509@xxxxxxx>
Duncan Mac-Vicar Prett wrote:
> On Friday 22 June 2007 15:54:27 Lukas Ocilka wrote:
>> Workflow for packages was never wanted and might be added later.
>> Currently we support only Add-Ons Products and Patterns.
>
> So we do it in a hacky way resolvable kind per resolvable kind. What will you
> do when we have to support it with packages?
>
> It has to be a clean way to trigger the workflow after *any* event has
> happened. As I said the idea is great and can improve user experience, but
> you are relating workflow to patterns (which have no relation, because you
> never know what a pattern adds) and because you are doing a second "solving"
> at YaST level (very similar to how selections were handled before)
...
Hi,
I was in NBG for a short while during the HackWeek where I asked Duncan
to clarify his (and others' ideas) about the proposed solution. Finally
he persuaded me that his (their) solution is better. Anyway, it it more
than I ever expected from libzypp to do, that's why my proposal was
about just extending the .pat files.
So, about the current solution:
Every single resolvable can have its own workflow (pattern, language,
product, package...). All these pieces of information also with the
order of workflows are written into single XML file for one product
(URL). Libzypp parses the file and if any resolvable having its own
workflow is either selected or unselected to be installed or removed
from the pool, libzypp useses a callback to YaST via package-bindings.
YaST then operates with the workflow itself and libzypp doesn't care.
What is it good for?
- It solves all possible resolvables at once
- Callbacks can be redefined without touching libzypp
- YaST doesn't need to use any hacks for querying which resolvables were
changes and which of them have a workflow etc.
See the attached proposal (changed and old one) or see this link:
http://users.suse.cz/~locilka/PatternsAndAddOnsWithWorkflows/
Anyway, we need a manager's blessing for such change of proposal :)
because now more things would be implemented in libzypp then I first
expected.
Thanks & Bye
Lukas
--
Lukas Ocilka, YaST Developer (xn--luk-gla45d)
-----------------------------------------------------------------
SUSE LINUX, s. r. o., Lihovarska 1060/12, Praha 9, Czech Republic
FATE #129: Framework for pattern based Installation/Deployment
==============================================================
About
-----
This new feature is comparable to workflows defined by Add-On products.
Currently, any Add-On used in installation can influence the installation
workflow by adding some new installation workflow steps, removing old ones,
or other modifications in installation settings (propose firewall to be
enabled by default...). Moreover Add-On workflow can be defined for case
when Add-On product is added later on a running system.
The main difference of this feature to Add-Ons is that it is about
patterns which can be also part of some Add-Ons.
Patterns are sets of required and optional RPM packages with additional
information such as supported architecture, dependencies on other
resolvables. Products have usually some patterns defined (Add-Ons are
also products).
Notice: After discussing with Libzypps (mostly Duncan), the plan has changed
in the way that not only patterns but also packages, languages, ...
can have their own workflow through an unified API (callbacks from
libzypp). Anyway, this document later on describes only patterns and
their workflows but keep in mind that the other resolvables are
covered as well.
Expected Result
---------------
* It should be possible to modify a workflow (installation/update/...)
by patterns.
* This new feature should not be in conflict with the current solution
of Add-On workflows.
Patterns vs. Add-Ons
--------------------
* Patterns can influence the installation workflow the very same
way as Add-On workflows do
* that's why Patterns should also use the same workflow definition
as Add-On Products (the very same XML format)
* and also why Patterns and Add-Ons should use the very same functions
for that (maximal unification)
* Workflow of patterns, languages, packages ... will be adjusted via
libzypp callbacks
Involved Parts of the System
----------------------------
- LibZYPP (data)
- YaST - Add-Ons (installation, running system)
- Installation / Upgrade (workflow)
- Packager (sw_single)
- Pkg-Bindings (to get data from LibZYPP)
Document mainly speaks of handling Patterns but Add-Ons will
be handled the very same or similar way...
Supported Scenarios
-------------------
1.) Product installation from media, network, etc...
- Can use several Add-On product, also with own workflow
and/or patterns
- Can use base-product patterns
2.) Add-On Product installation on a running system
(`yast2 add-on`)
- Can use own workflow
If hasn't own workflow, `yast2 sw_single` is called
- Can use own patterns
3.) Common Software installation
(`yast2 sw_single`)
- Can use patterns
Each installation type needs a special handling. Some scenarios
will be unified at a high level of functional API (sw_single).
All scenarios will be, at least, unified at low-level functional API.
Patterns Scenarios (technical proposal)
---------------------------------------
The symbol '$[x]' means: workflow definition including workflow order.
1.) Installation
These two callbacks can be called from libzypp:
* AddResolvableWorkflow (resolvable_type, resolvable_name, src_id, $[x]);
* RemoveResolvableWorkflow (resolvable_type, resolvable_name, src_id, $[x]);
At the end of the installation proposal, workflows are merged into
one which is then executed.
2.) Software Management (Installation / Removing)
This is `yast2 sw_single` on a running system.
The same two callbacks can be called here as well:
* AddResolvableWorkflow (...);
* RemoveResolvableWorkflow (...);
After the packages, patterns, ... are installed, sw_single checks whether
there are some workflows to be executed and adjusts the workflow wizard
if needed.
3.) Installation of an Add-On Product on a Running System
3.1.) Add-On Product Installation with Own Workflow
Some Add-On products can have their own installation
workflow defined in installation.xml (stage: normal,
mode: installation).
They are handled this way:
* Add Add-On Product as a normal installation source
* Call defined installation workflow
Patterns can be selected during the Add-On installation workflow
run, the solution is also simple:
The same callbacks can be called here too. At the end, they will be merged
into one workflow and the workflow executed.
3.2.) Add-On Product Installation without Own Workflow
These Add-On products don't have any own workflow.
They are handled this way:
* Add Add-On Product as a normal installation source
* Call sw_single
That's why we can solve them the very same way as
the case 2.) Software Management
Sorting of Additional Workflows
-------------------------------
Sorting means the order in which these patterns or add-ons workflows are
presented to users. Some kind of sorting can be done even in these
workflows themselves if one depends on another.
This solution should cover both patterns and add-ons workflows
* Add-Ons
An add-on's workflow, which is added first, is also merged first.
If some add-on is added by a dependency in the base-add-on, it's
also added later which also means merged later.
* Patterns
The least difficult order method is an integer value for each pattern
that has a workflow. At first, patterns would be sorted by the Add-On
which they belong to (At first, all add-ons are merged, then all patterns
from the first add-on, then all patterns from the second add-on...).
Second, by that integer variable (the lowest number the earlier they
appear). A missing order number would mean a random order (actually,
an order how pkg-bindings return them).
Example of Sorting
------------------
This is what we have:
* Base Product
- Base Workflow
* 1st Add-On
- Add-On 1 Workflow
- Pattern 1A, Order 35
- Pattern 1C, Order 6
- Pattern 1R, Order 355
* 2nd Add-On
- Add-On 2 Workflow
- Pattern 2A, Order 6
- Pattern 2F, Order 211
- Pattern 2G, Order 2
Simple solution:
(Patterns are sorted, firstlty by add-ons, secondly by pattern-order)
* Base Workflow
+ Add-On 1 Workflow
+ Add-On 2 Workflow
+ Pattern 1C, Order 6
+ Pattern 1A, Order 35
+ Pattern 1R, Order 355
+ Pattern 2G, Order 2
+ Pattern 2A, Order 6
+ Pattern 2F, Order 211
Problematic solution:
(Patterns are sorted by pattern-order shared among all add-ons)
* Base Workflow
+ Add-On 1 Workflow
+ Add-On 2 Workflow
+ Pattern 2G, Order 2
+ Pattern 1C, Order 6 (conflict!)
+ Pattern 2A, Order 6 (conflict!)
+ Pattern 1A, Order 35
+ Pattern 2F, Order 211
+ Pattern 1R, Order 355
I'd rather have a simple solution extensible in the future. The second
solution, that sorts all patterns at once, could make more errors.
If some Add-On is dependent on another one (or product), it knowns that
product's workflow then it can also provide an additional workflow that
would update (add/remove/replace) the workflow on a proper position.
The only thing we need to provide is a proper documentation with examples.
Low Level API Proposal (technical)
----------------------------------
This API uses some new terms that need to be explained:
* Workflow Store
- Kind of database of installation or configuration workflows
* Base Workflow
- The initial workflow defined by the base product
- In case of running system, this will be probably empty
* Additional Workflow
- Any workflow defined by Add-On or Pattern in installation
or Pattern in running system
* Final Workflow
- Workflow that contains the base workflow modified by all
additional workflows
Proposed set of functions unified for Add-Ons and Patterns:
* SetBaseWorkflow()
- Stores the initial base-product workflow
ProductControll::initial_workflows
ProductControll::initial_proposals
ProductControll::initial_inst_finish
ProductControll::initial_clone_modules
* AddWorkflow()
- Adds a new workflow definition into the Workflow Store
- Add-On workflows will be identified by source ID (unique)
- Patterns workflows will be identified by source ID plus
pattern name (unique)
* RemoveWorkflow()
- Removes an already stored workflow definition from the Workflow Store
* MergeWorkflows()
- Uses the Base workflow as the initial workflow and merges
all additional workflows in use
- Merging might need to follow some order in which these workflows
are merged to avoid conflicts and undefined behavior
* RedrawWizardSteps()
- Intentionally redraws the installation wizard steps to match the
current merged workflows.
* HaveAdditionalWorkflows()
- Returns whether some additional workflows were used or not
And, of course, some other helpers that will be used by functions
mentioned above.
Workflow Callbacks (from LibZYPP)
---------------------------------
* AddResolvableWorkflow ();
- Called from libzypp when any resolvable (that has own workflow)
is selected to be installed. WorkflowManager is then called to
propagate that workflow to YaST.
* RemoveResolvableWorkflow ();
- Called from libzypp when any resolvable (that has own workflow)
is removed from the pool. WorkflowManager is then called to remove
the workflow from YaST.
For YaST, it's better to have a path to some XML file instead of getting
the XML as string data from libzypp. However YaST can store the data
to some temporary file because it needs to do so anyway (just because of
adding/removing workflows to/from the main workflow).
Needed ZYPP Support for Pattern Workflows (technical)
-----------------------------------------------------
There needs to be an additional XML file with workflows of resolvables
on the media.
/resolvables_workflows.xml
This XML file would need to contain these pieces of information:
XML file
|
+- Workflow definition item (1)
| |
| +- Resolvable name
| |
| +- Resolvable type
| |
| +- Workflow order
| |
| +- Workflow definition (added, removed dialogs, etc.)
| (Or a link to file containing the XML definition)
|
+- Workflow definition item (2)
| |
| +- ...
|
+- Workflow definition item (n)
|
+- ...
All this needs to be sent via the callback to YaST. "Workflow definition"
and "Workflow order" doesn't need to be sent in case of removing the
workflow.
Needed Pkg-Binding Support
--------------------------
Pkg bindings are requested to propagate libzypp bindings to YaST.
> On Friday 22 June 2007 15:54:27 Lukas Ocilka wrote:
>> Workflow for packages was never wanted and might be added later.
>> Currently we support only Add-Ons Products and Patterns.
>
> So we do it in a hacky way resolvable kind per resolvable kind. What will you
> do when we have to support it with packages?
>
> It has to be a clean way to trigger the workflow after *any* event has
> happened. As I said the idea is great and can improve user experience, but
> you are relating workflow to patterns (which have no relation, because you
> never know what a pattern adds) and because you are doing a second "solving"
> at YaST level (very similar to how selections were handled before)
...
Hi,
I was in NBG for a short while during the HackWeek where I asked Duncan
to clarify his (and others' ideas) about the proposed solution. Finally
he persuaded me that his (their) solution is better. Anyway, it it more
than I ever expected from libzypp to do, that's why my proposal was
about just extending the .pat files.
So, about the current solution:
Every single resolvable can have its own workflow (pattern, language,
product, package...). All these pieces of information also with the
order of workflows are written into single XML file for one product
(URL). Libzypp parses the file and if any resolvable having its own
workflow is either selected or unselected to be installed or removed
from the pool, libzypp useses a callback to YaST via package-bindings.
YaST then operates with the workflow itself and libzypp doesn't care.
What is it good for?
- It solves all possible resolvables at once
- Callbacks can be redefined without touching libzypp
- YaST doesn't need to use any hacks for querying which resolvables were
changes and which of them have a workflow etc.
See the attached proposal (changed and old one) or see this link:
http://users.suse.cz/~locilka/PatternsAndAddOnsWithWorkflows/
Anyway, we need a manager's blessing for such change of proposal :)
because now more things would be implemented in libzypp then I first
expected.
Thanks & Bye
Lukas
--
Lukas Ocilka, YaST Developer (xn--luk-gla45d)
-----------------------------------------------------------------
SUSE LINUX, s. r. o., Lihovarska 1060/12, Praha 9, Czech Republic
FATE #129: Framework for pattern based Installation/Deployment
==============================================================
About
-----
This new feature is comparable to workflows defined by Add-On products.
Currently, any Add-On used in installation can influence the installation
workflow by adding some new installation workflow steps, removing old ones,
or other modifications in installation settings (propose firewall to be
enabled by default...). Moreover Add-On workflow can be defined for case
when Add-On product is added later on a running system.
The main difference of this feature to Add-Ons is that it is about
patterns which can be also part of some Add-Ons.
Patterns are sets of required and optional RPM packages with additional
information such as supported architecture, dependencies on other
resolvables. Products have usually some patterns defined (Add-Ons are
also products).
Notice: After discussing with Libzypps (mostly Duncan), the plan has changed
in the way that not only patterns but also packages, languages, ...
can have their own workflow through an unified API (callbacks from
libzypp). Anyway, this document later on describes only patterns and
their workflows but keep in mind that the other resolvables are
covered as well.
Expected Result
---------------
* It should be possible to modify a workflow (installation/update/...)
by patterns.
* This new feature should not be in conflict with the current solution
of Add-On workflows.
Patterns vs. Add-Ons
--------------------
* Patterns can influence the installation workflow the very same
way as Add-On workflows do
* that's why Patterns should also use the same workflow definition
as Add-On Products (the very same XML format)
* and also why Patterns and Add-Ons should use the very same functions
for that (maximal unification)
* Workflow of patterns, languages, packages ... will be adjusted via
libzypp callbacks
Involved Parts of the System
----------------------------
- LibZYPP (data)
- YaST - Add-Ons (installation, running system)
- Installation / Upgrade (workflow)
- Packager (sw_single)
- Pkg-Bindings (to get data from LibZYPP)
Document mainly speaks of handling Patterns but Add-Ons will
be handled the very same or similar way...
Supported Scenarios
-------------------
1.) Product installation from media, network, etc...
- Can use several Add-On product, also with own workflow
and/or patterns
- Can use base-product patterns
2.) Add-On Product installation on a running system
(`yast2 add-on`)
- Can use own workflow
If hasn't own workflow, `yast2 sw_single` is called
- Can use own patterns
3.) Common Software installation
(`yast2 sw_single`)
- Can use patterns
Each installation type needs a special handling. Some scenarios
will be unified at a high level of functional API (sw_single).
All scenarios will be, at least, unified at low-level functional API.
Patterns Scenarios (technical proposal)
---------------------------------------
The symbol '$[x]' means: workflow definition including workflow order.
1.) Installation
These two callbacks can be called from libzypp:
* AddResolvableWorkflow (resolvable_type, resolvable_name, src_id, $[x]);
* RemoveResolvableWorkflow (resolvable_type, resolvable_name, src_id, $[x]);
At the end of the installation proposal, workflows are merged into
one which is then executed.
2.) Software Management (Installation / Removing)
This is `yast2 sw_single` on a running system.
The same two callbacks can be called here as well:
* AddResolvableWorkflow (...);
* RemoveResolvableWorkflow (...);
After the packages, patterns, ... are installed, sw_single checks whether
there are some workflows to be executed and adjusts the workflow wizard
if needed.
3.) Installation of an Add-On Product on a Running System
3.1.) Add-On Product Installation with Own Workflow
Some Add-On products can have their own installation
workflow defined in installation.xml (stage: normal,
mode: installation).
They are handled this way:
* Add Add-On Product as a normal installation source
* Call defined installation workflow
Patterns can be selected during the Add-On installation workflow
run, the solution is also simple:
The same callbacks can be called here too. At the end, they will be merged
into one workflow and the workflow executed.
3.2.) Add-On Product Installation without Own Workflow
These Add-On products don't have any own workflow.
They are handled this way:
* Add Add-On Product as a normal installation source
* Call sw_single
That's why we can solve them the very same way as
the case 2.) Software Management
Sorting of Additional Workflows
-------------------------------
Sorting means the order in which these patterns or add-ons workflows are
presented to users. Some kind of sorting can be done even in these
workflows themselves if one depends on another.
This solution should cover both patterns and add-ons workflows
* Add-Ons
An add-on's workflow, which is added first, is also merged first.
If some add-on is added by a dependency in the base-add-on, it's
also added later which also means merged later.
* Patterns
The least difficult order method is an integer value for each pattern
that has a workflow. At first, patterns would be sorted by the Add-On
which they belong to (At first, all add-ons are merged, then all patterns
from the first add-on, then all patterns from the second add-on...).
Second, by that integer variable (the lowest number the earlier they
appear). A missing order number would mean a random order (actually,
an order how pkg-bindings return them).
Example of Sorting
------------------
This is what we have:
* Base Product
- Base Workflow
* 1st Add-On
- Add-On 1 Workflow
- Pattern 1A, Order 35
- Pattern 1C, Order 6
- Pattern 1R, Order 355
* 2nd Add-On
- Add-On 2 Workflow
- Pattern 2A, Order 6
- Pattern 2F, Order 211
- Pattern 2G, Order 2
Simple solution:
(Patterns are sorted, firstlty by add-ons, secondly by pattern-order)
* Base Workflow
+ Add-On 1 Workflow
+ Add-On 2 Workflow
+ Pattern 1C, Order 6
+ Pattern 1A, Order 35
+ Pattern 1R, Order 355
+ Pattern 2G, Order 2
+ Pattern 2A, Order 6
+ Pattern 2F, Order 211
Problematic solution:
(Patterns are sorted by pattern-order shared among all add-ons)
* Base Workflow
+ Add-On 1 Workflow
+ Add-On 2 Workflow
+ Pattern 2G, Order 2
+ Pattern 1C, Order 6 (conflict!)
+ Pattern 2A, Order 6 (conflict!)
+ Pattern 1A, Order 35
+ Pattern 2F, Order 211
+ Pattern 1R, Order 355
I'd rather have a simple solution extensible in the future. The second
solution, that sorts all patterns at once, could make more errors.
If some Add-On is dependent on another one (or product), it knowns that
product's workflow then it can also provide an additional workflow that
would update (add/remove/replace) the workflow on a proper position.
The only thing we need to provide is a proper documentation with examples.
Low Level API Proposal (technical)
----------------------------------
This API uses some new terms that need to be explained:
* Workflow Store
- Kind of database of installation or configuration workflows
* Base Workflow
- The initial workflow defined by the base product
- In case of running system, this will be probably empty
* Additional Workflow
- Any workflow defined by Add-On or Pattern in installation
or Pattern in running system
* Final Workflow
- Workflow that contains the base workflow modified by all
additional workflows
Proposed set of functions unified for Add-Ons and Patterns:
* SetBaseWorkflow()
- Stores the initial base-product workflow
ProductControll::initial_workflows
ProductControll::initial_proposals
ProductControll::initial_inst_finish
ProductControll::initial_clone_modules
* AddWorkflow()
- Adds a new workflow definition into the Workflow Store
- Add-On workflows will be identified by source ID (unique)
- Patterns workflows will be identified by source ID plus
pattern name (unique)
* RemoveWorkflow()
- Removes an already stored workflow definition from the Workflow Store
* MergeWorkflows()
- Uses the Base workflow as the initial workflow and merges
all additional workflows in use
- Merging might need to follow some order in which these workflows
are merged to avoid conflicts and undefined behavior
* RedrawWizardSteps()
- Intentionally redraws the installation wizard steps to match the
current merged workflows.
* HaveAdditionalWorkflows()
- Returns whether some additional workflows were used or not
And, of course, some other helpers that will be used by functions
mentioned above.
Workflow Callbacks (from LibZYPP)
---------------------------------
* AddResolvableWorkflow ();
- Called from libzypp when any resolvable (that has own workflow)
is selected to be installed. WorkflowManager is then called to
propagate that workflow to YaST.
* RemoveResolvableWorkflow ();
- Called from libzypp when any resolvable (that has own workflow)
is removed from the pool. WorkflowManager is then called to remove
the workflow from YaST.
For YaST, it's better to have a path to some XML file instead of getting
the XML as string data from libzypp. However YaST can store the data
to some temporary file because it needs to do so anyway (just because of
adding/removing workflows to/from the main workflow).
Needed ZYPP Support for Pattern Workflows (technical)
-----------------------------------------------------
There needs to be an additional XML file with workflows of resolvables
on the media.
/resolvables_workflows.xml
This XML file would need to contain these pieces of information:
XML file
|
+- Workflow definition item (1)
| |
| +- Resolvable name
| |
| +- Resolvable type
| |
| +- Workflow order
| |
| +- Workflow definition (added, removed dialogs, etc.)
| (Or a link to file containing the XML definition)
|
+- Workflow definition item (2)
| |
| +- ...
|
+- Workflow definition item (n)
|
+- ...
All this needs to be sent via the callback to YaST. "Workflow definition"
and "Workflow order" doesn't need to be sent in case of removing the
workflow.
Needed Pkg-Binding Support
--------------------------
Pkg bindings are requested to propagate libzypp bindings to YaST.
| < Previous | Next > |