[opensuse] How to import users and group ids?
In my effort to upgrade our servers to openSuSE12.3, I have yet again encountered a problem that required a lot of time to debug and track down. This has plagued me and frosted my tootsies before... I often copy data (whole directory trees) that is required for various servers from an older OS into the realm of the new OS that I am upgrading to. For example, the MySQL databases, the Apache and Tomcat web site data, James mail data, etc. In the process user and group ID's from the old version of files can "change" because the underlying integer value - UID/GID - for these IDs is different for the two different OS versions. This in turn can lead to hard to debug security problems. It seems to me that when installing a new version of openSuSE, there ought to be a way to import the /etc/passwd and /etc/group files from an older OS into the new OS BEFORE anything gets actually installed. In that way, the underlying integer UID/GID could be kept consistent and not lead to problems that can/will result if/when a particular user/group identifier changes in value. If there is a way to do this, I was unable to find it in the documentation or via Google, so pointers will be much appreciated. Or perhaps, is there a tool that can merge and update a system, and all its files, from one set of /etc/passwd and /etc/group files into another set? Again, I am not able to find a tool that does it.... I know that I can use YaST to change the UID/GID for each user or group ID individually, and it seems to work OK, but gets tedious. Hoping there a better way? Marc... -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/17/2013 09:12 AM, Marc Chamberlin wrote:
In my effort to upgrade our servers to openSuSE12.3, I have yet again encountered a problem that required a lot of time to debug and track down. This has plagued me and frosted my tootsies before... I often copy data (whole directory trees) that is required for various servers from an older OS into the realm of the new OS that I am upgrading to. For example, the MySQL databases, the Apache and Tomcat web site data, James mail data, etc. In the process user and group ID's from the old version of files can "change" because the underlying integer value - UID/GID - for these IDs is different for the two different OS versions. This in turn can lead to hard to debug security problems.
Yes, you should try to avoid UID/GID changes.
It seems to me that when installing a new version of openSuSE, there ought to be a way to import the /etc/passwd and /etc/group files from an older OS into the new OS BEFORE anything gets actually installed. In that way, the underlying integer UID/GID could be kept consistent and not lead to problems that can/will result if/when a particular user/group identifier changes in value. If there is a way to do this, I was unable to find it in the documentation or via Google, so pointers will be much appreciated.
Or perhaps, is there a tool that can merge and update a system, and all its files, from one set of /etc/passwd and /etc/group files into another set? Again, I am not able to find a tool that does it.... I know that I can use YaST to change the UID/GID for each user or group ID individually, and it seems to work OK, but gets tedious. Hoping there a better way?
useradd/groupadd have options to create the account with a specified uid and gid, instead of the default (just taking the next free one). If you know /etc/{passwd,shadow,group,gshadow} well enough - btw, there are manpages for that ;-) -, i.e., if you know what you're doing, then you could just copy the relevant lines from the old system to the new one. Clashing numbers/names have to be avoided, of course. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/17/2013 09:12 AM, Marc Chamberlin wrote:
In my effort to upgrade our servers to openSuSE12.3, I have yet again encountered a problem that required a lot of time to debug and track down. This has plagued me and frosted my tootsies before... I often copy data (whole directory trees) that is required for various servers from an older OS into the realm of the new OS that I am upgrading to. For example, the MySQL databases, the Apache and Tomcat web site data, James mail data, etc. In the process user and group ID's from the old version of files can "change" because the underlying integer value - UID/GID - for these IDs is different for the two different OS versions. This in turn can lead to hard to debug security problems. Yes, you should try to avoid UID/GID changes.
It seems to me that when installing a new version of openSuSE, there ought to be a way to import the /etc/passwd and /etc/group files from an older OS into the new OS BEFORE anything gets actually installed. In that way, the underlying integer UID/GID could be kept consistent and not lead to problems that can/will result if/when a particular user/group identifier changes in value. If there is a way to do this, I was unable to find it in the documentation or via Google, so pointers will be much appreciated.
Or perhaps, is there a tool that can merge and update a system, and all its files, from one set of /etc/passwd and /etc/group files into another set? Again, I am not able to find a tool that does it.... I know that I can use YaST to change the UID/GID for each user or group ID individually, and it seems to work OK, but gets tedious. Hoping there a better way? useradd/groupadd have options to create the account with a specified uid and gid, instead of the default (just taking the next free one).
If you know /etc/{passwd,shadow,group,gshadow} well enough - btw, there are manpages for that ;-) -, i.e., if you know what you're doing, then you could just copy the relevant lines from the old system to the new one. Clashing numbers/names have to be avoided, of course.
Have a nice day, Berny Thanks Berny for taking the time to reply, and yes how I KNOW that UID/GID changes can be a problem! ;-) I don't know about a file called gshadow, nor do I have any man pages on it, but I am aware of the other relevant files - passwd, shadow, and group and have studied the man
On 4/17/2013 12:48 AM, Bernhard Voelker wrote: pages for each. The approach you suggested, to change the actual entries in the passwd and group files to reflect the UID/GID that I would want for each ID will not work, if it is done after the fact that files and directories have already been created and installed. And I don't know of a tool that will use these sort of edits/changes and update all the files and directory properties, throughout the entire file system, to reflect the changes made to the UID/GID within the passwd and group files. As I mentioned YaST does do it, but as far as I know, only on an individual per ID basis, not wholesale, which makes it tedious to use. Thanks for the nice day wish, it IS a nice day today here in Washougal, Washington! You too! Marc.. -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Hi Marc, On 04/17/2013 06:19 PM, Marc Chamberlin wrote:
Thanks Berny for taking the time to reply, and yes how I KNOW that UID/GID changes can be a problem! ;-) I don't know about a file called gshadow, nor do I have any man pages on it, but I am aware of the other relevant files - passwd, shadow, and group and have studied the man pages for each.
Ah, sorry, gshadow does not exist anymore. It was similar to what the shadow file is to passwd, but for the group file. No worries.
The approach you suggested, to change the actual entries in the passwd and group files to reflect the UID/GID that I would want for each ID will not work, if it is done after the fact that files and directories have already been created and installed.
Right. I assumed that the accounts and the files did not yet exist.
And I don't know of a tool that will use these sort of edits/changes and update all the files and directory properties, throughout the entire file system, to reflect the changes made to the UID/GID within the passwd and group files.
If you only use plain UNIX permissions, i.e., not ACLs, then chown and chgrp is the right tool for that purpose. The following command will change the ownership of all files with the UID 1000 to 2000, restricted to the current file system (-xdev): $ find . -xdev -uid 1000 -print0 | xargs -0 chown -c 2000 Likewise for groups - changing the group from GID 100 to 200: $ find . -xdev -gid 100 -print0 | xargs -0 chgrp -c 200 Documentation about chown, chgrp: info coreutils "chown invocation" info coreutils "chgrp invocation" Ah yes, only root can do that ;-)
Thanks for the nice day wish, it IS a nice day today here in Washougal, Washington! You too! Marc..
Cool, that city looks like a nice place to live! Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2013-04-17 at 00:12 -0700, Marc Chamberlin wrote:
In my effort to upgrade our servers to openSuSE12.3, I have yet again encountered a problem that required a lot of time to debug and track down. This has plagued me and frosted my tootsies before... I often copy data (whole directory trees) that is required for various servers from an older OS into the realm of the new OS that I am upgrading to. For example, the MySQL databases, the Apache and Tomcat web site data, James mail data, etc. In the process user and group ID's from the old version of files can "change" because the underlying integer value - UID/GID - for these IDs is different for the two different OS versions. This in turn can lead to hard to debug security problems.
It seems to me that when installing a new version of openSuSE, there ought to be a way to import the /etc/passwd and /etc/group files from an older OS into the new OS BEFORE anything gets actually installed.
And there is, there is such a feature while you run the installer. There are two places. One, when you get the partition proposal, you can tell yast to instead read the fstab from an existing filesystem. You point yast to a partition, it looks around, reads fstabs, and displays the filesystem layout that would result. The next is during user creation: again, there is an entry to import the user list from a previous system. You can choose the partition to read from, and they are imported. However, this applies to users created with UID ≥1000, I don't know what happens below with users the system uses for services. When I have services like databases, mail service, web, etc, I do system upgrades instead, not fresh installs. <http://en.opensuse.org/SDB:System_upgrade> Online upgrade method <http://en.opensuse.org/SDB:Offline_upgrade> Offline upgrade method <URL="http://doc.opensuse.org/documentation/html/openSUSE/opensuse-startup/cha.update.html> Chapter 16. Upgrading the System and System Changes - -- Cheers, Carlos E. R. (from 12.1 x86_64 "Asparagus" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAlFudNUACgkQtTMYHG2NR9U1RQCgmAuMLkreFrhDGyk5EPrUxsWz B7YAn0kzsa7tmJij1UzK54dt+kz5PzN1 =WUoT -----END PGP SIGNATURE-----
Carlos - Thank you for your reply. I am aware that if I update an existing openSuSE system "in place" then I won't have these sort of UID/GID issues. Unfortunately, I have been burned in the past doing so, and because an upgrade can take quite awhile to complete, we want to keep the older version in a runnable state while we upgrade and test the new version. In particular we don't always upgrade our servers as each new version of openSuSE comes out, and often have to jump across several versions. So that means we must do fresh installs of each new version of openSuSE we upgrade to, in separate partitions and migrate services/data from the older version to the newer version manually. So that is why I am asking if there is an easy way to migrate UIDs/GIDs from an older system to a newer one, either during installation or after the initial installation is complete. I kinda suspect not, but will leave the question open hoping for enlightenment.. Meanwhile will continue to use YaST and bore myself to tears doing it on a case by case basis while avoiding conflicts... Cheers to you also.... Marc... On 4/17/2013 3:09 AM, Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wednesday, 2013-04-17 at 00:12 -0700, Marc Chamberlin wrote:
In my effort to upgrade our servers to openSuSE12.3, I have yet again encountered a problem that required a lot of time to debug and track down. This has plagued me and frosted my tootsies before... I often copy data (whole directory trees) that is required for various servers from an older OS into the realm of the new OS that I am upgrading to. For example, the MySQL databases, the Apache and Tomcat web site data, James mail data, etc. In the process user and group ID's from the old version of files can "change" because the underlying integer value - UID/GID - for these IDs is different for the two different OS versions. This in turn can lead to hard to debug security problems.
It seems to me that when installing a new version of openSuSE, there ought to be a way to import the /etc/passwd and /etc/group files from an older OS into the new OS BEFORE anything gets actually installed.
And there is, there is such a feature while you run the installer.
There are two places.
One, when you get the partition proposal, you can tell yast to instead read the fstab from an existing filesystem. You point yast to a partition, it looks around, reads fstabs, and displays the filesystem layout that would result.
The next is during user creation: again, there is an entry to import the user list from a previous system. You can choose the partition to read from, and they are imported.
However, this applies to users created with UID ≥1000, I don't know what happens below with users the system uses for services. When I have services like databases, mail service, web, etc, I do system upgrades instead, not fresh installs.
<http://en.opensuse.org/SDB:System_upgrade> Online upgrade method <http://en.opensuse.org/SDB:Offline_upgrade> Offline upgrade method <URL="http://doc.opensuse.org/documentation/html/openSUSE/opensuse-startup/cha.update.html> Chapter 16. Upgrading the System and System Changes
- -- Cheers, Carlos E. R. (from 12.1 x86_64 "Asparagus" at Telcontar)
-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux)
iEYEARECAAYFAlFudNUACgkQtTMYHG2NR9U1RQCgmAuMLkreFrhDGyk5EPrUxsWz B7YAn0kzsa7tmJij1UzK54dt+kz5PzN1 =WUoT -----END PGP SIGNATURE-----
-- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2013-04-17 at 09:38 -0700, Marc Chamberlin wrote:
Carlos - Thank you for your reply. I am aware that if I update an existing openSuSE system "in place" then I won't have these sort of UID/GID issues. Unfortunately, I have been burned in the past doing so, and because an upgrade can take quite awhile to complete, we want to keep the older version in a runnable state while we upgrade and test the new version. In particular we don't always upgrade our servers as each new version of openSuSE comes out, and often have to jump across several versions. So that means we must do fresh installs of each new version of openSuSE we upgrade to, in separate partitions and migrate services/data from the older version to the newer version manually. So that is why I am asking if there is an easy way to migrate UIDs/GIDs from an older system to a newer one, either during installation or after the initial installation is complete. I kinda suspect not, but will leave the question open hoping for enlightenment.. Meanwhile will continue to use YaST and bore myself to tears doing it on a case by case basis while avoiding conflicts...
Ok, understandable. Question: your problem is with UIDs above 1000, or below? I would suggest you create the adequate partitions in the destination system, but do not install. Then create the /etc directory, and put in it the groups and password files (and shadows). Then start the installation and tell it to import the users, as I explained previously. If that does not work, perhaps you have to clone the root filesystem at least before yast will recognize the user list as valid. - -- Cheers, Carlos E. R. (from 12.1 x86_64 "Asparagus" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iEYEARECAAYFAlFvANIACgkQtTMYHG2NR9XlkQCeOvp522YPJIrTvlqtIuaIPP2U cDkAn1nczrh8nMcxBN+0x7BE5esK1aAB =pMim -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 4/17/2013 1:06 PM, Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wednesday, 2013-04-17 at 09:38 -0700, Marc Chamberlin wrote:
Carlos - Thank you for your reply. I am aware that if I update an existing openSuSE system "in place" then I won't have these sort of UID/GID issues. Unfortunately, I have been burned in the past doing so, and because an upgrade can take quite awhile to complete, we want to keep the older version in a runnable state while we upgrade and test the new version. In particular we don't always upgrade our servers as each new version of openSuSE comes out, and often have to jump across several versions. So that means we must do fresh installs of each new version of openSuSE we upgrade to, in separate partitions and migrate services/data from the older version to the newer version manually. So that is why I am asking if there is an easy way to migrate UIDs/GIDs from an older system to a newer one, either during installation or after the initial installation is complete. I kinda suspect not, but will leave the question open hoping for enlightenment.. Meanwhile will continue to use YaST and bore myself to tears doing it on a case by case basis while avoiding conflicts...
Ok, understandable.
Question: your problem is with UIDs above 1000, or below? Both! System and User UIDs/GIDs...
I would suggest you create the adequate partitions in the destination system, but do not install. Then create the /etc directory, and put in it the groups and password files (and shadows).
Then start the installation and tell it to import the users, as I explained previously.
If that does not work, perhaps you have to clone the root filesystem at least before yast will recognize the user list as valid. OK. Thanks Carlos, I am down to our last 2 servers that need to be upgraded, so will give your suggestions a try when I get to them... Bernhard Voelker gave me a couple short shell commands that are proving helpful as well... Appreciate the suggestions...
Marc... -- "The Truth is out there" - Spooky -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (3)
-
Bernhard Voelker -
Carlos E. R. -
Marc Chamberlin