[opensuse] File system safety during crash - ext3 or ext4?
Hi! A system that is going to be running 24/7 (in practice), but will not have UPS (so, please, don't suggest that), which file system to run and how? Yes, it will have backups. Still, there is a data disk that should be kept alive... what is the safest way to go? I was initially thinking about ext3 + full journal of data and metadata. However, according to Wikipedia, this isn't safe from crash (http://en.wikipedia.org/wiki/Ext3#No_checksumming_in_journal). However, it says that: "The ext3 barrier option is not enabled by default on almost all popular Linux distributions[citation needed] except openSUSE" Is this true (11.1)? And what does it mean and how do I check it? At least I didn't see the mentioned "barrier=1" in YaST - Partitioner. Wikipedia then mentiones that ext4 is released and that it is to fix this thing. However, again according to wikipedia (http://en.wikipedia.org/wiki/Ext4#Caveats) it again has different, but similar problems. Of course, with ext4 I would use the same journaling. So, what to choose and how to set up? Disk/Partition is 2Tb. The data will be mainly large photographs (RAW) and video files. Performance is important, but safety during power outages is more important. Separate system drive (Ext3). -- HG. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2009-05-17 at 15:50 +0300, HG wrote:
A system that is going to be running 24/7 (in practice), but will not have UPS (so, please, don't suggest that), which file system to run and how? Yes, it will have backups.
...
So, what to choose and how to set up? Disk/Partition is 2Tb. The data will be mainly large photographs (RAW) and video files. Performance is important, but safety during power outages is more important. Separate system drive (Ext3).
There is no real, safe, alternative to having and UPS. Yes or Yes, use it. If you insist... suffer. I would suggest reiserfs, but it is in decline nowdays. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkoQCf8ACgkQtTMYHG2NR9ULDwCeJu/bRya7PxHGxjEV2gIQ8byD HZ8An1i5DdHfOXufKjyUD489Cj9uoX0D =Cbdn -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sunday May 17 2009, Carlos E. R. wrote:
...
There is no real, safe, alternative to having and UPS. Yes or Yes, use it.
If you insist... suffer. I would suggest reiserfs, but it is in decline nowdays.
UPS, like backups and brushing and flossing your teeth, are required only for the stuff (data or teeth) you want to keep.
-- Cheers, Carlos E. R.
RRS -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
HG pecked at the keyboard and wrote:
Hi!
A system that is going to be running 24/7 (in practice), but will not have UPS (so, please, don't suggest that), which file system to run and how? Yes, it will have backups.
_*NO*_ system can run 24/7 without a UPS. I cannot think of any filesystem that can survive repeated crashes without losing data. _NO_ business runs a data center *without* a UPS. You said not to suggest the obvious safety net but I did anyway. -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hi!
This is _*NO*_*BUSINESS*_ and no data center. There is no uptime
requirements, nothing. There are backups.
Now, with that in mind, do you know which one to choose ext3 or ext4
and how to set it up so that the full data journaling works as
expected?
Thanks,
HG
On Sun, May 17, 2009 at 4:12 PM, Ken Schneider - openSUSE
HG pecked at the keyboard and wrote:
Hi!
A system that is going to be running 24/7 (in practice), but will not have UPS (so, please, don't suggest that), which file system to run and how? Yes, it will have backups.
_*NO*_ system can run 24/7 without a UPS. I cannot think of any filesystem that can survive repeated crashes without losing data. _NO_ business runs a data center *without* a UPS.
You said not to suggest the obvious safety net but I did anyway.
-- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-- HG. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
HG wrote:
Hi!
This is _*NO*_*BUSINESS*_ and no data center. There is no uptime requirements, nothing. There are backups.
Now, with that in mind, do you know which one to choose ext3 or ext4 and how to set it up so that the full data journaling works as expected?
Given those circumstances, I would opt for ext4. /Per -- Per Jessen, Zürich (19.1°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
HG wrote:
Hi!
A system that is going to be running 24/7 (in practice), but will not have UPS (so, please, don't suggest that), which file system to run and how? Yes, it will have backups.
I have a number of systems running JFS - until recently none of them had UPSs, but we have also had less than one outage per year on average since 2001. The UPS is being installed as we speak, but it's mostly due to lengthy database recovery time, not file system recovery.
So, what to choose and how to set up? Disk/Partition is 2Tb. The data will be mainly large photographs (RAW) and video files. Performance is important, but safety during power outages is more important. Separate system drive (Ext3).
How about restart time? Even a journalled filesystem may end up requiring a full check, which on 2Tb could take quite a while. /Per -- Per Jessen, Zürich (25.2°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hi!
A system that is going to be running 24/7 (in practice), but will not have UPS (so, please, don't suggest that), which file system to run and how? Yes, it will have backups.
Still, there is a data disk that should be kept alive... what is the safest way to go? I was initially thinking about ext3 + full journal of data and metadata. However, according to Wikipedia, this isn't safe from crash (http://en.wikipedia.org/wiki/Ext3#No_checksumming_in_journal). However, it says that: "The ext3 barrier option is not enabled by default on almost all popular Linux distributions[citation needed] except openSUSE" Is this true (11.1)? And what does it mean and how do I check it? At least I didn't see the mentioned "barrier=1" in YaST - Partitioner.
Wikipedia then mentiones that ext4 is released and that it is to fix this thing. However, again according to wikipedia (http://en.wikipedia.org/wiki/Ext4#Caveats) it again has different, but similar problems. Of course, with ext4 I would use the same journaling.
So, what to choose and how to set up? Disk/Partition is 2Tb. The data will be mainly large photographs (RAW) and video files. Performance is important, but safety during power outages is more important. Separate system drive (Ext3).
-- HG. I have to agree with all of the other comments. A UPS is cheap insurance. I used to support over 800 servers, and always argued with my manager about
On Sunday 17 May 2009 05:50:50 am HG wrote: the costs at each site. There was a saying I used " Pay me now or pay me later". All you need is a ups that will give you enough time to shutdown orderly, no data lost. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hello!
So far I got 5 replies, but no answers. So, I'll try again... does
anybody know what the wikipedia articles are talking about in terms of
those data loss scenarios with ext3 and ext4? And more importantly,
which one I should use (to be safest) and how should I set up those?
All insight and help regarding ext3/4 setup is greatly appreciated.
Thanks,
HG
On Sun, May 17, 2009 at 3:50 PM, HG
Still, there is a data disk that should be kept alive... what is the safest way to go? I was initially thinking about ext3 + full journal of data and metadata. However, according to Wikipedia, this isn't safe from crash (http://en.wikipedia.org/wiki/Ext3#No_checksumming_in_journal). However, it says that: "The ext3 barrier option is not enabled by default on almost all popular Linux distributions[citation needed] except openSUSE" Is this true (11.1)? And what does it mean and how do I check it? At least I didn't see the mentioned "barrier=1" in YaST - Partitioner.
Wikipedia then mentiones that ext4 is released and that it is to fix this thing. However, again according to wikipedia (http://en.wikipedia.org/wiki/Ext4#Caveats) it again has different, but similar problems. Of course, with ext4 I would use the same journaling.
So, what to choose and how to set up? Disk/Partition is 2Tb. The data will be mainly large photographs (RAW) and video files. Performance is important, but safety during power outages is more important. Separate system drive (Ext3).
-- HG. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sun, May 17, 2009 at 12:04 PM, HG
Hello!
So far I got 5 replies, but no answers. So, I'll try again... does anybody know what the wikipedia articles are talking about in terms of those data loss scenarios with ext3 and ext4? And more importantly, which one I should use (to be safest) and how should I set up those?
Ext4 is an adolescent with great potential, but it still makes big mistakes routinely. I have been watching the ext3 / ext4 mailing list for a few months. Almost never a ext3 data loss complaint. Almost daily similar complaints about ext4. Especially on large filesystems. 2 TB is smaller than most of the complaints, but I would not trust my data to ext4 on any size filesystem at this point. Further the OpenSUSE release still has ext4 marked as experimental. That means, don't use if you don't want to lose your data. Greg -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hi!
Thanks Jdd and Greg!
I didn't know ext4 was still experimental - that definitely settles
this. Thanks for the info. I'll then go with ext3 and data journaling.
Thanks again.
Br,
HG
On Sun, May 17, 2009 at 7:38 PM, Greg Freemyer
On Sun, May 17, 2009 at 12:04 PM, HG
wrote: Hello!
So far I got 5 replies, but no answers. So, I'll try again... does anybody know what the wikipedia articles are talking about in terms of those data loss scenarios with ext3 and ext4? And more importantly, which one I should use (to be safest) and how should I set up those?
Ext4 is an adolescent with great potential, but it still makes big mistakes routinely.
I have been watching the ext3 / ext4 mailing list for a few months. Almost never a ext3 data loss complaint. Almost daily similar complaints about ext4. Especially on large filesystems. 2 TB is smaller than most of the complaints, but I would not trust my data to ext4 on any size filesystem at this point.
Further the OpenSUSE release still has ext4 marked as experimental. That means, don't use if you don't want to lose your data.
Greg -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sun, May 17, 2009 at 12:47 PM, HG
I didn't know ext4 was still experimental - that definitely settles this. Thanks for the info. I'll then go with ext3 and data journaling. Thanks again.
I have to second Carlos. I've used Reiserfs exclusively for about 8 or 9 years, and I have NEVER lost data due to a crash, and I've experienced consitent poiwer failures and lockups over the years. Even tho Reiser is being depreciated, I still use it and will continue to until the ext systems get up to Reiser's level. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Quoting Larry Stotler
On Sun, May 17, 2009 at 12:47 PM, HG
wrote: I didn't know ext4 was still experimental - that definitely settles this. Thanks for the info. I'll then go with ext3 and data journaling. Thanks again.
I have to second Carlos. I've used Reiserfs exclusively for about 8 or 9 years, and I have NEVER lost data due to a crash, and I've experienced consitent poiwer failures and lockups over the years. Even tho Reiser is being depreciated, I still use it and will continue to until the ext systems get up to Reiser's level.
YMMV, I have lost data too many times, and had the nailbiting experience too often of it going through its FS repair routine to repair a corrupted FS to put it on a new installation, even though it usually repaired the damage without apparent data loss. With the size of modern disks, its major advantage (IMHO- no fixed block size, so no wasted space with lots of small files) has very little benefit anymore. I think I have one computer left in storage with the home partition on ReiserFS, but no recent ones. Ext3FS in my experience has been much better behaved. Just my $0.02USD, Jeffrey -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Jeffrey L. Taylor wrote:
I have to second Carlos. I've used Reiserfs exclusively for about 8
YMMV, I have lost data too many times, and had the nailbiting experience too often of it going through its FS repair routine to repair a corrupted FS to put it on a new installation, even though it usually repaired the damage without apparent data loss.
So which is it? You lost data or didn't lose data? Scary processes are only scary if you don't understand them. I've been thru the Reiser repair mainy times over many years and it has always worked flawlessly. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sun, 2009-05-17 at 12:51 -0700, John Andersen wrote:
Jeffrey L. Taylor wrote:
I have to second Carlos. I've used Reiserfs exclusively for about 8
YMMV, I have lost data too many times, and had the nailbiting experience too often of it going through its FS repair routine to repair a corrupted FS to put it on a new installation, even though it usually repaired the damage without apparent data loss.
So which is it? You lost data or didn't lose data?
Scary processes are only scary if you don't understand them. I've been thru the Reiser repair mainy times over many years and it has always worked flawlessly.
I switched 2 years ago from ReiserFS to ext3 and have since never any data loss anymore. ReiserFS had many errors most time I rebooted and causes also some data loss. Since using ext3, never any loss anymore and booting is normal with normal recovery actions if needed. frans -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2009-05-17 at 22:01 +0200, Frans de Boer wrote:
Scary processes are only scary if you don't understand them. I've been thru the Reiser repair mainy times over many years and it has always worked flawlessly.
I switched 2 years ago from ReiserFS to ext3 and have since never any data loss anymore. ReiserFS had many errors most time I rebooted and causes also some data loss. Since using ext3, never any loss anymore and booting is normal with normal recovery actions if needed.
I have had data loss on ext3, xfs, and reiserfs. None is failsafe entirely; but in my limited experience, reiserfs is very resilient to power failures - and does the fsck very fast, several times faster than ext3. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkoQcIkACgkQtTMYHG2NR9U7CgCghfb6PrFKqhGCxQ7W/yaG+q2e nD8AnA5EDOVdasLKfL+eNRH2p+ZBwAsj =7Egi -----END PGP SIGNATURE-----
On Sunday 17 May 2009 03:16:08 pm Carlos E. R. wrote:
On Sunday, 2009-05-17 at 22:01 +0200, Frans de Boer wrote:
Scary processes are only scary if you don't understand them. I've been thru the Reiser repair mainy times over many years and it has always worked flawlessly.
I switched 2 years ago from ReiserFS to ext3 and have since never any data loss anymore. ReiserFS had many errors most time I rebooted and causes also some data loss. Since using ext3, never any loss anymore and booting is normal with normal recovery actions if needed.
I have had data loss on ext3, xfs, and reiserfs. None is failsafe entirely; but in my limited experience, reiserfs is very resilient to power failures - and does the fsck very fast, several times faster than ext3.
The lost data could be just lost in a file system. The nature of reiserfs, to sort all in binary trees can under certain circumstances funny effects. I had archive with old items stored in separate directory. After crash and forced file system repair run they appeared again in original places as subdirectories of themselves. For instance /home/username/Documents/images as /home/username/Documents/images/images . -- Regards, Rajko http://news.opensuse.org/category/people-of-opensuse/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sun, May 17, 2009 at 4:16 PM, Carlos E. R.
I have had data loss on ext3, xfs, and reiserfs. None is failsafe entirely; but in my limited experience, reiserfs is very resilient to power failures - and does the fsck very fast, several times faster than ext3.
I have never had data loss due to a reiser problem. Mine have always been hardware related. I was able to recover my laptop drive early last year and then clone it and repair reiser and get the data back. But, Spinrite(great tool - highly recommended) found that the problems are bad sectors. No file system is immune to bad sectors, especially when it his the main parts of the system. I hated having to fsck ext2 all the time due to power failures or hard lockups on overclocked systems(which a UPS won't help). I should invest in a cheap ups, but rent is more important atm. I haven't benched ext3 or ext4 much, mainly because reiser has always been so reliable and recovers very quickly from a power/lockup issue. However, all my drives are reiser. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Quoting John Andersen
Jeffrey L. Taylor wrote:
I have to second Carlos. I've used Reiserfs exclusively for about 8
YMMV, I have lost data too many times, and had the nailbiting experience too often of it going through its FS repair routine to repair a corrupted FS to put it on a new installation, even though it usually repaired the damage without apparent data loss.
So which is it? You lost data or didn't lose data?
Sometimes I lost data after ReiserFS corruption, sometimes I didn't.
Scary processes are only scary if you don't understand them. I've been thru the Reiser repair mainy times over many years and it has always worked flawlessly.
Corrupted FS is always scary unless the repair ALWAYS works. Which it didn't. And often when it didn't, I'd find out much later. Jeffrey -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Quoting HG
Hello!
So far I got 5 replies, but no answers. So, I'll try again... does anybody know what the wikipedia articles are talking about in terms of those data loss scenarios with ext3 and ext4? And more importantly, which one I should use (to be safest) and how should I set up those?
All insight and help regarding ext3/4 setup is greatly appreciated.
The answer has to depend on what are your costs for lost data. As a freelance programmer, a UPS has to only save me from ONE major restore from backups to be worthwhile. On my wife's laptop, with no income critical data, the answer is different. Though as soon as we move Quicken with the checkbook registers, and investments, the answer changes. Re-entering a month's worth of bank, mutual fund, etc. statements is probably more costly than a UPS with 10 minutes of power. Just enough time to save the files, close the apps, and shutdown. Though probably hiberating/suspend-to-disk is more cost effective. Jeffrey -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sunday May 17 2009, Jeffrey L. Taylor wrote:
...
The answer has to depend on what are your costs for lost data. As a freelance programmer, a UPS has to only save me from ONE major restore from backups to be worthwhile. On my wife's laptop, with no income critical data, the answer is different.
Laptops, having their own batteries and software for preemptively hibernating / suspending when the battery nears a critical level of discharge, the issue of UPS is essentialy moot: the device is effectively equipped with its own UPS. It would, in fact, be a bad idea to plug a laptop into a UPS, since its battery (the UPS's) is better used to give time to other systems for graceful shutdown (or, conversely, to justify the use of a less capacious UPS).
...
Jeffrey
Randall Schulz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Quoting Randall R Schulz
On Sunday May 17 2009, Jeffrey L. Taylor wrote:
...
The answer has to depend on what are your costs for lost data. As a freelance programmer, a UPS has to only save me from ONE major restore from backups to be worthwhile. On my wife's laptop, with no income critical data, the answer is different.
Laptops, having their own batteries and software for preemptively hibernating / suspending when the battery nears a critical level of discharge, the issue of UPS is essentialy moot: the device is effectively equipped with its own UPS. It would, in fact, be a bad idea to plug a laptop into a UPS, since its battery (the UPS's) is better used to give time to other systems for graceful shutdown (or, conversely, to justify the use of a less capacious UPS).
You are quite right, desktop habits die hard. A good surge suppressor is a better solution for a laptop. Jeffrey -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2009-05-17 at 12:38 -0500, Jeffrey L. Taylor wrote:
minutes of power. Just enough time to save the files, close the apps, and shutdown. Though probably hiberating/suspend-to-disk is more cost effective.
Yes, it is possible to have a machine hibernate automatically when power fails, which is much better than powering off: there is no lost data, and the machine recovers very fast, all services and programs active. Only network connections suffer, and not always. But this configuration is not enabled by default. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkoQbncACgkQtTMYHG2NR9XFkwCffNr67kQZzBjDFjYFyh2/X/EB 84IAnR1NirtjDGdUhDFQgbWt4lA8qKeY =dg5N -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos E. R. wrote:
On Sunday, 2009-05-17 at 12:38 -0500, Jeffrey L. Taylor wrote:
minutes of power. Just enough time to save the files, close the apps, and shutdown. Though probably hiberating/suspend-to-disk is more cost effective.
Yes, it is possible to have a machine hibernate automatically when power fails, which is much better than powering off: there is no lost data, and the machine recovers very fast, all services and programs active. Only network connections suffer, and not always.
But this configuration is not enabled by default.
Not only is it not enabled by default, its not even available in many machines. Most laptops have this ability because they have built in battery monitoring. And I have to point out for those singing the prises of UPSs, that you are very little better off with a UPS that does not have monitoring. You ride out the short disconnects but you crash just as hard when the batteries eventually die. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2009-05-17 at 13:14 -0700, John Andersen wrote:
On Sunday, 2009-05-17 at 12:38 -0500, Jeffrey L. Taylor wrote:
minutes of power. Just enough time to save the files, close the apps, and shutdown. Though probably hiberating/suspend-to-disk is more cost effective.
Yes, it is possible to have a machine hibernate automatically when power fails, which is much better than powering off: there is no lost data, and the machine recovers very fast, all services and programs active. Only network connections suffer, and not always.
But this configuration is not enabled by default.
Not only is it not enabled by default, its not even available in many machines. Most laptops have this ability because they have built in battery monitoring.
And I have to point out for those singing the prises of UPSs, that you are very little better off with a UPS that does not have monitoring. You ride out the short disconnects but you crash just as hard when the batteries eventually die.
Obviously! That is what I use. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkoQdHYACgkQtTMYHG2NR9XUhQCeIQFOeRbIilbo+TFTdGkFM70t FrkAoIqKG104yAKvE915ddiCyxgGNErT =wLTb -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (12)
-
Carlos E. R.
-
Frans de Boer
-
Greg Freemyer
-
HG
-
Jeffrey L. Taylor
-
John Andersen
-
Ken Schneider - openSUSE
-
Larry Stotler
-
Per Jessen
-
Rajko M.
-
Randall R Schulz
-
russbucket