[suse-mirror] openSUSE 12.1 Update Mirror Sync Problem
This is my first post here. I am not sure if this has been posted to the mirror mailing list or not. There is a problem with the 12.1 Update build process. RPM's are built and then two different copies exist with different signatures. Files have same Name, Size, and Creation Date. But have a different Time. The first file get put in the Mater Mirror and is synced to the remote mirrors. The second file, at a later date then enters the Master Mirror, but not all of the mirrors get the second version. I guess the sync method used might be filename only. This causes digest errors because the second file's checksum is in the repodata but the first file is in the repo. For mirror admins, until the openSUSE team can find the problem that is creating two RPM's please use time stamps or checksums for mirror syncs. openSUSE Forum Post http://forums.opensuse.org/english/get-technical-help-here/applications/4714... openSUSE Bug Report https://bugzilla.novell.com/show_bug.cgi?id=742772 Dave W -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
On Tue, 21 Feb 2012 12:37:17 -0500 David Westfall <david.westfall@red-inc.us> wrote:
This is my first post here. I am not sure if this has been posted to the mirror mailing list or not.
There is a problem with the 12.1 Update build process. RPM's are built and then two different copies exist with different signatures. Files have same Name, Size, and Creation Date. But have a different Time. The first file get put in the Mater Mirror and is synced to the remote mirrors. The second file, at a later date then enters the Master Mirror, but not all of the mirrors get the second version. I guess the sync method used might be filename only. This causes digest errors because the second file's checksum is in the repodata but the first file is in the repo.
For mirror admins, until the openSUSE team can find the problem that is creating two RPM's please use time stamps or checksums for mirror syncs.
openSUSE Forum Post http://forums.opensuse.org/english/get-technical-help-here/applications/4714...
openSUSE Bug Report https://bugzilla.novell.com/show_bug.cgi?id=742772
1. the problem on our side *should* be fixed. we have reports from multiple users that they dont see that problem anymore. 2. syncing even just the update tree with --checksum will not make you happy at all. I did the mistake and it will not finish in time. 3. If your own mirror (intermediate mirror) doesnt pick up the fixed packages. which should have newer timestamps, then just sync the 12.1 update tree with --checksum. That actually finishes in reasonable time. 4. In the bug you referenced, you wanted to contact the admin of the mirror, which you are using. Did you hear anything from them yet? 5. I verified stage.opensuse.org *and* rsync.opensuse.org that both have the correct repositories twice. (once just a few minutes ago) maybe we should remove your mirror from the rotation on download.o.o until you heard back from the admin? darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
I joined the mailing list, so the remote mirror admins would know to check their syncs. I am trying to get rsync access to opensuse.org. Mirrors that are broken. Kernel.org: Has not updated since Oct. Admin has not answered email. TDS.net: FTP side has blank directory, cannot get sync from. Admin answered but has not fixed. ANL.gov:Digest Errors. Admin no longer answering emails. Temple.edu:Digest Errors. Admin has not answered email. How many mirrors do, I as a user, need to test? According to the Mirror Infrastructure page, in order to be a registered mirror the site must give read only rsync access to openSUSE for scanning to update the redirector. In the forum post you will see that using aria2c to fetch RPM's, that had digest errors, the redirector pointed to temple.edu which also had errors. So it looks like your rsync scanner only looks for file names. Can the scanner be changed to look for bad files and build a list of mirrors that are either out of date or have bad copies and inform the admins. You could post the list here, openSUSE should make this list a requirement for being a registered mirror. Or send the admin an email, they gave you one when the mirror was registered. Right now a file I get from anl.gov has a bad signature but is in the meta4 file has having the correct version. 12.1/Update/x86_64 # rpm --checksig pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm: rsa sha1 (MD5) PGP MD5 NOT OK Update/x86_64 # grep anl pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm.meta4 <url location="us" priority="7">http://mirror.anl.gov/pub/opensuse/opensuse/update/12.1/x86_64/pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm</url> The openSUSE mirror scanner could be used to find these files. http://en.opensuse.org/openSUSE:Mirror_infrastructure#Registering_your_mirro... Dave W
1. the problem on our side *should* be fixed. we have reports from multiple users that they dont see that problem anymore.
2. syncing even just the update tree with --checksum will not make you happy at all. I did the mistake and it will not finish in time.
3. If your own mirror (intermediate mirror) doesnt pick up the fixed packages. which should have newer timestamps, then just sync the 12.1 update tree with --checksum. That actually finishes in reasonable time.
4. In the bug you referenced, you wanted to contact the admin of the mirror, which you are using. Did you hear anything from them yet?
5. I verified stage.opensuse.org *and* rsync.opensuse.org that both have the correct repositories twice. (once just a few minutes ago)
maybe we should remove your mirror from the rotation on download.o.o until you heard back from the admin?
darix
-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
-- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
On Tue, 21 Feb 2012 15:28:35 -0500 David Westfall <david.westfall@red-inc.us> wrote:
I joined the mailing list, so the remote mirror admins would know to check their syncs.
I am trying to get rsync access to opensuse.org.
Mirrors that are broken.
Kernel.org: Has not updated since Oct. Admin has not answered email. TDS.net: FTP side has blank directory, cannot get sync from. Admin answered but has not fixed. ANL.gov:Digest Errors. Admin no longer answering emails. Temple.edu:Digest Errors. Admin has not answered email.
How many mirrors do, I as a user, need to test?
none. you should use download.opensuse.org, if you encounter an error with a mirror you report the mirror and we remove it from the rotation.
According to the Mirror Infrastructure page, in order to be a registered mirror the site must give read only rsync access to openSUSE for scanning to update the redirector. In the forum post you will see that using aria2c to fetch RPM's, that had digest errors, the redirector pointed to temple.edu which also had errors. So it looks like your rsync scanner only looks for file names. Can the scanner be changed to look for bad files and build a list of mirrors that are either out of date or have bad copies and inform the admins.
no. this would be too expensive. it would mean downloading the files from the mirrors. a full scan over all mirrors in the DB already takes 6-7hours.
You could post the list here, openSUSE should make this list a requirement for being a registered mirror. Or send the admin an email, they gave you one when the mirror was registered.
so i will contact anl.gov, kernel.org and temple.edu off list to get things sorted. in the mean time I disabled anl.gov for kernel.org and temple.edu I would need to know which mirrors there you tried exactly. both run multiple mirrors.
Right now a file I get from anl.gov has a bad signature but is in the meta4 file has having the correct version.
12.1/Update/x86_64 # rpm --checksig pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm: rsa sha1 (MD5) PGP MD5 NOT OK
Update/x86_64 # grep anl pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm.meta4 <url location="us" priority="7">http://mirror.anl.gov/pub/opensuse/opensuse/update/12.1/x86_64/pdns-backend-postgresql-2.9.22-7.3.1.x86_64.rpm</url>
The openSUSE mirror scanner could be used to find these files.
http://en.opensuse.org/openSUSE:Mirror_infrastructure#Registering_your_mirro...
please use download.opensuse.org, it will redirect you to close mirrors. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
I now have rsync access to rsync.opensuse.org. I wrote a script to rsync the directories I want (12.1 i586 i686 noarc and x86_64). After the rsync I ran my repoVerify.sh script. Update # repoVerify.sh repo OS Name openSUSE repo OS Version 12.1 repo Name Update Comparing Checksums for all RPM's against repodata. Checksum for i686/glibc-2.14.1-14.14.1.i686.rpm does not match repodata. Comparing Checksums for all DRPM's against repodata. Checking all RPM's for bad signature's. Checking all DRPM's for bad signature's. Fetch Correct Files?y Fetching correct files. i686/glibc-2.14.1-14.14.1.i686.rpm 2012-03-20 10:08:49.380522 NOTICE - Download complete: /atlas/Distributions/openSUSE/12.1/Update/i686/glibc-2.14.1-14.14.1.i686.rpm.meta4 [#2 SIZE:0B/1.3MiB(0%) CN:0 SPD:0Bs] [Checksum:#2 768.0KiB/1.3MiB(54%)] 2012-03-20 10:08:49.478152 NOTICE - Verification finished successfully. file=/atlas/Distributions/openSUSE/12.1/Update/i686/glibc-2.14.1-14.14.1.i686.rpm 2012-03-20 10:08:49.478370 NOTICE - Download complete: /atlas/Distributions/openSUSE/12.1/Update/i686/glibc-2.14.1-14.14.1.i686.rpm Download Results: gid|stat|avg speed |path/URI ===+====+===========+=========================================================== 1| OK| 15.6KiB/s|/atlas/Distributions/openSUSE/12.1/Update/i686/glibc-2.14.1-14.14.1.i686.rpm.meta4 2| OK| 0B/s|/atlas/Distributions/openSUSE/12.1/Update/i686/glibc-2.14.1-14.14.1.i686.rpm Status Legend: (OK):download completed. **** Tested file before aria2c fetch 12.1/Update # grep 'name="glibc"' tmpFileList|grep i686 <package pkgid="13ab3d2dc7416ea949a1fe252fd83ddee600395dc74b79fa8a984a527ccc9a9e" name="glibc" arch="i686"> <package pkgid="af368a9d98cf593e77f5fbdca3a0ef963264ceee58a06b336bef300e03d93fff" name="glibc" arch="i686"> <package pkgid="31b30dea751822729275cfffc9a7619ad133217cd3fb54f4146e89b2ccb7981b" name="glibc" arch="i686"> 12.1/Update # sha256sum i686/glibc-2.14.1-14.14.1.i686.rpm 79ad1370628e518259510af7167476ec08a66c7d929b3e562eff2020b91c39e2 i686/glibc-2.14.1-14.14.1.i686.rpm 12.1/Update # l -h i686/glibc-2.14.1-14.14.1.i686.rpm -rw-r--r-- 1 root root 1.4M Dec 9 07:55 i686/glibc-2.14.1-14.14.1.i686.rpm *** After aria2c fetch. 12.1/Update # sha256sum i686/glibc-2.14.1-14.14.1.i686.rpm 79ad1370628e518259510af7167476ec08a66c7d929b3e562eff2020b91c39e2 i686/glibc-2.14.1-14.14.1.i686.rpm This tells me that after the cecksums for repodata have been calculated this file was replaced. I assume that rpm checksums are calculated for every file each time you update repodata and that repodata is built by and autoamted script. If so, it is like I said there are different versions of files floating around your servers and they find there way in to your master mirror at different times. I wrote repoVerify.sh on 28 feb and run it after every mirror update. This is the first time that I saw glibc. And after my script fetchs the correct file with aria2c I would run my script again and not get any errors in the repo. My script still says that glibc has the wrong checksum. Do you want me to add this to the bug report? Would you like a copy of my script? Dave W -- To unsubscribe, e-mail: mirror+unsubscribe@opensuse.org To contact the owner, email: mirror+owner@opensuse.org
participants (2)
-
David Westfall -
Marcus Rückert