[opensuse-buildservice] How to clean up /srv/obs/sources?
Hi, I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk... Then I found the culprit. It's the /srv/obs/sources folder: 6.4G repos 521G sources Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-( -- With best regards, Carsten Hoeger
On Tue, 30 Aug 2011 11:15:46 +0200, Carsten Hoeger <choeger@opensuse.org> wrote:
Hi,
I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk...
Then I found the culprit. It's the /srv/obs/sources folder:
6.4G repos 521G sources
Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-(
Hello, I use a script called from crontab In this example I clean /tmp/ files greater than 7 days old. # cat delete-tmp-greater7days-old.sh #!/usr/bin/sh echo "remove /tmp/ files older than 7 days after in various locations" /bin/df -h # list files then remove find /tmp/. -mtime +7 -exec ls "{}" \; find /tmp/. -mtime +7 -exec rm -rf "{}" \; echo "used after :" /bin/df -h Note:your lines would read find /srv/obs/sources/. -mtime +7 -exec ls "{}" \; find /srv/obs/sources/. -mtime +7 -exec rm -rf "{}" \; Cheers Glenn -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Aug 30, 2011 at 01:45:42PM +1000, doiggl@velocitynet.com.au wrote:
On Tue, 30 Aug 2011 11:15:46 +0200, Carsten Hoeger <choeger@opensuse.org> wrote:
Hi,
I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk...
Then I found the culprit. It's the /srv/obs/sources folder:
6.4G repos 521G sources
Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-(
Hello, I use a script called from crontab In this example I clean /tmp/ files greater than 7 days old.
# cat delete-tmp-greater7days-old.sh
#!/usr/bin/sh echo "remove /tmp/ files older than 7 days after in various locations" /bin/df -h
# list files then remove find /tmp/. -mtime +7 -exec ls "{}" \; find /tmp/. -mtime +7 -exec rm -rf "{}" \;
echo "used after :" /bin/df -h
Note:your lines would read find /srv/obs/sources/. -mtime +7 -exec ls "{}" \; find /srv/obs/sources/. -mtime +7 -exec rm -rf "{}" \;
This would delete all sources after 7 days of inactivity. Probably not what is wanted... ;) Ciao, Marcus -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Aug 30, Marcus Meissner wrote:
I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk...
Then I found the culprit. It's the /srv/obs/sources folder:
6.4G repos 521G sources
Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-(
Hello, I use a script called from crontab In this example I clean /tmp/ files greater than 7 days old.
# cat delete-tmp-greater7days-old.sh
#!/usr/bin/sh echo "remove /tmp/ files older than 7 days after in various locations" /bin/df -h
# list files then remove find /tmp/. -mtime +7 -exec ls "{}" \; find /tmp/. -mtime +7 -exec rm -rf "{}" \;
echo "used after :" /bin/df -h
Note:your lines would read find /srv/obs/sources/. -mtime +7 -exec ls "{}" \; find /srv/obs/sources/. -mtime +7 -exec rm -rf "{}" \;
This would delete all sources after 7 days of inactivity.
Probably not what is wanted... ;)
Hehe, no, not really. What I'd love to keep is: Just the files, that belong to what I still have in my published repos. Does anybody have some related sql queries which I could place into a script? Of course I will share the results. -- With best regards, Carsten Hoeger
Am Dienstag, 30. August 2011, 15:30:20 schrieb Carsten Hoeger:
On Tue, Aug 30, Marcus Meissner wrote:
I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk...
Then I found the culprit. It's the /srv/obs/sources folder:
6.4G repos 521G sources
Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-(
Hello, I use a script called from crontab In this example I clean /tmp/ files greater than 7 days old.
# cat delete-tmp-greater7days-old.sh
#!/usr/bin/sh echo "remove /tmp/ files older than 7 days after in various locations" /bin/df -h
# list files then remove find /tmp/. -mtime +7 -exec ls "{}" \; find /tmp/. -mtime +7 -exec rm -rf "{}" \;
echo "used after :" /bin/df -h
Note:your lines would read find /srv/obs/sources/. -mtime +7 -exec ls "{}" \; find /srv/obs/sources/. -mtime +7 -exec rm -rf "{}" \;
This would delete all sources after 7 days of inactivity.
Probably not what is wanted... ;)
Hehe, no, not really.
What I'd love to keep is: Just the files, that belong to what I still have in my published repos.
Does anybody have some related sql queries which I could place into a script? Of course I will share the results.
You don't need SQL for that, best is to write some perl code parsing all .rev files and compare that with stored files. And in best case another tool, which allows you to short the history before. -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
I've attached a tarball of a source cleaner that reads rev files, marks containers & source files for deletion. It leverages OBS services. Jeff Glanz Dell | PG Release Engineering Team office + 1 512 724 9509 -----Original Message----- From: Adrian Schröter [mailto:adrian@suse.de] Sent: Tuesday, August 30, 2011 8:42 AM To: opensuse-buildservice@opensuse.org Cc: Carsten Hoeger Subject: Re: [opensuse-buildservice] How to clean up /srv/obs/sources? Am Dienstag, 30. August 2011, 15:30:20 schrieb Carsten Hoeger:
On Tue, Aug 30, Marcus Meissner wrote:
I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk...
Then I found the culprit. It's the /srv/obs/sources folder:
6.4G repos 521G sources
Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-(
Hello, I use a script called from crontab In this example I clean /tmp/ files greater than 7 days old.
# cat delete-tmp-greater7days-old.sh
#!/usr/bin/sh echo "remove /tmp/ files older than 7 days after in various locations" /bin/df -h
# list files then remove find /tmp/. -mtime +7 -exec ls "{}" \; find /tmp/. -mtime +7 -exec rm -rf "{}" \;
echo "used after :" /bin/df -h
Note:your lines would read find /srv/obs/sources/. -mtime +7 -exec ls "{}" \; find /srv/obs/sources/. -mtime +7 -exec rm -rf "{}" \;
This would delete all sources after 7 days of inactivity.
Probably not what is wanted... ;)
Hehe, no, not really.
What I'd love to keep is: Just the files, that belong to what I still have in my published repos.
Does anybody have some related sql queries which I could place into a script? Of course I will share the results.
You don't need SQL for that, best is to write some perl code parsing all .rev files and compare that with stored files. And in best case another tool, which allows you to short the history before. -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
On Tue, Aug 30, Jeff_Glanz@Dell.com wrote:
I've attached a tarball of a source cleaner that reads rev files, marks containers & source files for deletion. It leverages OBS services.
Great!!! Thank you very much!
Jeff Glanz Dell | PG Release Engineering Team office + 1 512 724 9509
-----Original Message----- From: Adrian Schröter [mailto:adrian@suse.de] Sent: Tuesday, August 30, 2011 8:42 AM To: opensuse-buildservice@opensuse.org Cc: Carsten Hoeger Subject: Re: [opensuse-buildservice] How to clean up /srv/obs/sources?
Am Dienstag, 30. August 2011, 15:30:20 schrieb Carsten Hoeger:
On Tue, Aug 30, Marcus Meissner wrote:
I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk...
Then I found the culprit. It's the /srv/obs/sources folder:
6.4G repos 521G sources
Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-(
Hello, I use a script called from crontab In this example I clean /tmp/ files greater than 7 days old.
# cat delete-tmp-greater7days-old.sh
#!/usr/bin/sh echo "remove /tmp/ files older than 7 days after in various locations" /bin/df -h
# list files then remove find /tmp/. -mtime +7 -exec ls "{}" \; find /tmp/. -mtime +7 -exec rm -rf "{}" \;
echo "used after :" /bin/df -h
Note:your lines would read find /srv/obs/sources/. -mtime +7 -exec ls "{}" \; find /srv/obs/sources/. -mtime +7 -exec rm -rf "{}" \;
This would delete all sources after 7 days of inactivity.
Probably not what is wanted... ;)
Hehe, no, not really.
What I'd love to keep is: Just the files, that belong to what I still have in my published repos.
Does anybody have some related sql queries which I could place into a script? Of course I will share the results.
You don't need SQL for that, best is to write some perl code parsing all .rev files and compare that with stored files.
And in best case another tool, which allows you to short the history before.
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- With best regards, Carsten Hoeger
Hi Jeff,
I've attached a tarball of a source cleaner that reads rev files, marks containers & source files for deletion. It leverages OBS services.
It looks like it only deletes a very small fraction of stale files within the source folder (and took about half a day to figure that out ;-))... 129M from ~550G is not much. I started it using -preview and replaced the {$projectGlob} with "*" in the code. Does anybody have at least a basic documentation on how that .rev/.del/.mrev stuff works? I have the suspicion that it isn't possible at all to determine all unused sources. -- With best regards, Carsten Hoeger
participants (5)
-
Adrian Schröter -
Carsten Hoeger -
doiggl＠velocitynet.com.au -
Jeff_Glanz＠Dell.com -
Marcus Meissner