Hello,

During db compression, prometheus failed with:

caller=db.go:766 component=tsdb msg=reloadBlocks err="find blocks: open
/var/lib/prometheus/metrics/: too many open files"
(and many more of those)

After I restarted prometheus, it was able to recover itself nicely.

My prometheus instance is using 826 after a restart, with 581 being metric
chunks/index/wal and 132 sockets:

# ls -la /proc/6546/fd -l | wc -l
826

I increased prometheus db range to 1y. Maybe that was caused the fileno limit
to be reached. I manually increased the default 1024 limitnofiles to 2048 but I
think the service could consider a little bit more file limit by default for
that service.