size of firefox sqlite database files
Hello: Currently I use firefox 91.4.1esr in openSUSE 15.3. I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history. Why don't the file sizes decrease after deleting their content by deleting history? I have many profiles and want to decrease the profiles' size. I thought bu deleting history I can do it but not. Can someone explain this? Thanks, Istvan
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100 Hello: Currently I use firefox 91.4.1esr in openSUSE 15.3. I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history. Why don't the file sizes decrease after deleting their content by deleting history? From the SQLite docs [1]: When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse. But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you. -- Bob Rogers http://www.rgrjr.com/ [1] https://www.sqlite.org/lang_vacuum.html
From: Bob Rogers <rogers-suse@rgrjr.homedns.org> Date: Thu, 6 Jan 2022 11:13:44 -0500 . . . But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you. Don't do that -- Firefox in 15.3 appears to use its own built-in SQLite version, so if you use the openSUSE-packaged command-line client version, you might break your Firefox beyond all repair. -- Bob
Thu, 6 Jan 2022 11:21:38 -0500 időpontban Bob Rogers írta:
From: Bob Rogers <rogers-suse@rgrjr.homedns.org> Date: Thu, 6 Jan 2022 11:13:44 -0500
. . .
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you.
Don't do that -- Firefox in 15.3 appears to use its own built-in SQLite version, so if you use the openSUSE-packaged command-line client version, you might break your Firefox beyond all repair.
This one I cannot understand either. I do not have any intention to mess with the sqlite file, but theoretically: can't any sqlite file be modified by any sqlite file handlig utility, be it consoles or GUI? Thanks, Istvan
On Thu, 06 Jan 2022 21:57:01 +0100 Istvan Gabor <suseuser04@gmail.hu> wrote:
Thu, 6 Jan 2022 11:21:38 -0500 időpontban Bob Rogers írta:
From: Bob Rogers <rogers-suse@rgrjr.homedns.org> Date: Thu, 6 Jan 2022 11:13:44 -0500
. . .
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you.
Don't do that -- Firefox in 15.3 appears to use its own built-in SQLite version, so if you use the openSUSE-packaged command-line client version, you might break your Firefox beyond all repair.
This one I cannot understand either. I do not have any intention to mess with the sqlite file, but theoretically: can't any sqlite file be modified by any sqlite file handlig utility, be it consoles or GUI?
Theoretically yes, providing the utility can handle the particular file version. But the answer that matters is not what happens in theory but reality. Since you don't know for sure what Firefox actually implements, you don't know for sure whether any corruption might result.
Thanks,
Istvan
Thu, 6 Jan 2022 11:13:44 -0500 időpontban Bob Rogers írta:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse.
Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted?
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you.
"Firefox to get around to doing it" - what does this exactly mean? Thanks, Istvan
On 06/01/2022 21.51, Istvan Gabor wrote:
Thu, 6 Jan 2022 11:13:44 -0500 időpontban Bob Rogers írta:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse.
Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted?
There is an menu or tool in Firefox to refresh profile. Be sure to read the caveats.
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you.
"Firefox to get around to doing it" - what does this exactly mean?
That the FF programmers create the tool or button to do that database purge. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Thu, 6 Jan 2022 21:54:42 +0100 időpontban Carlos E. R. írta:
On 06/01/2022 21.51, Istvan Gabor wrote:
Thu, 6 Jan 2022 11:13:44 -0500 időpontban Bob Rogers írta:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse. Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted?
There is an menu or tool in Firefox to refresh profile. Be sure to read the caveats.
Sorry, but this info is vague. My problem is that when I delete history I expect that the history data is erased, and cannot be read anymore (by accessing the database file; not including hard disk forensic data recovery). If the data still can be read directly from the database after deleting history it is a privacy/ security issue.
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you. "Firefox to get around to doing it" - what does this exactly mean?
That the FF programmers create the tool or button to do that database purge.
And what is this tool exactly, and where can it be found? It is part of firefox or an addon to firefox, or a separate binary program? And if it is a tool that I have to activate (button) why do I have to wait until firefox gets around to doing it? I am confused. Thanks, Istvan
On 06/01/2022 22.07, Istvan Gabor wrote:
Thu, 6 Jan 2022 21:54:42 +0100 időpontban Carlos E. R. írta:
On 06/01/2022 21.51, Istvan Gabor wrote:
Thu, 6 Jan 2022 11:13:44 -0500 időpontban Bob Rogers írta:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse. Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted?
There is an menu or tool in Firefox to refresh profile. Be sure to read the caveats.
Sorry, but this info is vague. My problem is that when I delete history I expect that the history data is erased, and cannot be read anymore (by accessing the database file; not including hard disk forensic data recovery). If the data still can be read directly from the database after deleting history it is a privacy/ security issue.
I am aware, but this is how all databases work. Google "how to refresh firefox?" (because I don't remember how to do it). Answer: Refresh Firefox 1 Click the menu button. ... 2 Click Refresh Firefox then Refresh Firefox in the confirmation window that opens. ... 3 When finished, a window will list your imported information. ... 4 Select whether you want Firefox to restore all or some windows and tabs and click the Let's go! <https://support.mozilla.org/en-US/kb/refresh-firefox-reset-add-ons-and-settings> Be sure to read the link, because it has caveats you may not like.
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you. "Firefox to get around to doing it" - what does this exactly mean?
That the FF programmers create the tool or button to do that database purge.
And what is this tool exactly, and where can it be found? It is part of firefox or an addon to firefox, or a separate binary program? And if it is a tool that I have to activate (button) why do I have to wait until firefox gets around to doing it? I am confused. >
We are telling you that it does not exist. You have to convince the Firefox developers to create it. Good luck :-) -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
Thu, 6 Jan 2022 22:38:41 +0100 időpontban Carlos E. R. írta:
On 06/01/2022 22.07, Istvan Gabor wrote:
On 06/01/2022 21.51, Istvan Gabor wrote:
Thu, 6 Jan 2022 11:13:44 -0500 időpontban Bob Rogers írta:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse. Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted?
There is an menu or tool in Firefox to refresh profile. Be sure to read the caveats. Sorry, but this info is vague. My problem is that when I delete history I expect that the history data is erased, and cannot be read anymore (by accessing the database file; not including hard disk forensic data recovery). If the data still can be read directly from
Thu, 6 Jan 2022 21:54:42 +0100 időpontban Carlos E. R. írta: the database after deleting history it is a privacy/ security issue.
I am aware, but this is how all databases work.
Google "how to refresh firefox?" (because I don't remember how to do it).
Answer:
Refresh Firefox
1 Click the menu button. ... 2 Click Refresh Firefox then Refresh Firefox in the confirmation window that opens. ... 3 When finished, a window will list your imported information. ... 4 Select whether you want Firefox to restore all or some windows and tabs and click the Let's go!
<https://support.mozilla.org/en-US/kb/refresh-firefox-reset-add-ons-and-settings>
Be sure to read the link, because it has caveats you may not like.
I also found this in the meantime, and certainly this is not something I want to do.
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you. "Firefox to get around to doing it" - what does this exactly mean?
That the FF programmers create the tool or button to do that database purge. And what is this tool exactly, and where can it be found? It is part of firefox or an addon to firefox, or a separate binary program? And if it is a tool that I have to activate (button) why do I have to wait until firefox gets around to doing it? I am confused. >
We are telling you that it does not exist.
Sorry, I understood firefox as the program, not as the devs. Thanks, Istvan
On 06/01/2022 22.47, Istvan Gabor wrote:
Thu, 6 Jan 2022 22:38:41 +0100 időpontban Carlos E. R. írta:
On 06/01/2022 22.07, Istvan Gabor wrote:
Thu, 6 Jan 2022 21:54:42 +0100 időpontban Carlos E. R. írta:
...
Google "how to refresh firefox?" (because I don't remember how to do it).
Answer:
Refresh Firefox
1 Click the menu button. ... 2 Click Refresh Firefox then Refresh Firefox in the confirmation window that opens. ... 3 When finished, a window will list your imported information. ... 4 Select whether you want Firefox to restore all or some windows and tabs and click the Let's go!
<https://support.mozilla.org/en-US/kb/refresh-firefox-reset-add-ons-and-settings>
Be sure to read the link, because it has caveats you may not like.
I also found this in the meantime, and certainly this is not something I want to do.
Me neither... Well, if your goal is privacy, what you can do is encrypt your home directory or partition. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On Fri, Jan 7, 2022 at 12:07 AM Istvan Gabor <suseuser04@gmail.hu> wrote:
Thu, 6 Jan 2022 21:54:42 +0100 időpontban Carlos E. R. írta:
On 06/01/2022 21.51, Istvan Gabor wrote:
Thu, 6 Jan 2022 11:13:44 -0500 időpontban Bob Rogers írta:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse. Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted?
There is an menu or tool in Firefox to refresh profile. Be sure to read the caveats.
Sorry, but this info is vague. My problem is that when I delete history I expect that the history data is erased, and cannot be read anymore (by accessing the database file; not including hard disk forensic data recovery). If the data still can be read directly from the database after deleting history it is a privacy/ security issue.
Then you asked the wrong question. Even if Firefox overwrites deleted data with zeros or some random pattern the file itself will certainly not shrink. If you have these concerns you will need to a) verify that deleted content is still accessible and b) make this question/request where it belongs - Firefox bugzilla, not some random list.
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 22:07:37 +0100 Thu, 6 Jan 2022 21:54:42 +0100 id pontban Carlos E. R. �rta:
On 06/01/2022 21.51, Istvan Gabor wrote:
Thu, 6 Jan 2022 11:13:44 -0500 id pontban Bob Rogers �rta:
From: Istvan Gabor <suseuser04@gmail.hu> �� Date: Thu, 06 Jan 2022 12:23:47 +0100
�� Hello:
�� Currently I use firefox 91.4.1esr in openSUSE 15.3.
�� I noticed that in the profile directory the size �� of some sqlite database files does not change �� after deleting history. Namely, places.sqlite, �� favicons.sqlite, cookies.sqlite files' size remains �� the same after deleting history.
�� Why don't the file sizes decrease after deleting �� their content by deleting history?
From the SQLite docs [1]:
����When content is deleted from an SQLite database, the content is ����not usually erased but rather the space used to hold the content ����is marked as being available for reuse. Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted?
There is an menu or tool in Firefox to refresh profile. Be sure to read the caveats.
Sorry, but this info is vague. My problem is that when I delete history I expect that the history data is erased, and cannot be read anymore (by accessing the database file; not including hard disk forensic data recovery). If the data still can be read directly from the database after deleting history it is a privacy/ security issue. That history can no longer be read by using the sqlite command-line tool or other sqlite database access methods. However, until firefox uses the sqlite "vacuum" command in a month or so (and there seems to be no way to force it to do so sooner), fragments can still be found by using the "strings" command on the database file (see "man strings"), which will easily pick out URLs in the history. This could certainly be a privacy issue, but only if you are worried about someone getting access to your ~/.mozilla directory (which is read-only to you) or to unencrypted backups that include it. -- Bob
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 21:51:36 +0100 Thu, 6 Jan 2022 11:13:44 -0500 id pontban Bob Rogers �rta:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse.
Does this mean that the sqlite file still contains the data which has been deleted from history? That is, someone who has access to the file can read history data, despite is has been deleted? Potentially, yes. Here's the full paragraph from which I excerpted the quote above: When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse. This can allow deleted content to be recovered by a hacker or by forensic analysis. Running VACUUM will clean the database of all traces of deleted content, thus preventing an adversary from recovering deleted content. Using VACUUM in this way is an alternative to setting PRAGMA secure_delete=ON.
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you.
"Firefox to get around to doing it" - what does this exactly mean? Thanks, Istvan At some point Firefox will tell SQLite to do the cleanup and compression. I tried to find out how often Firefox does this, but couldn't come up with anything definite. Curiously, there isn't an obvious configuration item for it. One place I found [1] suggests it happens in the background every month or two, but it's a more of a work proposal, and the page hasn't been updated for more than a decade. Other "how to speed up Firefox" pages I googled were from about that era as well. I did find a more recent Reddit post [2] with a Python recipe for doing the vacuum job to compress the databases, but I would strongly recommend stopping Firefox and backing up ~/.mozilla/firefox first. ================ From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 21:57:01 +0100 Thu, 6 Jan 2022 11:21:38 -0500 id pontban Bob Rogers �rta:
Don't do that -- Firefox in 15.3 appears to use its own built-in SQLite version, so if you use the openSUSE-packaged command-line client version, you might break your Firefox beyond all repair.
This one I cannot understand either. I do not have any intention to mess with the sqlite file, but theoretically: can't any sqlite file be modified by any sqlite file handlig utility, be it consoles or GUI? Thanks, Istvan Yes, but. The version incorporated into Firefox is very likely to be an older version, and if the openSUSE version uses a different file format and silently upgrades the files, Firefox may no longer be able to use them, and will be unable to start. -- Bob [1] https://wiki.mozilla.org/Firefox/Projects/Places_Vacuum [2] https://www.reddit.com/r/firefox/comments/di51ps/a_surprising_way_to_gain_so...
On 1/6/22 10:13 AM, Bob Rogers wrote:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse.
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you.
-- Bob Rogers http://www.rgrjr.com/
I wrote the following to vacuum the sqlite files in both .mozilla and .thunderbird. sqlite is a horribly wasteful database format when used over a long period of time without vacuuming the stale space. If the attachment doesn't come though, I'll post to paste.opensuse. Should work when run from konsole or an xterm just fine even though it was written several years ago (it looks for sqlite3) -- David C. Rankin, J.D.,P.E.
On 1/8/22 12:09 AM, David C. Rankin wrote:
On 1/6/22 10:13 AM, Bob Rogers wrote:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
Currently I use firefox 91.4.1esr in openSUSE 15.3.
I noticed that in the profile directory the size of some sqlite database files does not change after deleting history. Namely, places.sqlite, favicons.sqlite, cookies.sqlite files' size remains the same after deleting history.
Why don't the file sizes decrease after deleting their content by deleting history?
From the SQLite docs [1]:
When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse.
But unless you want to use the SQLite command-line client to do this for yourself, you'll have to wait for Firefox to get around to doing it for you.
-- Bob Rogers http://www.rgrjr.com/
I wrote the following to vacuum the sqlite files in both .mozilla and .thunderbird. sqlite is a horribly wasteful database format when used over a long period of time without vacuuming the stale space.
If the attachment doesn't come though, I'll post to paste.opensuse. Should work when run from konsole or an xterm just fine even though it was written several years ago (it looks for sqlite3)
I got the attached file
On 08/01/2022 09.09, David C. Rankin wrote:
On 1/6/22 10:13 AM, Bob Rogers wrote:
From: Istvan Gabor <suseuser04@gmail.hu> Date: Thu, 06 Jan 2022 12:23:47 +0100
Hello:
...
I wrote the following to vacuum the sqlite files in both .mozilla and .thunderbird. sqlite is a horribly wasteful database format when used over a long period of time without vacuuming the stale space.
If the attachment doesn't come though, I'll post to paste.opensuse. Should work when run from konsole or an xterm just fine even though it was written several years ago (it looks for sqlite3)
Came fine, thanks. I understand it seeks and runs on every thunderbird and mozilla *.sqlite file found - that may be a problem, I have several profiles and I would want to run it only on one. I might modify it to work on the current directory only, for simplicity. -- Cheers / Saludos, Carlos E. R. (from 15.2 x86_64 at Telcontar)
On 1/8/22 5:09 AM, Carlos E. R. wrote:
Came fine, thanks.
I understand it seeks and runs on every thunderbird and mozilla *.sqlite file found - that may be a problem, I have several profiles and I would want to run it only on one.
I might modify it to work on the current directory only, for simplicity.
Yes, I should probably update it. I just looked at the file date: -rwxr-xr-- 1 david david 2056 May 11 2012 ../adm/mozsqlcln.sh That was a few more years ago than I recall -- getting old... :) -- David C. Rankin, J.D.,P.E.
participants (7)
-
Andrei Borzenkov -
Bob Rogers -
Bruce Ferrell -
Carlos E. R. -
Dave Howorth -
David C. Rankin -
Istvan Gabor