[opensuse] Help - Howto Recover SD Card (at least any jpegs still on the card)?
All, I have a trail-cam with a 16G card that I use to monitor the predators coming into the yard so I can keep the cats from getting eaten (long story). This month when I pulled the card, it can't be recognized in any of my readers (Linux or the kids Windows boxes) It is seen and the full 16G is recognized, but blkid reports no filesystem on it. No clue what happened, lightning, low-battery, who knows. But now that leaves me with the problem of how best to try and recover what I can. Do I just image all 16G with dd_rescue and then use a scanner that will recognize the jpeg headers and try a recovery that way. What tool? Instead of dd_rescue, can I just open the device as a raw file and read the number of bytes reported? If I'm just going to scan through it, then I don't see why a dd_rescue write to an image would help. (other than it's multiple attempts to read from each memory location) However, I'm not worried if I miss an image or two because one of the header bytes wasn't read or I get a few off colored pixels. Anybody run into a similar issue and find a decent way to recover what is recoverable? -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 9/18/20 10:44 PM, David C. Rankin wrote:
Update: I have run testdisk and photorec against the card and absolutely nothing is found. I made two-passes with photorec, the first looking for all file types, the second looking only for jpg files. Nothing was found at all. So I guess this card is just toast. -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 19/09/2020 06.54, David C. Rankin wrote:
Not dd-rescue, there is no benefit by trying to read multiple times. Just plain dd. Then run the scan on the image with photorec - which I see you already used.
Gah... :-( Google "photo card recovery" suggests: <https://www.cnet.com/how-to/how-to-recover-deleted-photos-from-a-memory-card/> But that thing works similarly to photorec. If there is no data, it will not work. If I google for "sd card recovery" instead, I get: <https://www.easeus.com/data-recovery/sd-card-data-recovery-software.html> which is more promising. I remember reading a name similar to "F3" but I find nothing, probably got the name wrong. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
Le 19/09/2020 à 08:42, Carlos E. R. a écrit :
I remember reading a name similar to "F3" but I find nothing, probably got the name wrong.
f3 is the linux name for h2testw, software to check a sd card to see if it's plain good or fake jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 19/09/2020 08.49, jdd@dodin.org wrote:
Ah, different purpose, but I got the name right. <https://software.opensuse.org/package/f3> f3 Fight Flash Fraud / Fight Fake Flash This package contains tools for identifying fake flash drives (primarily USB sticks and memory cards). A fake flash drive fraudulently inflates its apparent storage capacity (far) beyond the physical capacity of its flash memory. Not surprisingly, using such a flash drive will, sooner or later, result in data loss and/or corruption. The main tools in this package are an open-source implementation of the H2testw algorithm. Some extra tools are also provided, among them one for using the actual storage capacity of fake drives as safely as possible. Version 7.0 Size 73.4 KB openSUSE Leap 15.1 with more info here <https://h2testw.en.lo4d.com/windows> But I do not see a description of the algorithm. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
Le 19/09/2020 à 09:00, Carlos E. R. a écrit :
But I do not see a description of the algorithm.
IMHO pretty simple. Fill the disk with numbered files. Read to see if the numbers are there... jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 19/09/2020 à 13:06, Carlos E. R. a écrit :
what's sure is that after the operation the disk is filled, and this works only if the disk is mounted then the simple explanations I gave is an algorithm :-)) jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 19/09/2020 05:29, jdd@dodin.org wrote:
Simple but smart. Write 1 megabyte numbered files. Along the way it tests the writeability/readback looking for, not only the 'high water mark' that is the drive's capacity, but any errors along the way. Yes, I suppose you could use a script that does a stepwise DD+offset. Yes I suppose using DD you could have input files with different byte patterns and try each of them before moving onto the next bock on the drive. Yes you could prove just how creative a shell programmer you are. Yes, you set up something like this for one of the 256G devices I have. It might be slower than F3, which took almost 3 days to scan and prove one of my 256G devices. OK, so that chip wasn't fast, but for my tablet, books and movies, it was fast enough. It also, it seems, was fast enough for video recording on my camera which surprised me. But then it's an older, smaller camera; a modern HD 4/8K 10-bit camera has a greater demand! But it's not as if I'm vblogging or making short movies for YouTube. The bottom line here is that I ALWAYS test my 'portable storage devices' beforehand. I do not want to find out they were failures or 'not as advertised' AFTER I've used them to take unrepeatable photographs. -- All tyranny needs to gain a foothold is for people of good conscience to remain silent. -- Thomas Jefferson -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 19/09/2020 à 18:17, Carlos E. R. a écrit :
I've never found a bad stick or bad card. Then I buy at reputable places...
I often do :-( jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 19/09/2020 18.28, jdd@dodin.org wrote:
And you get bads? I typically buy at Media Mark my sticks, or a local big place. I bought 3 USB-3 sticks the other day, I will test one with F3 to see how it goes. (22/22) Installing: atril-lang-1.20.0-lp151.2.4.noarch ............................................................................................................................................[done] Telcontar:~ # f3 If 'f3' is not a typo you can use command-not-found to lookup the package that contains it, like this: cnf f3 Telcontar:~ # cnf f3 f3: command not found Telcontar:~ # cnf F3 F3: command not found Telcontar:~ # zypper se f3 Retrieving repository 'EXT: Packman Repository' metadata ..........................................................................................................................................[done] ... Loading repository data... Reading installed packages... S | Name | Summary | Type --+-------------------------------------+------------------------------------------------------------------------+----------- | f3 | Fight Flash Fraud / Fight Fake Flash | package | f3-debuginfo | Debug information for package f3 | package | f3-debugsource | Debug sources for package f3 | package ... Telcontar:~ # So it is there. Telcontar:~ # zypper install f3 ... (1/1) Installing: f3-7.0-lp151.2.3.x86_64 .........................................................................................................................................................[done] Telcontar:~ # Telcontar:~ # f3 --help If 'f3' is not a typo you can use command-not-found to lookup the package that contains it, like this: cnf f3 Telcontar:~ # What? Telcontar:~ # rpm -ql f3 /usr/bin/f3read /usr/bin/f3write /usr/sbin/f3brew /usr/sbin/f3fix /usr/sbin/f3probe /usr/share/doc/packages/f3 /usr/share/doc/packages/f3/LICENSE /usr/share/doc/packages/f3/README.rst /usr/share/doc/packages/f3/changelog /usr/share/doc/packages/f3/examples /usr/share/doc/packages/f3/examples/f3write.h2w /usr/share/doc/packages/f3/examples/log-f3wr /usr/share/man/man1/f3read.1.gz /usr/share/man/man1/f3write.1.gz Telcontar:~ # NAME f3write, f3read - test real flash memory capacity ... EXAMPLE To write over a flash drive mounted at /media/TEST: $ f3write /media/TEST To read this flash drive: $ f3read /media/TEST It has to be mounted? Ok... I have to format it first, it contains the 15.2 iso. Done. Telcontar:~ # time f3write /media/DESTROYABLE/ F3 write 7.0 Copyright (C) 2010 Digirati Internet LTDA. This is free software; see the source for copying conditions. Free space: 7.02 GB Creating file 1.h2w ... OK! Creating file 2.h2w ... OK! Creating file 3.h2w ... OK! Creating file 4.h2w ... OK! Creating file 5.h2w ... OK! Creating file 6.h2w ... OK! Creating file 7.h2w ... OK! Creating file 8.h2w ... OK! Free space: 0.00 Byte Average writing speed: 9.52 MB/s real 12m41.392s user 0m1.896s sys 0m8.135s Telcontar:~ # Telcontar:~ # ls /media/DESTROYABLE/ 1.h2w 2.h2w 3.h2w 4.h2w 5.h2w 6.h2w 7.h2w 8.h2w Telcontar:~ # ls -lh /media/DESTROYABLE/ total 7.1G -rw-r--r-- 1 cer users 1.0G Sep 19 18:55 1.h2w -rw-r--r-- 1 cer users 1.0G Sep 19 18:57 2.h2w -rw-r--r-- 1 cer users 1.0G Sep 19 18:59 3.h2w -rw-r--r-- 1 cer users 1.0G Sep 19 19:00 4.h2w -rw-r--r-- 1 cer users 1.0G Sep 19 19:02 5.h2w -rw-r--r-- 1 cer users 1.0G Sep 19 19:04 6.h2w -rw-r--r-- 1 cer users 1.0G Sep 19 19:06 7.h2w -rw-r--r-- 1 cer users 25M Sep 19 19:06 8.h2w Telcontar:~ # time f3read /media/DESTROYABLE/ F3 read 7.0 Copyright (C) 2010 Digirati Internet LTDA. This is free software; see the source for copying conditions. SECTORS ok/corrupted/changed/overwritten Validating file 1.h2w ... 2097152/ 0/ 0/ 0 Validating file 2.h2w ... 2097152/ 0/ 0/ 0 Validating file 3.h2w ... 2097152/ 0/ 0/ 0 Validating file 4.h2w ... 2097152/ 0/ 0/ 0 Validating file 5.h2w ... 2097152/ 0/ 0/ 0 Validating file 6.h2w ... 2097152/ 0/ 0/ 0 Validating file 7.h2w ... 2097152/ 0/ 0/ 0 Validating file 8.h2w ... 51048/ 0/ 0/ 0 Data OK: 7.02 GB (14731112 sectors) Data LOST: 0.00 Byte (0 sectors) Corrupted: 0.00 Byte (0 sectors) Slightly changed: 0.00 Byte (0 sectors) Overwritten: 0.00 Byte (0 sectors) Average reading speed: 39.88 MB/s real 3m0.373s user 0m3.705s sys 0m1.917s Telcontar:~ # So, it is good. The thing is, on the package, 8GB from Trascend. It is not. Telcontar:/data/storage_b # fdisk -l /dev/sde Disk /dev/sde: 7 GiB, 7558135808 bytes, 14761984 sectors Disk model: Transcend 8GB ... It is 7.558GB, or 7GiB. No way 8GB. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
Le 19/09/2020 à 19:25, Carlos E. R. a écrit :
sure, and get refunded :-)
I typically buy at Media Mark my sticks, or a local big place. I bought 3 USB-3 sticks the other day, I will test one with F3 to see how it goes.
yes, but you pay 4x the real price :-)) jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 19/09/2020 19.58, jdd@dodin.org wrote:
Depend what you call real price. I also get the assurance they work. 32GB for about 6€ is Ok. If I pay less and some fail, that is not the real price for me. The USB3 one was bought at Amazon and was a bit more expensive, 8.54€ -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On 19/09/2020 13:25, Carlos E. R. wrote:
It is 7.558GB, or 7GiB. No way 8GB.
So fdisk says 14761984 sectors So f3 says 14731112 sectors A 39872 sector difference. If I was asked to speculate on the 'why' I'd first suggest that the 'shortfall' are what might be termed "error correction lookaside sectors" and the metadata to make it work. Back in the days when rotating rust took two men to life, I implemented a error correction system, the likes of which I'm sure is built into modern drives transparently. It revered a few cylinders that substituted for one gone bad. The driver had a look-aside table. A seek for the bad cylinder was redirected to the lookaside. Yes it slowed things down, yes it gave up some disk capacity. Yes you had to make a best effort to copy the stuff from the sector-gone-bad to the lookaside. I'm sure its a lot more sophisticated now. I'm sure its built into the firmware of the device. That would be my first guess. A second guess isn't going to happen until after a second pot of coffee and isn't going to happen on a Sunday. -- Your eyes are weary from staring at the CRT. You feel sleepy. Notice how restful it is to watch the cursor blink. Close your eyes. The opinions stated above are yours. You cannot imagine why you ever felt otherwise. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 20/09/2020 14.40, Anton Aylward wrote:
f3 looks at the partition. fdisk looks at the entire device. The full output was: Telcontar:/data/storage_b # fdisk -l /dev/sde Disk /dev/sde: 7 GiB, 7558135808 bytes, 14761984 sectors Disk model: Transcend 8GB Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xa952c083 Device Boot Start End Sectors Size Id Type /dev/sde1 2048 14761983 14759936 7G b W95 FAT32 Telcontar:/data/storage_b # Thus, 14759936 the partition. Somewhat less writeable space. I could look at "df", but the stick now contains the 5.2 ISO. I can try with another stick, uonopened. Ok, it has: Telcontar:~ # fdisk -l /dev/sde Disk /dev/sde: 7 GiB, 7558135808 bytes, 14761984 sectors Disk model: Transcend 8GB Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0xc3072e18 Device Boot Start End Sectors Size Id Type /dev/sde1 56 14761983 14761928 7G b W95 FAT32 Telcontar:~ # The number of sectors vary, but this is the original formatting by the manufacturer, while the previous stick was done by gparted. Telcontar:~ # df /media/Transcend/ Filesystem 1K-blocks Used Available Use% Mounted on /dev/sde1 7364580 4 7364576 1% /media/Transcend Telcontar:~ # Multiplying by two - or: Telcontar:~ # df --block-size=512 /media/Transcend/ Filesystem 512B-blocks Used Available Use% Mounted on /dev/sde1 14729160 8 14729152 1% /media/Transcend Telcontar:~ # f3 said 14731112, this is similar. I could reformat with parted and see. [...] Done: Telcontar:~ # fdisk -l /dev/sde Disk /dev/sde: 7 GiB, 7558135808 bytes, 14761984 sectors Disk model: Transcend 8GB Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x7d890169 Device Boot Start End Sectors Size Id Type /dev/sde1 2048 14761983 14759936 7G b W95 FAT32 Telcontar:~ # Telcontar:~ # df --block-size=512 /media/BORRAME/ Filesystem 512B-blocks Used Available Use% Mounted on /dev/sde1 14731120 8 14731112 1% /media/BORRAME Telcontar:~ # Which is the same figure as F3 gives. QED :-D
:-) -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On 19/09/2020 13:25, Carlos E. R. wrote:
I typically buy at Media Mark my sticks, or a local big place. I bought 3 USB-3 sticks the other day, I will test one with F3 to see how it goes.
Leave one in a drawer unused until after the new year and test it again then. Test one now and then leave that in the drawer until after the new year, then test it again. I did this a while ago by happenstance. The chip I used worked fine and kept working fine. The unused one was a disastrous fail. The tested once tested OK the first time, tested OK after I took it out of the drawer and then failed intermittently after a month's service. Although I tested them on my PC I reformatted and used them in my DSLR, used a carrier/USB to transfer/copy into the PC for photo-editing, then reformatted again in the camera. I can't speculate on why 'drawer living' is destructive buy repeated use isn't. -- Context is Everything -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 9/19/20 8:26 AM, Anton Aylward wrote:
This card was used fine for 4-5 years, and as mentioned, it is just used to monitor the foxes and coyotes in the backyard to minimize the number of cats we are feeding them. (kids seem to get upset when fluffy goes missing) So data isn't critical (except from the cat's perspective...) Lots of great pictures over the years. It's just like the filesytem evaporated. Lightning nearby? There are power lines withing 30 ft or so, so a close lightning strike could produce one hell of a magnetic field while it is running to ground. (or it was just a crappy SD card that came with a predetermined TITSUP date. 5 more ordered for $14. That should get us though the cats remaining lifetimes. ddrescue log is blank all the way through: # Mapfile. Created by GNU ddrescue version 1.25 # Command line: ddrescue --reopen-on-error --min-read-rate=400k /dev/sde trailcam.img trailcam.map # Start time: 2020-09-19 03:07:32 # Current time: 2020-09-19 04:31:57 # Copying non-tried blocks... Pass 5 (forwards) # current_pos current_status current_pass 0x00120000 ? 5 # pos size status 0x00000000 0x00120000 * 0x00120000 0x00050000 ? 0x00170000 0x00020000 * 0x00190000 0x00170000 ? 0x00300000 0x00020000 * 0x00320000 0x002F0000 ? 0x00610000 0x00020000 * 0x00630000 0x005F0000 ? 0x00C20000 0x00020000 * 0x00C40000 0x00BF0000 ? <snip more of same to end> -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 9/23/20 2:33 PM, Carlos E. R. wrote:
Nothing in the system log related to 'dd_*rescue' or 'sde'. Since the block device /dev/sde is the only thing reported and nothing related to the partition /dev/sde1 I am reasonably satisfied at this point the card just died some type of death. I put an old card from another camera back in until the new cards arrive and it is working fine -- so I am satisfied that we have run this into ground. When photorec with a raw read cannot even find a jpeg signature anywhere attempting to read the whole card, I don't see me reinventing the wheel and opening /dev/sde for a raw read from a file descriptor doing any better. (if open didn't fail outright) Thanks for all the suggestions. Will chock it up to a coronal mass ejection or some other phenomena ;p -- David C. Rankin, J.D.,P.E.
On 23/09/2020 22.10, David C. Rankin wrote:
This is very strange, there should be some type of error in the log.
photorec fails because dd* fails.
Thanks for all the suggestions. Will chock it up to a coronal mass ejection or some other phenomena ;p
Did you try reading with a different card reader? -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
On 9/19/20 1:42 AM, Carlos E. R. wrote:
Gah is right... The card is toast. After running testdisk and photorec, I also attempted ddrescue for fun. It was able to recover '0' bytes. The card appears to be T O A S T !! photorec scans what is reads from the device for the JFIF header looking for the sequence '0xff 0xd8 0xff' (I grabbed the source). So if no filesystem or superblock can be found, lsblk -f, blkid -p, (I also tried fdisk -- can't read device) and testdisk, photorec, and ddrescue all can't find a bytes of data or JFIF header -- then It's T O A S T !! -- David C. Rankin, J.D.,P.E.
On 9/20/20 5:26 AM, Carlos E. R. wrote:
The filesystem is still there?
Because if it is gone, some tools can not seek on it.
No, no filesystem. The card still reports its size as 16G, but that's all (I doubt that has anything to do with partition tables or filesystems, but rather some initial bytes of the SD card information.) So nothing can be read from the card at all, not even opening the block device directly (/dev/sde). You couldn't even wipefs if you wanted to and the card cannot be formatted under Windows -- it's just toast. -- David C. Rankin, J.D.,P.E.
Le 21/09/2020 à 09:39, Carlos E. R. a écrit :
On 21/09/2020 06.56, David C. Rankin wrote:
So nothing can be read from the card at all, not even opening the block device directly (/dev/sde).
dd if=/dev/sde of=card.img bs=16M status=progress
if you can't open the block device... jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 9/21/20 2:58 AM, Carlos E. R. wrote:
dd isn't an option because it will exit on the first read error. I think my last option is just writing a small C source to open the block device and use a low-level 'read()` from the open file descriptor. Though if I was a betting man, I would be the read with proper validation would fail and it would behave like dd. I'm saving that for a rainy day. I order more SD cards for the trail cam. One more cat down.... Damn foxes. -- David C. Rankin, J.D.,P.E.
Le 22/09/2020 à 20:02, David C. Rankin a écrit :
dd isn't an option because it will exit on the first read error. I think my
dd-rescue don't stop jdd -- http://dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2020-09-22 21:56, jdd@dodin.org wrote:
dd's default is to stop. But you can tell it not to with "noerror" ex. # dd if=... of=... conv=sync,noerror status=progress Mind you. The sync option pads the block with 0x00 (null). In essence: dd is for copying and nothing more dd-rescue is for rescuing data from damaged media A couple of tools that have some years on the neck and for slightly different purpose. dcfldd is for making forensic verifiable images dc3dd same as above -- /bengan -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 22/09/2020 20.02, David C. Rankin wrote:
That's fine, we want to see what error it prints and writes to syslog, if any. Besides, using "conv=sync,noerror" it will not abort, as Bengt Gördén wrote.
No need. dd has options for almost any need. dd-rescue is not appropriate, because retry read on a card doesn't work.
I order more SD cards for the trail cam. One more cat down.... Damn foxes.
Ow. Just order the cards from a reputable site, surprises with photos are just not worth the savings. By the way: there are applications on Android to trigger the camera on movement. -- Cheers / Saludos, Carlos E. R. (from 15.1 x86_64 at Telcontar)
participants (5)
-
Anton Aylward -
Bengt Gördén -
Carlos E. R. -
David C. Rankin -
jdd＠dodin.org