Hello, I use quite often SD card for video recording. I have problems with card flow rate measuring, because video is sometime very demanding, so I try to verify the card before use. however, cards are chepaer nowadays, but the quality is variable. problem is that some tools works and give data, others do not work, and specialy rsync!! normally rsync gives at the end of the work a summary with the flow rte (often refered as "speed") the target file system is fAT32 (no choice) the card si test here failed on real record in a 5DMKIII, probably because the flow rate of the device (around 32Mb/s) was too high. but look at these results: http://susepaste.org/27536079 rsync gives an error after some time, but f3write do not and I have similar results for some days, now, sometimes error, sometimes not NB: f3write is a specialized tool to verufy SD cards, similar to the windows tool h2testw any idea? thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2015-06-01 08:36, jdd wrote:
Hello,
I use quite often SD card for video recording. I have problems with card flow rate measuring, because video is sometime very demanding, so I try to verify the card before use.
There is a labeling in those cards that specifies the write speed: Class 4, class 6, class 10... but I don't know how to google it. Ah! Found it, here: <http://en.wikipedia.org/wiki/Secure_Digital#Speed_class_rating> But then read the section "Real-world performance", it becomes interesting. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Le 01/06/2015 12:37, Carlos E. R. a écrit :
On 2015-06-01 08:36, jdd wrote:
Hello,
I use quite often SD card for video recording. I have problems with card flow rate measuring, because video is sometime very demanding, so I try to verify the card before use.
There is a labeling in those cards that specifies the write speed: Class 4, class 6, class 10... but I don't know how to google it. Ah! Found it, here:
<http://en.wikipedia.org/wiki/Secure_Digital#Speed_class_rating>
But then read the section "Real-world performance", it becomes interesting.
yes, but it's completely bullshit. no metering method gives same result and no result is acording to what is written on the cards :-( jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2015-06-01 12:43, jdd wrote:
Le 01/06/2015 12:37, Carlos E. R. a écrit :
But then read the section "Real-world performance", it becomes interesting.
yes, but it's completely bullshit. no metering method gives same result and no result is acording to what is written on the cards :-(
The paragraph I mentioned last may explain why. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 2015-06-01 13:22, Carlos E. R. wrote:
On 2015-06-01 12:43, jdd wrote:
Le 01/06/2015 12:37, Carlos E. R. a écrit :
But then read the section "Real-world performance", it becomes interesting.
yes, but it's completely bullshit. no metering method gives same result and no result is acording to what is written on the cards :-(
The paragraph I mentioned last may explain why.
You may have to resort to write a script to do the testing, writing files of similar sizes to those your camera uses. Or test write speed for several sizes, on empty card, and on half used card. Or measure the speed variance as it gets filled. I don't know if the overwrite speed (write the same file again) is interesting. Or change a sector in the middle of file. I would do that with dd and a script. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-06-01 14:00, jdd wrote:
Le 01/06/2015 13:37, Carlos E. R. a écrit :
but f3write succeed and rsync fails.. why?
Maybe because rsync does clever things, like crc checks, copy partial sections when needed... it is not what I would use to measure speed. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlVshBIACgkQja8UbcUWM1z1JwEAntxSdkR4iRZXwIyUj3lhKk/n LlBcjemYMuesQxHhttYA/1LdP398GhehaJY7ubQt3TQu9MHWVbV9ujN1PQyYbdsR =Q4yD -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 01/06/2015 18:10, Carlos E. R. a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2015-06-01 14:00, jdd wrote:
Le 01/06/2015 13:37, Carlos E. R. a écrit :
but f3write succeed and rsync fails.. why?
Maybe because rsync does clever things, like crc checks, copy partial sections when needed... it is not what I would use to measure speed.
f3write is to verify the disk is good, so yes it makes very fine verification :-) don't know for rsync... there may be some ghost in the card :-) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-06-01 18:49, jdd wrote:
Le 01/06/2015 18:10, Carlos E. R. a écrit :
but f3write succeed and rsync fails.. why?
Maybe because rsync does clever things, like crc checks, copy partial sections when needed... it is not what I would use to measure speed.
f3write is to verify the disk is good, so yes it makes very fine verification :-)
don't know for rsync... there may be some ghost in the card :-)
Mmm. Maybe I misunderstood the issue... - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlVsvnoACgkQja8UbcUWM1x/JgEAh2A9DjDX1Gy0gD/JugWLKyFm Gz6+1slepWzEyRA9XQkA/34ZMehfdeK+MFVsy68NGfk/7J4xTzTLHmE1Pj67mZt5 =W6Kq -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/01/2015 08:00 AM, jdd wrote:
Le 01/06/2015 13:37, Carlos E. R. a écrit :
I would do that with dd and a script.
yes, it's what f3write do.
but f3write succeed and rsync fails.. why?
Fails? I rsync'd my ~/Music to a car and rsync reported a lot of errors, telling me it couldn't write to "dot" files. The dot files weren't in the source and weren't in the destination. best guess is that they were temp files of some sort, by what at the destination .... Ran a directory diff[1]. All there. All the same sizes. So I don't understand the 'errors'. maybe should use the "--temp-dir=" option maybe should use the "--info=" option [1] I could have run a rsync again which would have updated anything missing, in theory, but the errors the first time don't instil confidence. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/01/2015 02:36 AM, jdd wrote:
however, cards are chepaer nowadays, but the quality is variable.
ROTFLMAO! I have a 64G card in my phone. My phone keeps saying that its corrupt "Would you like to reformat it?" After a couple of these I put it in a carrier in a USB adapter and 'dd'd the hell out of it ... back and forth, back and forth. I also dusted out the contacts on the phone[1]. Still the problem persists. Maybe the pone expects some kind of synchronous response ... but this is a class 10 card. heck, other class 6 chips work. "Variable". That's a good word. [1] Sometimes I miss the ultrasonic bath full of trichloroethane that I had in the lab at university for cleaning electronic components. And things. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/01/2015 02:36 AM, jdd wrote:
the target file system is fAT32 (no choice)
Actually you do have a chouce ... most likely. There are supposed to be (out of Samsung) devices that use more modern, Linux derived, file systems. Maybe the influence of Android, who knows. Certainly the Linux file systems and even NTFS don't work with "Older" video devices, or most phones. But fat32 has its limits. Most notably a 32G limit https://support.microsoft.com/en-us/kb/184006 That wasn't too bad when my camera was living on the then expensive 16G cards :-) But the spec sheet said that it can handle 64G and 128G cards. So, apparently, can my phone. (see other posting.) So I end up using http://en.wikipedia.org/wiki/ExFAT And yes there is a 3rd part implementation for openSuse: Information for package fuse-exfat: ----------------------------------- Repository: home:Reki Name: fuse-exfat Version: 1.0.1-2.1 Arch: x86_64 Vendor: obs://build.opensuse.org/home:Reki Installed: Yes Status: up-to-date Installed Size: 43.7 KiB Summary: Free exFAT file system implementation Description: This driver is the first free exFAT file system implementation with write support. exFAT is a simple file system created by Microsoft. It is intended to replace FAT32 removing some of it's limitations. exFAT is a standard FS for SDXC memory cards. There is also a non-fuse imnplementation https://build.opensuse.org/package/show?project=home%3Aenzokiel&package=exfat-nofuse and https://build.opensuse.org/package/show?project=home%3AX0F%3AHSF&package=exfat-nofuse Information for package exfat-utils: ------------------------------------ Repository: Packman Repository Name: exfat-utils Version: 1.1.1-1.1 Arch: x86_64 Vendor: http://packman.links2linux.de Installed: Yes Status: up-to-date Installed Size: 181.7 KiB Summary: Tools for the exFAT file system implementation Description: This project aims to provide a full-featured exFAT file system implementation for GNU/Linux and other Unix-like systems as a FUSE module and a set of utilities. YMMV. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 01/06/2015 17:43, Anton Aylward a écrit :
On 06/01/2015 02:36 AM, jdd wrote:
the target file system is fAT32 (no choice)
Actually you do have a chouce ... most likely.
you may be right, extfat can be possible, as my deivces also read 64Gb SDXC cards with extfat, but is this a good thing, is extfat better then Fat 32, I have no idea
And yes there is a 3rd part implementation for openSuse:
there is a codec in the OBS, I already installed it, works like a charm.
Information for package exfat-utils:
I think I have this one. just tested, may be extfat is a bit faster than fat32, but not that much, and I use to format the card with the device that write on it, so... thanks jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/01/2015 12:44 PM, jdd wrote:
you may be right, extfat can be possible, as my deivces also read 64Gb SDXC cards with extfat, but is this a good thing, is extfat better then Fat 32, I have no idea
My opinion: FAT was not a good thing vFAT was not a good thing FAST32 was not a good thing exFAT is not a good thing. Basic reasons. * Single "superblock" and file allocation table is vulnerable. * No journalling. The development community[1], mostly using Linux as a platform, have done a lot of work on how file systems affect performance of SSD/Flash and developed different file systems to expereiment with this and better address strengths and weaknesses. Some of these do better than BtrFS. It's all out there for a quick google. Actually its overwhelming the amount of work that has been done. <quote src="http://www.tomshardware.com/news/NAND-Flash-Flash-Friendly-File-System-F2FS-Jaegeuk-Kim,18229.html"> Last week Kim said the company chose a log-structured file system (LFS) approach, which it adapted for new form factors like SSDs, SD cards and more. ... "Because a NAND-based storage device shows different characteristics according to its internal geometry or flash memory management scheme aka FTL, we add various parameters not only for configuring on-disk layout, but also for selecting allocation and cleaning algorithms," Kim added. </quote> More at http://www.tomshardware.com/reviews/ssd-file-system-ntfs,3166.html http://en.wikipedia.org/wiki/F2FS https://lwn.net/Articles/520003/ And the big YES-BUT My phone and camera won't handle F2FS. Well _M_A_Y_B_E_ there will be ROM for my phone, if I want to buqqer up some other functionality. http://nexus5.wonderhowto.com/how-to/convert-any-kitkat-rom-your-nexus-4-5-f... Great. What about my S2 and S3? Perhaps someone could comment, my French isn't up to this: http://www.galaxys-team.fr/viewtopic.php?f=91&t=43494 [1] That includes researchers at Intel and other semiconductor firms as well as at universities -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 01/06/2015 19:27, Anton Aylward a écrit :
* Single "superblock" and file allocation table is vulnerable.
there are two of them :-) and I have no control over the file system used by ma cemeras and camcorders :-( if not I would like better ext2 :-) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-06-01 19:59, jdd wrote:
Le 01/06/2015 19:27, Anton Aylward a écrit :
* Single "superblock" and file allocation table is vulnerable.
there are two of them :-)
True.
and I have no control over the file system used by ma cemeras and camcorders :-(
True.
if not I would like better ext2 :-)
Actually... these flash devices (not SSDs) are optimized for FAT. The start sectors of the stick or card get smaller actual sectors (not the proper name, true), because in that area there are much more writes; for each file that changes something, there is a corresponding change in the FAT, the directory table, or both. So they make that region more resilient to multiple writes. I had a link that explained this. Found another one: <http://lwn.net/Articles/518988/> There is a mention in the comments (by the article author) about this, and that f2fs uses the same region. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlVsvaEACgkQja8UbcUWM1zwUAD+IjPW9d0nN+vWasbhc8Qya0KD LeNYjqtEY8gL5MTWjcQA/2Od+zJj8vD5mILHCHy556b0UdDRULpxwyYSOrP0wZVX =LQCO -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-06-01 22:16, Carlos E. R. wrote:
On 2015-06-01 19:59, jdd wrote:
There is a mention in the comments (by the article author) about this, and that f2fs uses the same region.
Here: <https://wiki.linaro.org/WorkingGroups/KernelArchived/Projects/FlashCardSurvey?action=show&redirect=WorkingGroups%2FKernel%2FProjects%2FFlashCardSurvey> «The cards take advantage of this knowledge by optimizing for the access patterns that are observed on FAT32, which unfortunately can lead to worst-case access patterns when using ext3 or other Linux file systems. In particular, the following (mis-)optimizations are commonly seen on flash media:» - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlVsvikACgkQja8UbcUWM1yRHAEAhKR6s76No67x1YGs/Wllmn3r 0fSZSh+Jtr96a2Pdlo0A/i7T8wOuqzkHofNWfc2oahXPIgJFhQIHuFOqvgaAu6hc =KBfN -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2015-06-01 17:43, Anton Aylward wrote:
On 06/01/2015 02:36 AM, jdd wrote:
the target file system is fAT32 (no choice)
Actually you do have a chouce ... most likely.
There are supposed to be (out of Samsung) devices that use more modern, Linux derived, file systems. Maybe the influence of Android, who knows. Certainly the Linux file systems and even NTFS don't work with "Older" video devices, or most phones.
...
So I end up using http://en.wikipedia.org/wiki/ExFAT
Very proprietary. Aggressively protected by microsoft, the evil empire :-p This one is better: <https://en.wikipedia.org/wiki/F2FS> on which Samsung had a big say. And Linux. There are others: <https://en.wikipedia.org/wiki/Flash_file_system> - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlVstvQACgkQja8UbcUWM1wd5AEAklRzNVR0AdZ5n+Fb0fqzT+XA CbnG08u6llqo7tJ2Z+UA/0F9w2OHDXj7MC2mh7khATIWTflBsOJQVx4OW+X9ZgBd =F9ax -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (3)
-
Anton Aylward -
Carlos E. R. -
jdd