Results 1 to 6 of 6
Hello. Briefly the problem is slowness of copying a big file into flashdisk.
It is a new HP v210w 32 gb flashdisk. It is FAT32 formatted. When I try to ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
- 12-16-2012 #1Just Joined!
- Join Date
- Dec 2012
- Posts
- 2
Weird cache flush problem with 32 gb flashdisk
Hello. Briefly the problem is slowness of copying a big file into flashdisk.
It is a new HP v210w 32 gb flashdisk. It is FAT32 formatted. When I try to write a big file (bigger than about 50 MB) into it, after writing about half of data, it begins to struggle.
An example, it is a 321 MB file, 4th coloumn is write speed in KB/sec, as one line per second appended to output:
No problem for 390 MB of 98 files:Code:> iostat -k -t 1 |grep sdc sdc 0,06 1,24 0,00 8661 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 66,00 0,00 7618,50 0 7618 sdc 84,00 0,00 10008,00 0 10008 sdc 85,00 0,00 9792,50 0 9792 sdc 84,00 0,00 10008,00 0 10008 sdc 83,00 0,00 9960,00 0 9960 sdc 82,00 0,00 9840,00 0 9840 sdc 83,00 0,00 9960,00 0 9960 sdc 85,00 0,00 9665,00 0 9665 sdc 83,00 0,00 9960,00 0 9960 sdc 81,00 0,00 9696,00 0 9696 sdc 81,00 0,00 9720,00 0 9720 sdc 85,00 0,00 9702,00 0 9702 sdc 79,00 0,00 9480,00 0 9480 sdc 93,00 0,00 9836,00 0 9836 sdc 44,00 0,00 4436,00 0 4436 sdc 0,00 0,00 0,00 0 0 sdc 26,00 0,00 2532,00 0 2532 sdc 81,00 0,00 7744,00 0 7744 sdc 0,00 0,00 0,00 0 0 sdc 8,00 0,00 820,00 0 820 sdc 92,00 0,00 9195,50 0 9195 sdc 16,00 0,00 1508,00 0 1508 sdc 1,00 0,00 24,00 0 24 sdc 28,00 0,00 2820,00 0 2820 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 25,00 0,00 2400,00 0 2400 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 78,00 0,00 7556,00 0 7556 sdc 48,00 0,00 4856,00 0 4856 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 35,00 0,00 3472,50 0 3472 sdc 1,00 0,00 20,00 0 20 sdc 13,00 0,00 1284,00 0 1284 sdc 0,00 0,00 0,00 0 0 sdc 5,00 0,00 448,00 0 448 sdc 54,00 0,00 5364,00 0 5364 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 12,00 0,00 1176,00 0 1176 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 44,00 0,00 4340,00 0 4340 sdc 0,00 0,00 0,00 0 0 sdc 3,00 0,00 348,00 0 348 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 41,00 0,00 4068,00 0 4068 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 32,00 0,00 3276,00 0 3276 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 63,00 0,00 6400,00 0 6400 sdc 45,00 0,00 4392,00 0 4392 sdc 0,00 0,00 0,00 0 0 sdc 22,00 0,00 1956,00 0 1956 sdc 53,00 0,00 5320,00 0 5320 sdc 0,00 0,00 0,00 0 0 sdc 64,00 0,00 6156,00 0 6156 sdc 14,00 0,00 1480,00 0 1480 sdc 0,00 0,00 0,00 0 0 sdc 8,00 0,00 724,00 0 724 sdc 19,00 0,00 1920,00 0 1920 sdc 23,00 0,00 2492,00 0 2492 sdc 43,00 0,00 4812,00 0 4812 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 9,00 0,00 760,00 0 760 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 13,00 0,00 1368,00 0 1368 sdc 0,00 0,00 0,00 0 0 sdc 1,00 0,00 52,00 0 52 sdc 3,00 0,00 232,00 0 232 sdc 0,00 0,00 0,00 0 0 sdc 3,00 0,00 356,00 0 356 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 14,00 0,00 1248,00 0 1248 sdc 0,00 0,00 0,00 0 0 sdc 67,00 0,00 6700,00 0 6700 sdc 16,00 0,00 1604,00 0 1604 sdc 0,00 0,00 0,00 0 0 sdc 6,00 0,00 508,00 0 508 sdc 37,00 0,00 3764,00 0 3764 sdc 11,00 0,00 916,00 0 916 sdc 91,00 0,00 9388,00 0 9388 sdc 82,00 0,00 8556,00 0 8556 sdc 0,00 0,00 0,00 0 0 sdc 1,00 0,00 76,00 0 76 sdc 7,00 0,00 768,00 0 768 sdc 0,00 0,00 0,00 0 0 sdc 11,00 0,00 952,00 0 952 sdc 51,00 0,00 4856,00 0 4856 sdc 1,00 0,00 108,00 0 108 sdc 86,00 0,00 8652,00 0 8652 sdc 27,00 0,00 2996,00 0 2996 sdc 16,00 0,00 1484,00 0 1484 sdc 3,00 0,00 360,00 0 360 sdc 0,00 0,00 0,00 0 0 sdc 11,00 0,00 980,00 0 980 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 4,00 0,00 384,00 0 384 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 52,00 0,00 5284,00 0 5284 sdc 49,00 0,00 5008,00 0 5008 sdc 0,00 0,00 0,00 0 0 sdc 82,00 0,00 7832,00 0 7832 sdc 7,00 0,00 744,00 0 744 sdc 0,00 0,00 0,00 0 0 sdc 16,00 0,00 1528,00 0 1528 sdc 97,00 0,00 9652,00 0 9652 sdc 29,00 0,00 3284,50 0 3284 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0
Also no problem when I write the file with "dd" with "sync" flag.Code:> iostat -k -t 1 |grep sdc sdc 0,49 1,19 44,90 8726 328987 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 56,00 0,00 5955,00 0 5955 sdc 0,00 0,00 0,00 0 0 sdc 2,00 0,00 160,00 0 160 sdc 37,00 0,00 4360,00 0 4360 sdc 84,00 0,00 9872,00 0 9872 sdc 84,16 0,00 9308,91 0 9402 sdc 84,00 0,00 9822,50 0 9822 sdc 83,00 0,00 9219,00 0 9219 sdc 81,00 0,00 9609,00 0 9609 sdc 87,00 0,00 9205,50 0 9205 sdc 82,00 0,00 9587,00 0 9587 sdc 85,00 0,00 9403,50 0 9403 sdc 89,00 0,00 9074,50 0 9074 sdc 82,00 0,00 9768,50 0 9768 sdc 84,00 0,00 10004,50 0 10004 sdc 87,00 0,00 9307,00 0 9307 sdc 80,00 0,00 9543,50 0 9543 sdc 89,00 0,00 9366,50 0 9366 sdc 6,00 0,00 574,50 0 574 sdc 0,00 0,00 0,00 0 0 sdc 38,00 0,00 4508,00 0 4508 sdc 84,00 0,00 9080,50 0 9080 sdc 16,00 0,00 1920,00 0 1920 sdc 79,00 0,00 9280,00 0 9280 sdc 84,00 0,00 9261,50 0 9261 sdc 82,00 0,00 9714,00 0 9714 sdc 81,00 0,00 9072,00 0 9072 sdc 80,00 0,00 9345,00 0 9345 sdc 82,00 0,00 9840,00 0 9840 sdc 84,00 0,00 8949,50 0 8949 sdc 81,00 0,00 9556,50 0 9556 sdc 83,00 0,00 8665,50 0 8665 sdc 82,00 0,00 9749,50 0 9749 sdc 83,00 0,00 9944,50 0 9944 sdc 85,00 0,00 8827,50 0 8827 sdc 81,00 0,00 9491,50 0 9491 sdc 85,00 0,00 9136,50 0 9136 sdc 77,00 0,00 9238,50 0 9238 sdc 83,00 0,00 9058,00 0 9058 sdc 80,00 0,00 9170,00 0 9170 sdc 83,00 0,00 8861,50 0 8861 sdc 83,00 0,00 9908,00 0 9908 sdc 86,00 0,00 9056,00 0 9056 sdc 82,00 0,00 9590,50 0 9590 sdc 82,00 0,00 8719,50 0 8719 sdc 82,00 0,00 9644,50 0 9644 sdc 83,00 0,00 8997,50 0 8997 sdc 78,00 0,00 9314,00 0 9314 sdc 72,00 0,00 7935,50 0 7935 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0 sdc 0,00 0,00 0,00 0 0
I use OpenSuse 12.2, kernel is 3.4.11-2.16.
Mount options of disk:
/dev/sdc1 on /media/BALIKHAFIZA type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=100,fmask=0 022,dmask=0077,codepage=cp437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remou nt-ro,uhelper=udisks)Last edited by Behzat; 12-16-2012 at 04:01 PM.
- 02-01-2013 #2Linux Guru
- Join Date
- Apr 2009
- Location
- I can be found either 40 miles west of Chicago, or in a galaxy far, far away.
- Posts
- 10,148
This is not an uncommon problem, which is why I don't use USB flash drives, but class 10 SD cards with a USB carrier. These are the (currently) highest performing flash drives available for write performance. Many, small files will normally work ok (a lot of that is cached with write-behind behavior), but a few really big files will force the system to write directly to disc. As for "dd" with "sync" flag, that will write each sector to disc as it is passed to the operating system, so it will utilize the hardware bandwidth more effectively. As an experiment, if you were to write all those small files with something like "cp", when it was done, try executing the "sync" command (as root), and see how long before the data is REALLY written to the device! A lot longer than you expect, I think.
So, get the fastest flash devices you can if write performance is critical - that means using class 10 SD cards and NOT normal USB flash drives.Sometimes, real fast is almost as good as real time.
Just remember, Semper Gumbi - always be flexible!
- 02-02-2013 #3Super Moderator
- Join Date
- Jun 2006
- Location
- Chandigarh, India
- Posts
- 24,722
Have you tried using default options in mount command?
Code:mount -t vfat /dev/sdc1 /media/BALIKHAFIZA -o defaults,umask=0
It is amazing what you can accomplish if you do not care who gets the credit.
New Users: Read This First
- 02-02-2013 #4Linux Guru
- Join Date
- Apr 2009
- Location
- I can be found either 40 miles west of Chicago, or in a galaxy far, far away.
- Posts
- 10,148
Interesting DC. Can you explain what you mean about this? What do these default options change about how the flash drive will behave?
Originally Posted by devils casper 
Sometimes, real fast is almost as good as real time.
Just remember, Semper Gumbi - always be flexible!
- 02-02-2013 #5Just Joined!
- Join Date
- Dec 2012
- Posts
- 2
But they tend to be fragile and comlicated, so not for my pocket Originally Posted by Rubberman 
Indeed I tried them before. They have their problems, harder to solve. For example I couldn't boot computer. I couldn't use it on satelite receiver's video recorder. It is a featured high quality usb carrier. I think it is hard to find a carrier which emulates with full features.
It worked Originally Posted by devils casper The problem is the "flush" option. Adding it to options repeats the issue.
Before this topic I suspected of it in the first place and experimented with options without "flush" but it behaved with the same problem. Or somehow it fooled me.
Thanks
- 02-02-2013 #6Linux Guru
- Join Date
- Apr 2009
- Location
- I can be found either 40 miles west of Chicago, or in a galaxy far, far away.
- Posts
- 10,148
Glad you got this sorted out. FWIW, I use SD and micro-SD cards in USB carriers all the time (they look like thumb drives, but with a slot in the back for the SD card) and can install and boot/run Linux on them without any trouble. In fact, I have a little old chocolate tin that I keep a bunch in, each with a different live distribution installed, so I can easily demonstrate a number of distributions to people or in classes without any trouble at all. IE, if I am teaching a class that is server related, I boot the CentOS one. If I am teaching a class that is related to general user services, then I boot an Ubuntu or Mint one. I just use a Sharpie to mark them so I know which is which... Originally Posted by Behzat But they tend to be fragile and comlicated, so not for my pocket Indeed I tried them before. They have their problems, harder to solve. For example I couldn't boot computer. I couldn't use it on satelite receiver's video recorder. It is a featured high quality usb carrier. I think it is hard to find a carrier which emulates with full features.
It worked The problem is the "flush" option. Adding it to options repeats the issue.
Before this topic I suspected of it in the first place and experimented with options without "flush" but it behaved with the same problem. Or somehow it fooled me.
Thanks Sometimes, real fast is almost as good as real time.
Just remember, Semper Gumbi - always be flexible!
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
