Truecrypt alignment workaround, I increased usb flash drive speed by 50%

[jom1000]

My USB drive writes at 16Mbyte/sec. Using Truecrypt to encrypt the partion, it went down to around 8Mbyte/sec. By moving the start of the partition by exactly 60 sectors, I increased the write speed to 13Mbyte/sec.

 

Explanation:

My flash drive can not write individual bytes or sectors, it can only write 'blocks' of 64 sectors (=32KB) at a time. The FAT32 filesystem likewise write 'clusters' of 64 sectors (=32KB) at a time.

 

The location of the each cluster must correspond to the start of a 'block', so that they are aligned. If the start of the cluster is in the middle of a block and the end of the cluster is in the middle of the next block, then when the cluster is written to the flash, two blocks are written instead of one. In other words, to write a single 32KB cluster, two 32KB blocks are written. This reduces the write speed.

 

The problem with Truecrypt is that is does not take the alignment into account. In the truecrypt user guide it says that the first sector "is stored in the last 512 bytes of the first logical drive track". Because there is no relation between the start of a track and the start of a block, the truecrypt partition usually starts part way into a block, which is why the write performance is bad.

 

To work around this problem, I repartitioned my drive. The partition originally started at sector 8192. This gave 16MByte/sec. This is the "magic number" - if the partition starts at 8191 or 8193, then it overlaps two blocks and the write speed goes down to around 8MByte/sec.

 

When I encrypted the partition with Truecrypt, I only got 8Mbyte/sec because it does not place the volume at sector 8192, but instead it places the volume at the end of the track which 8192 belongs to. I have 63 sectors per track, so sector 8192 is the second sector of the 130th track. Truecrypt started its volume at the end of this track (sector number 8252), which is 60 sectors too far. So the solution was to move the partition back by 60 sectors, so the partition started at 8132 instead of 8192. This caused the first sector of the Truecrypt volume to be located at the magic sector 8192.

 

Note: this only works with a "Standard Truecrypt Volume". If you use a "Hidden Truecrypt Volume", then the start of the inner volume is dependent on what files you have copied to the outer volume. I havent worked out how to align this.

 

Note: my tests were done with FAT32 copying a 500MB file on a 16GB Corsair Flash Voyager GT.

 

Note: the size of the cluster can be specified when formatting, and should be set to the block size of the flash drive - in my case 32KB.

 

How did I work all this out? Here is a useful link:

 

http://www.hjreggel.net/cardspeed/cs_calign.html

[pawluk]

My USB drive writes at 16Mbyte/sec. Using Truecrypt to encrypt the partion, it went down to around 8Mbyte/sec. By moving the start of the partition by exactly 60 sectors, I increased the write speed to 13Mbyte/sec.

 

Explanation: TRIMMED - see original post

 

An extraordinary post, jom1000.

 

I too have a 16GB FV, though non-GT version, and I would like to investigate improve its Write Speed.

 

Having no experience with hex editors or TrueCrypt software, I read the link that you suggested and the various articles associated with it.

 

To begin, after a brief Google search, I chose to download and install the free release of Hex Editor Neo, version 4.73.0.2639, and while I am reasonably confident that the free version will not help me move my partition, it may give me a little insight on how this is accomplished. Thus far, I discovered the Shift Operation. Unfortunately, it does not provide any detail on how this operation is implemented.

 

Perhaps I am being foolish in my quest to improve the overall performance of my 16GB. Formatted to FAT32 with 32K cluster size, its Read speed is fine at 32.98MB/s, but I want to improve its Write speed, currently at 9.75MB/s. These results were achieved using HDBench 3.40 Beta 6 with its default 100MB file.

 

Do you have any recommendations regarding a quick tutorial that can provide me with the procedure necessary to verify the placement of its volume at sector, and how to shift or move it with a hex editor?

 

Oh, RAM GUY, I am well aware of your suggestions including but not limited to HP USB Disk Format Tool, and your lame SD-CARD Organization software recommendation. The SD-CARD Organization software is an abomination and should not be used with USB drives. Either software's formatting can effectively and more efficiently be achieved using Window's FORMAT command line syntax. Politely keep your lame, misguided suggestions and related comments to yourself.

 

Many thanks,

 

Pawluk

[Wired]

pawluk, please keep your post on topic. Ram Guy has not mentioned that software in this thread, let alone even posted in this thread. If you have other issues, please post them in a new thread.

[jom1000]

please keep your post on topic. Ram Guy has not mentioned that software in this thread

 

Although I didnt mention it when I started this thread, funnily enough, it was the SD-CARD Organization software that put me on the right track!

 

When I first got my USB stick, it had write speeds of 16MByte/sec, the maximum speed according to spec.

 

However during my messing about, the speed dropped to 8MByte/sec (this was before I tried Truecrypt). I tried getting it back to 16MByte/sec Window's FORMAT with different cluster sizes, with no success.

 

Then I discovered this forum, tried the SDCARD software - and yes, it was back to 16MByte/sec! I took a look at what it did, and found it was because it placed the partition at the "magic" sector 8192. I'd be interested in more info about how this program works. (The HP software did not help).

 

Then when I tried Truecrypt, the speed was back down to 8MByte/sec. By this time I knew I could make it faster - as you can see from my first post.

 

 

Do you have any recommendations regarding a quick tutorial that can provide me with the procedure necessary to verify the placement of its volume at sector, and how to shift or move it with a hex editor?

 

 

Use a partition program! I dont think you can use a sector editor (maybe you can, but it is hard work). I simply used a partition program that allowed me to easily place the partition at sector 8132. (The program I used was parted under Linux, though there are probably windows partitioners that will allow you to do the same thing). Note: I used windows for everything else.

[CurlySue]

Use a partition program! I dont think you can use a sector editor (maybe you can, but it is hard work). I simply used a partition program that allowed me to easily place the partition at sector 8132. (The program I used was parted under Linux, though there are probably windows partitioners that will allow you to do the same thing). Note: I used windows for everything else.

 

I have looked at several free partition managers and none seem to recognize my 16GB FV. Easeus Partition Manager 3.0 appeared to be the best of the bunch, it pegged the drive at 10.75 GB regardless of flash drive size. I tried several, from 1GB King's to 8GB and 16GB Corsairs, all were supposedly 10.75 GB drives. Similar problems exist with Cute Partition Manager 0.9.8, Paragon Partition Manager Express 9.0, and Ranish Partition Manager 2.44 partition managers.

 

If you don't mind me asking, which partition manager program are you using to achieve these magic results, jom1000?

 

Have you had any luck accomplishing the "magic" with any freeware partition manager?

 

Thanks.

 

CurlySue

[jom1000]

If you don't mind me asking, which partition manager program are you using

 

I used parted

 

http://www.gnu.org/software/parted

 

It is free and comes with Linux. (I use Ubuntu in a virtual machine in Windows).

[RAM GUY]

Great information jom1000 and thank you for taking the time to share that with everyone!

[SoundEmporium]

I used parted http://www.gnu.org/software/parted It is free and comes with Linux. (I use Ubuntu in a virtual machine in Windows).

 

Since 90+% of PC users run Windows, I downloaded PARTED as gparted-livecd-0.3.4-11.iso for Windows. Sadly, unlike your experience, PARTED was USELESS in Windows. It provided nothing like you describe. My bet is your experience was funded by CORSAIR.

[RAM GUY]

Since 90+% of PC users run Windows, I downloaded PARTED as gparted-livecd-0.3.4-11.iso for Windows. Sadly, unlike your experience, PARTED was USELESS in Windows. It provided nothing like you describe. My bet is your experience was funded by CORSAIR.

 

SoundEmporium,

I can tell you that we had absolutely nothing to do with this the user did this all on his own.