Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×

Fast File Encryption for Windows? 117

Posted by Cliff from the speed-over-security dept.
cryptoz wonders: "I've used numerous encryption applications for both Windows and Linux over the past few years and have always been satisfied. Until I realized I needed to start encrypting large files (say 10 to 30 GiB), or at least a large number of small(er) files. I found that everything I use seems to take hours and hours to compress, encrypt and shred. Not to mention decompressing, decrypting and deleting on the other end. Every web search I do on the topic seems to turn up mostly closed-source applications or snake oil, neither of which is acceptable. Does Slashdot have any suggestions for fast file encryption? I should make it clear that in my particular case, I do not need to have a perfect key or incredibly secure encryption, since it is not the weakest link (as I am susceptible to hardware key-loggers, CRT eavesdropping and the like). The encryption needs to be just strong enough, but most importantly, *fast*." This is a worthwhile question, but when dealing with files in the 10s of GB, can anything really be considered to be "fast"?
This discussion has been archived. No new comments can be posted.

Fast File Encryption for Windows? More

Fast File Encryption for Windows?

Comments Filter:

  • TrueCrypt (Score:4, Informative)

    by RemovableBait ( 885871 ) * <[slashdot] [at] [blockavoid.co.uk]> on Monday June 12, 2006 @04:50PM (#15519517) Homepage
    I'd say your best bet'd be TrueCrypt [truecrypt.org].

    You linked to it yourself, so you should be aware of the strengths of the application. It does on-the-fly disk encryption with either whole partitions or disk image files, has absolutely no problem with massive disks (I have a 40GB image on a USB drive), and is pretty fast. My benchmarks come up with 50MB/s average throughput (around 56MB/s encrypting, 47MB/s decrypting) for 256bit AES encryption on my machine. TrueCrypt seems to cope well with files of any size, and while I can't say I've tried 30GB, 4.7GB DVD images work very well indeed.

    One thing that really makes it stand out in your scenario is the ability to use keyfiles. This allows you to select one or more files that will be used (hashed?) with your password to secure your data against those hardware keyloggers. (Although, I would question whether encryption is really required if you aren't that bothered about security.)

    The best part of TrueCrypt is that it is completely open-source. No closed/proprietary systems and no snake oil. For encryption on Windows, when the built in stuff doesn't cut it, TrueCrypt is the only way to go, IMHO.
    • One thing that really makes it stand out in your scenario is the ability to use keyfiles. This allows you to select one or more files that will be used (hashed?) with your password to secure your data against those hardware keyloggers. (Although, I would question whether encryption is really required if you aren't that bothered about security.)

      It all depends on the threat model. I could see desiring encryption without being bothered by keyloggers if you're worried about someone breaking into your car an
    • Agreed. One point worth noting is that I can't think of many ways of producing a dataset that large where the data is produced faster than TrueCrypt can encrypt it. Don't store it on an unencrypted partition and then encrypt it for processing, produce it directly on the encrypted partition and then move the resulting data. Similarly, don't decrypt to local storage at the other end, use the file directly from the encrypted partition; chances are your consuming application (presumably some kind of data ana
  • Security is the antithesis of comfort/ease of use.

    Also, security can be increased to downright unusability, too.
    • I would consider that a feature. If what you are working with is so super-double-secret, maybe it isn't safe for you to see it either.
      a la "I got an idea, an idea so smart my head would explode if I even began to know what I was talking about."
    • Security is the antithesis of comfort/ease of use.

      Not really: There are three offerings: cheap, secure, useable... Pick any two

      You may combine ease of use and secure, but this will cost...

      Regards, Martin

  • It's what I use and it's quite fast. I get about 52 MB/sec encryption speed and I'm loving it.

  • Dunno why noone mentioned it... (Score:5, Funny)

    by Monokeros ( 200892 ) on Monday June 12, 2006 @04:52PM (#15519533)
    But you should look at TrueCrypt

  • Yes... (Score:5, Funny)

    by GillBates0 ( 664202 ) on Monday June 12, 2006 @04:53PM (#15519541) Homepage Journal
    ...when dealing with files in the 10s of GB, can anything really be considered to be "fast"?

    Yes, a station wagon filled with tapes of 10GB+ files doing 80mph on a highway is going at a pretty fast clip in my opinion. YMMW.

    With apologies to AS Tanenbaum.

    • Actually, that's probably the fastest method of data transfer. Faster than pigeons or snails, even.
    • I can imagine how the shredding is done, but how does the station wagon encrypt the tapes?
      • I can imagine how the shredding is done, but how does the station wagon encrypt the tapes?

        Obviously with a transubstantiation cipher[1]. Sheesh, kids these days.

        [1] you don't put labels on, and you just throw the tapes sort of haphazard into the back of the wagon - then at the far end you rely on pulling some sort of miracle out of your backside to get the data off in order.
    • station wagon filled with tapes of 10GB+ files doing 80mph on a highway is going at a pretty fast clip in my opinion.

      Or a 747-400 Freighter full of 750GB HDDs flying at cruising speed.

      2,565 HDDs fit in a cubic meter. (http://www.westerndigital.com/en/products/Product s.asp?DriveID=137 [westerndigital.com])

      The 747-400 Freighter has 777 m^3 of cargo space. (http://www.montereypeninsulaairport.com/747specsh eet.html [montereype...irport.com])

      This gives 2565 * 777 = 1,993,005 HDDs.

      750GB == 698.5GiB

      This gives us a grand total of 1,392,113,992.5 GiB or 1.392

      • Re:Yes... (Score:2)

        by mph ( 7675 )
        So that plane carries 1.392 ZiB in ~4.5 hours (16200 seconds) for a total of 85.9PiB per second.
        Yeah, but I'd still hate to ssh over that link.
      • Sure, but think about the latency.

      • You may want to double check the weight of 1.9 million hard drives vs. the maximum cargo weight allowance for a 777-400 freighter. A long time ago I did this same math for DLT cartridges and a 747 cargo plane, and I noticed that you couldn't fill the cargo space more than 60% full with DLTs before the plane was too heavy to take off.

        • I noticed that you couldn't fill the cargo space more than 60% full with DLTs before the plane was too heavy to take off.

          DLTs, eh? You must be a DECcie.

          Good point about the weight. The -400 Freighter can carry 112,490kg of cargo, and each HDD weighs 0.6 kg, meaning 187,483 hard drives.

          Bummer, that's less than 10% of the cargo space, and "only" 130.96 EiB.

          Since the latest SuperDLT tapes have 300GB (279.4GiB) raw capacity, SuperDLT600 tapes would give you much better bandwidth.

  • SureCrypt (freeWare) (Score:5, Informative)

    by neonprimetime ( 528653 ) on Monday June 12, 2006 @04:54PM (#15519552)
    Ever tried this? SureCrypt [wiredfile.com]

    SureCrypt is an ultra small encryption program designed for fast processing of extremely large files. It can encrypt or decrypt files as fast as Windows Explorer can copy them. SureCrypt presents a flexible user interface with detailed record of all operations.

  • Just ROT2 the bits. (Score:1, Informative)

    by Anonymous Coward
    Just ROT2 the bits. Or is it ROT1?

  • -1, Pedantry (Score:2, Insightful)

    by Gothmolly ( 148874 )
    GiB? Dude, just say GB, we all get it. It's a buttload of data.
    • We all know the "G" is the important letter there, but why on earth would GB be preferrential to GiB? You don't give any reason. Obviously, you don't think typing 1 extra letter is much of a reason, since you pounded out several for your post.
      • We're all used to seeing just "GB", so when you see "GiB", it throws you off because it doesn't look right. Plus it looks like a word, in fact is a word (slang) in online FPS games. Plus it's a way of "showing off" for the technically "elite", aka the snobs who think they are so fucking smart and they just can't believe other people would prefer "gigabyte" over "gibibyte" :P
      • We all know the "G" is the important letter there, but why on earth would GB be preferrential to GiB?

        1. Because GB is a more well known shorthand for a data amount.
        2. Because a difference of 73,741,824 bytes doesn't matter in this article.

        • Re:-1, Pedantry (Score:4, Insightful)

          by Lord Ender ( 156273 ) on Monday June 12, 2006 @09:00PM (#15520936) Homepage
          "1. Because GB is a more well known shorthand for a data amount."

          That is a reason TO use GiB. It promotes awareness so that there is no confusion when it DOES matter.

          "2. Because a difference of 73,741,824 bytes doesn't matter in this article."

          Um... that supports the argument that it doesn't matter one way or the other which one is used, making the initial complaint seem pointless.
      • Because if people started using GiB and the likes, they couldn't bitch about harddrive manufacturers giving them less bytes than they expected.

        If people want to continue using GB when they really mean the ISO-defined GiB quantity, go right ahead. But don't complain about people choosing to use the correct measurement unit.
  • TrueCrypt works like a v-twin on a golf cart.

  • Who's asking? (Score:3, Funny)

    by UnknowingFool ( 672806 ) on Monday June 12, 2006 @05:04PM (#15519627)
    Dear cryptoz, We'd like to discuss you encrytption concerns. With our vast experience in encryption and decryption, we believe through our highly effective questioning we can find the right product for you. Please arrive at our facility at Fort Meade at any time. Ask for the Best Interest of National Security Special at the front desk when you arrive. Sincerely, The NSA

  • Another nod for Truecrypt (Score:3, Interesting)

    by cptgrudge ( 177113 ) on Monday June 12, 2006 @05:04PM (#15519628) Journal
    Personally, I used truecrypt [truecrypt.org] on Windows before I moved to Ubuntu, and use the same now, though it's a little more work to get it running. It looks like you've used it before, though. I'm not sure why truecrypt wouldn't work.

    As far as shredding files goes, that isn't really connected with the encryption process, but more to your hard disk speed. Writing random bits to a 10-30 GiB file is going to take a while no matter what program you use.

    • If the crypto is done right, shouldn't destroying the keys and maybe some random parts of the file be enough to completely remove any hope of recovering the data?
      • If the crypto is done right, shouldn't destroying the keys and maybe some random parts of the file be enough to completely remove any hope of recovering the data?

        If I understand TrueCrypt's technology and assuming that you didn't let an attacker copy your TrueCrypt volume header... overwriting the first 512 bytes of a TrueCrypt volume destroys the key for that volume. Unless you have a backup of the volume header, your data is lost and unrecoverable unless you get lucky and can break the encryption key.
        • I'm pretty sure that this is correct. Although, this of course assumes that the proverbial Men in Black haven't procured your computer before you've had a chance to blow away the volume header and render the data unusable.
      • Not for the truly paranoid. Truecrypt does have a plausible deniability feature (aside from normally encrypted data looking like random noise), where you have a hidden volume [truecrypt.org] with your "real" data in case you are forcibly made to reveal your keys. You can reveal your "key" to the person that is forcing it from you, but all they see are some "semi-private" documents that you've put there in the dummy volume. Your "real" key is still safe, and all they see is random gibberish for your other data. Since th
    • I personally prefer an 8 to 10lb, solid steel "shredder" for hard disks. You also get a visceral feedback when it's working.
      • Yes, that would be much quicker, though it renders the disk permanently unusable, obviously. Much more fun, though.

        Reminds me of dangerous fun had with a potato cannon a buddy and I built a few years back. Something about giving a potato the same kinetic energy as a 15 pound bowling ball travelling at 125 MPH and slamming it into stationary objects still brings a smile to my face.

  • Security costs CPU cycles (Score:3, Insightful)

    by Ckwop ( 707653 ) * on Monday June 12, 2006 @05:26PM (#15519780) Homepage
    I found that everything I use seems to take hours and hours to compress, encrypt and shred. Not to mention decompressing, decrypting and deleting on the other end.

    XOR against a repeated key would be ultra-fast but woefully insecure. When will people learn that it takes CPU cycles to encrypt that much plain-text? In just about every other field you don't get something for nothing; why should Cryptography be any different?

    Simon

    • Biham and Seberry's "Py" can clip along at approaching 2 cycles/byte. That means a high-spec machine could be decrypting that 30GB file in around ten seconds - far faster than it can read it from disk, in fact.

    • Yes, but there is a huge difference between unencrypting the entire file to read one bit or something that intelligently breaks the file up so you can randomly read bits of it without having to deal with the entire file. With modern computing processing speeds you can achieve decent encryption in almost the time that the average HardDrive can read with only a slighly delay.

  • Hardware acceleration. (Score:5, Interesting)

    by wild_berry ( 448019 ) on Monday June 12, 2006 @05:26PM (#15519788) Journal
    I suggest getting some hardware acceleration: the VIA EPIA boards use electrical interference in their traces to suppy entropy to a hardware encrypt/decrypt enginge that can achieve 25 Gb/s encryption. This [via.com.tw] is a 1.0GHz passively-cooled board with SATA ports, hardware MPEG2 decoding and all on a 17x17 cm^2 board.
    • Quick fact-checking: the present VIA Padlock site says it can securely hash data at 20 Gb/s, in contrast to my memory telling me of VIA's pages showing a benchmark of encryption at 25 Gb/s on a CPU at around 1GHz, quadrupling the encryption capability of a P4 at 3GHz.
    • I don't suggest EPIA if what you're after is speed. They're small and low-power, but not fast. The 25 Gb/s is a joke, I don't even think the chip even has that much memory bandwidth: you might get it for ECB mode cache->cache, but the whole system will crawl.

      If you count other things your OS will be doing, an Athlon system will be faster and cheaper. However, the EPIA will be half the size and draw a third the power.
    • I have to suspect that VIA is overstating their hardware's capability. If they aren't, why in the hell aren't they putting these chips onto PCI cards and selling crypto accellerators for web transactions? Assuming that the crypto chip is 20% of the cost of an EPIA board, you could pack 4 of the chips onto each card and still be able to sell it at a profit for $150.

      LK
      • That would be an interesting idea. I believe that the encryption functions are part of their fork of the x86 instruction set, and run on their CPU's. That would add complexity to making a PCI encyption-acceleration device, but could be a profitable venture.

        I'm also suspicious of its capabilities -- it seems feasible to have a large source of entropy and perhaps a hashing or XOR acceleration engine -- I doubt it has the bandwidth to read and hash at 20 Gb/s as the present marketing web pages claim Padlock

  • You're all terrorists and kiddie-diddlers (Score:3, Funny)

    by QCompson ( 675963 ) on Monday June 12, 2006 @05:35PM (#15519847)
    Everyone who uses encryption is a terrorist and/or a child molester. If you're not doing anything wrong, what do you have to hide?

    Personally, I videotape all my daily activities and archive them in case a law enforcement agency wants to know what I was up to on a particular date. I suggest you all do the same. Think of the children and 9/11!!!
  • Check BestCrypt [jetico.com]. I've been using it for years, and like it (haven't tried 10Gb though). TrueCrypt looks like the same concept, so use the trial to speed test for comparison. It's not free, but it is available for windows and linux.

    Also, be aware that your encryption choice will affect speed greatly. 448-bit is slower than 224 bit, etc. Also some algos are optimized - twofish is a pentium-optimized version of blowfish.
  • Is it specifically the encryption, or are the compression and shred (both of which often do not scale well) taking the most of the time?

  • Seagate recently released a self-encrypting hard-drive... does hardware level encryption at S-ATA link speed, or so they claim. More info: http://www.apcstart.com/site/dwarne/2006/06/263/se agates-self-encrypting-hard-drive [apcstart.com]
  • A fast and reliable way to encrypt a file is to sweep a strong magnet across your hard disc. Decryption of the files is more difficult and time comsuing, scientist are still working hard to find the final solution.
    • There are two *very* strong magnets right next to my hard disk's platters. The hard disk is pretty unimpressed. Chances are that you don't have any magnets available that would destroy your hard disk.

      Use a sledge hammer.
  • NTFS encryption seems to be pretty fast -- we use it for doing encrypted backups onto portable USB hard drives. The bottleneck seems to be the hard drive speed rather than the encryption. I think we put about 40GB on the USB drives in under an hour or so. Mind you, the drives aren't that fast, they're the little laptop drives, plus there's the USB overhead. Perhaps, rather than looking for a faster encryption program you need faster/more hard drives. Lots of data is slow to push around and if the sourc
    • The big problem with NTFS encryption (a.k.a. Windows EFS) is that:

      - The keys are tied to the machine (you can't take those USB drives and mount them on another machine, at least not when I tested it)

      - The keys are a PITA to backup, the management interface is clunky

      - It's not the strongest system in the world (I believe there are numerous issues with how it was implemented)

      That being said, it's generally better then nothing for when you want to protect semi-confidential data. Most attackers won't t
      • Yep, the key issue is something that people must be aware of before using EFS for backups. The keys can be copied and moved. There are good instructions for this on MSDN or MSKB, or probably both, not too hard to find. And remember to keep a couple copies of the key on floppy or CD or something. And TEST the restore procedure, preferably more than once! The interface to backup and restore keys is a tad clunky, but not too bad. You only need to do it once on the backup machine, once on a test machine,
        • Sounds like your company is about the size of ours (less then 50 employees). We have a variety of backup schemes, depending on the user.

          For our remote workers using laptops, version-control software for corporate data and staying in sync with the other workers in their department. Combined with SecondCopy + TrueCrypt partitions on the USB/FireWire drives. The local USB/FW drive handles things like backing up their personal files or e-mail. We also recommend that they make use of a tool like Acronis Tru

  • Are you sure it's the encryption? (Score:3, Insightful)

    by cow-orker ( 311831 ) on Monday June 12, 2006 @06:16PM (#15520091)
    Assume a sustained transfer rate of 30MB/s, which is quite good for a single hard disk. You won't get that much when transferring lots of small files. Reading 30GB takes 1000s or about 18 minutes, writing it back another 18 minutes, doing both takes longer, because interleaving both processes will lower the transfer rate. If you're shredding the old data, you can add in another 20 minutes per pass. So encrypting 30GB takes 60 minutes, probably a lot more, and there's nothing you could do about it in software.

    Encryption itself... I seem to remember that TwoFish needs 26 clocks to encrypt 8 bytes on a Pentium. So your 2.6GHz CPU can encrypt 8GB/s (but the bus cannot deliver that much, I suspect). Add in some fudge factors for OS overhead and other tasks, and you're still two orders of magnitude below the IO time.

    You need faster disks.

    • Encryption itself... I seem to remember that TwoFish needs 26 clocks to encrypt 8 bytes on a Pentium. So your 2.6GHz CPU can encrypt 8GB/s (but the bus cannot deliver that much, I suspect). Add in some fudge factors for OS overhead and other tasks, and you're still two orders of magnitude below the IO time.

      BTW, TrueCrypt includes a little benchmark tool to allow you to calculate throughput rates for the various algorithms (as implemented inside of TrueCrypt). Useful for seeing just what the best-case ra
      • Blowfish (47) > Twofish (41) > CAST5 (35) > Serpent (34) > AES (33) > Triple-DES (12)

        Blowfish, Twofish, Redfish, Bluefish!
      • How old is that Opteron? I'm running an Athlon 64 300+(2Ghz, 754), XP Pro (not 64 Bit) and I get means of Blowfish (48.2), Twofish(42.3), CAST5(35.2), Serpent(35.6), AES(33.8), Triple-DES(12.6).

        LK
        • It's an Opteron 246 chip (well, actually a pair of them, but TrueCrypt 4.2 isn't multi-threaded yet) from around early 2005 [pricescan.com]. I know it wasn't top of the line when I bought it (I think the 248s were out by then).

          It's the 2GHz core with 3GB of PC3200 RAM running WinXP Pro 32bit. The motherboard is a Tyan Tiger K8W S2875 with a slightly odd memory path. Only one of the Opterons is connected to the memory, the 2nd Opteron routes its memory access through the first one. It's not ideal, but it was the small
    • You're assuming that he's doing nothing else with his CPU. He could be running CPU-intensive stuff at the same time. It's impossible to tell from the description, though.
      • I'm also assuming he's doing nothing else with the disk. Also, what's not to understand about "order of magnitude"?! Dude, it's I/O bound, no matter what else he's doing. And frankly, I cannot imagine why anyone would read/decrypt/write a chunk of 30GB at a time. Completely useless, doing it in the background while using the data (must be pr0n flicks anyway) would make much more sense.
    • Doh, my math skills seem to be a bit rusty, and nobody notices. I also remembered the numbers incorrectly. Bruce Schneier says, Twofish costs 17.8 cycles per byte, so a 1.7GHz cpu could encrypt 100MB/s. The bus is still hard pressed to shovel that much data back and forth and it is still a lot more than a hard disk can deliver, especially if it is read and written at the same time.
  • Hi,

    I would recommend "KRYPTO", or more precise "KRYPTO 2.0/2006 Professional Multi User Professional Data Fullbit Coding Program". The program uses the best encryption possible (called 256 bit fullbit encryption). Read up on it here:

    Kryptochef [kryptochef.net]

    The application even sports a friendly GUI that is easy to use and allows even novice users to encrypt files.

    Cheers, Fogger
  • PKZIP has built-in encryption, both their old-style proprietary algorithm as well as AES. It works, it's fast, and it has all sorts of other benefits. I use it all the time and I'm very happy with it.
  • I don't know if it's of any use in this situation but you might want to look into AxCrypt

    http://axcrypt.axantum.com/ [axantum.com]
  • Check out Steve's SecurityNow! podcast 41
    to hear why & more about it:

    http://media.grc.com/sn/SN-041.mp3 [grc.com]

    For slow modem users, here's the transcript:

    http://www.grc.com/sn/SN-041.pdf [grc.com]

    A list of his other podcasts:

    http://securitynow.info/ [securitynow.info]
  • The real solution is don't encrypt individual file. Encrypt the whole disk including free space. Takes awhile to initially encrypt but not a big performance hit on use.

  • Consider Tiny Blowfish - skip the shredding step (Score:1, Informative)

    by Anonymous Coward
    Bear with me for a moment while I try to get to the point as directly as I can.

    There's a program called "Tiny IDEA" which implements the IDEA cipher. It's written in assembler for DOS, and comes with source code; the executable is about 500 bytes. It was originally written by Fauzan Mirza (who has credibility in that he also won Bruce Schneier's $10,000 award for best attack on Twofish during the AES competition). It was later further optimized and improved by someone named Mark Andreas, who I've never hear
  • But have you seriously considered using Windows built in Encrypting File System (EFS)? WHEN CONFIGURED PROPERLY, it can be both very secure and speedy.

    Not sure what you are doing with the files (i.e. staying on your machine or being distributed, etc.) but the EFS might be a very simple and effective option. Microsoft's website actually has some fairly good articles about it's usage beyond the stupid-user stuff.

    What's important to remember is that you MUST use Window's SYSKEY program in mode 2 or 3 in orde
    • First, read the forums and learn about the people who have lost all their EFS data because of the sloppiness of Microsoft.

      In some cases EFS is tied to the computer on which it is installed. You cannot restore it to another computer, even if you have all the keys.

      Were you thinking, oh this time Microsoft won't be sloppy?
      • Author never said he/she was moving files to/from different computers, as noted in my comment. EFS can be safe, secure, and FAST (what the author wanted) if you take the time to set it up properly.

        Author has a problem, I'm trying to offer a viable solution... a solution that I have found to work well on the enterprise level. So spare me the anti-M$ rhetoric, please.
  • I found that everything I use seems to take hours and hours to compress, encrypt and shred. Not to mention decompressing, decrypting and deleting on the other end.

    It sounds you don't know what TrueCrypt really does. Real-time transparent encryption does not "compress" nor "shred" anything.
  • Nice troll you got going there. Real nice.

    Anyone else notice that the submitter is called 'cryptoz', or that his linked website, http://www.sheehy.ca/crypto/ [sheehy.ca], is called "The Cryptography Center"?

    Also the little matter of his website's description saying "This website is designed as a location for as many cryptography resources as possible. The intent is to collect a large number of articles for those who are interested in learning more, practical computer applications to download, lists of other resources,

Slashdot Top Deals

It is easier to write an incorrect program than understand a correct one.

Close