Crypto Algorithm Question - Securing a remote backup

Hello,

I have a project I'd like to add a level of security to, but since I'm totally new to encryption, I could use some advice before proceeding.

The goal of my project is to allow me to synchronize/backup my desktop with a remote server, kind of like rsync. The difference is that I'd like the data to be encrypted on the server as it is not under my control. In searching for a ready made tool to do this I came across rsyncrypto (sorry, can't post a link to it as I'm a new user), but it is not available on the server, and the admins are not willing to install it. So I'm looking at writing my own solution, and using the Rijndael cipher to encrypt the remote data.

From what I've read, using a unique random IV for each encrypted file is very important, and then the most common method for preserving the IV for decryption is to prepend it to the encrypted file. But I actually want to encrypt each file in 10k increments (or some other size, haven't decided on that yet), so for example if I change the contents of a padded ID3v2 header in an MP3 file, only the first 10k block needs to be resynchronized, not the entire 5meg file (I realize this won't help much for text files, as the block containing the change as well as every subsequent block will have to be resynchronized if the file size changes, but it sounds fun to implement anyway)

So my first question is, I guess in the above situation, I should actually use a unique IV for each block of each file, correct?

Then my next question is, I've seen a few implementations take a short text password, salt it (and prepend the encrypted file with the salt to preserve it, just like the IV), and then hash it to generate the key to be used for encryption. Is this any more/less secure than just choosing 8 random 32bit numbers and using the combined 256 bits as a key? Obviously it would be harder to remember that, but I can store it at work or some other off-site location to avoid losing it.

And assuming the password + salt + hash route is the best way to go, is there a benefit to using a different random salt for each file? If not, I guess it could just be hard coded into the program and then it wouldn't need to be prepended to every encrypted file like the IV would need to be.

And lastly, I've seen one implementation that prepends the plaintext with 128 bits of salt before it goes through the encryption algorithm, and I'm just wondering what the reason for doing this might be since the IV is already different for each file, so even if 2 files begin with the same chunk of data, their encrypted outputs should be different, correct?

Thanks in advance,
Rick

You should use a package, rather than writing your own.
Look at GPG, its free, open source, well trusted.

Take the files on the server/computer you trust, run
gpg -c on them, to encipher them.
Then transfer/copy them to the untrusted host.

When you want the file, copy/transfer the file from the untrusted host to your trusted computer, decipher it using gpg and the same key, and use as you want.

Nearly all implementations use the output of one block's cipher text as the IV for the next. But that is an implementation details.

I strongly discourage folks from writing their own crypto until they have used and understand pre-packaged solutions. Its easy to mis-use crypto and end up with weak solutions.

Actually I did look at that, but from what I read it has the drawback that when the header of an MP3 file changes, the encrypted outputs of the two files will be completely different, so I'll be transferring 5+ megs for each 1 byte change (the player I use tracks how often a song is played, so that's why the header changes often). I know I could just send the MP3s without encryption, since it's not important to protect them, or I could just disable the tracking feature, but I'd rather not do either if I can avoid it.



Yes with all the options available I can definitely see that happening if one doesn't know what they're doing, which is why I thought I'd ask a few questions on here before starting. I don't suppose there is any way to determine that your crypto solution is weak (other than having someone break it, of course)?

It is a feature of strong cryptography that changes to one bit in the cleartext file totally changes all (most) of the bits in the resulting cipher file.

If you are just moving music files, most folks just setup a SSH tunnel, or use a VPN.

With ubiquitous broadband, moving 3MB of data is not normally considered an issue

Right, I understand that. So I guess basically my question is: Is it less secure to encrypt the cleartext file in chunks, for example instead of encrypting a 5 meg file all at once, encrypting the 5 meg file in 500 smaller 10k chunks?



The music files were just an example of why I'd like to be able to encrypt the files in chunks, but my plan is to store all kinds of data, some potentially more sensitive than others, so I'd like it encrypted on the remote server, not just in transfer to it.



You're right, 3MB isn't so much, but for example I recently added a custom tag to the header of each song to indicate which of the people in my household listens to it (to aid in the creation of playlists, I was tired of turning Random play on and getting Britney Spears or whatever!). So just to use round numbers, say I have 10,000 songs that are 5MB each, thats 50 gigs to transfer. If I encrypt in 10k chunks, that's only 10,000 songs * 10KB each, so only 100 megs to transfer, which is much preferable in my opinion. It's an extreme example since I'm not likely to change every single song like this again, but you never know.

You are touching on one of my hot topics, see the SlimDevices forums for more.

The idea of putting tags inside MP3 files was OK for 1999, but has outlived its useful life. The idea that tags defined the artist, title, etc. implied that they were fixed. But what is happening is that preferences, dynamic associations, volume/replay settings, etc. are being placed as "tags" in the file. They really belong in a separate database.

So you can send the fixed data once, and send just the little bits and pieces that change in a second channel.

While MP3 tags are typically in either the start or end of the file, FLAK and OggVorbis tags can go anywhere. This will break your chunk idea.

Its hard, and usually dangerous, to make security judgements about things such as your 'separate chunk' encryption, without serious study, time and effort. If the keys and IV are independent, I would expect that it does not weaken the cipher. But now instead of one key for the 5MB, you have 30. And key management is the real weakness in most security schemes

Right, I 100% agree on that. When I first started ripping my CD collection I was using a lossless codec called WavPack, and it stored the album information in a separate .CUE file. I quickly ran out of HD space and was forced to convert the entire collection to a lossy codec to save space, which is why I have the MP3s I do now.



Would separate keys really be necessary? If I use GPG to encrypt an entire directory of files, would I have to supply a unique password for each file? To me encrypting each file in a directory separately is comparable to encrypting each 'chunk' of a file separately, so the requirements should be the same.

Its really more a question of what you want. You can tar up the whole directory, gzip it and encipher it all. Or do individual files, or chunks.

Its a bit of how paranoid you want.

Its normally considered stronger to use separate keys. Again, its just a part of security. But there are 'known plaintext' attacks that smell like they could be used if the bad guy knew that you were securing Rick Astley's "Never Gonna Give You Up".

But key management gets hard quickly, so there is no easy answer.