A quick note to your argument about how with regular encryption you know when you have found the right key because regularities will appear: You can easily circumvent this*, by encrypting the data multiple times with different keys and possibly with different algorithms.
That's how you know the security is solid.
Or that there is nothing there that can be broken...
Imagine for example if I write in chinese and you had no reference for chinese. You do not know how to read it and you don't know of any similar languages to aid in your decoding of the chinese.
First something general about using human languages for the purpose of security.
1) If you invent a language, your vocabulary and grammar will basically be the key. Admittedly, it will probably be a pretty big key, but partial knowledge of the key will likely allow partial deciphering of the messages in that language.
2) Inventing languages for the purpose of security is not a well studied field. If you do it ad-hoc, you are unlikely to get a great result.
3) In general, when using human languages, similar messages will look similar. As information rarely exists in a vacuum (the Voynich manuscript pretty much does, yes), this may allow an attacker to make inferences about your messages.
To use your example of Chinese, consider the following scenario. You are using Chinese to communicate with a friend. In this world, Chinese is a language known only to you and your friend. I am a well funded adversary and capable of observing your general daily behaviour in a way that only requires communication meta-data and a guy who watches your house and one who watches the house of your friend.
On day 1:
You send your friend a message: æ'äçç¦ä½æ"ääç"èã
I observe:
- You clean the windows.
- You go buy things from the grocery store.
- You give your friend a phone call.
- You take a walk.
- Your friend goes buy things from the supermarket.
- Your friend receives a phone call from you.
On day 2:
You send your friend a message: æ'äfçäåååZçoeæ'æoeåäã
I observe:
- You take a walk.
- You chat with a neighbour.
- You call your mother on the phone.
- You go visit somebody.
- Your friend receives a phone call from somebody.
- Your friend goes eat at a restaurant.
On day 3:
You send your friend a message: æ'æoeåç¦ä½æ"ääç"èã
I observe:
- You buy things from the grocery store.
- You take a bath.
- You take an evening walk.
- Your friend receives a phone call from the person you visited the day before.
- Your friend goes swimming.
(Please go here for a copy where /. hasn't messed up the Chinese characters.)
By now, I will be able to make some guesses about the meaning of various parts of the Chinese language. If you communicate more, I will be able to refine those guesses. I suggest you mull over it a bit and see if you can come up with a few hypothesis yourself. And yes, it's proper Chinese.
You're ignoring my point which is that the cryptography is theoretically breakable. Where as the things I'm talking about are not.
I am not ignoring that. I merely hold the opinion that the difference in practical security is negligible. To reiterate the pros and cons of using OTPs instead of regular, modern cryptography:
+ Properly used, OTPs can be perfectly secure.
o This means, if you compare the security of an OTP with that of a conservatively designed cipher using 128bit keys, you have reduced the probability that your cryptography will be broken by a bit less than 0.00000000000000000000000000000000000001%.
o Using an OTP does not defend against attacks that circumvent the cryptography instead of attack it, which is basically all of them.
- You need a secure random number generator, which is not backdoored by the NSA and can quickly generate sufficient amounts of random data.
- You need to securely exchange OTPs with everybody you want to communicate with. This is hard, if you want to maintain the perfect security provided by the OTP:
* You cannot send unencrypted it over the internet. If you send it encrypted, it will be no better than using said encryption.
* You cannot entrust it to the postal service, because it may be intercepted.
* If you don't know your communication partner in person, even meeting up may not be secure as you might meet a similar looking adversary with a fake id card.
* Either of you might get "mugged" on the way to the meeting.
* You need to repeat this whenever you run out of OTP.
o Admittedly, exchanging keys for regular encryption faces similar problems, but there are some established techniques for verifying identities in a probabilistic way (e.g. the web of trust with PGP). If you are doing realtime communication with somebody already, it is probably sufficiently safe to quickly tell them a fingerprint or public key.
- You need to securely store your OTPs. For this you need heavy physical security. If you just encrypt them for security, they won't really be more secure than the encryption you used.
- You give up useful techniques such as Diffie-Hellman key exchange for a slight increase in security.
- You cannot transmit basically arbitrarily large amounts of data with a single key exchange.**
And if those people are out of reach or unknown then the message will remain secure... and here is the point "forever".
Do not skip over that last word please like you did last time.
My point is that we don't need it to last forever. It's enough if it lasts until the universe loses itself in entropy. Hell, it's probably enough if it lasts 500 years. Unless I become immortal, I suppose I would be fine with my secrets from now being revealed in 500 years. Might be nice for the historians then. It is with this in mind that I argue about the sufficiency of non-OTP cryptography.
For any (un)reasonable time span, 256bit keys are more than enough. Heck, if we go with 512bit keys, an attacker that turns every fundamental particle in the universe into a computer that works at breaking your key will still take longer to break the key than the universe has existed for so far.
You can use OTPs to achieve "perfect" security. Yes. It will be impossible to tell, which of the 2^(messagelen) messages is the correct one if you do not have access to the OTP. However, the practical application of OTPs is difficult and the gain in security is unlikely to affect anything in the real world.
As to your point about getting things decrypted by torturing the people that encrypted it... well yes, that always works but the point is to make that a literal requirement.
It basically is a requirement already. That's the reason the UK has laws that allow them to throw you into prison for not giving them your encryption keys. That's the reason the US government went and demanded that Lavabit hand over their encryption keys so they can get a Snowden's emails.
* In that case, your sequence and number of encryptions will become part of the key. At some point, structure will always appear. At the very least in the brain of the receiver.
** There are usually limits for the number of blocks or bytes a single key can be used with a cipher, but you can always negotiate a new one securely through your existing, secure channel.