I said that transaction malleability was exploited by hackers; it was.
I thought I was pretty clear when I said it wasn't.
The fact that the Bitcoin software no longer has this bug does not change the fact that it once did have this bug, and that this bug has been exploited.
Again no, as far as I know it was never exploited. But I can see you prefer to believe an internet echo chamber confirming your world views over me over me, who is saying you are just plain wrong. More on the dangers of doing that later. For now I assume you really are willing to discard your tin foil hat if you understood what happened. Unfortunately that is going to require going into some detail.
The transaction malleability problem we are discussing here is actually about how the transaction signature is represented. As I said, there are other causes of malleability, some of which haven't been fixed. The transaction signature is particularly important because the bitcoin protocol uses it to identify the transaction. When used in that manner the same piece of information is called a transaction id. Because it does uniquely identify a transaction once it is accepted into the block chain bitcoin exchanges sometimes use the transaction id to match for transactions they have generated.
The different ways of representing a transaction id doesn't effect the core operators of bitcoin, so it was never regarded as serious. The reason it didn't effect bitcoin is two otherwise identical transactions with different transaction id's look like a double spend. Naturally the bitcoin protocol rejects all but the first attempt, so it doesn't matter how many different transaction id's you throw at it. Bitcoin is based on the premise that there is one and only one true and correct transaction history – and that is the block chain. You can throw any rubbish you like at it (and there have been many attempts at DDOS it by doing just that), but as far as bitcoin is concerned the only transactions that exist are the ones that get appended to the block chain. So if there are transactions with multiple id's, it is the id that gets into the block chain that is the official one. The rest never happened.
So far I expect this matches your understanding of the root cause of the problem. It is about now we depart from that.
The transaction signature / id is a ECDSA signature. Here is a real one:
770a723381d3edbcbfd06cecdd7b9f8569e9691d3a06a8a9c8972dd6fcbc8493 . It looks remarkably like a fixed length SHA checksum doesn't it? It's not. An ECDSA signature is two large numbers, which in bitcoin is encoded in DER format.
DER format is used because, quoting from that Wikipedia link: “DER is a subset of BER providing for exactly one way to encode an ASN.1 value the shortest possible length encoding must be used”. Which sort of begs the question “how it be malleable”? It isn't. But, the software the reference bitcoin software uses to produce and decode these signatures is openssl, and like all good internet software openssl follows Postel's Law: “"Be liberal in what you accept, and conservative in what you send”. So OpenSSL always generates valid bitcoin signatures, but it accepts invalid ones, and in particular numbers with leading zeros. Whether you call this a bug or feature is more a matter of taste than anything else.
This bug / feature was noticed by the bitcoin developers some 3 years ago. It wasn't viewed as serious. As I said, it doesn't effect the bitcoin protocol at all, and even if an exchange make themselves susceptible to the malleable transaction id, it's damned difficult to exploit. The only way to exploit it is to pick up a transaction from the “to be processed pool”, modify it, then re-issue it. Then you have to get very lucky, because you are relying on some miners seeing your version before they see the original – and then those miners winning the block. It's made doubly difficult because exchanges like mtgox are very close to the miners in internet terms. Still, they decided it should be fixed, and did so about 1 year ago. Thereafter miners rejected any transactions signatures that weren't properly formatted DER. Bitcoin is effectively an open source project, so this like any bug was discussed in open forums, a patch issued on a public platform, and then noted in the release notes. They developers claim they sent mtgox an email about it.
Now read this very carefully: To my knowledge this malleability issue was never exploited before it was fixed.. And of course after it was fixed it could not be exploited.
So what did happen? The next key fact is mtgox doesn't use the reference bitcoin software. They wrote their own. They may have had a good reason for doing this, but nonetheless it gave them the opportunity to introduce brand new bugs that weren't in the reference implementation. And they did introduce at least one, because it appears they treated the ECDSA signature like the SHA checksum it resembles, and treated it as fixed length. That's a guess. What isn't a guess is mtgox was generating transactions with malformed signatures – they had leading zeros. This is the bug number 1 I referred to earlier. Assuming the guess at the reason is correct there was a 1 chance in 256 of the leading byte in the signature being 0, and so 1 in every 256 bitcoin transactions created by mtgox has an invalid signature. It might be higher – remember there are two numbers in the signature.
Thus one year ago the miners started dropping mtgox transactions. Maybe around 1 in 256 of them. The real effect show up like this. When you trade bitcoins on mtgox there will be a stage when the traded bitcoins sit in their accounts. This is no different to any other broker – be it bitcoins or stamps. To get them back into your control you have to ask mtgox to transfer them to you, which is quick and easy because they provided a web page to do just that. But after the malleability fix, 1 in 256 of those transfers would fail. If this happened to you, when you noticed the transaction didn't appear on the block chain you had to contact mtgox. When you did they generated a new transfer. Unless you were 65536 times unlucky, that worked, and everyone was happy.
Contacting a understaffed corporation in a foreign country isn't easy at the best of times, and some bright spark hit upon an idea. He could see the malformed transaction in the mining pool. All he had to do was delete the offending leading 0 and send it to the mining pool, and it would be accepted. There is no race here – his version would always win, and he would get paid without the hassle of contacting mtgox. But then whoever did this must have noticed something very odd. Even though he had been paid, at mtgox it looked for all the world like it hadn't happened. Mtgox has said that was because there were relying on the transaction id (which hasn't been accepted by the block chain, thus wasn't trustworthy) to identify the transaction. Bug 2. At this point an honest person would have contacted mtgox and told them what was going on. Evidently that didn't happen. Instead they contacted mtgox, and asked them to fix the “problem”. Which they did, without doing a full check on the audit trail first. Bug number 3. And we had our first double spend.
Now at this point it becomes important to appreciate the scale of what is happening. Mtgox does millions upon millions of transfers over the course of a year. Hundred's of thousands of transactions must have been effected over the year once invalid transactions were dropped. Mtgox said they knew something usual was going on, but geeze how does a company worth $30M manually authorise $350M worth of double spends? We don't know, but however they pulled it off is bug number 4. Without bug numbers 3 and 4 this would be minor news. Had mtgox done what any other sane organisation does upon noticing anomalies in their accounts and audited them, the losses would have been minor. Whether you still insist this is a bitcoin protocol bug or not, the fact is the effect should have been no more than an irritation.
That it's not only a minor irritation, but mtgox doesn't know where the money has gone speaks volumes. In order to buy and sell bitcoins you have to transfer money in or out of mtgox. Either way you run into various countries Anti Money Laundering laws. The end result is you have to send mtgox id. Not just any id – passports, drivers licences and credit cards. Ergo they should be able to track down the double spends, and get the money back right? Well no, not if upon noticing the numerous bugs, someone did bitcoin transfers without triggering the laws. In order words, if they didn't transfer money into or out of mtgox. But that would mean they transferred bitcoins into mtgox, then transferred them out again in small lumps, hoping to hit the 1 in 256 bug. You only need give mtgox a fake email address to do that. But no one legitimately does that on a large scale because all it does is generate fees for the miners. Yet mtgox still didn't notice.
Eventually they did become alarmed, and they finally shut down and did a full reconciliation of their accounts versus the one true and correct record – the block chain. They never re-opened.
So back to your tin foil hat. You understand that mtgox is a commercial organisation, and when they issued their statements they were hoping to re-open, yes? All statements they issued weren't full and correct explanations of what happened. So far, they haven't bothered to issue anything remotely resembling that, and so we can't be sure what really happened. Instead they published PR fluff, designed to prepare their customers for their triumphant return. Granted you would never know that given the reception they got from the internet echo chamber who seems to falling over themselves to accept mtgox's version of events, but it should be obvious. No thinking person would base their opinion on bitcoin's viability on mtgox's press releases to date, which blame anything but mtgox. But apparently you do.
But maybe you thought that like Enron, mtgox was a upstanding and competent company, whose PR releases were done in good faith. I could sort of swallow that. But then we come to Tech Crunch's re-posting of Silk Road's 2 blog post. Silk Road 2 is a anonymous criminal organisation. They say their customers, who have no way of tracking down Silk Road 2, have lost all their bitcoins due to the malleability problem. And you accept this blog post at face value? Even more unbelievable, you then go onto to claim because mtgox could not add up two numbers to save themselves, and a pack of criminals say they “lost” their customers bitcoins the underlying bitcoin ecosystem is not ready for prime time?
Look, I have some news – what makes the news is the surprising stuff. Like Enron losing billions of investors money, and Lehman Brothers losing even more. It's surprising precisely because it's unusual. Granted bitcoin is unusual. Granted, there has been the occasional failure, they are listed here. I count around 30 of them. In that same time period, there have been over 30 million transactions, all recorded without error in the block chain. It's a pity my bank wasn't so accurate.