Software Is Great; Software Has Bugs

By James Kwak

I’m not qualified to comment on the internals of Bitcoin; I’m neither a programmer (OK, Alex, not much of a programmer) nor a computer scientist. But I do know that Bitcoin exists because of software that people wrote, and every means by which we use Bitcoin also operates because of software that people wrote. The problem here is the “people” part—people make mistakes under the best of circumstances, and especially when they have an economic incentive to rush out products. That’s why, while we love what software can do for us, we also like having a safety net—like, say, the human pilots who can take over a plane if its computers crash. This is the subject of my latest column over at The Atlantic. Enjoy.

10 responses to “Software Is Great; Software Has Bugs

  1. Had I only attended MIT instead….. no make that Princeton, no wait…. Harvard.

    “February 26th 2014: Dear MtGox Customers,
    As there is a lot of speculation regarding MtGox and its future, I would like to use this opportunity to reassure everyone that I am still in Japan, and working very hard with the support of different parties to find a solution to our recent issues.
    Furthermore I would like to kindly ask that people refrain from asking questions to our staff: they have been instructed not to give any response or information. Please visit this page for further announcements and updates.
    Sincerely,
    Mark Karpeles
    ________________________________________
    February 25, 2014: Dear MtGox Customers,
    In light of recent news reports and the potential repercussions on MtGox’s operations and the market, a decision was taken to close all transactions for the time being in order to protect the site and our users. We will be closely monitoring the situation and will react accordingly.”

  2. PGP Signature

    To Whom It May Concern:

    “Over the past several weeks, I too have been following the unfolding story of the latest Gox problems, from the outside. I was one of the earliest critics of Mt.Gox, calling on people to abandon the exchange in April of 2013 during one of their previous (of several) self-induced meltdowns…. In April of 2013 I said:

    “Magic The Gathering Online Exchange is a systemic risk to bitcoin, a death trap for traders and a business run by the clueless.” https://bitcointalk.org/index.php?topic=172991.msg1801451#msg1801451
    That same week, I participated in a podcast for LetsTalkBitcoin where we advised customers of Gox to leave and never trust that exchange again with their funds.

    April 2013 was also the last time I did business with Mt.Gox, moving all my funds to blockchain.info which allowed me to maintain control of my keys without the possibility of theft from or by the site operator.
    Since then I have made repeated public statements advising people to avoid Gox and commenting on their latest mess (USD withdrawals, lag, DDoS etc).

    Approximately two weeks ago, Gox claimed that Transaction Malleability “a bug in bitcoin”, which was known since 2011, was forcing them to suspend withdrawals. I publicly excoriated Gox’s incompetent and clownish management and disputed their claim that their problems were due to a “bug in bitcoin”.

    A few days later, a bot started using Tx-Mal as a broad DoS attack against all exchanges, aiming to uncover if other exchanges were vulnerable. In response, some exchanges temporarily suspended withdrawals to investigate their implementations and confirm they were robust. I was part of the team helping to coordinate between the other exchanges to ensure that they could quickly resume operations which they did no more than 48 hours later. Some exchanges were in fact completely unaffected, revealing as false Gox’s claims that this was a bug in bitcoin.

    Blockchain.info staff jumped in to help from the very first moment, even though blockchain.info’s wallet and block explorer were not affected by Tx-Mal. While I offered coordination and assistance with the media response, blockchain.info staff developed a new API on blockchain.info to present a partial transaction ID (NTXID) solution developed by core developer “”sipa”, based on the immutable inputs of at transaction. We offered that on blockchain.info as an independent location to look up and verify transactions with by ntxid. During this time, Mark Karpeles was active on the forums and developer boards and appeared to be implementing fixes to Gox software to address Tx-Mal. This solution helped many exchanges accelerate their technical fixes to their infrastructure and between this and other industry efforts, all the other exchanges resumed normal operations in less than 48hrs. I am very proud of the role blockchain.info team played in providing technical asisstance to many across the bitcoin industry at a time of crisis.

    As I watched Karpeles post updates on public channels (like #bitcoin-dev on IRC) about the NTXID solution, I became more optimistic that a technical solution to Gox’s code problems was imminent. As we started seeing Gox transactions posted on the public blockchain ledger, as reported on reddit and other sites, it appeared to me as if Gox might recover from their latest mess. During this entire time, I had no information about Gox’s financial state other than what Mark Karpeles posted publicly. I don’t know Mark Karpeles personally and I don’t think I have ever met him or communicated with him directly.
    At this point I felt bittersweet: I wanted Gox to recover and allow customers to retrieve their funds as quickly as possible. Yet, I knew and had publicly expressed, that the underlying problem would not be fixed: Proven management incompetence, expressed by a variety of massive failures, and their failure to hire a competent security and management team. My primary concern was for the funds of customers trapped on Gox and therefore I kept hoping for the best: a way for people to get out of this horribly mismanaged business.
    In interviews, throughout last week, I stated that while I had serious misgivings about the competence of Mt.Gox executives and especially Karpeles, I had not seen any indication of bad faith or fraud in the past two years. Furthermore, Gox had stated publicly that they kept the majority of funds in “cold storage”, so I believed that even if the exchange had been defrauded because of their poor implementation of withdrawals-based-on-transaction-ID, the damage would be limited to the “hot wallet”.

    Yesterday afternoon at approximately 3pm PST, Monday February 24th, I heard unconfirmed reports that Gox was in crisis mode and their funds were mostly, if not entirely, gone. This was the first hint I had of any solvency issues. Less than 30 minutes later, I found a public blog post with a lot more detail here:

    http://two-bit-idiot.tumblr.com/post/77745633839/bitcoins-apocalyptic-moment-mt-gox-may-have-lost

    Based on this blog, it appears their “cold storage” was not in fact “cold” – which is either a stunning misrepresentation of their security or an outright lie. “Cold storage” does not “leak”. The idea that the funds were stolen, unnoticed, from cold storage, due to Transaction Malleability, strains the credulity of even the most gullible observers.
    As I write this, Tuesday Feb 25th at 9:00am PST, I still have no official confirmation and I know nothing beyond what is in that article, but I fear the worst. Everything I see makes me believe that Gox will never recover and that the funds are most likely lost.

    I am devastated by the impact this will have on customers of Gox and I am angry at the irresponsible behavior of Mt. Gox and especially Mark Karpeles that will damage the lives of many people.

    Even though I read the blog posting about Gox’s problems sooner than most, I did not sell any bitcoin and will not sell any in the coming weeks. I continue to be committed to bitcoin’s future and I am confident that the bitcoin industry and community will add Gox, along-side SilkRoad, as a lesson and move towards the future, stronger.
    Last night, I took action to help rebuild some of the trust that this community desperately needs after reports of a massive breach of trust. I spent the evening and into the late night at the offices of Coinbase, reviewing their funds security with my own eyes. They invited me and I volunteered, even though they are a competitor and I have no professional relationship with the company, because they felt that an independent review would quickly put customers at ease. My statement on Coinbase:

    http://antonopoulos.com/2014/02/25/coinbase-review/

    We will face a storm of negative media, conflating Gox with bitcoin and hurting the bitcoin community in the short term. First and foremost, we must all be thinking of the people affected by the loss of funds in Gox and I extend my heartfelt sympathy to them all. We must honestly and directly address the concerns of all users and interested parties, emphasizing the difference between a decentralized trusted system (bitcoin) and the failures of a single company that did not use the trust mechanisms offered by bitcoin’s blockchain technology.

    Gox represents a the failure of a poorly managed exchange that had full centralized control of customer funds, in custodial accounts, off the bitcoin blockchain. By keeping the funds off the blockchain, Gox removed the protections of transparency and end-user control and replicated the model of a centralized bank without any of the controls and oversight such institutions require.

    There is a better way: bitcoin companies can maintain customer funds on the bitcoin blockchain with full transparency and accountability. We can offer client-side key-management solutions that put full control in the hands of the customers and remove them from the control of custodians, be they exchanges, markets or web-wallets. If a bitcoin company keeps custodial access to customer funds (holds their keys), then they can and must offer cryptographic-proof of solvency through the blockchain.

    I will continue to work tirelessly with the rest of the industry to build trustworthy, transparent, decentralized and cryptographically provable solutions that protect customer funds and allow them to retain full control.

    We must all draw hard lessons from this experience and use them to make our community stronger. A few individuals cannot nullify the positive and inspiring work of thousands.

    I continue to believe in this community, full of hope, innovation and creativity and acting in good faith to promote this incredibly important technology that can improve the lives of billions.”
    — Andreas M. Antonopoulos

  3. Bitcoin Gox - Not Too Big, To Fail

    http://finance.fortune.cnn.com/2014/02/27/bitcoin-janet-yellen/ :
    “The Fed chair says the regulator will take a hands-off approach towards the cryptocurrency.”….. That’s why bitcoin boosters should have let out a sigh of relief when Fed Chair Janet Yellen said in no uncertain terms that her institution will not be regulating the currency anytime soon. “It’s important to understand that this is a payment innovation that’s happening outside the banking industry,” Yellen said at a Senate Banking Committee hearing Thursday morning. “The Federal Reserve simply does not have the authority to regulate bitcoin in any way……The answer came in response to West Virginia Senator Joe Manchin’s questions about bitcoin, which belied intense distrust of the currency. Manchin called bitcoin an “unstable currency” that he believes is being used mostly for illegal activities. Manchin was seemingly disappointed with Yellen’s statement that the Fed had no authority to regulate bitcoin, saying he believed there would be an intersection between bitcoin and Fed-regulated banks in the near future…………..Yellen didn’t budge, telling Manchin that if he wanted further oversight of the currency, Congress could take action to require it. Otherwise, she said, any regulation would be under the purview of the Justice Department and the Treasury Department.”
    ———-

    “The venture capitalist Marc Andreessen, whose firm has invested millions of dollars in Bitcoin-related start-ups, drew a comparison to MF Global, the brokerage firm that filed for bankruptcy in 2011. “This is like MF Global, not some huge breakdown of the underlying technology or other exchanges,” Mr. Andreessen said in comments aired by CNBC on Tuesday morning. “Bitcoin protocol is unchanged and other Bitcoin exchanges and companies are doing fine.”…………. But his comparison to MF Global struck some observers as less than comforting. The saga surrounding MF Global, which collapsed when investors and ratings agencies became unnerved by its large position in European sovereign debt, raised broader questions about the safety of customer funds. In MF Global’s case, $1.6 billion of customer money went missing, and clients questioned the oversight of the firm.”
    ————

    “Several Bitcoin powers-that-be distanced themselves from Mt. Gox, saying they always thought it was a risky place to hold Bitcoins and promising to “collaborate with other industry leaders to establish better controls to ensure that events like this are not repeated.” Gigaom’s Jeff John Roberts suggested wealthy Bitcoin owners/proselytizers like the Winklevoss twins might be intervening in the market to stop Bitcoin’s slide, acting like the Federal Reserve or something…… But the reputational damage is done, coming hard on the heels of wild swings in the currency’s price and other recent public Bitcoin controversies. Those include the shutdown of the Silk Road online exchange and the arrest of Charlie Shrem, vice chairman of the Bitcoin Foundation and a recipient of millions of dollars in funding from the Winklevii.”
    —————–

    “My thoughts and my heart also go out to all of those others in the community who lost a lot more than me today,” said Rick Falkvinge, the 42-year-old founder of the Pirate political party based in Sweden. Despite losing what he said was $80,000 in Mt. Gox, Mr. Falkvinge said that he remained “absolutely bullish on Bitcoin.”

  4. No one defends themselves so loudly, as the GUILTY!

  5. Thank you for articulating my skepticism, James. While bitcoin can (in theory) provide certain benefits in the ether world, it is a product of software creation. As a former software developer I refuse to believe that any software created currency can be foolproof. To embrace such a currency, you have to believe and trust that it is indeed ingenious and cannot be manipulated, or in other words, totally unlike any software that has ever been created. If you believed in the infallibility of Reagan’s crazy Star Wars defense which was centered around millions of lines of untested code, then by all means you should buy bitcoins. For the rest of us who laugh at the utopian concept that software can infallible, stay away from bitcoin.

    And yes, audit trails are important in the world of business. While some view the lack of an audit trail as Nirvana (Hey look ma, no regulation!) they provide a high degree of trust within the business and investment world. Without audit trails, without a banking system, without vigilant regulators, without transparency and without third party verification, what could go wrong? In the words of that great MAD philosopher Alfred E. Newman, “What, me worry?”

  6. @ If you believed in the infallibility of Reagan’s crazy Star Wars defense which was centered around millions of lines of untested code,

    Now you see, no one ever won an argument with a false premise, either by illusion, poor translation, or interpretation.
    But a lot consequences have yet to be paid by those who act on their false premises, as we are seeing around the globe. Once the financial manipulations come to a swift conclusion. Those of guilty conscious will have seen there bad deeds come to light, and then it’s just a matter of time before they realize their true demise and the fallacy of their ways.

  7. BTC 2.0 - Intro to Gox'ing

  8. @Edwardus, “…And yes, audit trails are important in the world of business. While some view the lack of an audit trail as Nirvana (Hey look ma, no regulation!) they provide a high degree of trust within the business and investment world. Without audit trails, without a banking system, without vigilant regulators, without transparency and without third party verification, what could go wrong?….”

    http://rt.com/news/us-military-ukraine-radicals-440/

    Wonder if the hooliganists got paid with BitCoins? They got paid in gold to take out the Czar…

    Indeed, what could go wrong with global bankster mercenary armies being richer than “Middle Class” in “democracies” and haundling with their own imaginary software currency…? Smells like Blackwater, don’t it?

    NEW WORLD ORDER = Back to picking through the garbage to get the financial info you need to steal it ALL….

  9. Interesting Article.