"The advantage is that it will not grow outside the very limited environment that provides the necessary arsenic. So if you accidentally spill the toxic tank the bug is not going to propagate and contaminate the rest of the world."

Unfortunately this is not necessarily true, as the organism also could incorporate phosphorus if it was given some in the media (and not given arsenic). However, it is certainly most likely to survive in the environment it has adapted to (i.e. a high-arsenic environment).

However, I still agree that this could be useful for mediation, since arsenic-laiden DNA, proteins, lipids, etc. may be less toxic than arsenic itself.

Not necessarily, the original source (doi:10.1038/nature09250) mentions two different possible sources of the error, either the Rydberg constant is off or the theory gives an incorrect prediction. Though they mention both possibilities in the abstract, the authors only mention the former in their conclusions (maybe an oversight, maybe they think that is the actual cause).

Disclaimer: I am not a physicist