I'm trying to stay concise, but I'll spell it out for you.
NYC has a TON of underground infrastructure that goes down as deep as 600-800 ft (depending on where you are and how you measure). For a train traveling 700MPH, you are going to need to pick a depth and more or less stick to it over a distance as small as Manhattan (maybe 3 miles wide at most?). The higher you go, the more trouble you will have finding a clear straight run. You can't get too close to the infrastructure above, or the rock will not support the weight and you'll have to make expensive reinforcements. For instance, when putting the LIRR (Long Island Railroad) connection in at Grand Central ("East Side Access" if you want to Google it), they opted to dig 140 feet down so that they would not have to reinforce the foundations of the station above. The F train is about 100 feet down so it can pass under the other subways. I can tell you from experience that the escalator ride is no fun down to the F train, and I would avoid it if I could. Sometimes the escalator would be out of service and you got to use them as stairs. Fun. There is an infamous subway entrance at Washington Heights 180ft deep that is accessible only by elevator (well, also stairs). The elevators are troublesome and one of them has - I shit you not - a human operator who just sits there and presses the automatic buttons. The other (identical) elevators are fully automatic, they just still hire this one guy. Sorry, tangent.
Finally, there is the water infrastructure. It is the deepest as far as I know, down something like 600ft. Maybe there is some way to "thread the needle" and get the hyperloop tunnel between the water infrastructure and the subway infrastructure. That'd be great, but it's still really far down. If he can dig all the way from Washington to NYC it will seem like a relatively minor thing, but lets not pretend it's trivial. It took them 4 decades to dig the water tunnel.