Try and look at this from the other point of view, leaving emotions aside for a minute.
You have a man accused of committing sexual crimes in Sweden. Sweden wants that one man to return to the country for questioning, in the country where he's accused.
That man says screw you, instead you have to figure out the logistics and spend a lot of taxpayer money to send over a whole team of interviewers to an entirely different country, so they can then enter a third country's embassy to suit his desires. This will invoke a whole host of legal, territorial, and logistical questions, based on who has sovereignty at what point, as well as cost a lot of money.
So... to avoid a situation that "will invoke a whole host of legal, territorial, and logistical questions, based on who has sovereignty at what point, as well as cost a lot of money", we now have him holed up in an embassy of a country that's granting him asylum, with the host country threatening to revoke diplomatic privileges in what may or may not be a treaty violation with far reaching implications on the sovereignty of embassies worldwide. While their lawyers figure out whether that's legal, the police are surrounding the building to make sure he doesn't get out and to keep the protesters at bay. Until they figure out an answer, diplomats on all sides are rattling their sabers vying for the upper hand. Meanwhile there's talk of whether it's legal to smuggle him out in a diplomatic bag as a way of crossing the British sidewalk that stands between the foreign soil of the embassy and its sovereign embassy car. All this coming after months of legal proceedings and requests between countries.
If Sweden's goal in not questioning him abroad was to avoid complication and expense, it seems like they failed miserably.
If you let go of your emotions for a moment, there are plenty of valid reasons for Sweden to want him to return to Sweden to be interviewed about a crime he's alleged to have committed in Sweden against Swedish women.
I'm all for having him interviewed and charged if there were crimes committed, but surely provoking an international incident isn't the most effective way to do that.