The Opportunity of SOAP

A reader writes: "There's an interesting piece on DaveNet concerning SOAP and some of Dave's predictions for it. He also talks about SOAP being in a flexible phrase, and the potentials for it, if it is used correctly. The point about the multiplicity of Dot.Nets is a good mental exercise too. Will development happen that way?"

On hold with Microsoft for 6 months

Dave sez: "Microsoft says they love developers"

I have an example:

I remember when NT4 first came out, trying to use their documented "Streams" support.

Nothing the MS NT4 documentation said was accurate. Files that were supposed to exist didn't, functionality claimed wasn't there.

So, I paid for Microsoft support on the issue.

The MS flunky couldn't answer my questions... said only developers could, and they were busy trying to release NT5 (which sliped schedule for another 3 years, finally released as W2K).

He said he would keep trying to get an answer, but after six months, I asked for my money back, and haven't used a Microsoft product since.

Re:Yet he forgot...

Biztalk is an application. It defines a number of XML schemas which map onto the same concepts as EDI, like purchase orders and manufacturing requests. It includes mechanisms for description and discovery (UDDI) of such schemas, as well as various transports to transmit them. Biztalk is there to replace EDI (and if you've used EDI, you'll raise quite a cheer at the prospect of its demise)

SOAP is a wire protocol. It applies the XML schema types to arbitrary data and provides some basic messaging on top of it, such as envelopes, metadata, and named endpoints. It's hardly even a protocol, much less a full-blown application.

But don't let me stand in the way of gratuitous microsoft bashing or anything.
--

Re:SOAP's real technical benefits

Traditional RPC protocols require one to know that a method is of the form:

foobar(int a, int b, char[] c)

With XML, each parameter is defined by a "tag". This allows position independence -- one only has to state the name of the parameter that the piece of data is for. (Insert analogy to Smalltalk method parameters here.)

SOAP is kind of ambiguous on the issue of positional or named parameters.

If you want to bind existing code to RPC interfaces, you must support positional parameters and you must ignore parameter names because most existing languages use positional parameters. An RPC binding for Java or C that uses parameter names is simply incorrect because it doesn't reflect language semantics.

"void f(int x)" and "void f(int y)" are the same interfaces in C/C++/Java. If you haphazardly used named parameters, you'd be missing a parameter. Or, if you declare a function as "int f(int x,int y)" in one place and "int f(int y,int x)" in another, you'd be calling the function with the wrong arguments if you used named parameters. The only way you can introduce named parameters for C/C++/Java interfaces is if you add new mechanisms for declaring how named parameters map onto the positional parameters of those languages. In different words, if you adopt a named parameter scheme, you cannot build an automatic mapping from C/C++/Java interfaces onto SOAP interfaces (unless you call your "named parameters" something like "arg0", "arg1", ...).

Note that Smalltalk parameters are, in fact, positional: "x put: y at: z" is not the same as "x at: z put: y"; in the first case, the method is "put:at:", in the second case, the method is "at:put:".

Traditional RPC protocols also require you to statically bind to an interface (.h file, .java file, IDL file). This means that when an interface changes, you need to recompile. This sucks.

Yup, dynamic interfaces are nice. Of course, lots of systems had them before CORBA and DCOM side-tracked the industry.

XML RPC frees you from having to use a standard object model, or a standard language. You can implement an XML RPC endpoint in ANYTHING you want -- there's no model to bind to. Every call is dynamic.

First of all, XML RPC is not the same as SOAP. In any case, not specifying the object model is not the same as not having one. In fact, SOAP has an object model, it just doesn't specify it. And SOAP's object model maps quite poorly on the object models of existing languages. That isn't freedom, it is merely forcing you to put in a lot of effort in exposing your interfaces in ways SOAP can actually handle them.

[SOAP] defines a standard type encoding (for arrays, structs, etc)

It doesn't really. The current SOAP spec doesn't tell you unambiguously how to represent common datatypes like arrays, strings, or structures. It gives you types that kind of are roughly analogous, but that's all. If these types interoperate between different implementations, it's because of conventions that are outside the SOAP spec, not because SOAP defines a standard.

SOAP is actually pretty typical of vendor standards: it specifies enough to give people a warm and fuzzy feeling, but it isn't sufficient for people to write truly interoperable implementations. That's great for the people who set the standard, it's useless for the industry.

HTTP provides the final value-add in that it's ubquitious, firewall-friendly, and clear-text.

It's too bad that SOAP doesn't commit to HTTP as a transport protocol. Because it doesn't, you can't really rely on HTTP mechanisms when implementing SOAP or defining SOAP-based functionality.

Why SOPA is so important

Put simply, using SOAP, two or more 'alien-things' can communicate to each other as long as they agree on the SOAP protocol - which is the ONLY requirement when doing client/server using SOAP.

Sure CORBA, DCOM, COM+, Java, etc. allow you to enable two different components to talk to each other, but those technologies do it in such a way that you must have a 'piece' of the server (called the client) to be delivered and used by the client developer. Thus, to talk with a 'server' you must meet the needs of the 'server' when using CORBA, DCOM, COM+, Java, etc.

With SOAP, this is all eliminated. As long as the server publishes its API via the SOAP protocol, I can write my client to talk with the server using what ever I want. This frees me from having to 'embed' in my client a piece of the server -- thus there is no longer any 'hard-coupling' between two 'things'.

In short, using SOAP, we now enable a true 'smart' data-exchange-protocol between two systems such that development is now at the level of "data-exchange" rather than API, SDK, language, etc.


---------------
Sig
abbr.

Open Source has also discovered SOAP

Of course Dave points out that open source has noticed XML RPC (his baby), but the open source community has also discovered SOAP: http://xml.apache.org/soap/ [apache.org].

SOAP's real technical benefits

Taking away the hype, here's a simple look at: "Why SOAP for RPC?"

1. The real value comes from XML.

   XML as a data representation has 2 major differences between it and the other more popular RPC protocols:

   1. It ships semantics with the data (i.e. it's a semantic data stream)
   2. It does not require static interface for 2 endpoints to communicate.
   Let me elaborate:
   • Traditional RPC protocols require one to know that a method is of the form:

     foobar(int a, int b, char[] c)

     I.e., you need to know the positions of the parameters, and their types.

     With XML, each parameter is defined by a "tag". This allows position independence -- one only has to state the name of the parameter that the piece of data is for. (Insert analogy to Smalltalk method parameters here.)

   • Traditional RPC protocols also require you to statically bind to an interface (.h file, .java file, IDL file). This means that when an interface changes, you need to recompile. This sucks.

     With XML, the interface for an RPC is defined by an XML Schema Definition, which is just a type/structural description of an XML document. No binding. No recompiling. No registry hell due to immutable COM interfaces.

   • Traditional RPC's were "static first, dynamic second". One could call COM components dynamically through Automation -- but this was an afterthought, and was crufty. Similarily, CORBA had the DII -- but it too was way of "tacking on" dynamic method invocations on top of what was a static model.

     XML RPC frees you from having to use a standard object model, or a standard language. You can implement an XML RPC endpoint in ANYTHING you want -- there's no model to bind to. Every call is dynamic.

2. SOAP's actual value (besides XML) is limited:
   • It's an envelope for an XML document (which can be used as an RPC call) that says "you should know these schemas in order to understand this message".
   • It defines a standard type encoding (for arrays, structs, etc)
   • Can target to specific proxies (actors) application extensions (XML namespaces & schemas)
   • SOAP runtimes will ease development for particular languages. Runtimes already exist for Java, .NET, and other languages.
3. HTTP provides the final value-add in that it's ubquitious, firewall-friendly, and clear-text.
4. Different quality of service needs do exist, so SOAP isn't dependent on HTTP.

Re:Um...

.NET is revolutionary for VisualBasic programmers, because it essentially puts them in the same territory as Java programmers. The cost of this, however, is that a huge chunk of their VB6 stuff breaks - the cost of turning VB into a 'safe' language.

On the other hand, if someone has already invested in Java, .NET really doesn't give you anything except a 1.0 RSN platform. The features it does have (including SOAP and 'web objects') could be added to Java (and probably will be) by the time MS ships.

My feeling is that C# is really there for propaganda value, both so that .NET doesn't get pigeonholed as a "VisualBasic" technology, and so that MS can tout it's language-independance. Plus it gives them the JUMP Java migration program.

VB has lots of users behind it, and MS is planning to ride .NET in on VB programmers. Too bad Sun didn't come up with the idea of a ObjectBASIC for the Java Platform a couple years ago.
--

Hall o' mirrors

As an old Mac guy, this column gives me the willies.

Let me explain. I have issues with Dave Winer entirely apart from his writing style (best described as stoned out of his keyboard); back in the day he was a Mac developer of considerable influence who became the press's go-to-guy whenever they needed a quote about the imminent death of Apple. The man (in those days at least) felt he had an ax to grind that nobody bothered to bring up; he had major influence in the creation of Apple's scripting architecture and then threw a hissy fit because they didn't do it his way (using Frontier, of course); at least that's how I've heard the story told. Dave spent the next few years whining to anyone who would listen that Apple would go down the tubes for not doing this or that thing that he thought should be done. Saying how nice Microsoft was to him (I remember him saying something in a letter to MacTech about them sending him "bouquets") was part of his spiel. Yessir, that Dave Winer was a popular one in the Mac community.

Now here he is saying how much he hates Microsoft. This probably shouldn't shock me as much as it does, though; Dave has always been one to drift with the prevailing winds. I don't think he's one to be taken terribly seriously (admittedly he makes a number of good points in the article; it's just that coming from him they ring a bit hollow).

/Brian

Re:SOAP's real technical benefits

3. HTTP provides the final value-add in that it's ubquitious, firewall-friendly, and clear-text. (emphasis added)

Simple firewalls work because different protocols use different port numbers. What we now see starting to happen on the internet is that everybody is starting to tunnel their funky new protocol over port 80 because firewalls usually leave that open.

Now, I don't call that firewall-friendly, but firewall-unfriendly. The people who are trying to maintain a security policy soon will not be able to filter port numbers any more.

Re:Um...

SOAP is a lightweight protocol for exchange of information in a decentralized, distributed environment. It is an XML based protocol that consists of three parts: an envelope that defines a framework for describing what is in a message and how to process it, a set of encoding rules for expressing instances of application-defined datatypes, and a convention for representing remote procedure calls and responses. SOAP can potentially be used in combination with a variety of other protocols; however, the only bindings defined in this document describe how to use SOAP in combination with HTTP and HTTP Extension Framework

from http://www.w3.org/TR/SOAP/

Re:Why SOAP is so important

Put simply, using SOAP, two or more 'alien-things' can communicate to each other as long as they agree on the SOAP protocol - which is the ONLY requirement when doing client/server using SOAP.

Sure CORBA, DCOM, COM+, Java, etc. allow you to enable two different components to talk to each other,

SOAP isn't an RPC/DO mechanism comparable to CORBA, DCOM, or RMI because SOAP doesn't specify how to encode many common datatypes found in programming languages. It doesn't even tell you how to encode arbitrary strings [w3.org].

So, it's hard to say what SOAP is. It's not a mechanism for exchanging arbitrary XML documents (and you can do that much more easily via HTTP anyway). And it's not a mechanism for implementing even the simplest forms of RPC involving real programming languages. It's something that looks like an RPC mechanism but doesn't really deliver.

What it really is is whatever Microsoft does and because the spec is so incomplete, you can't reliably interoperate with it. What you can do is reverse engineer a little more easily. Let's be thankful for little favors, but let's not grow overly enthusiastic.

With SOAP, this is all eliminated. As long as the server publishes its API via the SOAP protocol, I can write my client to talk with the server using what ever I want. This frees me from having to 'embed' in my client a piece of the server -- thus there is no longer any 'hard-coupling' between two 'things'.

This stuff isn't new at all: people had dynamically typed remote interfaces and service descriptions for years. CORBA/DCOM/RMI actually represented a step backwards when they came out.

Dynamically typed remote interfaces and service descriptions are nice and have all sorts of benefits, but they don't eliminate coupling.

In short, using SOAP, we now enable a true 'smart' data-exchange-protocol between two systems such that development is now at the level of "data-exchange" rather than API, SDK, language, etc.

Smart data exchange would require actual smarts: rules, constraints, inferences. There is nothing in the SOAP spec or implementations that supports that. It's extremely naive to assume that those problems will get magically solved merely by using some complicated XML scheme for describing data and services.

The XML community is roughly where the Lisp community was in the 1960's: they are awed by the power and convenience of a universal syntax and dynamic type system and they haven't woken up to the fact that the problems themselves are hard and aren't solved just by having convenient, powerful data types.

SOAP on a ROPE

I think it's funny that Microsoft called one of their SOAP tools the Remote Object Proxy Engine. Someone over there must still have sense of humor.

Yet he forgot...

to mention BizTalk. We all know Microsoft's SOAP implementation won't be the most compatible of them all. Heck, I develop with it, I know this myself. But he seems to overlook the fact that when Microsoft realizes a bug, a problem, a milieu if you will, they don't fix it. No, they release another program to do it for them. Honestly, this has it's own strength and weakness, weaknesses such as you now have to pay to "license" another peice of software (as with MS, it seems you can never own what you pay for), that it's not fully integrated into whatever other product suite, and that it can mean yet another peice of software to learn. However, the strength of having a dedicated development team on it, developing and extending the features and making sure that any single program doesn't get to feature rich (hint: this is where you make a crack about Word's feature bloat) balance those negatives out.

Which brings me to the topic at hand. BizTalk. Microsoft's goal with BizTalk is to serve as a translator to allow interop with XML documents. Now, you and I both know that a well written XSL can handle many of the features BizTalk does, and if you read into enough, that's really all BizTalk does is collect XML files, translate them with an XSL, and put the resultant XML files somewhere else. Now, tell me, how is this any different from SOAP/XML-RPC, in which the calls are transmitted and received through XML? Yes, Microsoft want it's spoonfed developers (sadly including me) to use their no doubt individual implentation which just happens to be different from everyone else. But they aren't going to isolate their developers, instead offering BizTalk to handle the translation from x,y, and z sources to MS Standard (ooh, shame on those rogue sources following a strict, compatible implentation rather than doing it the MS way!)

On the same vein, but for Linux, how long will it be until someone writes a set of libraries that handle translation between XML-RPC and whatever method a develop choses to implement SOAP to/from the MS standard on their end? It isn't that hard to see how the requests are formed, sent, and received (it's just XML data over HTTP), so surely someone will release a library or a set of XSL files. Yes, I know, this is more work, but this is not the end of the SOAP standard for open source developers, I don't believe, although from reading the article the author would seem likely to disagree with me.

SOAP is...

By digging one more layer down, you can find this [w3.org] at W3 [w3.org]:

Abstract

SOAP is a lightweight protocol for exchange of information in a decentralized, distributed environment. It is an XML based protocol that consists of three parts: an envelope that defines a framework for describing what is in a message and how to process it, a set of encoding rules for expressing instances of application-defined datatypes, and a convention for representing remote procedure calls and responses. SOAP can potentially be used in combination with a variety of other protocols; however, the only bindings defined in this document describe how to use SOAP in combination with HTTP and HTTP Extension Framework.

Re:Um...

Am I the only one who thinks this is kinda stupid ?

I mean, they are basicly doing tunneling corba/RMI/etc. calls over HTTP .. for what reason ? right ! to go behind firewalls ! So .. I build my firewall, allow only HTTP, then they start using it as a tunnel device !!?

Re:Um...

Sun have blown hot and cold over it for the last 6 months or so.

Well, SOAP started out as an extension to XML-RPC. Sun liked XML-RPC. Sun didn't like the initial version of Microsoft's extensions to XML-RPC because they thought they favored Microsoft (surprise surprise). IBM didn't like the initial version of the SOAP spec either, and proposed some changes, which became SOAP 1.1. Sun likes IBM's revised spec (both Sun and IBM are Java proponents, amongst other things). So Sun really hasn't been terribly inconsistant about things. Also, many people have taken Sun's involvement with other, sometimes similar technologies (like Jini, CORBA, etc) as well as them not dropping RMI as being cold towards SOAP. However, I think Sun is just hedging their bets and trying to embrace as many things as makes sense for them to do.

Currently they say it's a good thing

Like I said, the SOAP spec changed, unless the spec changes again or something else comes along, I wouldn't expect Sun to change their position again.