Tim Bray on Microsoft Office

jgeelan writes "The co-inventor of XML, Tim Bray, has been talking about the newly XML-enabled version of Microsoft Office, code-named 'Office 11' and tells XML-Journal that 'when the huge universe of MS Office documents becomes available for processing by any programmer with a Perl script and a bit of intelligence, all sorts of wonderful new things can be invented that you and I can't imagine.'"

Yay Evil Monopoly Of Doom!

Wow, I was way off when I predicted that Microsoft would further obfuscate their Word format. This seems to be in all respects a Good Thing.

StarOffice has used XML for their native file formats for some time now; I wonder if this means we'll see an even better-quality translator between the two formats?

Re:Yay Evil Monopoly Of Doom!

So, what happens when somone want's to email an XML enabled Word document...... Does it somhow become encrypted on its way out of the database, remains scrambled on it's way over the internet, and reassembles itself into nice XML once it arrives on the recepients computer?.... Doesn't sound like XML to me?!

Re:Yay Evil Monopoly Of Doom!

Not that I totally disagree with your point, but with ".Net" people will be discouraged, or it will be far more difficult to send the actual document. My guess is that some future version of Office will default to "Send the shortcut".

Now they of course will change Office for the Mac to read from those servers... The data WILL be stored in XML on those servers, so coders will have an easy time with it.

You bring up an interesting point about paranoid people and Microsoft. I have followed Microsoft fairly closely over the last ~18 years and feel comfortable saying that they have never worked with any "standard" out there. They have ALLWAYS developed their own. Can you name an example of any "standard" software technology they have adopted and not changed? A perfect example of this would be ZIP. Why doesn't Microsoft use it instead of CAB files? There are many many more I could use as examples if you would like.

Microsoft has an internal saying "If it is not ours destroy it".

My point is this. A company that has for 18 years been trying to lock people in to their technology, will cause some people to be a bit paranoid.

Re:Yay Evil Monopoly Of Doom!

Okay, so it'll be harder to mount a windows partition effectively, but this doesn't affect transmission of documents, especially if they're stored in an XML format. As for me, I think it's more valuable to have files that I can read outside of their native filesystem rather than have a readable filesystem filled with unreadable files.

Re:Yay Evil Monopoly Of Doom!

What a bunch of pseudo-technical garbage!

I have a Masters in Computer Science with a focus on databases and storage technology and very little of what you said makes any sense to me. There's nothing easier than getting at data stored in SQL. Where I work, we've shipped a few products where we didn't document the schema because it was too complex and we didn't feel we could support it. Within weeks, almost all of our major customrs had it reverse-engineered anyway. SQL is very easy to get at!

kernel level SQL data

There's no such thing. SQL data is stored in tables. You use queries to get at it. Period.

Also, your story doesn't make any sense. The article says Office 11 is in Beta already. IIRC, the SQL Server and Palladium stuff in the OS doesn't come until Longhorn. Do you think they will actually release a version of Office which won't work until their next OS (who knows when that will be) is released and adopted? How will they make money off all the people who recently upgraded to Windows XP then?

Re:Yay Evil Monopoly Of Doom!

I don't beleive any of this crap is goingto happen from MS. Not for a New York second.

You'll be DMCA'd out of the loop for trying, and the format will validate itself with 'Palladium' features in software, or some such.

However, the mind reels at the idea of managing PowerPoint and Excel files from emacs!

Re:Yay Evil Monopoly Of Doom!

I don't beleive any of this crap is goingto happen from MS. Not for a New York second.

Dark-masked B.Gates approaching you:
"I find your lack of faith....disturbing."

Re:Yay Evil Monopoly Of Doom!

No you were not. MS routinely uses XML to encapsulate (proprietary) binary data. In the case of the MSOffice file format, this is especially true, but to a lesser extent this also goes for stuff like BizTalk etc (that has a terrible license attached to it). If Ms is *really* serious about using open formats, and using XML in their Office suite, they should put their money where their mouth is and join in the OpenOffice File format project [openoffice.org]. Most of the opensource players are working their already, and the EU is also set to join. I assure you that mature participation of Microsoft would be very welcome.

Of course, this will never happen. Instead, MS will continue to push their own "open" XML based file formats. Microsoft Kerberos, anyone?

However...

Knowing Microsoft, they don't like to give up that format lock-in. They'll find some way to make MSXML difficult or impossible to access, assuming that they haven't already.... encrypted data or something like that.

An eXaMpLe of MSWord XML...

<?xml version="1.0"?>
<Data>
MSWORD$$g$%jk$%sxx"d$%^$% ^($%^m$k^a$%j^$%B$JJ"£"$M £K
</Data>

Re:However...

They'll find some way to make MSXML difficult or impossible to access, assuming that they haven't already.... encrypted data or something like that.

What?!?!?!? You mean they'll try and pass something off as a a "security feature," when it's really intended to protect them?

Nah, that's not really their style.

Read the article?

This bloke said he has had extensive access to the alphas and betas. He also said how great it was.

So unless your mind has been slashdotted to the extent that you think that Microsoft is going to suddenly change the file-format completely between beta and release, then we know that it is perfectly easy to read.

And if you do believe they will change the format, then you are a moron.

Re:MOD PARENT UP

They already done that with other products and wrote it in thier halloween-document. Where do you have any indication that they should change stratagy now ?

Because they have competition now, and thats makes people do funny things.

Did you ever think that maybe all thing things MS has done in the last 24+ months that show an increasingly high level of support for interoperability and standards compliance is more than just show?

Did it ever occur to you that maybe, just maybe, that the head people at MS read the halloween documents and discarded them as garbage?

The format is XML, its going to be open, and its going to be great. I am willing to make a cash bet on it. Yeah, sure, some data inside it will probably be a binary representation of an object - maybe graphs, charts links to external programs, images, etc. If you want to take that bet, e-mail me.

Most people here on slashdot know that even with a level playing field of open file formats that Office will *still* dominate the marketplace for Office software.

Re:MOD PARENT UP

Slight difference this time. Now, we actually have a beta release, seen by independent parties (such as Tim Bray) who have praised the file format. As I said, unless you seriously think they will completely change the file format between beta and release...

Now, whether they can license the format so as to make it illegal for other apps to use it, I don't know. However, I suspect this is not the case as it more or less removes the advantage to having invested in XML in the first place. Well, sure there's good publicity, but how long would that last when people immediately discover it is worthless?

And of course, the vast majority of people don't care about file formats. The only people to whom this news is of interest are those who will want to either access Office docs themselves, or use other apps (e.g. Open Office) to view Office docs. If this sector are banned from doing this, why did MS spend so much money on using XML in the first place?

Re:MOD PARENT UP

If this sector are banned from doing this, why did MS spend so much money on using XML in the first place?

You might similarly ask, "If MS didn't intend to comply with web standards, why did they spend so much on Internet Explorer"

Please tell me I don't have to spell the answer out for you.

Incompatibilities Once Again

.... I guess it's just MSXML rather than THE standard XML. But we can figure it out with some "intelligent guesswork" now because the file would be human-readable.

Re:Incompatibilities Once Again

It's just like the old SGML module for Word they used to have about 6 years ago. My guess is that there will be some significant drawback to saving documents in XML, such as loss of some formatting information. That would convince users not to save in the XML format... but that isn't the important thing to Microsoft.

More significantly, there might be small incompatibilities, or ways that Word-created XML documents divert slightly from what is normal and proper in XML. Perhaps Word will make some (intentional) mistakes when reading back XML files generated in other applications, just like Word's old SGML module would choke on many proper SGML documents.

Make no mistake: the fact that almost everybody is using Office and the associated file formats makes it very hard for a new contender to enter the office suite market. Microsoft must be aware of the power they have over the market with their Office file formats. Think of it: when you exchange files with other businesses, you have two realistic choices of file formats: Office or plaintext. And now Microsoft is introducing compatibility with an open and well-defined markup langauge, in favour of their proprietary language? I'll believe it when I see it.

Re:Incompatibilities Once Again

Think of it: when you exchange files with other businesses, you have two realistic choices of file formats: Office or plaintext

I think PDF is a viable (growing even) third option. Adobe is "evil" just like MS (remeber Sklyarov)... regardless, PDF is nice and it works well, and the files are way smaller than word docs.

Re:Too good to be true

Because it doesn't matter if everyone is able to read, modify and generate Office-compatible files. People will us Office products in future. Opening the file formats doesn't change anything.

XML makes it easy to create programs that will depend on MS Office. So this only makes it easier to create programs which depend on Microsoft products.

Re:Too good to be true

Except I will look to xml.openoffice.org to write some xslt transformations to take Microsoft office documents and liberate them once and for all.

Once I can move my team of 20 people to open office with no real worries or complaints about 'interchanging' files with lusers still using Microsoft, I will.

BUT, have you ever looked at an HTML file generated by Microsoft word? It is a GREAT example of how they can pollute a standard into something unreadable.

I suspect that they will copyright or otherwise lock up their DTD/Schema, and try to lash out at anyone that uses them in other than 'approved' ways.

Re:Too good to be true

Maybe they need a migration path away from the win32-based format they use now. .NET also seems to follow that path. Remember that MS needs access to other platforms than the i386/desktop in the future - mobile devices for instance. Keeping a format that is basically a binary image from a PC is good for locking out competition, but not when you have to start competing with yourself.

What will be the default save format?

The most important question, besides if the MS Word XML format will be well-documented enough, is if it will be the default saving format. Most MS Office users simply don't care enough to save MS Word documents in RTF, for example, even if it's more than good enough for the vast majority of the documents.

Not the main issue on the article, but it is unfair to single someone as the inventor of XML, which is just a streamlined version of SGML which is an evolution from IBM's GML.

Re:What will be the default save format?

Stop right there.
If you continue with that line of reasoning, someone's gonna demand that it be called SGML/XML.

Grr.

Re:What will be the default save format?

According to this [com.com] article on ZDNet, it wil probably NOT be the primary file format:

To make that happen, Microsoft is turning to what some analysts say is a risky strategy. The company is adopting Extensible Markup Language (XML) as a second file format in all Office applications, to enable better data exchange between the productivity suite and back-end software, such as databases.

I doubt it.

I really have my doubts about wether Microsoft will allow "any programmer with a Perl script and a bit of intelligence" to muck around with Office documents.
I'm guessing their XML document format will be just as hard to decyper and the current office formats.

Re:I doubt it.

I really have my doubts about wether Microsoft will allow "any programmer with a Perl script and a bit of intelligence" to muck around with Office documents.

Why not? After all, the high-quality ActiveState port of Perl to Win32 exists because Microsoft paid for it, and you can download it for free. Not only that, but if you want to write your own code to manipulate Office documents, you have been able to do that for years in VBA - all the Office programs expose rich APIs. In fact, they are composed of Objects that you can instantiate and use in your own programs if you want - all MS care about is that there is a licensed copy of Office on the user's machine. One of the easiest ways to do charting is to simply reuse a bit of Excel, for example. From there it's a short hop via COM to any program you want.

I'm guessing their XML document format will be just as hard to decyper and the current office formats.

The fact that Office documents have been in a proprietary format in the past is actually unimportant, since the interfaces to the applications (and hence their documents) are well documented (check MSDN or Barnes & Noble if you don't believe me). So the reason that Microsoft are doing this is that they lose nothing and gain from making the platform even more attractive to developers.

Re:I doubt it.

The fact that Office documents have been in a proprietary format in the past is actually unimportant, since the interfaces to the applications (and hence their documents) are well documented

So you can read Office documents with other programs as long as you have Office and MS dev tools?

You do see the folly in that, right?

-Kevin

Re:I doubt it.

I'm guessing their XML document format will be just as hard to decyper and the current office formats.

There are 2 problems with the current format of Microsoft Office file:

1. Give the correct interpretation to the bytes representing the document content, in order to import the Office document in some other office suite using a different representation.
   This is mostly solved (thanks to years of trials and errors).
2. Give the correct interpretation to the bytes representing the document itself AND all the extra cruft having nothing to do with the document contents that the Microsoft Office suite puts in, in order to generate documents readable by the various versions of the Office suite.
   This is definitively more difficult, as nobody knows Office internals and how they expect such additional data to be. StarOffice guys managed to make an acceptable job, at the price of years of trials and errors. It's like watching at a dump of your computer's memory, guesssing what's code, what's data, what's padding and the meaning of every byte...

Now, do an XML format simplifies things? Well, yes, just as an RTF text is easier to manage than a pure binary format, but nothing prevents putting extra cruft in an XML document, so it's just that instead of having to use a hex editor, you now may use a text editor, but giving a correct interpretation of tags and attributes is something that only Microsoft can do, unless it publishes the full specifications (present and future: after all, XML is eXtendible, right?)

Personally, I think that:

• Microsoft is realizing that the current Office formats are getting out of control, so it wants to get rid of them, because mantaining backwards compatibility is becoming too much painful.
• An XML-based format may be the right answer for Microsoft, in that all the subtles of parsing binary data simply disappear, while it may still make difficult to everyone else to understand what's the real meaning of data. Let's say <obscuretag_42 foobarizer="xyzzy"/>
• Microsoft was not giving out the specifications of the formats of its Office suite before: should we now suppose it's giving out the DTD/Schema AND a good explanation of how to interpret it? I'd hope the answer is yes, but giving the company's precedents...

Historical turningpoint?

I just thought about someone saying that somewere, when you look back in history, you can see some historical turningpoint where tings just went wrong or right.

One small such point is when IBM gave out the specs to their hardware for PC allowing everyone to clone it, while Apple did not.

This could be such a point. Maybe in 10 years we'll look back at this and ask ourselves "Why the heck did MS XML-enable their Office app, releasing the hold that they had"

Only time will tell I guess.

.haeger

I Play Hattrick [hattrick.org]

Re:Historical turningpoint?

IBM release th framework in which to do so because of the governmental investigation they were under at the time.

They didn't do it out of the goodness of their hearts, but they did indeed do it. It wasn't the complete bios though so Compaq had two teams...one team looking at the specs, and another (that could never look) building a clean room implementation.

*when* ?

when the huge universe of MS Office documents becomes available for processing by any programmer

I beg you pardon? Smelly programmers can keep their hands off my documents. If I wanted you to have them, I'd have emailed them to you as plaintext. I wasn't aware the the Office license meant my documents were common property....

The right time for MS

MS is trying to time this right.

Right now they are seeing diminishing sales, possible shrinking market share. Most of the danish public sector is looking to save money using OpenOffice/StarOffice.

MS needs to increase their compatibility with other options, as they would otherwise force customers to convert every single user away from MS at once, instead of OpenOffice coming in slowly.

They can also hope, that their format is setting the standard, and the other companies will have to play catch-up rather than the other way around.

imagination

...all sorts of wonderful new things can be invented that you and I can't imagine...

When will MS ever learn that we don't WANT to imagine how wonderfull the MS Office Universe is ?

WTF????

from the article;

The most important question, besides if the MS Word XML format will be well-documented enough,...

WTF!? XML shouldn't need to be documented. The whole point is to create a human readable file that is parseble by computer. If MS Word delivers an XML file that I can't figure out, it's not XML.

Re:WTF????

Have you ever seen some complex XML file? Without documentation it could be as difficult as binary to reverse-engineer!

Re:WTF????

The point of XML is not for it to be human readable, but to allow easy automatic processing of various kinds.

With XML Schema and DTD's, you can validate various aspects of the data without writing a custome validator.

With XPath and XPointer you can refer to parts of an XML document without needing to understand what the document contains.

With XSL you can translate all or parts of the document from one format to the other without your application needing to know the structure, and without needing to understand more of the format than the parts you are extracting.

With SAX and the DOM you can programmatically traverse and extract information from an XML file without having to write a custom parser.

With CSS an editor or viewer for instance can use a standard mechanism of applying styles to elements without hardcoding the style attributes for elements anywhere.

With XML namespaces, you can intersperse data in various formats in the same file, and the components handling each of the vocabularies need not know anything about the other components - an example would be embedding SVG in HTML: The HTML renderer doesn't need to understand any of the SVG tags, only that it should delegate contents with other namespaces to another component. And the SVG renderer couldn't care less about the HTML.

And this doesn't even touch on the benefits of all the various interchange formats that have been specified on top of these base technologies.

The importance of XML is that it opens up the doors for building interchangable components that operate on data without needing any hardcoded application specific knowledge of the data.

Most of the time, you still have to write some code to tie it all together, but you don't have to build your own parsers, your own document object model, your own styling system, your own way of handling contained data of other types, your own way of transforming data between formats, etc.

For me as a software developer XML delivered years ago. I use XML technologies daily, and it saves me work.