California Joins Open Document Bandwagon

Andy Updegrove writes "A legislator in California has decided that it's time for California to get on the open formats bandwagon. If all of the bills filed in the last few weeks pass, California, Texas, and Minnesota will all require, in near-identical language, that 'all documents, including, but not limited to, text, spreadsheets, and presentations, produced by any state agency shall be created, exchanged, and preserved in an open extensible markup language-based, XML-based file format.' What type of formats will qualify? Again, the language is very uniform (the following is from the California statute): 'When deciding how to implement this section, the department in its evaluation of open, XML-based file formats shall consider all of the following features: (1) Interoperable among diverse internal and external platforms and applications; (2) Fully published and available royalty-free; (3) Implemented by multiple vendors; (4) Controlled by an open industry organization with a well-defined inclusive process for evolution of the standard.'"

Minnesota also

Minnesota also is considering open documents. [slashdot.org]

Re:Minnesota also

From TFS

'all documents, including, but not limited to, text, spreadsheets, and presentations, produced by any state agency shall be created, exchanged, and preserved in an open extensible markup language-based, XML-based file format.'

The one thing that stands out to me is specifying that it is XML-based. I have no problem with that, the two competing formats are indeed XML based but shouldn't the mandate of the state simply to specify the Open and Cross-Platform aspects?
What happens for instance if tomorrow all of us wonderful Slashdot readers co-developed a magical format that not only was open and cross platform but inexplicably worked with all currently available office suites without modification... ...but it wasn't XML. Does that disqualify it? I would rather see the politics of this issue left with the politicians, the tech issues left with the techies. I've seen other combinations go horribly wrong DMCA

Re:Minnesota also

If it is an open XML based format then doing a conversion to whatever new format arises should be trivial (maybe not fast, but fairly easy with XSLT). SO better to put it into XML now and worry about what better format may arise later.

This is good news... why be negative about it?

Re:Minnesota also

If, and when, such a format comes into play, and has a large enough subset of tools available for it, then the laws can be revised. With all documents already in XML, converting to the new format should be nearly painless, and more likely, both formats could be used.

The tech needs to be spelled out clearly in the law, otherwise vendors like Microsoft will be able to say their format qualifies and lobby until enough tech-clueless legislators agree to it.

No brainer

I don't get what all the hoo-haw is and why we need courts or lobbying for any of this. I find it very difficult to write anything when my term paper or [insert your document here] isn't open. Sounds like a bunch of people just need to learn how to double-click.

What does XML have to do with it?

As long as the format meets criteria 1-4, I don't see why it's necessary to specify that it must be XML-based. Keep it simple, and all that...

Criteria n3

As long as the format meets criteria 1-4

In other news, Microsoft is quickly subsidizing 3 small companies to write quick and meaningless stupid plug-ins using OOXML as input, just to pretend that their format is "Implemented by multiple vendors" and on "diverse (...) platforms" (ie.: Windows 98, Windows ME, Windows 2000, Windows XP *and* Windows Vista)...

Re:What does XML have to do with it?

There's only one reason, and that's because the higher-ups think that EVERYTHING should be XML based. Of course, they have no idea what this actually means. They just know that it needs to have XML in it, because that's what the other guys are doing.

This reminds me of my boss, who keeps saying that we need to publish things in XML, but can't give me any reason why we should. Then again, two years ago I kept on hearing about how our company needed a blog, again with no justification as to how it would help us. Thankfully, that passed. Eventually, the XML thing will, too. Of course, this isn't meant to belittle the things out there that actually can benefit from utilizing an XML format.

Wtf? Why "XML-based", specifically?

Why not just require the format to be in ANY published standard format? "XML" by itself is meaningless, "extensible" is a loaded term (and a very bad idea when trying to write a way to keep things compatible). Why do lawmakers always have to over-specify things until the purpose of the law is lost?

Re:Wtf? Why "XML-based", specifically?

Why do lawmakers always have to over-specify things until the purpose of the law is lost?

Job security. If they wrote clear, concise, and sharply targeted bills, we wouldn't need to keep electing a fresh crop to fix the mess left by the last one.

For one thing...

XML means it is readable by humans. You don't even NEED any kind of a program to get the text.

Dominoes

The dominoes are beginning to fall.

Text in XML?

...text, spreadsheets, and presentations, produced by any state agency shall be created, exchanged, and preserved in an open extensible markup language-based, XML-based file format.

Why would text need to be in an XML-based format, when it can be in a... text format? If you have a text document that doesn't require any formatting, just make it text!

Re:Text in XML?

Well, obviously it doesn't need to be xml, but XML does have one nice self-documentation property that plain text lacks: the character encoding.

If you've looked at project gutenberg texts, you can see why this is a problem. Not a huge problem, but a problem. When a source text has a non-ascii character in it, they have to put some sequence of ascii characters which will suggest what the glyph is supposed to be. This doesn't really preserve the information in the source document, nor does it make the document easy to read.

So, you could have a trivial text XML format that has only one defined tag. It's still useful:
<xml version="1.0"? encoding="us-ascii">
<text>
This is my text. It has no wacky glyphs so ASCII is fine.
</text>

vs.

<?xml version="1.0"? encoding="utf8">
<text>
This is my text. It has wacky glyphs therefôre ascii sücks for it!
</text>

<xml>we should do this too</xml>

<xml>
<user="wwwillem">
<subject>we should do this too</subject>
<content>
    What is good for government documents is also good for Slashdot posts. :-)
</content>
</xml>

What type of formats will qualify?

Format is irrelevant - since these documents will contain legal-speak, they'll be unreadable anyway. ;)

XML panacea

N00b: Hey we have this data representation problem, we'll use XML!
Greybeard: Son, now you have two problems.

Why is the victim silent?

Well why corporate America is so silent on this issue? After all almost all the revenue of MSFT comes from corporations. If the palying field becomes level and multiple vendors compete to serve them while being fully inter-operatble, it will be the corporate America that will benefit most. Technically they are the victim of the monopolistic deeds of MSFT. Still they remain silent, and the Govenment, after protecting the citizens from their own stupidity (seat belts, airbags, spacing between crib railings) now comes to rescue corporate America?

If Government intervention is what it takes to force a level playing field, I will accept it. But still I would prefer it if market forces create a level playing field instead of government mandates.

Called my rep

Just called my CA Assembly rep to ask them to support the bill. Look yours up here [ca.gov].
It may not be perfect, but is a move in the right direction.

Does MS's format qualify?

MS made a format that fits the very definition of what they said will be required in this bill. Is this bill just going to lead to government organizations upgrading to the new Office? Technically, all of these things apply even if the implementation of the "standard" will later be forked by MS with their extend and extinguish model. In short, does this really mean truly open formats will get a boost? Or that MS's new format will seem like the solution to a problem they have practically invented?

Woohoo, ODF soon to be here

I think the only document format that would qualify is ODF (by OASIS). It's the only well known document format, based on XML and extensible, open and implemented by different vendors and office suites.

Microsoft's open XML format:

Just specifying XML doesn't mean much, really:

<document>
Description of MS Open Format
<![CDATA[
37642364 78346478 23465789 34657834 65783465 78934653 47895634 78563478 65347856
56347825 63478256 34786578 34567893 45678934 65783456 78465783 46578346 57834567
34895723 48957348 90578934 75890347 58934758 93475892 ... more binary crap...
]]>
</document>

Texas House and Senate bills

HB 1794 [state.tx.us]
SB 446 [state.tx.us]

So far, each bill has been filed and referred to the appropriate committee. However, the legislative session just started in January and things don't usually start happening until after the filing deadline on 2007-03-09.

Oh Please!

There is one particular spreadsheet we have to email for a report - it is loaded with Active-X controls and the like, it Works with Excel on Windows and nothing else. One of the insances where we have to pull out 'the Windows laptop' to do the report once a month.

But since that is a reporting metod with contractors (not the general public), I bet it would be an exception.

Along with that there are other Windows specific gotchas - one is an the Access DB that another program has required us to use, and the third instance, a reporting site that is loaded with IE specific ActiveX code (even if you spoofed as IE, it doesn't work).

Every other state report we do sanely accepts either a, delimited text uploads, plain old paper reports, or a 'most browser friendly' web form.

Here's my problem:

There's one company missing from the Open Document party here, and without it playing along, I don't see this lasting long.

I am, of course, talking about Microsoft. They refuse to accept the Open standard.

Until that happens, there will be problems. Yes, you could have .odt documents sent internally, but what if someone has to send a document to someone outside the company? Microsoft Office does not recognize .odt, and if you think that you can train someone to remember to send .doc files to outside users, and keep internal documents to .odt, then I have a bridge to sell you.

Is Arnie still governor?

What next? Hasta la vista, Vista?

Two Possible Outcomes

If California passes this resolution I can see two outcomes. 1 The state recognizes that ODF has to be used and scraps Office and loads OpenOffice or StarOffice. Big win for the citizens of California big loss for Microsoft. 2 The state recognizes that ODF has to be used and because older versions of Office won't work with ODF they purchase Vista and Office 2007 for all state agencies. Huge loss for the citizens of California huge win for Microsoft. Guess which is more likely?

Reality vs. Concepts-

The absolute unfortunate truth in this case is that it will not matter what requirements the state of California sets forth, because in the end it serves not the people but the income of the government.

You see, in California, we have this precedence of hiring under-motivate, under-educated, people into roles to fulfill status-quo on the premise of serving equality.

This results in a rule that I call "Factor 4" where by you can take the initial cost of any related project, service or resource requisition, and thereby multiply it by factor of 4 in order to obtain the actual cost to the government.

Sadly, Factor 4, is a direct result of the mediocrity that has taken up residence within all of our government agencies. I cannot imagine a bigger nightmare than this one that I just read about. Half of the institutions within the government are filled with people that have no idea what that means, and lack the education to understand it.

With this being true, we open the door to committees, educational round-tables to determine educational requirements, requisitions for training, then post-comittees to evaluate if the needs were met, then another comittee to determine if the proper mixture of minority members were upheld, then further we'll add layers of evaluation to insure that all submissions qualified with the sole purpose of perpetuating a verification process that checks itself sometimes 3 times over- with absolutely no guarantee that said process is: accurate, predictable, or effective.

All this does is allow state governments the ability to ask for additional funding, which they will earmark with non-related items, and then fund other programs with the initial request.

Translation- the greater good for which said items are presented will be moderately served.

Outcome: Ho hum and whatever. Can't we think of better things to do with my tax money than fuddling around with this area of business? I say they throw out this status quo requirement and start paying people what they are worth so that we can get some really bright minds into our state governments.

Good news for some

This should bring in some big bucks for certain projects [sourceforge.net] at least...

And the answer is ...

What is needed is an ODF compatible version of WordStar.

The idea that at an enterprise level there are multiple vendors (real vendors, not distributors) of a word processor and spreadsheet program is a joke. There are perhaps three, and the two I know of are OpenOffice/StarOffice and Microsoft. And there are huge questions about the enterprise viability of OpenOffice that have yet to be answered.

Also, the level of complexity for ODF is such that it is unlikely that every implementation is going to render it the same. This means you create a document with one application and all the form fields are lined up. It is then printed with a different application - still using ODF - and the form fields are shifted over slightly. Maybe just enough to move from column D to column E on the form.

The level of complexity is utterly absurd for any cross-application compatibility. Micrsoft at least understands the problem and clearly indicates that such compatibility isn't going to happen. Reading their standard shows that. Without a committee overseeing development and implementation and certifying implementations, there will never be the level of compatibility that is required.

Dept Tag--"Lobbiest"?

Lobbiests would be people who are most like lobbies, I take it?

Funny you should mention that -

I just noticed that the shiny new SCSI controller cards I got yesterday had a big fat "no lead" symbol prominently featured on the boxes...

/P