Creating and Using XML-Based Internal Documents? 179
Richard Emberson asks: "Once again into the breech...or at least the ground floor in a new startup. This time around, I would like to have all of the Engineering
documentation internally online: a unified, internal, CVS-ed, web-based, development organization document tree covering the engineering process, methodology, coding standards, nightly build/test reports, FAQs, new hire information and help pages and the documentation for each project.
Recently I've written documentation (on Linux of course) using the Apache XML-stylebook tags, stylesheets, and Ant-base publishing - and I like it.
So my questions are: Has anyone done this and, if so, how were the links between documents managed?" Does your workplace use XML in its internal documentation? If so, how well does your system work, and what advice would you pass on to anyone else attempting something similar?
"If you start out with only one project (product), how do you structure it so that when new projects come into existence they can easily be integrated? Are there documentation templates out there upon which I can base the various development documents (like requirements, product development plan, design, coding walk-thru standards, etc.) and not have any of this swell too be so large that no one will be able to produce, maintain or read it?"
InfoImaging and Dig35 meta data uses XML (Score:3, Informative)
Good luck- it's quite impressive once you get the trees set up correctly
XML is good (Score:1, Insightful)
Developers,Developers,Developers,Developers....
-teknopurge
http://techienews.utropicmedia.com [utropicmedia.com] help us beta!!!
Re:XML is good (Score:3)
XML is simply a notation for structured information. It's not the first, it won't be the last, it's not the most compact (lisp is far compacter), and it may well not be the best.
But it's around, there are some robust libraries to parse it, so you might as well use it.
It's all directed acyclic graphs, really...
Jules
Re:XML is good (Score:1)
Re:XML is good (Score:1)
You know, you could've just said "It's all trees, really", and made it a lot simpler. Speaking of which, XML may simply be a notation for hierarchial and mixed data, but it happens to be a fairly simple one that extends well into multiple namespaces, is unambiguous with regards to parsing, and in general is human readable.
While I agree XML may not be the best at at accomplishing specific tasks, its general nature makes it a pretty good fit for general, all-around use. Uniformly using XML for a lot of things helps out us humans more than anything else.
XML (Score:1, Offtopic)
XML is not a replacement for HTML.
It is important to understand that XML is not a replacement for HTML. In future Web development it is most likely that XML will be used to describe the data, while HTML will be used to format and display the same data.
My best description of XML is as a cross-platform, software and hardware independent tool for transmitting information
Re:XML (Score:1)
Re:XML (Score:2, Insightful)
2) XML is not a replacement or a complement for HTML. HTML has nothing to do with XML. XML is a extensible markup language which can be used to transfer an infinitum of data forms, HTML happens to be one of th emany many uses it has, but not nearly the most important. XML is more commonly used for databases, RPC calls, log files, EDI, or new languages than as a complement for HTML.
Re:XML (Score:3, Interesting)
I'll curse the brain dead moron that first suggested using XML for log files for the rest of my days. It is completely unsuited to the task. I have to wade through megabytes of pointless XML every day, searching for errors that are obfuscated by the sheer volume of crap that surrounds them.
Re:XML (Score:2)
Really, I'm asking: how much work?
Because from where I stand (as a programmer who does some sysadminning on the side), it doesn't look that hard.
Re:XML (Score:1)
It is STUPID IN THE EXTREME to store textual data in anything other than the simplest text format you can get away with. Use the lowest common denominator, but I guess they don't teach that anymore in school, just "use the latest XYZ!" nonsense that's almost taken over this field.
If you want to use XML, fine, translate text to XML, but it is IDIOTIC to store it that way.
Re:XML (Score:2)
There are databases optomized for loading and running queries on XML data based only on the included metadata. There are graphical XML editors which allow a structured view of arbitrary data, given a DTD (which is also standardized and has tons of tools capable of processing).
Writing a custom parser for every single data file one might wish to store because one chose to use a text format that looked easier strikes *me* as stupid in the extreme.
Btw, I didn't learn this stuff in school -- I learned it the hard way, writing conversion tools for other people's "simple" proprietary formats.
Re:XML (Score:2)
More than you'd imagine, because it's not pure XML. It's XML intermixed with other data in different formats. But still not that hard. The point is, though, how much less work would it be if they'd just logged stuff in a sensible format to start with?
Re:XML (Score:2)
<a-big-configuration>
<a-section>
<a-section-name>
whoopee
</a-section-name>
<a-section-comment>
here is an irrelevant comment!
</a-section-comment>
<a-value>
<note-about-this-value>
in case you didn't know, this is a value - watch out!
</note-about-this-value>
<a-value-name>
bloatedness.factor
</a-value-name>
<a-value-value>
<a-value-value-type>
<a-value-value-type-value>
integer
</a-value-value-type-value>
<a-value-value-type-description>
an 'integer' is a whole number...ya moron...
</a-value-value-type-description>
</a-value-value-type>
<a-value-value-value>
1000000000000
</a-value-value-value>
</a-value-value>
</a-value>
</a-section>
</a-big-configuration>
Maybe in the old days... (Score:2)
XML is not a replacement for HTML.
I disagree rather strongly with this. I don't know what your experience is with XML but there are lots of shops that use XML for both presentation and data interchange because of its versatility. An XML document can be presented using an XSLT stylesheet or parsed using a DOM, SAX or whatever API. So the same document that can appear on a website because it has a stylesheet to transform it to HTML to maker it viewable is the same document that is used by your applications as a config file, data file, database updategram, etc. with zero modifications to the file.
This is a very, very powerful aspect of XML. In my opinion, HTML is dead and considering that there's been an XHTML Recommendation [w3.org] for close to two years I wonder why people are still clinging to XHTML (Yes, I know it's because the browser developers have dropped the ball).
Re:Maybe in the old days... (Score:1)
The 'T' in XSLT stands for 'Transformation.' By the time the data described in the XML has reached you, it might not be (and probably isn't) XML any more. It's been transformed into something else.
Re:Maybe in the old days... (Score:2, Insightful)
The problem is (Score:2, Insightful)
Good luck taking on the Microsoft Monster.
Re:The problem is (Score:1)
Doc Book (Score:5, Informative)
Have Fun
Re:Doc Book (Score:3, Informative)
DocBook is excellent, no question about it.
Re:Doc Book (Score:2, Interesting)
We have a very common environment of mixed levels of technical expertise. MS-Word is the standard that everyone has, and everyone can use. Currently it is the dictated standard for internal documentation within R&D. The only way to get something in to replace it, is to provide a UI which is at least close to its simplicity of use
Any solutions out there for DocBook?
Word to XML? (Score:3, Insightful)
True enough, but it doesn't answer the original question. Can MS-Word (or even StarOffice, if you want a non-MS option) output XML? I know it will output HTML, but it's such a bastardized mass of proprietary and font/format specific crap that it's essentially useless unless you extensive filter it first.
The whole point here is not to create pretty formatted documents, but to leverage the power of XML to add context and meaning to the content of the documents in order to create a rich and interlinked heirarchy of information. Conventional word processors just create blobs of information -- pretty formatted blobs, but blobs nonetheless...
Re:Word to XML? (Score:1, Informative)
Re:Word to XML? (Score:2)
blob of binary data
with
<ms-office type="word">
blob of binary data
</ms-office>
Technically a well-formed XML file (assuming that the DTD shows the ms-office tag as having CDATA content), but it's useless as a shared document format.
Star Office (Score:3, Interesting)
Re:Doc Book (Score:1, Interesting)
http://www.arbortext.com/
It's not quite as simple as I make it sound: you'll have to develop a DTD and Stylesheets... but for the content-people, is easy-breezy.
Don't know how well this falls into the scope of a fully XML-based internal doc environment though.
Different problems (Score:3, Informative)
DocBook, and any decent document XML DTD, gives you the ability to tag your text with some description of what it means. It might be "chapter" or "list," or it might be domain specific like "files," "bugs" and "see also" (for man pages). The presentation details are left to the processing software to handle.
MS-Word, in contrast, is nothing but a paint tool for words. You can certainly give your styles names that have some domain meaning to you, but it's still ultimately nothing but a set of style instructions.
For a single document, this isn't a big issue. But if you have a lot of documents and you want to reuse content, it's impossible with the MS-Word approach. With DocBook, in contrast, it's easy to set up your documents so that the same file can be reused in multiple places, but only selected content will be reused.
IMHO, if your technical writers can't make the shift to meaningful tags, you're better off without them. (The writers, not the tags.) If they can't handle this level of structure, their writing is undoubtably muddled and confused no matter how pretty it looks.
Re:Doc Book (Score:2, Interesting)
Re:Doc Book (Score:1)
Re:Doc Book (Score:3, Interesting)
Linking features between documents can be achieved using <olink> and <ulink> tags. The exact details would depend on your repository setup, but the things you want to do are catered for.
Best of luck.
Re:Doc Book (Score:3, Insightful)
In short, for editing, Emacs/psgml mode worked fine, but the complexity of keyboard commands, my aging brain, and deadline pressure meant that I could only learn to use a small chunk of the possible features, especially of psgml mode which, unlike Emacs, was new to me. I opened one Emacs frame on one page on a 4 x 7 desktop for each chapter and then skipped from desktop page to desktop page with a full screen for each. Hint: Make a tags table so that you can then do search and replace operations across all files that make up your book. (See 22.16 Tags Tables in the latest GNU Emacs Manual (14th edition).)
Using docbook-xml, I found that tables were difficult, too difficult. I was unable to make a simple 3 colum x 5 row table within the hour I'd alloted and finally gave up. Ended up using the tag to get literal text. Fortunately, only one or two tables were desired. Checking the archives on the docbook-related mailing lists, I was not able to put Norm Walsh's advice (given to someone with a similar table-related problem) to go read the info on the new table model to practical use within my limited timeframe.
Also, index generation was a bitch. I used collateindex.pl in conjunction with openjade. There must be a better way, but I couldn't find it in the limited time I had.
No graphics were included, but some of the traffic regarding the problems of others gave me a headsup that I should allow plenty of time for experimentation before actual work could begin, learning how to insert both vector and bitmap graphic files in a document that would be turned into a
Incidently, I was also able to produce MIF files for FrameMaker from the same XML source with openjade, a client requirement.
YMMV.
s/again/more (Score:1)
Re:s/again/more (Score:1)
If we are getting picky, Henry V's speech before Harfleur actually starts "Once more unto the breach". :-)
Anthony
Re:s/again/more (Score:2)
if you're going to correct someone using regex, do it right
Re:s/again/more (Score:1)
breech? (Score:1)
Why Are You Asking Me? (Score:5, Insightful)
It sounds to me like you're already a step ahead of the rest of the world, for the most part.
My workplace uses a hodge-podge of formats including "special" ASCII text files, Framemaker, HTML and Microsoft Word. Needless to say, it's a mess. No open, standard, consistent tools to examine all of our documentation. Yeah, you can grep HTML, but the others are a pain. And don't even think about automatic script language based conversion among these formats.
I suspect you're more advanced in your thinking than 90% of the places out there. Why not continue with your thinking and let the rest of us know what you decide?
Re:Why Are You Asking Me? (Score:5, Interesting)
The question I would ask is, Is that because the tools are inadequate? Or because human thinking and creative processes are a mess?
I have worked in highly structured engineering shops where everything is done according to procedure and every document stored in a structured manner (basically using procedures laid down in 1920!). These shops excel at delivering well-scoped projects in understood knowledge realms (the mythical bridge) on-time and on-budget. They do not do so well at handling projects in poorly understood knowledge realms, or projects where the environment and/or requirements changed rapidly.
I have also worked in loose, no-standards, anything goes engineering shops without any structured document/knowledge processes. I wouldn't hire one of these shops to build a bridge on time, but in fast-changing environments they do much better than the first type.
Conclusion: Be sure you understand what type of shop you are supporting before tying everything down with highly structured processes.
sPh
Re:Why Are You Asking Me? (Score:2)
It doesn't.
You can run a highly creative shop and still achieve ISO9001 compliance. When documenting the processes, you "build in" the flexibility that's required to maintain the creativity, while at the same time avoid regulating it to such an extent that you squelch the creativity.
It can -- and has -- been done. Standards are a *very* good thing in all work environments. Standards that require source-cause fixing of "bugs" (errors, mistakes, mis-steps, call it what you will) are even better.
I love how MS is dealing with XML (Score:2, Interesting)
they will most likly us it for communicating between thier office applications.
of cource they don't tell you that in the press releases, they just say "Office will be moving to XML because we want to support standards" to bad they arn't using XML in a standard way (I know I will get flamed for that last part
Re:I love how MS is dealing with XML (Score:2)
Re:I love how MS is dealing with XML (Score:2, Informative)
Re:I love how MS is dealing with XML (Score:2)
If you ever use the Word 2000 object model to contact MS Word you will get XML streams in and out of it.
Re:I love how MS is dealing with XML (Score:1)
Re:I love how MS is dealing with XML (Score:1)
Re:I love how MS is dealing with XML (Score:2)
Re:I love how MS is dealing with XML (Score:2)
blob of binary data
with
<ms-office type="word">
blob of binary data
</ms-office>
Or something like that. Just like that joke about the guy lost in the helicopter, they provided a format which is technically correct, but totally misses the point.
Re:I love how MS is dealing with XML (Score:2)
The question is if Office XP has peeled back a few layers of the onion.
Re:I love how MS is dealing with XML (Score:2)
Word2K saves its files in an ASCII format that *looks* like XML, but isn't. It's jam-packed with, you guessed it!, proprietary "extensions." It's not parseable as pure XML.
Add in that the Word DTD isn't, to my knowledge, a publicly-available DTD, and the XML mess that Word produces is just as opaque as the hodgepodge mess that they called RTF.
Re:I love how MS is dealing with XML (Score:2)
Zope (Score:5, Informative)
XML as Html.
Re:Zope (Score:2, Interesting)
CVS is designed for the group. Revision control is handled in Zope primitives, but to be able to back out of something, while diff'ing another thing would be a pain to code for it while maintaining its (admittedly half-decent) UI.
Re:Zope (Score:2)
The Zope people have been tinkering with XML for some time, and have had some interesting capabilities, but they are a bit behind the curve (IMHO) on supporting XML standards like XSLT. Perhaps there's a bit of impedence mismatch, since Zope has alway been about content/presentation separation, but does it in a different way than XML. I'd say that Zope is currently rather more web centric than most people seriously interested in XML need it to be. People will want the capabilities to generate printed manuals and PDFs from their documents. On the other hand, Zope rocks for making HTML. For example, I think that for transforming XML into various forms of HTML pages, Zope's Page Templates is going to be a far simpler and better for HTML designers to deal with than Apache Turbine or WebMacro. On the other hand it's apples to oranges, because there is a lot more flexibility built into the servlet model than into Zope.
It's not saying you can't do everything with Zope, just that you have a lot more that you will have to either develop or wait for somebody else to do before you really have a complete solution.
Re:Zope (Score:2, Informative)
Three relevant links to read in considering Zope for XML are:
Creating XML Applications With Zope [xml.com]
Create a XML Based Document Repository [zope.org]
Cant Handle Humongous XML [zope.org]
In some data management scenarios, using Zope obviates the need for XML markup. In practice, content management issues like security, revision control, and online access through a browser are bigger issues than markup. Zope provides solutions to all these problems.
My main caveat in using Zope is that finding all the relevant documentation for XML or anything else is a veritable Easter egg hunt. The Zope API doesn't seem to be documented in one place. More than once a Zope tutorial seriously proposed that the reader read the Python source code for further information.
Stylebook is dead (Score:3, Informative)
I use stylebook for internal documents at planetu.com, but stylebook seems to be dead. Docbook is a much better choice. I just have not got around to converting to it.
The really nice thing about using XML is that I can automate some of the documents. Such as the list of valid form fields for a HTML/jsp page.
For the nightly builds, change logs and javadocs, I use Alexandria [apache.org].
Linking between documents (Score:3, Informative)
related: hire a librarian (Score:1, Interesting)
Don't associate librarians with "the library"; there's more of them in businesses than public/school libraries. These are specialists trained in information organization, retrieval, and distribution.
And, being a recession, you may find this investment quite affordable right now.
Re:related: hire a librarian (Score:1)
Cut back a little on the puffery and spin, and I might be more persuaded.
Re:related: hire a librarian (Score:1)
Ctimes2
PS - if you took offense to this... lighten up.
Our company is using XML for all documentation. (Score:5, Informative)
We standardized all our early documentation on XML, and it's been working great. Admittedly we're using Perforce [perforce.com], not CVS, but we're doing something very similar to what you want to do.
All our documentation is in XML format, in a DTD that we defined. We then have XSLT transformation scripts that convert that documentation to HTML format, and scripts that automatically update our development intranet whenever changes get checked in, along with scripts that invoke javadoc and doxygen on all the code to convert that to HTML format. We're in the process of being able to convert the same documents to PDF format to be able to publish those same documents, in the same formatting, to pretty-formatted documents for printing.
This, aside from the simplicity of not having to worry about formatting documentation when you write it, is pretty cool. It's easier for me (as an engineer) to write a very sparse, structured XML document that will end up looking very good on screen, than to learn enough HTML to make my documents look good. And it's easier for us to enforce a standard look and feel across all documentation this way, because only the XSL transforms have to change if we change our formatting.
But the real advantage is coming out with more advanced uses. For example, when we have configuration files, we have a special DTD that we define the documentation for the configuration files in, and then any documents that describe the configuration files are automatically converted both to the HTML documentation, AND to an example configuration file for users. We can also mark some things as only visible internally, so that the same document can have data that's visible to end-users, and data that's visible to employees (so if we have advanced configuration options that we don't want customers mucking with because they're for debugging the system, we can document them in the same place, and just hide them from customers but let our support and professional services people in on the secret).
The best part is that because our XML DTD is very structured, someone like me (an engineer), will actually use it because it ends up being easier for me than writing in plain text, whereas I wouldn't do it in HTML (or if I did, it would just look like crap). It also makes it much easier to do integrations across branches of code: because we know the DTD for our XML documents very well, it's more likely that integrations will go smoothly, which helps keeping multiple branches of code and docs in sync automatically. If you go with a binary format, you're not going to be able to do that, and every time you make a change, you're going to have to manually change the documentation for each and every branch. With ASCII or HTML, everybody's going to produce documents that look a little different, so you're not going to be able to have as easy a time in integrating between branches.
Our docs infrastructure person can pipe up in terms of the particular technology that we're using, but it's turned out to be one of the best infrastructure decisions that we (actually, she) made, and it's saved uncountable hours and actually made it more likely that people will write documentation themselves, because they don't have to pull out some crazy windows tool: just edit the document in emacs, and it'll still look pretty for the customers.
Re:Our company is using XML for all documentation. (Score:2)
UML (Score:2)
That one gets fed into a custom repository system which automatically generates all the code we need for using the objects, including javadocs and XML documents, when then go into the rest of the channel for documentation. The repository system also generates multiple types of storage back-ends (Oracle, Postgres, Java Serialization, In-Memory-Only, etc.) so that the engineers never have to worry about data access code, and we can change what type of storage we're using at a customer site.
So in that respect, we're using Visio or MagicDraw just for the documentation of the UML model, but the "real" model is always in XML and only used by the repository system we've developed.
UML in XML (Score:2)
Sounds like a nice system -- and a well organized team. You hiring?
The docs infrastructure person pipes up (Score:2, Informative)
This is what we do. It works, and it solves many of the issues associated with traditional documentation "systems".
1. Define a DTD of the elements you will need in your documents. You can go with a predefined DTD like DOCBOOK, or you can roll your own. Because my goal was to get engineers to write as many documents as possible (thereby making my life easier), I chose to create my own DTD - There's about 60 elements. Special documents (like projects, or resumes, for example) have their own
DTD (and XSL) in order to keep the main documentation DTD as simple as possible.
2. Create a general XSL sheet that transforms all
documents which conform to the DTD into HTML for your internal web-site - all of the presentation
logic is in this one XSL sheet (although it can use xsl:include if you'd like to break it up by header, table of contents, content, footer, etc).
3. Set up documentation sub-branch(es) in the source control system. (In the
code base and QA/infrastructure/whatever else you plan to document and publish). The closer it is to the code, the more likely that engineers will add reviewcomments, fix errors, update it, etc.
4. Anyone and everyone writing documents does so
according to DTD and checks them into the designated directory structure in the source control repository. (As far as they are concerned, this is the end of their work -- it just magically appears on the site)
5. Set up Ant to use one of the Trax processors (I use Xalan2).
6. Write general purpose ant targets to convert
documentation with general XSL sheet, build indexes, etc. Write build.xml files that call these tasks and convert/index/etc all the documentation in each sub-branch.
7. A monkey or chron-job converts the documentation and pushes it to the website.
Benefits:
Tables of Contents, Index pages, indexing, image-links, etc. are all generated. There is a much lower chance of links being broken.
Once you get the XSL right, you don't have to worry about consistency with the look and feel.
You really can just concentrate on content.
In my experience, engineers are much more likely
to use VI/emacs to edit/review a document than Word or Framemaker. The GUI XML editors are getting better...
Drawbacks:
Someone has to own the DTD, beat their head against the wonderful syntax of xslt, and be willing to decipher the Ant stack traces when things inexplicably bail.
It's not yet commonly done (or at least they don't post on may newsgroups), so often you are charging ahead without much feedback.
Technical writers are difficult to hire. We haven't tried to find another one yet, but I imagine asking people to write in XML will limit our talent pool. Of course... in theory I could make all this work with Framemaker+SGML...
Introducing XML documentation in the company (Score:3, Informative)
A way to introduce XML-based documentation in a company is to prove what it can do (and not just speak about it). In a previous company, I expressed the desire to have a documentation generated from the source code, but nobody seemed really interested. So I did it myself, and when they saw it, they loved it.
The idea was to parse our source files (which were in various languages, more or less easy to parse) and generate an XML documentation for the APIs. In a second step, other programs read the XML documentation and transformed it in RTF (Word) and HTML, using SAX and XSLT (I tried both and preferred SAX).
The HTML version was installed on the Intranet and the developers used it as a reference documentation in their everyday work. They knew they could trust it more than any other documentation, because it was regenerated every night. They also liked it because, unlike Javadoc, the source code parser worked very hard to gather information from the code without forcing the programmer to use constraining comment conventions.
The Word version was delivered to the clients as an API documentation.
Other documentations were written directly in Word. The system worked very well, and ensured a good-quality and up-to-date API documentation without too much work.
I also used the intermediary XML documentation for other purpose, including some code generation, which proved the versatility of XML.
Try Conglomerate (Score:4, Informative)
More worthy, full document management system than the efforts being put into word processing.
Conglomerate [conglomerate.org]
Re:Try Conglomerate (Score:2)
But I have to say - check the docs section for an interesting quote:
Now surely this is a terrible inditement of a system that is intended to be used as:
Re:Try Conglomerate (Score:1)
XML Document Architectures (Score:5, Informative)
- Plain XML, without schemas
- Plain XML, with schema (or DTD)
- Database -> XML
- Repository -> XML -(XSLT)-> HTML
- Repository -> XML -(XSLT)-> XML -(XSLT)-> HTML
There are of course variations. Check out IS Architectures - Organizing the Web Server [martinfowler.com] for more details when one of your outputs is HTML.XML is a markup langauge that is supposed to be human readable. Thus anyone can whip up an XML document that describes some data (e.g. documentation on software). It helps if you have standards to make the XML consistent.
Creating schemas for all you different types of documentation is probably the first big pain in the butt you will deal with, but it is pretty essential to get a project like you describe to work. It helps by setting common standards which all participants in your org can use to understand the docs they are looking at. Now you also get some tool support for creating and validating your XML documents.
Store all your documentation data in a database and use common db tools to extract it and format in XML. Why bother? Tool support! Lots of software development project tools support using a db as a repository for the various work products (documentation and code and stuff). This also allows you to have somewhat easier methods for serving your content to interested parties with appropriate security constraints.
Here we add the ability to transform the human-readable-but-cumbersome syntax of XML into html for viewing on a browser. The big effort for this sort of architecture is that you have to create the XSLT for all your different document types and you need some way of linking-to/searching your documents from the html into the repository. Some application and web servers help with this. I'm most familiar with the Java space, and Tomcat with various xml libraries can be made to do this.
This is the most flexible architecture in which pure data XML is transformed into an intermediate form which represents an abstract presentation of the XML and which is then transformed into HTML (or WML or PDF or whatever). The first stage of transformation you need one XSLT style sheet for each document type to convert it into the presentation XML. Then for the second stage you need one stylesheet for each display format. The big advantage here is that if you need to publish to a new document format, you don't need to re-write _all_ of your first stage transformations, you only need to add one new second stage transformation.
Why XML? (Score:3, Informative)
A good place to start is Open Source XML Database Toolkit [wiley.com] by Liam Quin.
The key point is that the best approach depends upon how the data will be accessed, used and updated. There does not appear to be an off-the-shelf, one-size-fits-all solution, even if you go to a commercial platform.
The advantage of XML is that you can start with a simple approach and migrate to a more complex approach without having to do an expensive data conversion.
The disadvantage is that XML can be quite expensive to set up on legacy documents and expensive to maintain as well. For documents that change frequently, have multiple uses, or require precise retrieval strategies, XML is the way to go. It's particularly useful when version control must be tracked at the paragraph level.
If version control takes place at the level of the whole document, retieval is done by keyword, and documents are displayed in one form only, XML may not add anything but trouble.
Process follows team (Score:1)
Let the team gel together. When you see a problem, then bring in some process and tools. Just as with when there is a conflict between the law and custom it isn't law that wins, so it is with process and tools dictated from on high. Process and tools need reasons for existing. By introducing them later, the purpose is clear to the team lead, and more importantly, the purpose is clear to the team. It also allows team members to bring to the table their favorite tools and processes, and the team lead may learn something.
It sounds reactive rather than proactive, but as with XP, it's really a meta-level up where you're planning ahead to be reactive, and you have a quiver of tools and processes at the ready.
Files Easy, Editing Hard (Score:4, Insightful)
I still maintain that the biggest hurdle in any standardized document system (especially if you include multiple concurrent authors) is the front-end editor. I wrote a simple (and highly buggy, I'll admidt, so you who know me keep your traps shut!) VB application that provided a multi-user front end to a database. The back-end (PHP) pulled all the appropriate rows for any given doc together and mashed it into a nice, navigable HTML document. I even had PDF support at one time (but it was even flakier than the GUI).
However, it was not XML, so it was REALLY limited in how easy it was to create new views on the data. The biggest problem I ran into was trying to find a good GUI editor -- this thing was written for security engineers, not HTML experts, and I wanted them to concentrate on content, not tags. I eventually settled (and settled is the right word) on the Microsoft DHTML control. Worked well enough for the time (two years ago at this point), but I still think half my problems stemmed from that widget, or bad interface programming to it. The advantage? WYSIWYG (more or less) editing. Seamless multi-user editing of the same document (well, okay, we had some record locking issues.
Since then, I've wanted very much to rewrite the thing to handle full XML, and I understand there's an effort underway to do just that (I've since moved to different pastures), but it's slow going. I've looked at current technology (ABIword, for example), and i'm just not convinced that it's going to be easy to get a good semi-WYSIWYG XML editor going. At least not on the cheap.
Some time ago was posted here an app called Conglomorate [conglomerate.org], which I still think has about the best approach to visually representing an XML document. But it hasn't been updated in forever, and was slow/buggy the one time I played with it. More recently, the XMLmind XML Editor (XXE) [xmlmind.com] has shown a lot of promise, even including CSS files for editing DocBook XML. They even have source available. Again, goes a long way to letting you edit diverse XML files in a logical way -- not by forcing you to look at ugly tree-views of an XML file, like so many first-generation editors. Finally, the latest XML Spy [xmlspy.com] editor beta goes a bit father even than XXE, using a full XSLT transform to provide a WYSIWYG format for XML files. Theoretically, with this, you should be able to display any of your documents in whichever approach you like -- full WYSIWYG, tables, trees, block labels, whatever.
Of course, neither of these latter tools work in a concurrent editing fashion. But that's a "minor" enhancement -- put together a robust DB back end, allow for good record locking, editor-to-editor communications for lock management, transaction log to allow back-out of changes, etc. Lots of possibilities. Take XXE, put this kind of capability on the back-end, an integrated login and document management system, and you've got a kick-ass document solution. Work the backend to allow for multi-stage review and publishing, and provide output engines for HTML, PDF, WAP, whatever, serve different subtrees of the system to, say, internal project web servers, external web servers, sales and marketing (for glossies), etc., and everyone can manage everything, real-time, GUI, with one tool.
But I dream.
(seriously -- if anyone's really working this, I'd love to help. I just wanna use it at home for my own web pages.)
Re:Files Easy, Editing Hard (Score:2)
DocBook (Score:1)
I think that the most important thing you can do though is make certain that people do not have to edit XML directly.
Create something that allows them to work without remembering all the tags and brackets and everybody will be happier.
Use a wiki (Score:1)
Re:Use a wiki (Score:3, Informative)
Forget CVS; start a Wiki! (Score:3, Informative)
The Wiki Wiki Web is a set of editable, cross-referenced web pages. Anybody can view them and anybody can edit them, and they are searchable. Wikis are pretty useful for internal documentation projects. It should be possible to extend the concept to add the security that is typically required and to add support for XML. Of course, all that means I am practically suggesting you write your own custom Wiki, which may take too long for you. But you could probably start with an existing Wiki and get good results. I have set up UseModWiki [usemod.com] (which is a CGI script written in Perl) and gotten good results.
Hope this helps!
Re:Forget CVS; start a Wiki! (Score:1)
Editing wiki pages looks like a HUGE STEP BACKWARDS from the editing that most of us are familiar with. I can't imagine trying to get more than one or two of the people where I work using it, and how much good can a collaborative tool do if most of the intended collaborators won't?
One word ! Wiki ! (Score:4, Insightful)
Re:One word ! Wiki ! (Score:2)
Re:One word ! Wiki ! (Score:2)
we use XML for our knowledge base (Score:3, Informative)
As soon as XML standards and derived technologies and languages (XSLT, DOM, and more) started to be strongly established nearly 2 years ago, we moved whatever existing documentation we had into XML, conforming to internally developed DTD's and specs, after a couple guys and I built a handy HTTP-based authoring tool that leverages technologies built-in Internet Explorer 5.0 which I've previously described right here [slashdot.org], allowing writers to not have to know anything about XML, and simply click their way thru easy interactive forms, in a fairly compelling user interface
With all of our information stored in XML, we can easily present it to various audiences, may it be our members who can search it by keywords to help themselves in our online support area or our technical support reps who can browse directory trees to specific XML documents and have access to more detailed information about hardware and platform configurations, document revision information and more.
The bottom line is this system works really well. And we have the amazing peace of mind of having GREAT information in a format that can never become completely obsolete, and that is always a couple XSLT [w3.org] stylesheets away from fitting just about *any* need.
Whether you make up your own DTD's or follow existing standard DTD's like DocBook mentioned in other posts, as long as you put some thought into structuring your XML data at the beginning, you can only win in the long run: XML documents can easily be processed into other XML DTDs/formats to represent the information in a way that better fits another application, and/or transformed into other documents made of a markup language meant for presentation like HTML or WAP.
yea. XML is nifty.
Similar problem here... (Score:2, Informative)
At my company (in fact it's a local branch of an US based corporation) we have similar problem. There is a team here developing a system designed specifically for a customer. As one can expect along with such a system goes all the documentation - everything you could expect starting from the analysis, through functional specification and coding guidelines to end user and administrator's manuals. To make things more complicated part of the development - and the documentation - is being done by a subcontractor (which happens to be on another hemisphere) - and it is being prepared in English, but some parts of it (especially the manuals) have to be translated into local language.
Up until now it has been a growing mess with documentation being written in Word (with all the usual problems Word has with large files, with lots of graphics - screens, no versioning etc.), with no standards, with people getting into one another's way while trying to update the numerous documents.
Recently together with a friend we have came up with the idea to switch all that into neat XML/SGML files, with CVS based versioning and everything based on open standards and free software as much as possible. To our surprise the management liked the idea and we got a green light to do some research. And then the problems have begun.
First, the editor. Coding XML files with vi or alike might be nice for a hacker - and is great for creating and testing XML formats used then for data storage etc. - but it is out of the question for documentation authors. And it is pretty understandable - to be able to concentrate on the content, on the text itself, the author needs to see only the contents, as nicely rendered as possible - no tags getting into way in each sentence, no learning for years how to use the editor (thus Emacs with its psgml mode is not an option - don't flame me, it's just a fact). After a long search I have to say that there is no working, finished GNU/free editor that would match our requirement of almost-WYWSIG presentation of an XML/SGML file. As to commercial ones the only two that look good are XML Spy 4.0 [xmlspy.com] - but it is just a poorely working beta for now - and Arbortext's [arbortext.com] Epic [arbortext.com] - which is almost exactly what we need, but is a bit expensive at around $700 a license.
Nevertheless, with no other options left we decided to go for the Epic when it comes to the editing side. We got an evaluation package and begun testing.
Now, we were from the start convinced that DocBook DTD [docbook.org] & tools that go along with it are the best choice for the kind of problem we faced. Epic supports the DocBook but comes along with their own version, which in turns doesn't work well with the Linux sgml tools that we use for translating the XML/SGML files to useful end formats. On the other hand not all Epic's features can be used when one just tries to edit the document based on an "external" DTD. To enable things like being able to see the graphics files inserted into the document one has to hm... "customize" the Epic by creating some additional configuration files (like .FOS files) using yet another expensive tool Arbortext sells - the Epic Architect [arbortext.com].
But that is not the end of the problem, because the stylesheets currently available for translating the Docbook based XML/SGML files into useful formats are not well documented and partially don't work (for example tags related to inserting pictures in the document are ignored when trying to generate a printable document). There is for example a project on Sourceforge [sourceforge.net] that develops XSLTs and DSSSLs for translating Docbook based XML into various formats, but so far I was not able to make them work - and there is no documentation. Also the DSSSL based machinery for translating SGML files that comes with various Linux distros is far from perfect - HTMLs are generated mostly OK, but printed documents (.tex and .pdf) leave much to be desired.
So, from our point of view it looks like we will have to buy an expensive editor and then someone would have to spend a month or so tweaking the editor, modifying the stylesheets for our needs, developing procedures and so on. And that someone would have to be quite a competent person (with deep knowledge of the subject), someone, who could be probably better used directly in the development project.
As for now the future of our little plan of switching from mess to neat XML based solution is uncertain. Mainly because we would have to build that neat solution ourselves, as what we can get from outside at the moment are some bits and pieces that - although nice by themselves - just don't fit together.
(And, BTW, I haven't even touched the nice catch with CVS - to be really useful in the kind of environment that we envisioned it would have to be integrated with the editor - and that doesn't seem likely).
Re:Similar problem here... (Score:1)
take a look at my comment [slashdot.org], you could basically spend a couple days to a coupl weeks max building a trivial to complex tool to edit all your documentation
-c
Link CVS yourself (Score:1)
XLink (Score:2)
biggest barrier in a small shop/startup (Score:2)
The biggest barrier I've encountered in small or startup environments to something like this is organizational "buy in". For example, at $workplace[-1] I wrote a defect tracking system custom tuned for us. Worked pretty well on a technical level ... but nobody used it becuase with < 10 people, it was easier to just turn around and hand somebody a postit-note bug report.
Then again, that is characteristic of one of the central challenges facing a small organization, namely how to grow the structures to make a larger organization maintainable. I.e. overcoming the "this is a pain in the ass, why do we need this now?" factor. The answer to this depends strongly on the people involved, but if you can make the system about as painless as typing into an ASCII file or scribbling on a postit, you stand a better chance of success.
Use XHTML (Score:2)
You really don't want to get involved in building browsing and editing tools for an arbitrary schema, its not worth the time.
XML is huge threat to Microsoft (Score:2, Interesting)
Microsoft have been very clever getting where they are today. One of the principal means they have got there is using the interfaces between different functional elements like keys, to lock customers in, and lock competing technologies out.
XML is a simple, standard way of formatting diverse information types so that it is easy to exchange data between applications, and easy to write programs for. It is brilliant in it's simplicity, and anyone who has studied it will know that it is 'not just another format', but one of the most important standards ever developed in the history of computing.
This represents a huge threat to Microsoft as it threatens one of their main strategies. I believe that everyone in the open source world should learn XML and it's associated standards, and use them as far as possible in their programming work. If, for instance, the open source community adopts DocBook, or Sun's forthcoming XML standard for documents, for all open source word processors, I don't think it will be long before there are so many useful document manipulation applications available that there will be a compelling business reason to move from Microsoft Word to an open source alternative.
Learn XML folks!!!
(There are probably a few reading this who are thinking - 'but Microsoft says that their office file formats are now XML based'. To you all I can say is that you should learn XML, and then you'll realise that what Microsoft is doing doesn't really have anything to do with what XML is all about).
XML/XSL - You know it makes sense (Score:1)
We solved the interconnection of the files by having one uber XML structure file that works like a tree contents so you can navigate around because the XSL generates links to the other documents as layed out in the XML file using the document().
Get into XSL it's loverly.
As an aside we even have an XML that contains directives to run external XSLs on the source XML and stitch the results back into the original XML and re-apply. It's a great way to expand things.
We use it for website construction.
Editor? (Score:2)
Clearly having the document stored as XML is superior because you can convert it to whatever you want, and the tags represent the document's meaning rather than its appearance. But producing the document using raw tags is not friendly; it discourages documenters.
What editor do you use?
Re:Editor? (Score:1)
Tree Views are my bane (Score:2)
We need something better. We need an interface like kword or abiword or lyx, all three of which offer:
but not a single one of which offers docbook/xml import, which is critical if you are writing documentation in XML. XML is designed to be the source, the font from which all formatted, renderable versions (read: pdf, html, rtf, txt, etc.) spring. Hence, it must be the saved version, the one you keep in your source/document control system. The others are just conversions from XML. Yet no editor imports XML! This ought to be the easy part! You just write an XSL stylesheet to convert XML-of-your-choice into your editor's internal format.
XML: what problems should you expect (Score:1)
OK, time to play the devil's advocate (I am actually a big XML fan): here are the problems you should expect when switching all your docs to XML:
In short XML is probably a good choice, it gives at least independance from the word processing software and allows you to include/retrieve data automatically in/from your documents, but don't underestimate the trouble you will have to go through to get to your "integrated-wonderful-all-encompassing system" (which doesn't exist right now, so you will get it in version 1.0...
How we do it around here: (Score:2)
Everything is the only solution (Score:2)
I am preparing to start a new project. I may have to install [Everything] first to see if it will work. Just have to make a couple of rules like all requirements must have a [Requirements] link so that they can all be found. Or better yet, instead of using Person, Place, Idea or Thing you could use Requirements, Use-Cases, Test Cases etc... Interesting idea. Customize [E2] to be a requirements/documentation management system.
Re:XML and CVS (Score:2)
What you really need is a source code system that can recognize that the input is XML and convert the document to canonical form before applying the diff. Unfortunately you would have to write code, the C14N for XML is, well utterly unsuited for that task.
Alternatively you could switch to a source code manager that used compression across the different archived versions rather than a simple diff. Unfortunately that probably involves writing the code yourself.
Re:XML and CVS (Score:2)
Re:Nobody Uses XML (Score:1)
One of the great things about the Unix philosophy is the notion that programs should use text for their input and output. This allows for programs to be piped together, and large problems can be solved by combining many small solutions.
As data becomes more complex and is shared among many different machines and architectures, straight text files are no longer sufficient. This is where XML comes in. Not only is it compatible and universal across machine types and OS's, but it is also able to be shared across any program.