Netscape Dumps Critical File, Breaks RSS 0.9 Feeds

An anonymous reader writes "In the standard definition of RSS 0.91, there are a couple of lines referring to 'DOCTYPE' and referencing a 'dtd' spec hosted on Netscape's website. According to an article on DeviceForge.com quite a few RSS feeds around the web probably stopped working properly over the past few weeks because Netscape recently stopped hosting the critical rss-0.91.dtd file. Probably someone over at netscape.com simply thought he was cleaning up some insignificant cruft." Some explanation has been offered by a Netscape employee.

Ack

I would've seen this post sooner, but my RSS feed was broken... something about a 404?

Why would this break RSS readers?

I don't see how this would break RSS readers. DTDs pretty much never get read except by validators. Normal SGML and XML parsers just treat the DTD URL as an opaque string, not as something that can be retrieved.

Re:Why would this break RSS readers?

Actually the DTD is loaded up by pretty much every proper XML library even if validation is "off".

The DTD contains more than just the element definitions and hierarchy. Its also used to define entities (&...;) that are non-standard to XML but may be expected in the file. HTML has tons of pre-defined entities but XML only has the core 4. All others are defined in DTDs and loaded on the fly as part of the processing.

There are ways to turn it off at the lowest levels, but higher-level abstractions/libraries might not give access to that. For example, with JAXP + SAX you can turn off DTD loading, but Jakarta Commons Digester doesn't give a setting where you can trigger that, so Digester tries to load the dtd, and even with validation off you can't change that. My only recourse is to take the DTD lines out of the various config files. (Reason: My JBoss server is deployed in private networks where the server can't reach the internet).

Re:Then they're broken!

Fetching the spec is idiotic.

First of all, that's a needless operation. It can take time; don't forget the DNS lookup and all.

Second of all, it's not as if you could handle any random DTD. Software doesn't work that way. (this is one of the reasons why XML itself is a mostly-lame idea) If the XML doesn't match expectations, you can't convert it to your own internal representation. You probably have a C struct that you need to fill in. Even in some wild interpreted language like perl, you just won't have any use for unexpected data structures and you damn well need the expected data structures.

the probably web

what is the probably web?

Ugh

According to an article on DeviceForge.com quite a few RSS feeds around the probably web stopped working properly over the past few weeks because Netscape recently stopped hosting the critical rss-0.91.dtd file.

STOP, Grammar time. Ooooh whoooaaa oh oh...

Probably someone over at netscape.com simply thought he was cleaning up some insignificant cruft."

Or Netscape got tired of people using their bandwidth. Regardless of the reasons: if you reference a file on someone's site, it's hardly their fault if they move/change/delete it, and it breaks your stuff.

The point...

Is to have a common component shared among many documents without replication.

Class paths is java are the perfect example to say how it *should* work. Java CLASSPATHs in every application/installation I have seen are site-local, all paths accessible without going over the internet to another site to get classes.

To be similar, an RSS site should copy this DTD to their local server, or to a server with which they have a concrete understanding of the relationship. Either a commercial agreement with a peer or at least using a server from an organization who explicitly defines the purpose of hosting to be a common place to promote it as a standard.

Did netscape promise itself to be an organization sharing that DTD explicitly, or did site developers get in the practice because 'it just always was there'?

DTDs are different

A DTD (document type definition) is a file that describes how an SGML document is structured. In this case, the DTD that went missing defines RSS 0.91, which is used by Navigator 4 for "channel" subscription.

It is expected that DTDs are hotlinked. For example, if you ever look at html source of a web page, you would see:

<!DOCTYPE ...>

on the top, and the hotlink goes to somewhere on w3.org. That is because W3 is the authority body that defines the html.

Since Netscape is the authority body that defines RSS 0.91, it is a bit strange how they stopped hosting the definition.

In any case, the missing definition won't affect software that processes RSS feeds. It only affects software that checks whether a SGML document is structured properly according to that missing DTD.

The main interest to this article seems to be the speculation how a deprecated web 1.0 company could end up hiring a clueless webmaster who deletes important files without recognizing its importance.

Obligatory Lamport quote on distibuted systems

"A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable."

SPOF?

Or maybe some smart person at Netscape decided to teach some people a lesson about using a 3rd party as a single point of failure?

Doesn't AOL own Netscape?

And if so, why would anyone rely on AOL to make something on the web work?

HAHA

Suck that, Web 2.0!

Host it yourself!

This is the precise reason why I host everything myself including my own series of tubes, dubbed the Internets. I host not only every file that my site uses, but I also have a program that regularly crawls the entire Internet and compresses it onto my own distributed system. That way I can browse the Internet by myself without worrying if someone else's system will fail. Although I do need to replace systems every now and then. But that's not a problem, b/c the distributed system has 3-5 copies of the Internet, each copy in a different place. Wait, isn't their some other company that does that? I can't quite place the name.

Seriously though, relying on some other system so your site will work is a recipe for disaster. It's similar to relying on someone to take you to work everyday. After a while, you get used to that fact that someone else is driving you that you don't even think about it. Then your driver gets deleted somehow. And you're stuck with no way to work.

So let me get this straight

Whenever someone accesses a RSS file, Netscape would know the IP for every access? How stupid can that be? Why don't the readers just cache the DTDs and fetch only if there's a problem?

This is a major problem,

This could seriously affect both of the guys using Netscape.

W3C doctype

How is this any different than the W3C doctypes ala

? Most web tutorials tell you to use the remote file....but if it ever goes down what happens? Is there a reason for it being taught this way?

Netscape Says No RSS 0.91 For You

From April 2001, "Netscape removed the RSS 0.91 DTD from their website. This means that all RSS feeds which depend on the RSS 0.91 (many, MANY news sites) cannot be used with a validating parser." [slashdot.org]

It seems as though it just took them 5+ years to follow up on the threat? Primary links are broken, but of course the lively /. discussion (which, um, I haven't read) remains.

Sorry about that

my.netscape.com is undergoing a redesign, and when we announced the redesign [netscape.com] about 10 days ago, the DNS entry for my.netscape.com was changed to point to the new server where My Netscape will be living. This had the effect of making anything under the old my.netscape.com unavailable, since the only thing public on the new server is a splash page. (Nobody on the team was especially aware of this DTD file since all of the old Netscape employees were let go last year around the time Netscape.com was redeveloped; anybody working at Netscape now was hired since then.)

Now, why this file was living under my.netscape.com is anybody's guess, but we'll have it restored ASAP. I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

Christopher Finke
Netscape Developer

Re:Sorry about that

What's the official way to let you know about this sort of thing? I'm not trolling -- the better you can inform folk like us about how to interact with you, the more likely it is you'll get a response when you need it. For example, a quick scan of the Help and FAQ pages linked to off of the Netscape home page shows no mention of how to contact folk like you.

Re:Sorry about that

What's the official way to let you know about this sort of thing?

You're correct that contact information appears to be MIA in the Netscape Help pages; I'll make sure to remedy that ASAP.

For something as serious as this, a user could have checked the profile of one of the Netscape Anchors or developers, where many of them list their screennames or websites, and subsequently, their e-mail addresses. (At least, I know I do [netscape.com].) Alternatively, any Netscape.com member could use Netscape sitemail to contact any of the staff members. Obviously, these are unacceptable for normal circumstances, but I wouldn't call this situation a normal circumstance.

You gotta be kidding me...

This blast is not squarely aimed at you, but you triggered it. Treat this in the spirit it is meant please (if I didn't give a crap at all, I wouldn't comment. Show this to your insulated bosses who don't know the first thing about community and transparency. Kudos to you BTW for showing initiative and acting on a Slashdot post. Honestly, I would not have given the "new Netscape" that much credit.).

>I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

This rankles.

Have you EVER tried contacting Netscape from the outside world? Seriously, I can count the number of times:

*) When my.netscape.com locked out Konqueror (1998?)
*) When my.netscape.com WITHDREW the ability to embed RSS feed on your "my" page -- actually this was PRE-RSS if I recall. Way before it was commonplace, you could embed Slashdot and Linux Today feeds. Then they killed it, presumably because they got enough users or some pointy haired reason. 1999.
*) When my.netscape.com adopted a shitty policy of DELETING all your mail if you don't login for 30 days. This did not seem to be publicised by an actual email. They don't seem to delete the mailbox itself, which violates RFCs I'm sure and basically insinuates the mailbox is active. I lost tons of mail from 1996-2003 (yeah yeah backups. Some things I didn't think I would need later). ?? Happened in 2003. Note that mailboxes were only 5MB still, so I quickly bailed for a 100MB Yahoo account.
*) The 2001 deletion of Netscape Developer. This lost a ton of Netscape copyrighted Javascript documentation.

Just TRY contacting Netscape from their page. The best you can do is use the WRONG FORM to submit to some contracter who won't forward it. Or, oh yeah - there's a 900 number for by the minute Support.

Back when it mattered, there was no 'Google Guy' for Netscape, who would act as an unofficial liason. After Jamie Z left, no one internally tried to fill the shoes of a community facing employee.

While I'll be eternally grateful for Netscape's open sourcing of their browser. What a different world it is now. Too bad that step is something the current management would never have allowed (that's the perception). I can't think of a more opaque Internet company than today's Netscape. I'm sure there are people who disagree or wish it could be changed (you're here..) but that and a $1 gets you a cup of black coffee. Show this to your boss - there are suggestions here :-)

Re:You gotta be kidding me...

You make several good points that I want to respond to more fully, but I've got to run out, so I'll have to do that later. In the meantime, I'll put this out there: my e-mail address is [e-mail] [mailto]; my screenname and other contact information is available at my website [efinke.com]. Anyone who wishes to do so can contact me regarding issues with any of the Netscape websites or the Netscape browser; if I can't solve your problem, I can definitely get you connected with the right person.

Re:Sorry about that

(Nobody on the team was especially aware of this DTD file since all of the old Netscape employees were let go last year around the time Netscape.com was redeveloped; anybody working at Netscape now was hired since then.)

Now, why this file was living under my.netscape.com is anybody's guess, but we'll have it restored ASAP. I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

Ummm, maybe I'm mising something here, but I would think that your web log would show a spike in 404 errors for this file, right? In my experience, it is helpful to assume that I do not know what I don't know, and to put procedures in place to help make those omissions stick out. So, a scan of your log files not only for this file, but for any others that also have a high number of 404's (especially from a multitude of referers) would be worth investigating.

BTW, best of luck on the redesign!

Re:Sorry about that

Hi, Christopher. First off, full marks for stepping up and explaining things honestly. You have done more good for Netscape than a dozen PR people. I'm sure you'll take a lot of crap from my fellow Slashdotters, but don't let it throw you. Listen to and acknowledge their legitimate complaints and you'll do fine.

I only wish that someone had brought it to our attention so that I didn't have to find out about it from Slashdot.

If you are looking to learn a lesson from this, how about this one: URLs are forever!

Whenever I make a change to a live server, my biggest concern is to not break existing usage. If I ever change an URL, I make sure to redirect old usage to new usage that's just as good. And if I'm ever not sure something is used, I generally look back at least three months in the logs. Especially if you've inherited a pile full of mystery, good analytical tools for your server logs are vital. Trying to run even a modestly-sized site without them is like running a large store without tracking your inventory: your life will become a series of unfortunate surprises.

Re:Sorry about that

URLs are forever!

Indeed, words to live by. I wouldn't pin this mistake on one person not checking the right logfile though; in a company as large as AOL, when an entire 150-person workforce is laid off and a new (much smaller) team is brought in to manage the old properties, things sometimes get lost in the shuffle. The entire my.netscape.com service happened to be one of those things. I'm sure that this incident will act as a reminder to never let this type of thing happen again.

And BTW, it appears that the DTD file will be restored early tomorrow morning at the latest.

Huh?

Netscape? What's a Netscape?

Seriously bad programming

If I would create a reader that was dependent on version 0.91 of the distribution, it sure as hell would include the DTD in local storage. It makes no sense to create a reader that can also use, say, version 0.92 since you would not know what had changed (and there is no such thing as inheritence between versions of a DTD afaik). Actually, as other readers noted, it would be terribly stupid to make your web-server or client rely on a third party computer for which you cannot guarantee the uptime.

These URL's are mainly there for their Uniqueness, not so much as for a place of quaranteed storage. Of course, they are also a nice place to look for the actual definition, but after that you would need a local repository. This is the first thing an XML library should support, and the first thing a moderately intelligent programmer should look at. I get *very* annoyed if this kind of basic rules are ignored. And I've even seen them ignored by people pointing to the XML digital signature definitions, where security and reliability should be the first requirements in the design.

Also, what would happen if w3c.org or netscape.com go the way of the Gopher? If they go bust? It's a quickly changing world out there.

Logs?

not trying to be a troll here.. but.. one would think that that file would have been accessed quite often and that would have shown up in the logs...

If I was a new hire at some old company where everyone else had been let go, I'd at least check out the logs and see what is being used? and then if some file is being hit 1,000's of times a day.. maybe ask a few questions..

IE7 is already broken anyways

Last time I looked, if the RSS 0.91 feed references the DTD, IE7 refuses to display it anyways... it's only of the DTD references is removed that IE7 "works" "properly"

A rewording...

Once upon a midnight dreary, while I websurfed, weak and weary, Over many a strange and spurious bookmark of 'free news galore', While I clicked my fav'rite feed, suddenly there came a warning, And my heart was filled with mourning, mourning for my dear amour. "'Tis not possible," I muttered, "give me back my free news source!" Quoth the server, "404".

127.0.0.1, anyone?

Anyone remember the incident, maybe 8 years ago, when the root DNS servers dropped the entry for localhost, 127.0.0.1? We had a lot of random code break because of that.

Deja Vu (Super Dupe?)

Wow.

I'd swear today I was looking at the /. frontpage from 4+ years ago...

After a little searching, I've found the exact same story, from April 2001:

http://slashdot.org/article.pl?sid=01/04/28/211921 1 [slashdot.org]

Those who forget history are doomed to repeat it.

I look forward to reading this story on /. again in 2013.

Content based addressing

The web needs some scheme for content based addressing. Like the urn:sha1 scheme used in gnutella. This (and some sort of reasonable caching scheme) would do a lot to alleviate problems like this. It could also help a lot with the Slashdot effect.

why don't...

RSS readers just cache the DTDs and refer to them locally instead? It's not like they're going to change or anything...

This Explains Why People Hate the IT Department

Referencing the other topic today...

You mean to tell me that every RSS reader references - and actually tries to FIND and DOWNLOAD - a specific SPECIFICATION hosted on ONE SITE ON THE PLANET?

Are you people utter fucking morons or what?

I can't believe design decisions like this.

I'm especially irritated because I have just spent the last week trying to find an rdiff-backup or rsync that functions on Windows WITHOUT A FUCKING 2GB or 4GB FILE SIZE LIMITATION! Even the Cygwin people could only tell me to "try it"...How about putting it in the fucking FAQ what the file size limits are?!

We HAVE LARGE FUCKING FILES THESE DAYS, MORONS!! We've HAD THEM FOR FIVE YEARS!!

Oh, nobody can be bothered to support librsync anymore...right...

Only Fedora Core 5 bothered to patch it so rdiff-backup can handle large files...

Backup isn't "sexy" like 3D wobbly windows, so nobody cares...

Chimpanzees. Jesus Baron von Fucking Christ!

re:netscape rss

OMGZ! Netscape is still around?

Catalog files...

I'm no RSS expert, and I don't spend that much time dealing with it, honestly.

However, I do spend a LOT of time dealing with XML. And many environments offer quite an elegant way of dealing with this: the catalog file.

The catalog file provides an alternative mapping between the public identifier and a local, system referenced file SO THIS SORT OF THING DOESN'T happen.

If the RSS standard is to be dependant on external doctype definitions, the RSS readers should either:
(1) Include the doctype hosted on their own system and change all of their doctype public URI's or (2) The RSS readers should include the requisite files locally, and contain a catalog that maps the public URI's to the locally deployed model.

Like I said before, I'm not so into RSS, so I'm not sure which is applicable, but certainly, I don't think that Netscape should have to host this...

Resolved

The file is now restored, but it will not be available forever. See this post at the Netscape blog [netscape.com] for the full details.

Re:Hmm....

Welcome to the new Quantum Web Computing Experience. Blink again, and there's a 50/50 chance it'll be back to normal - or your computer will disappear.

Incomplete Quote

I believe the quote you're looking for is:

Damn you, Bill Gates! Damn you all to DRM!

Re:Bull

"RSS readers don't do anything with the DTD or try to access it at all. This didn't break anything."

You, sir, are wrong.

XML parsers (which RSS readers use to... you guessed it, parse the XML) will try to look at things referenced in the DOCTYPE declarations.

I guess I can vaguely understand the desire to have the 'authoritative' provider of the DTD in one place, but it's always made me uneasy to have this single point of failure for the potentially many consumers of a DTD.

It's apparently also made many of my employers over the past 10 years uneasy. I've had to take control of this situation many times at many jobs and change it so we host copies of DTDs ourselves (or remove the DOCTYPE altogether from documents).

There have also been times I've worked on software that otherwise did not need to have an internet connection available. This DOCTYPE business is a problem in that case.