Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
DEAL: For $25 - Add A Second Phone Number To Your Smartphone for life! Use promo code SLASHDOT25. Also, Slashdot's Facebook page has a chat bot now. Message it for stories and more. Check out the new SourceForge HTML5 Internet speed test! ×

Comment Re:What do you people expect? (Score 1) 62

Where I used to work, we called this the "Stack Overflow Effect" because so much bad code written by well-meaning people was floating around Stack Overflow that did things in dangerous, security-risky ways, such as telling people to disable TLS chain validation so they could use a self-signed cert for their test environment, then wondering why so many apps shipped with chain validation turned off in the production versions of the app.

I've actually written security documentation whose primary purpose was to provide a single set of code snippets that were known to do things in the right way so that we could plaster Stack Overflow with links to the doc. Then, when people say, "but can't I just...", we can say, "No", and point them atdocumentation explaining why so that at least when they do something stupid anyway, we can say, "Dude, what part of 'no, that is incredibly dangerous' didn't you understand?"

Comment Re:An Insider's View (Score 1) 57

First of all, never call your product a "competitive product". You know what this means? Essentially what you're saying is "the others are just as shitty, so why try harder?" Another thing is that the message is not what you say but what your audience hears. It's nice that you feel like your customer has a seat at your table, but this does not arrive at your customers. They do not feel that way. And if you care about how your customers think about you, this is what matters.

One thing is certain: Goodwill goes a long way, and it takes a long, long time to rebuild from ruins. And let's be honest here, Comcast's goodwill is in the gutter. You have a long uphill battle in front of you if you really care.

Comment Re:The current system is stupid. (Score 1) 154

No, that won't work. Changes may have taken place in-between the two copies of robots.txt.

An example: A newspaper.
At the first fetch of robots.txt, an article might not exist. The first version of it has not yet been verified, and is published with a new robots.txt that tells robots not to crawl it. Then, the article is modified and verified, and a new robots.txt published that now allows crawling it.
Yet, a spider may have caught the first robots.txt from before the article, the article while it was in bad shape, and the second robots.txt from after it was corrected. Both robots.txt files agree that it can be cached, yet the copy that was crawled was never meant for caching, and the robots.txt at the time it was published even said so.

Comment Re:The current system is stupid. (Score 1) 154

The problem with robots.txt is that it doesn't contain a validity period.

Say I add mustnotbecrawled.html, a link to it in existingpage.html, and a modification to /robots.txt that bans crawling of mustnotbecrawled.html. The problem is that a robot might have downloaded robots.txt right before my publishing, and does not see that it shouldn't crawl it. So it does.

It could be argued that a crawler should always re-load robots.txt if encountering a document newer than the last server transmit time for robots.txt, but that adds a lot of extra requests.

Some propose using the meta tag for excluding browsers, but that has its own problems. Like only working for XML type documents. And being applied after the fact. If I have a several megabytes HTML, and want to exclude it to save bandwidth, the meta tag won't work. It adds a little bit extra bandwidth.

I think this should be handled at user-agent level, where crawlers identify themselves as a crawler, and the web server can make the decision on whether to serve them based on that.

Submission + - Second parchment manuscript copy of Declaration of Independence found (bostonglobe.com)

Okian Warrior writes: Two Harvard University researchers announced Friday that they have found a second parchment manuscript copy of the Declaration of Independence in a tiny records office in southern England.

The only other parchment copy is maintained by the National Archives in Washington, D.C., researchers Emily Sneff and Danielle Allen said in a statement.

The newly discovered document — which the two have dated to the 1780s — was found in the town of Chichester archives, and is believed to have originally belonged to Duke of Richmond who was known as the “Radical Duke,’’ for the support he gave to Americans during the Revolutionary War, the researchers said.

Slashdot Top Deals

Intel CPUs are not defective, they just act that way. -- Henry Spencer

Working...