When Your Site Ceases To Exist

El Lobo writes with a sobering account of how Javalobby dropped off the face of Google last month. The site had been attacked by forum spammers and Google indexed some of their spew before the Javalobby guys could remove it. According to a post in Rich Skrenta's blog, Google is now the de-facto front page for the Internet, accounting for anywhere from 70% to 78% of the search market. The power this conveys is hard to overstate. From the Javalobby saga: "We had completely disappeared from Google's main index! If you run a website, then you know how serious a problem this is. On any given day over 10,000 visitors arrive at Javalobby as a result of Google searches, and suddenly they stopped coming! ... Suddenly we no longer existed in the eyes of Google."

Never heard of them before, so nothings' changed

Javalobby? Another slashvertisement ...

Re:Never heard of them before, so nothings' change

OP might have a point that this is slashvertisement. javalobby is on the top for the 4 primary search engines.

http://search.live.com/results.aspx?q=javalobby&sr c=IE-SearchBox [live.com]
http://www.ask.com/web?q=javalobby&qsrc=0&o=333&l= dir [ask.com]
http://search.yahoo.com/search?p=javalobby&ei=utf- 8&fr=b2ie7 [yahoo.com]
http://www.google.com/search?q=javalobby&rls=com.m icrosoft:en-us&ie=UTF-8&oe=UTF-8&startIndex=&start Page=1 [google.com]

Re:Never heard of them before, so nothings' change

yeah, and if you search for KillerBob on Google, my site comes up at the front. If you type my real name, my personal website isn't even on the front page. On the second page, there's a couple of scripts I wrote over 10 years ago, and a story I submitted to BBSpot years ago, but my personal website still doesn't show up. Selection of keywords. If you type the name of any specific site, you'll get that site first. If you type what the site does, you may find that it's much lower on the page ranking. They probably aren't worried about traffic from people who search for the word "javalobby", because those people probably already know about their site.

They're worried about the people who search for terms like "java help", which is what somebody who *doesn't* already know about their site would be searching for. In my case, it's quite deliberate. I'm using robots.txt to tell GoogleBot to ignore my personal website. It's *personal*. All it is is an e-mail gateway, anyway; the blog is restricted access. There's no point in having it in Google, so the robots.txt reduces my daily traffic.

Re:Never heard of them before, so nothings' change

How many days after a site has been transformed by hijackers/forum spammers/whoever into a pile of crap should it come off the top of googles search results? A day? A week?

60 days but you can request reinclusion sooner with Google Webmaster tools [google.com]

Re:Never heard of them before, so nothings' change

Just because they show up when you enter the name of the site doesn't mean they haven't lost lots of PageRank.

They probably mean that they used to show up when you searched for "Java", but because the spambots created so many outgoing links they lost their PageRank and now you have to search for "JavaLobby" to get them.

Re:Never heard of them before, so nothings' change

Dude. Slashdot is the last place I'd want to advertise. Their site will be down in minutes (what with being on the front page, and the article unabbreviated).

Re:Never heard of them before, so nothings' change

Yes, downtime will come and go, but the page rank effects will be everlasting!

Re:Google needs to do more of this.

Well, anyone stupid enough to buy SCOX deserves what they get ... but I notice the #1 article is boston.com (Boston Globe) - the same people who did the hatchet job on Peter Quinn for advocating ODF for Massechussetts [groklaw.net]

We know who was behind THAT one ... Microsoft. And of course they're behind the SCOX stuff ... perhaps this is just another Team99 tactic?

What's the problem...?

I just typed in "Javalobby" in the Google search and their link came up on top. If there was a problem, it looks like it's fixed.

Re:What's the problem...?

I just typed in "Javalobby" in the Google search and their link came up on top. If there was a problem, it looks like it's fixed.

Phew. At least when you're caught in the crossfire in the spam war, it's just a flesh wound. The seem to be on the third page [google.com] of a Google search for java.

Re:What's the problem...?

If you don't know what the URL is exactly, you may pop the name into Google to find it first. That's very important if you're browsing at work and you don't want to pull up a web page of nude chicks serving java in a lobby. :)

Seriously, I know a lot of people who Google first to find the link to the website (i.e., type "CNN" to go to the CNN webstie). Some people are too lazy or ignorant to type out the full URL.

Man, I thought it was bad when I lost 50 places...

My joke site (SSLI: Search for Satanic Lyrics [dimspace.com]) used to be the number one result for "Satanic Lyrics, but about two months ago, ZAP! Gone from the frone page of Google. It's something like number 50 now, so instead of getting... ummm... three visitors a day, I get something like one a week :-)

Re:Man, I thought it was bad when I lost 50 places

I just visited your site just so I could joke around about being your single weekly hit.

Joke's on me and my poor eyes; I can't believe that you are ranked so high up at 50.

Re:Man, I thought it was bad when I lost 50 places

I had a similar, but opposite experience. I started setting up Yet Another Job Site, but I never got around to making it useful (see Click. Hired! [clickhired.com]). Google decided that it sort of liked it for a while, sending some traffic my way. I went from making nothing on my google ads to a few bucks a day. It wasn't much money, but it was fun seeing the traffic come in. Then google decided it was the crappy site that it was and my traffic went back to its deserved trickle. I wrote an article about it with pretty graphs:

What Google Giveth, Google Can Taketh Away [lot42.com]

I should have submitted it for a slashvertisement. :)

The Quick Fix

1. Move all forums to Javalobbyforums.com or equivalent
2. ???
3. Hire 'little people' in multicoloured pointy hats to help generate traffic for your site not that it is now google acceptable
4. Profit!

Quick Fix Redux

5. Have midgets properly proofread all posts

What problem?

They're on the Slashdot front page, I don't think they'll mind being off Google for a little while.

Maybe...

Maybe you should stop relying on a single source for you advertising.

Maybe you should actually monitor your forums. You know, in case your customers need your help or a SPAM-bot goes on a rampage.

Maybe you should actually have a site that people care about so they'll keep coming back.

Maybe you should slashvertise and ... wait, you did that.

If your site is worthwhile, dropping off Google for a week won't affect it that much, and you'll actually have control over your forums.

Re:Maybe...

Maybe you should RTFA - they're not relying on Google for "advertising"
Maybe you should RTFA - they DO actively monitor their forums. They deleted the messages very quickly - but too late, because Googlebot beat them to it.
Maybe you should RTFA - they DO have a site that people care about and frequently visit. But they want people searching for solutions that appear in their FORUMS to find those postings via search engines.

Re:Maybe...

I don't see a double standard, yet. I don't know the GPs opinion on the MS Monopoly.

It's their own fault...

If they could have implemented one layer of security or verification to prevent spambots from registering (similar to phpBB or vBulletin), they would have prevented all this. But they didn't. There is no image verification on their forum registration page. All it takes is a spammer with a source of disposable e-mails such as dodgeit.com to spam your page to hell.

Pay

Did you miss the memo? Google owns your ass now.

This is why people don't like monopolies much.

Is this normal?

The problem is indeed deeper than just a headache for a webmaster or two. Let's face it: just as the desktop software market depends on MS Windows, and a lot of software companies will vanish overnight in case Microsoft introduced a new trick [like, signed - for a price - executables only, or backwards-incompatible API, etc], so the web now depends on Google. Should all the Google system administration team take a week off - and voila, you get no new customers, because they don't know where to go, and you're lucky if somebody from your old clients returns using his browser's history. Of course, there's Yahoo, MSN, Nigma, and a hundred of startups, but all of them combined hardly have the same significance that Google enjoys alone. So let's either keep our fingers crossed and hope that Google will not do anything more evil than it does now, or... heh, I don't really know even what else could we do.

Warning: Site crashes Safari

Pinwheel of death!

This is just stupid

From the title, I thought this was going to be finding a mirrored copy of your website after you stop maintaining it and your host drops you. But being nolonger indexed?? That doesn't make your site dissappear - what a drama queen. Untill Google becomes the only search engine, or becuase a government institution, people need to stop being so dramatic. Websites existed before search engines as far as I understand.

well,

whilst some people may have a point about the *cough* slashvertisment this article has made me think about Google and monopolies, should I now change my search engine of choice because having many players in any market is better or is a monopoly acceptable when they are (pretty much) the best... even if they do sometimes change where, and if, they list sites

Alternative result types?

Maybe this is where Google needs to provide multiple indexing algorithms. The idea by giving different result types ( most linked, closeness to keywords, flashiness, highest rated, totally random, etc ), this would make it harder for site spammers to know which algorithm to be targeting.

The Wahbulance is on it's Way

I refuse to even click the link. This site, based on what I see here, deserves anything bad that happens to it. Millions of sites see their traffic rise and fall every day. And none of them take up our valuable time to post a sniveling bitch about it to the front page of Slashdot.

Well....

That'll teach em'.

Google = Advertisement

Could they have prevented this? I think so. But let's face it: Google results are a great way to advertise on the internet. Do you find the products you're looking for on banner ads from other sites? I always use google to search for products and services that I want to find online.

It's back...

Good news somebody, it's back on google.

Opinions are like diapers...

I don't care f'r Google for personal reasons undisclosed, so I don't use their products.

They're not MY de facto site, nor do I consider TFA any more than fanboy buzz. Just like other search engines we've used over the years of 'net usage, they're just the one on top right NOW. Give it 10 years. They might be the next big monopoly, or the next Webcrawler.

Personally, I prefer the meta-search engines; more baskets means more eggs.

Ceases to Exist, but... site is now on Slashdot?

See the irony?

I'm going to go out on a limb here and say that your lame site is getting more traffic than its ever received in a single day.

Which means that you've just been depending on Google too heavily for too little in return.

Digg it. Sig it. Promote the hell out of it.

I'd say this is a non-story, but the irony is that it was ultimately a wonderful short term solution to the author's issue.

Google does *not* own the Internet unless you depend solely on Google.

Of course, google has already re-listed them...

In the comments are some strings that one writer of theirs expects to find on their site when searching google, but didn't. I just searched for the "jgoodies data binding" and their site comes up the 7th top level listing on the first results page.

It seems to me that google worked perfectly here. When 50,000 spam and phishing messages were posted to that site, the ranking of it went way down. When they cleaned them up, the site ranking came back.

What, would the site owners have google preserve their site ranking even though the content on the site went in the toilet? As a google user, I'm quite happy that google de-listed these folks for a bit, because otherwise these and other searches would have been severely polluted.

Sean

That's why I clusty...

I'll clusty before I'll google. Clusty needs some tuning, but it picks up the far-web that google doesn't even parse.

It exists...

Try typing any mis-spelling of javalobby. Anything. Google offers you the alternative of 'javalobby'. They *so* do not recognise this website... so much so that they dare to *suggest* it as an alternative to a common mis-spelling of the forbidden site. Bastards! How deep does their vitriol run?

It's simple... Build a better search

Build a search engine which reads my mind better than google does and brings me results which are more relevant. Perhaps something which learns what I want and what I don't.
 

Ask Matt!

If you would have tried doing even a little research, you would have found out that Google penalizes hacked sites [mattcutts.com] and even makes an attempt to contact the webmaster to alert them to the problem. Not only that, they'll relist you if you remove the spam.

1. Fail to follow even basic internet precautions standard since 1998
2. Whine loudly on Slashdot when search engine behaves as advertised
3. Get lots of new traffic
4. Profit

Snowboarding2.com

This has occurred with Snowboarding2.com as well. It use to offer a subdomain feature where snowboarders could create their own website. A spammer used a few subdomains and had cialis and other drug links placed to it all over the net. The subdomain service was ended a year ago and all of those subdomains have timed out for over a year as a result yet the site continues to be sandboxed by Google. A site that was on the first page of Google results since '99 is no where to be found. There is a difference between showing up on page 10 and being sandboxed completely. You can type in snowboarding2.com itself into Google and the website itself does not even show up. Google has been contacted several times regarding this and nothing has been done. A link campaign was also performed to overpass the amount of bad links with good links and that search term to no avail. With the recent Google update it is now a PR0 website when it was a PR5 for a very long time.

Overdramatic

I think that the article is overdramatic, and maybe a bit of self-promoting.

According to ALexa [alexa.com] (look at Reach), they dropped by roughly a factor of 2 to 3, from 100 to 150 per million, depending on the base period chosen, to about 50 per million. A factor of three variation in site traffic over a few weeks is large, but it's not the end of the world.

The Freedom Watcher

I'm glad that Google watches the Web so that we don't have to fear anything. See my small cartoon [typepad.com]. Bye, Oliver

Alex Chiu

Join the club, Alex Chiu has been blacklisted by Google for years.

http://www.alexchiu.com/spread.htm [alexchiu.com]

A choice quote:

"Google controls 50% of the world's searches. This famous website is so controversial that it has been banned by the most popular search engine in the world 'Google'. That's right. You cannot find alexchiu.com in Google system. Some very important people don't want you to know about Alex Chiu. Alex Chiu is on more than 30 TV interviews, 250 radio interviews, and in business ever since 1996. Yet AlexChiu.com cannot show up on Google?"

How Google handles hacked sites

How Google handles hacked sites [mattcutts.com]

As it turns out, Google is very professional on this issue, notifying webmasters, putting timeouts on the "sandboxing", etc ..

Subject

"Suddenly we no longer existed in the eyes of Google."

Then you should have gotten your shit together and been more proactive on the spam front.

This Is An Easy Fix!

It's extremely easy to get reincluded to the Google Index. Just follow the steps on their help: http://www.google.com/support/webmasters/bin/answe r.py?answer=35843 [google.com]

Idiots.

Making your whole business reliant on a single vendor is just stupid.
Especially a vendor that you don't even have a contract with.

People act like Google is a public service, Google is a business and as a business there is no reason why they have to index your site.
 

Don's cry. Want a kleenix?

Here

Let this be a lesson to all of us who have websites. Your sacrifice will not be forgiven. RIP

Javalobby: good riddence

Javalobby dropped off the face of Google last month.

Good riddence! I dared criticise Java and OOP there and it started a long involved discussion. When the discussion ranked too popular on their traffic ranking system, the editors yanked it. They couldn't handle Java criticism so they pulled a "China".

It feels good when censorship aholes get what they deserve. Cheers!
   

Maybe Javaloby Should .....

Maybe Javalobby should watch for the googlebot and not deliver new posts until they're known to be attractive to Google? If they do indeed clean up the posts "quickly," then why not set the policy on their web servers to only deliver posts that are 2 X "quickly" old to clients with the googlebot user agent? just a thought.

the road to hell is paved with good intentions

What they say is law these days. It is NEVER that good to give one entity so much power.

Focus on your real customers

Visitors from Google are not your real customers, they are more like guests. You should service them well, of course, and they do contribute a lot to your profitability (if you are commercial) or your popularity as a site. But your real customers are those who remember your URL by heart and visit you again and again, posting to your forum and buying from you repeatedly. You should focus on them. Have an pot-in mailing list where they can learn about your news and make sure they are interested to know what new things you have to offer.

Google Not-So-Great Anymore

Is it just me, or do you find the first page returned isn't that good anymore on Google due to people exploiting their algorithm? I've started going to the 3rd or 10th page returned to start seeing results less "gamed."

Another question

I have a website that got around 600 visitors a day with a certain domain name (.fr).
When my host had to renew the subscription for the domain name, they didn't, even though I paid. Then someone "stole" the domain name when as it was free.
Now I had to buy another domain name (.com) and some a**hole put ads on my former domain.

Does anyone know what I can do to have google index the new one and give it the position my former domain name had ?

old news

This happened last year. There've been several follow ups to the original blogpost on how the situation was resolved. There's even a guy from google who showed up in the forum and offered his helped to fix things. By now the damage has been undone of course but for some time Google stopped returning any javalobby results.

Btw. it's javalobby.org, they're not for profit and sort of pay the bills with the advertisements. Just like slashdot. I've been a member of their site since 1998 and they're good guys.

Google's right

It is entirely in the rights of Google to alter their indexing at will, it is unfortunate if your business depends on a free service, so much more is it in your interest to secure your site against abuse.

The same shit happened to me more than two years ago. I not only removed the offending pages, but actually return error code 410 GONE which should mean that all references to the resource should be removed. Despite that, I still see robot queries for these pages, as late as December Google tries to fetch the offending pages, and I also see real user requests to the offending pages - somewhere on the Internet, links must still exist, despite these pages only existed for a month. But I can't find them, and they doesn't show when searching for links to my page.

My site is indexed now, but I have never regained the ranking.

What Google should offer - knowing problems with forum and blog spam - is to temporarily derank the site. If a site is deranked because of such spam a flag should be made in the index, such that next time they come around they will check if the page is still offending.

Google does offer in their webmaster tools both the opportunity to report spam on web pages, to get these pages removed, and to request reinclusion.

Isn't the real problem

Google's indiscriminate indexing techniques? Shouldn't Google just be proactive and exclude areas prone to spam invasions such as site forums? (After all, Google is supposed to be the origin of all innovation on the internet).

When Your Site Ceases To Exist

Post an article about it on Slashdot.

Re:Anti-trust against Google?

On what grounds? Google isn't stifling competition.

Re:Anti-trust against Google?

And guess where live.maps.com is on Google's search? Go look... no it's not on the first page.... Go to the second page of results... Ah yes half way down.... HMMMM I think Google has a case to answer here, I simply don't believe Microsoft maps can possibly legitimately be ranked where it is.

Because you are an idiot. Go back to live.com and see where it shows up in the *search* results for maps (sponsored links DO NOT count, duh!). I tried, and the site appears nowhere in top TOP 50 results.

Hilarious, come on all you Google fanboys/MS anti-fanboys.... try and spin this one into yet another Microsoft bashing session I dare you, then I can see something truly imaginative.

You've already succeeded all on your own.

Re:Anti-trust against Google?

Better off not. Or else all government web pages might suddenly cease to exist.

Re:Concentration of power unoticed by Google Fanbo

Nah... Is there anything worse than Apple fanboys?

Re:Anti-trust against Google?

Being a monopoly isn't illegal. Abusing a monopoly is. When Google starts using OEM contracts to force their competitors in another market off the desktop, then maybe you have a case.