Follow Slashdot stories on Twitter


Forgot your password?
The Internet

Judge Bars eBay Crawler 168

matty writes: "A judge has said that Bidder's Edge could no longer use its crawler to gather information from eBay. 'Even if its searches use only a small amount of eBay's computer system capacity, Bidder's Edge has nonetheless deprived eBay of the ability to use that portion of its personal property for its own purposes.' So what about Yahoo! and all the other search engines? Don't they use similar technology? Read the article and see for yourself." Or maybe it's not such a bad precedent; it'd be interesting if such a ruling helped discourage hard-drive searching by software which searches for "undesirable" content without your consent or knowledge.
This discussion has been archived. No new comments can be posted.

Judge Bars eBay Crawler

Comments Filter:
  • Just because a resource has been made available to the general public, DOES NOT mean that the owners of that resource shouldn't retain every right to be able to bar people from using it, for whatever reason.

    The ability to say "You are no longer permitted to use my server in that way, stop it." is an essential one. It means, for example, that even though slashdot is open for public posting, Rob, Jeff et. al. still have the right to bar someone who they feel is abusing that right from their servers.

    Charles Miller
  • This ruling, if it stands, could have rather far-reaching consequences for the industry. Here are a few scenarios, both good and bad:

    1. [Good?] Programs which sift the Internet for e-mail addresses (for later spamming) will be banned because such sifting "use[s] a portion of that property," wherein "that property" is defined as any webpage or newsgroup - ANY on-line resource - that can legally be defined as someone else's property.

    This creates a puzzle: webpages can be defined as property, but a domain name can't? So, the address which signifies and, in the mind of the consumer, effectively "houses" said property isn't property itself? So, you can steal my property deed, which _isn't_ property so no theft has occurred, and sell it to someone else who then has the right to profit from the webspace or other virtual property which the URL represents?

    2. [Bad?] Deep linking has been okayed by the courts. So, deep linking is okay, which inarguably "use[s] only a[n] ... amount of [system X's] computer system capacity," but automated search systems aren't? They both use part of system X's capacity, but the automation makes it evil? Is this because the automated searches potentially use up more bandwidth? If so, will we see future rulings where judges determine that linking or searching is okay as long as it doesn't exhaust more than BLANK percent of a system's capacity?

    3. [Good? Bad?] Search engines which use web spiders are now feasibly verboten. I know that robots.txt has long existed, but many millions of sites don't use it. Will they now be legally required to if they don't want to be searched? Can I sue, then, if you index my site and I don't want you to, because indexing my site uses up some portion of my bandwidth that I didn't want to give?

    Just my .02 cents worth.
  • Bidders' Edge were doing something to Ebay's site that Ebay didn't want them to do.

    I hereby prohibit the RIAA and their appointed representatives from checking to see if I have DeCSS on my web site...
  • The judicial system is not based on hard and fast rules like many people would like to think. Certainly, there are some rules which are very strong, but nothing (ABSOLUTELY NOTHING) is inviolable in the judicial system, because there will always be an exception to every scenario thought up.

    The courts decide these complicated, redundant, and confusing matters on a CASE BY CASE BASIS. They get to decide every time it comes before them whether or not someone was accessing your site as a potential customer or not. To this extent, they'd allow offline browser caches under most circumstances. In this case, however, Bidder's Edge was clearly and blatantly redistributing E-bay's information, thus taking advertising revenue away, and also depriving E-bay from the basic control of its content. I personally think E-bay might be well-served to allow Bidder's Edge to spider them, but they are perfectly within their right to object, (after all, IANA-MBA) and the court is perfectly well within its authority to uphold this principle.
  • Ebay isn't interested in protecting AD revenue. They are trying to prevent people from discovering other auction locations. They have allowed other companies to spider their site, but only when the ebay content is placed on a page separate from other auctions sites.

    They want to inhibit the ability to co-mingle data from multiple sources, so that they can create an maintain a monopoly in the online auction space.

  • Several people here have pointed out search engines, but there's a big difference: AltaVista/Google/etc. honor robots.txt. According to some posters here BiddersEdge appears not to have honored it.

    Ebay does not have a /robots.txt. Pretty silly that they don't because then I think that most of the community would agree with the rest of your stated position.

  • The real problem is that there is a significant difference between a person viewing a page and a bot "viewing" a page. It is reasonable to assume that a person might (however remotely) bid on the listing. Bots can't bid.

    Many ebay auctions have hit counters. I've seen those counters at a ratio of 80 views per 1 bid. That is a pretty remote chance. In the not so distant future we may have bots bidding on items.

  • I admit that I have a strong feeling against advertising, at least against what is going on under that name today. I think we are treated more and more as consumers only and people start finding enjoyment only in acquiring things or paying for luxury services that they would not need/like/care for normally. I think ads play a major role in this problem.

    That said I did not say that using the ads is unethical. I said that I do not see ethics on the side of eBay in this dispute. That is I do not think that Bidder's Edge acted unethically. They tried to make money of their own and reading other comments it is not clear that eBay suffered or gained because of it. And even if eBay suffered then it might be justified by the consumers winning. For me this economical system (based on having property) is justified to the extent while it is beneficial to the society (mankind?) in its entirety. (I am aware that this is not the mainstream school of thought.)

    Again, my only little point wanted to be that I am not sure that ethics should be dragged into this dispute to make a judgment, it looks to me that what happens is two companies fighting over profit/loss or rather suspected/potential profit/loss.

  • First, that 80 views includes bots (granted that it is probably a very small percent). But the ratio of bot views to bot bids is 1:0. We'd be more inclined to take a friendlier view towards bots if they actually bid.


  • Do I get to sue people who use my public web pages? Odd. I thought once you placed them in the public that the spider was out of the bag, but that might be a webism.
  • But what about re-publishing the contents of that database?
    That's always been perfectly legal. Phone books are a good example - those community phonebooks that you get are nothing but re-published information from the phone company's version. The phone company has copyright on the compilation, not the individual entries. There was a recent attempt to change this with some really really horrid legislation, but IIRC it was defeated.
  • I think that the robots.txt could be seen as a "No trespassing" signs. For those who don't know robots.txt is a file that advertise limits for robots and spiders.

    I have just one little doubt, how can one define what is a robot? If I have a cgi-bin that gets information from a site and presents this info to user on request (pehaps in a simplified way or otherwise processed way) woudn't this be a user-agent? If you say no, then maybe you should be aware that you just made proxy-servers a robot. But then you may say that a proxy dosen't change the information, so it appears as the author intended (not aways true, but...), then you just said no to lynx, since almost every site want you to see the graphical adds.

    This question is rather hard to answer. I myself think that a cgi-bin could harverst information on the web. I think that if ebay is so worry about this matter, they should simply make security matter to forbidden such access, or at least make them harder.

    That's my R$ 0,04 (this is more or less 2 cents of a dolar in Real the Brazil's currency)

    "take the red pill and you stay in wonderland and I'll show you how deep the rabitt hole goes"
  • It just trawls up the question of whether it is ethical to use sites whose services you use, without viewing their ads.

    Is it ethical to change the channel when a commercial comes on during a tv show? Uhhhh, yes(I know websites count 'hits' and tv shows are surveyed not 'hit' so dont give me that crap). I don't have to go out of my way to earn money for the fact, I can go out of my way to deny them money. It's up to them to find a viable business model and make a profit. As far as ebay...I kind of side with them and kind of don't...if you are posting content, expect it to be viewed. I don't know if copyright can be involved in this or not? Repackaging information from places has been around for awhile.
  • Let me preface this post with this

    :: I don't know what the fuck I'm talking about ::

    Having said that, are they truly stealing ad revenue from eBay? The crawlers count as hits yes? Is it not these hits that are used for the marketing of ad space? I don't know how this works, but I'm guessing it's something like this.

    Customer : I'd like to purchase ad space on eBay... How much does that cost?

    eBay : Well, for your ad to be viewed by 100 people, it'll cost $100. For 500 people, it'll cost $200. For $1000 people, it'll cost $250.

    Yes, I understand that customers going through Biddersedge are customers that perhaps AREN'T going through ebay, and they are therefore losing that revenue. Still though, you would think that eBay would have tried to come up with a way to get money from that, rather than shut down a potential source of revenue. Something like:

    eBay : Hey, biddersedge... how many people do you have viewing our auctions?

    Biddersedge : Well, we usually get about 100 a month.

    eBay : Hmm... you're stealing our ad revenue. You should pay us the money that we would make off of that ad revenue. Lets see, we charge $100 for 100 viewers, you owe us $100 this month.

    This would enable eBay to reap the benefits of having someone else advertise for them. Now, if Biddersedge couldn't, or wouldn't reimburse, and refused to stop crawling their servers, which is unlikely, then you would move to a robots.txt file, or perhaps DENY host settings. Combat a high-tech problem with high-tech solutions, not legal ones.

    Anyway, this is just my take on things. Feel free to correct my mistakes.

  • number of "advertisements" (feel free to use Visa, ipix imaging services, powered by sun) that ebay no doubt gets paid to put there.
    If what you actually said there is true, then they aren't losing any money at all, because nothing has to do with the user. What I'm guessing you meant to say was '...ebay no doubt gets paid when you click on'. Even so, eBay's not really losing money, just not gaining any, the same way that they wouldn't if you went to the page and didn't click on any of the ads. And since when does not clicking on any of the ads make them 'lose' money? How could they possibly 'lose' money because of a web-bot going through their page?
  • The problem itself could easily have been resolved with a robots.txt

    eBay has a robots.txt, and it clearly prohibits bots from reading the listings.

  • Or maybe it's not such a bad precedent; it'd be interesting if such a ruling helped discourage hard-drive searching by software which searches for "undesirable" content without your consent or knowledge.

    It's pleasant to think that this ruling has potential to only be used for Good Things®.

    However, the stark reality is that if it can be used to prevent breaches of privacy à la Aureate Media aka Radiate [], it can also be wielded as a weapon in cease-and-desist letters as well as in the courtroom when a site doesn't like another (a competitor, perhaps?) crawling them on a regular basis like the recent AuctionWatch [] brouhaha.

    Gibson Research Corporation has a fantastic links page [] regarding privacy. Many links on this page really ring true with this ruling.

    On another note, Slashdot is really on the ball -- eBay's Press Releases page [] has absolutely nothing to say about this (as of when I wrote this, of course).

    "Give him head?" ... "Be a beacon?"

    "One World, one Web, one Program" - Microsoft Ad
  • (usual disclaimer about my total lack of legal training, and the face that this is only my opinion)

    This decision appears to be legally shaky and could get reversed on appeal. The judge has ruled that BE cannot crawl eBay because the crawler is using some small percentage of eBay's server resources, and eBay has the right to limit who uses their "personal property" (in this case, their server resources). However, there are some limitations to the application of law regarding protection of personal property.

    1. The property should be private, i.e. not a public resource. While it can be argued that eBay servers are the property of eBay, these servers serve web pages to the public Internet. eBay constantly advertises asking people to come to their site. It could be argued that eBay's website is an open accommodation. This is a somewhat weak legal argument due to the disclaimers that are sure to be somewhere on, but legal disclaimers will not apply if the appeals court rules that their website is an open accommodation.
    2. To classify an open accommodation as private, the owner of the item should have taken reasonable precautions to inform the infringing party of their trespass. This is why land has "No trespassing" signs on it. I don't see any such signs on eBay, and in any case it can be argued that accessing auction information on eBay is retrieving the information that eBay is designed to serve, and cannot be termed trespass.
    3. If the property is ruled an open accommodation, the owner must show that he has suffered reasonable loss by the trespass. This is why, if I leave my house door open by mistake and you come in and steal my stuff, I still have legal recourse to recover it (assuming I find you). In the case of software which can be copied freely, this law is applied by the copyright holder under the (perhaps dubious) justification that it causes them economic loss. It appears eBay attorneys have claimed protection under this law, saying that their servers are under load from BE's spider. However, if it is ruled that the primary purpose of the accommodation is the only purpose to which the infringing party puts the accommodation to use, this should not be a valid argument. The situation is akin to a bulletin board you put up outside your house. The space in front of the bulletin board is a limited resource (since if you are not standing in front of it, you cannot read the notices). If somebody walks by once every hour, stands in front of the board for five minutes and reads the notices, this isn't trespass until you can show that this person is causing you reasonable loss. If he stood in front of the board all the time, thereby preventing anybody else from reading it, that would be a valid argument.
    Just my $0.02
  • But how do you know if you're allowed on a web page until you actually bring it up? It's like posting a billboard and saying only certain people are allowed to look at it. Plus, this case sounds like one aimed at killing off spiders completely. It starts off with just Ebay, then other companies decide they want to control who can visit their sites, and so on. Pretty soon, the people writing the spiders don't know which sites they can legally visit. That's what robots.txt is for, BTW.

    Generally, if you put up a web page, it meant for everyone to see. If you want to limit who can see it, add logins and passwords. Think about sites like NY Times -- they require a free login to get access. This is one way to keep the spiders out, though I suspect it is also used to keep statistics on how many people view their stories.

  • The problem may be easily solved if Bidder's Edge made use of some sort of distributed HTTP cache [] which would already have the relevant HTTP objects stored and would avoid requests of eBay's servers. These caches are populated by other individuals making use of them through other local caches (you ARE making use of a local HTTP cache [], aren't you?)

  • by ( 142825 ) on Thursday May 25, 2000 @09:23PM (#1047007) Homepage
    Since spammers use cycles of pop3 servers and SMTP servers stealing cycles, bandwith, and disk space. This ruling may apply.

    This is not a precident!

    For it to be a precident, it has to be ruled on by an appeals court. Unless, it is something that an appeals court does not usually rule on (ie. a motion to remand).

  • by carlfish ( 7229 ) <> on Thursday May 25, 2000 @09:24PM (#1047008) Homepage Journal
    The "using CPU cycles" is not a new legal argument. Back when there were no specific laws against computer cracking, I believe one of the earliest convictions came from proving that a cracker "stole electricity" by using cycles on the machine they'd broken into.

    You'll first note, from following the provided link, that this was simply an injunction. There was no mention of damages. There is nothing in this that sets any precedent against webcrawlers, or deep linking. All the case says is that yes, the owners of a webserver may choose how that server is used.

    Bidders' Edge were doing something to Ebay's site that Ebay didn't want them to do. Ebay asked them to stop. Bidders' Edge didn't want to stop. So it went to court. The court told Bidders' Edge that what they were doing constituted a use of the Ebay system (their CPU cycles) against the wishes of its owner, and ordered them to stop.

    Now what, exactly, is wrong with that? Do we really want to set the alternative precedent, which is that as soon as you put a public resource on the net, you have no say in how that resource is used? I can imagine all the IRC script kiddies drooling at the thought of that one. "Hey! It's a public server, my clonebots were only a few hundred out of the thousands of users online, so I can't have been using a significant amount of the net..."

    Charles Miller
  • The above is probably the most reasonable post in this whole topic.

    There is no guarantee that the companies that prospered during the first wave of net commerce will prosper forever. Now that eBay is the incumbent, it has an incentive to preserve the status quo. When you're making money today, you want to make sure that the changes to come in the near future are not "that" near. (I don't know off the top of my head whether eBay is profitable or not, but they are certainly generating a lot of revenue.)

    Reasonable people can disagree on whether that be a wise move in the long- or medium-term. I think anything other than short-term planning is becoming more and more of a lost art. Either way, I also say, "Let 'em!"
  • Sure eBay pages have advertising, you just don't recognize it. It's not third-party ads; it's internal. Go browsing through eBay and every category will have some "Featured Auctions." What makes these auctions featured? The sellers pay extra to have them listed, and also to have little icons listed with the items. If you go through someone else's search engine, you'll miss eBay's biased search results.
  • The worst effect of this decision will be on meta-search engines and shopping bots. Those sites rely on the ability to act as a proxy on behalf of the user to request pages from other sites in real time and add value to this data by presenting it in one screen (and removing duplicate listings, etc.). Bidder's edge does not use a "spider" and is thus not concerned with robots.txt. Instead, it is simply acting as an intelligent proxy server--requesting information from multiple auction sites on behalf of its users.

    Ebay's objection to this practice had nothing to do with ad-revenue. They were upset that they were being compared equally with other auction sites. It is as if Amazon wrote to the companies operating the shopping bots with a statement that they were "Amazon" and should not be compared with mere booksellers. Ebay's brand is their major asset and they are willing to protect it at all costs.

    As a result of the injunction that Ebay obtained, consumer choice and fair competition can be undermined by any e-commerce site with a superiority complex.
  • The site isn't dumb, if anyone was dumb, it was me. I was the one who submitted the story. How many stories have you submitted?

    first of all, Yahoo is a directory, not a search engine

    Thanks for the correction.

    secondly, the issue here is that one site is trying to make a profit by doing nothing but taking the information off another site and presenting it in a different form...

    Umm...isn't that exactly what all the directory and search engine services do? Make a profit by taking info off of other websites and presenting it in a form usable to someone looking for something?

    Feel free to respond and disagree, but I find it a bit extreme to say that a site is 'dumb' just because you don't see things exactly the same way. (It's not dumb to call a directory a search engine, it's a pretty fine line, IMO)

  • I know it isn't exactly the same thing, but one can hope can't we?
  • ya know, if eBay actually followed the standards on this here internet thing, instead of just trying to exploit it then maybe I could understand this... in fact I'd even cheer for it. BUT [] is 404 so the whole thing just lost are respectability.
  • The supermarket would have to require customers to "log on" and "block" those who never buy stuff (Would that be legal?).

    As far as I am aware, that would be perfectly legal.

    Do not forget that shops, despite being open to the public, are still private property. If you don't want to let someone in your shop, you don't have to. If you want to bar people that come in without buying anything too often, you can - although I would expect it to be bad for business, it certainly wouldn't be illegal.

    You do not have a God-given right to enter a shop, or any other privately-owned premises.


  • Reality check time: What isn't being taken over and controlled by corporations, or somebody trying to make a buck off you. This is the state of our entire culture, and as far as I remember, it always has been. Every aspect of Western culture has been bought, sold, and paid for numerous times. I'm slowly realizing where the 'punk' in cyberpunk comes from... Gibson wasn't glorifying or romanticising that world, it's a warning, probably too late to be much more than a prophecy though... FTM
  • This really isn't much different from a retail store hiring someone to go around to other retail stores and price check (which they do). However, in this case they are actually costing the spied on store money through the use of bandwidth and cycles. E-Bay is perfectly within their right to deny someone access to their computers just as a retail store has the right to throw out spies when caught (which they also do).
  • by Anonymous Coward
    This could create a really scarry legal precedent. It is unfortunate that public policy is, more and more, being decided in through litigation and not legislative bodies with input from the communities effected. I hope that the judge eventually removes this injunction.
  • Ebay isn't really hurting themselves. One thing to remember is that eBay, like many other websites probably gets almost as much money from adds as it does from the transactions that go on on the site. (cf: TV and radio stations)

    At the heart of the reason for their complaint is that they're being deprived of the advertising revenue while being asked to support their competition with their research/admin time AND their CPUs/bandwidth.

    At the same time they've managed to limit the damage that this would do because they're not banning EVERYBODY from their site -- just this specific company. For this rulling to affect you you have to get eBay's attention first. They have to ask you to not use their site before you can be forced not to.

    I can think of two analogies that would apply (in different ways) here. One would be a radio/TV repeater that filtered out all the adds and put their own in. I think (hope) that most people can understand how that is problematic (from the station's point of view).
    Another analogy would be you walking around a shopping mall handing out leaflets with price comparisons between that mall and the one across the street -- "See how many prices are lower across the street!". -- The mall would be completely in it's rights to tell you to go show off your price advantage on somebody else's property. Shopping malls (like websites) are only PSEUDO public property.
    Although they generally invite and allow the whole public on their site, they have the right to tell you to go away and don't ever come back again. I don't think that they even have to tell you why. You are expected to self-police yourself on that. If you come back, and they recognize you, they have the right to go after you for trespassing.

    All that this ruling does is extend the principle of trespassing into the electronic domain in a reasonably straightforward and sane manner.

  • No, I think this is more like if /. started mirroring content on their own servers, then it would be the same thing.

    Linking to articles on C|Net or Wired, the way /. currently does so, increases their(the news services') page views, not vice versa.

  • First off, I'm profoundly disturbed by government intervention into an issue that can undoubtedly be solved with code.

    Indeed eBay owns it's publically accesible servers, but who owns the content?

    After all, can eBay rightfully edit, modify or remove the listings in question? (Remove perhaps, upon poster's misconduct).

    If not, then the seller still owns the content, in my opinion. Without that content, what have they got?

    So, now what -- tie this ruling and the DMCA together and sue AOL for using proxy servers to 'aggregate' content?

    "You can't download this page, cuzz.. I don't like you."

    If this flies, can I restrict access to my site to those with T1's?

    "Sorry, due to your slothful connection, you've exceeded the maximum allowable download time, you can expect a call from our lawyer."

    Apache::Slothful now available at your local CPAN mirror!

    -Jay J

  • Y'know with a <LIMIT> directive or something? If I were the judge, I would have required eBay to use reasonable technical defenses. If those failed, then, perhaps a legal solution could be pursued.

  • Honestly, I can't see how they can determine how much wasted resources is too much..Like matty said, what about these search engines like yahoo..don't they do the same thing (except perhaps on a smaller scale) ?

    In other words, where do you draw the line between acceptable and not acceptable..or can you?

  • Everyone is whining about how this ruling prevents spiders/robtots from traversing the web and how this somehow impacts free speech and infringes on our rights. That's crap. The analogy to yahoo/google/etc. not being able to index the web because of this ruling is even worse. What this ruling does affect is whether spiders can traverse eBay, and only eBay. And I heartily agree.

    Who wants a search engine cataloging dynamic sites like eBay? Not me. Oh, look! There's that watch I wanted!


    [click] Drats! That auction ended over a year ago. I can't remember once browsing to an auction site from a search engine and hitting a valid link. It just doesn't happen.


  • I agree with your observation regarding the contents being for public use on the eBay servers. The solution should be technical, by telling exactly who and how can access this information, which could be enforced by legal means of course.

    On the ethical issue: I am not sure that a site has the ethics behind it to dump advertising on me in exchange for information they provide. (They lawfully can do so of course.) I thought eBay makes a cut from the items sold via its service. I see no ethical problem of someone using a different program than eBay's own to watch that data and filter out the crap. Would it be a problem if the same thing would be done to e.g. IBM's patent server
    I see no significant amount of ads there by the way - only IBM logo and info on IBM, no banners.

    So I think that eBay should be satisfied if their name (and a maybe a link to their site) would be mandatory for everyone operating a search service based on their public database. Or alternatively they should implement some kind of registration on their site.

  • that's why all rulings should be made immediately available online.
  • We reserve the right to refuse service to anyone
    Yeah, but I was refering to the way in which they did it. I've seen those signs, but I wouldn't expect to walk into a store and suddenly be slapped with a lawsuit. eBay should have taken the technical solution, a deny flag in the webserver config file or something of the like, rather than a lawsuit. In addition, from their User Agreement,

    You agree that you will not take any action that imposes an unreasonable or disproportionately large load on our infrastructure.

    Now, since when was a web-bot, one process, an unreasonable or disproportionately large load? I'd bet that it makes up less than a half of a tenth of a thousandth of a percent of the traffic that eBay gets.
  • I know that ebay makes a small fraction on each of the transactions made through them. That's how they make money, blah blah...

    Anyway, I don't see why the search engine would hurt them. In fact, I would think that buyers would spend less time looking for what they're searching for and therefore decrease the amount of time between each transaction == higher number of transactions per given period of time == higher profit for eBay.

    In fact, since buyers spend less time wading through unwanted info, less resources would be consumed on eBay's side.

    This should be a win-win situation for eBay. I don't see how it hurts them at all.
  • by roystgnr ( 4015 ) <roystgnr&ticam,utexas,edu> on Thursday May 25, 2000 @09:32PM (#1047029) Homepage
    If EBay wants to stop a particular client from using it's services, they hardly need to go to court to do it! They can weed out requests based on browser IP, based on whether the client is a spider (robots.txt), based on click-thru agreements...

    Basically, imagine that every day, I ask you for a few dollars and you give it to me. Should you then take me to court to force me to stop asking? No! You should just stop handing me your money!

    If EBay has such a system in place, and BiddersEdge is ignoring robots.txt, lying about it's client type, or otherwise circumventing it, then fine, moderate this to (-1, Idiot)... but it's late, I'm too lazy to read the article before going to bed, and I've used EBay once or twice without them asking to make sure I wasn't a competitor first.
  • (Now, if the search engine provider doesn't respect the ROBOTS.TXT exclusions, I say go after them with legal means.)
    Wimp. When did the age of Technology for Technology end and the age of Technology for Profit begin? Sysadmins shouldn't go crying to the DOJ every time somebody they don't like pings their servers. A true Sysadmin would have just added a DENY tage to the httpd config file, rather than trying to complain to the courts.
  • Search engines spider your site and you get more traffic (and maybe more money).

    Bidder's Edge spiders your site and you get less traffic (and maybe less money). See the difference?

    So will everyone please stop suggesting that this applies to search engines!
  • by Kris_J ( 10111 ) on Thursday May 25, 2000 @09:35PM (#1047032) Homepage Journal
    • No public/anonymous browsing - have to log on to see eBay content.
    • Make it against usage agreement to use bots
    • Make the site bot-friendly either through optimised pages or a separate connection (ODBC-esq)
    • Enter into specific licencing agreements with 3rd parties to allow sorting, sifting and filtering of eBay contect. If they breach the licence, pull their access.
    Easy. Then the courts don't have to rule that the public aren't allow to use publicly accessible information if the provider doesn't like the way you look at them.
  • smart2000 wrote:

    Ebay does not have a /robots.txt. Pretty silly that they don't because then I think that most of the community would agree with the rest of your stated position. and don't, but does.

  • Ah! I see ... I always use the search function, and never go directly into the catagories, so I never see the "featured" section and forgot about it.
  • Umm...isn't that exactly what all the directory and search engine services do?

    actually, no...a search engine grabs the title and description for a page (or uses one you submitted to them) in order to help the searcher decide if the page would be something they are looking order for your statement to be true, the search engine would have to take a selected portion of the content and repackage it so as to give a searcher the information from the site without the searcher having to visit it. This is what the case is trying to prevent happening and I think eBay and others are correct to put a stop to it.

    Let me ask you this...if you built a site and spent a hundred hours a week updating and refining until you had a great following of repeat visitors and then I came along and decided that using a neat little script I would spider your site daily and copy all the best parts off it to create my site, what would you do? Even better, what if I used this stolen content to sell advertising on my site and I started raking in big bucks off it? Hmmmm? Nothing wrong with me stealing the fruits of your labor here and making mucho dinero off you??? Bah!

  • Because Ebay makes their money from ad revenues. To use an analogy: Slashdot is cool and harbors extrememly libertarian views about the web, but everytime someone views their headlines on or any one of a million other places that posts them, they are deprived of a hit to their homepage. Which means that advertisers pay them for one less impression. Over time, these can add up - they probably have cost /. a nice chunk of change in the long run.

    This is the same exact thing that happened to ebay. By mirroring only the content of their site (the ad listings), less people are going to to see that information. Hence, less are viewing the ads on, which means they get less money in the long run.

    The fourth clause for fair usage under the copyright act states "(4) the effect of the use upon the potential market for or value of the copyrighted work." Obviously, fair use has to end here - what this site was doing obviously drove down the "market value" of Ebay advertising, because it lowered their impression count. The fact that it is publicly accessable has nothing to do with it; there are plenty of publicly accessable things out there that are still copyrighted. At the bottom of every page you still find "Copyright © 1995-2000 eBay Inc. All Rights Reserved."

    There has been a widescale trend by the courts in narrowing the scope of commercial fair use recently, and this case fights right in with that trend.

  • It seems to me that a system stops being "Personal property" when I put a web server on it for puplic use. If I understand the ruling, anytime I acess a web site I am stealing CPU time from whom ever owns the machine.
  • On the other hand, I think what Bidder's Edge does is really indefensible from an ethical standpoint and I am rooting for them to lose because they are in effect *competing* with eBay for advertising dollars by *using* eBay's content. If you view content from ebay through Bidder's Edge, that's advertising revenue eBay doosn't get which BE does.

    How much of ebay's content are they using? If it is nothing but item names and bid prices, to hell with ebay. Those are facts, and facts aren't copyritable. They should have known that for markets to work there's got to be easy access to pricing information, and people are going to make it so if ebay doesn't. If they had any brains they'd set up some sort RDF-like publishing system, so others can grab it quickly without sucking all their CPU cycles. And let others submit bids in large batches on behalf of third parties, for a small fee of course.

    As for using server cycles, who are they kidding? In ten years, every person is going to haven dozens of software agents doing their bidding. What's the differnce in my robot scouring auction sites and Bidder's Edge doing it for me? Bidder's Edge is actually more efficient because it removes the duplication of effort. How would ebay feel if all it's thousands of users turned their own robots loose on the the servers?
  • by hawk ( 1151 ) <> on Friday May 26, 2000 @05:20AM (#1047039) Journal
    I am a lawyer, but this is not legal advice. If you need legal advice, contact an attorney licensed in your own jurisdiction.

    > While IANAL, I have a friend that is and I think that giving out legal
    > advice (which can consistitue answering legal questions as such)
    > without charging for it can be considered a violation of the Bar

    No; that is an ancient joke--though if you go back far enough, you find Barristers robes with a pocket in the back so that they could conveniently be given a payment--as it was improper/unethical/ungentle to ask--Esquire is a derivitave of Squire, a class that wasn't supposed to work for a living.

    Anyway, free advice is legal, and so is free work (pro bono publico--for the public good).

    That said, anywone dispensing free advice in a public forum like this is nuts. There is *no* way to adequately gather enough information from a short posting to give competant advice (save in the most trivial situations). Additionally, proving what you did or didn't say is difficult. WHether you charge or not, you are still liable for the advice.

    Extreme example: voice on phone asks question, gets answer, does something, comes back and sues. You have no idea who the person was, or if this is the person you talked to. But you're on the hook, and you insurance carrier is less than happy. This is why I have a hard, no exceptions, policy that I only give advice to clients.

    On the other hand, I do frequently comment on legal issues around here--but never in a way that could be legal advice.

    Lastly, note that the legal questions that get asked are generally ones for which a proper answer would take several hours of research. I'm not going to do that every time a complete stranger asks. If someone wants to pay my hourly, sure--but I'm not going to do several hundred or a couple of thousand dollars of free work on a regular basis :)
  • [Note: I work for a different online auctioneer].

    We've been discussing whether or not Bidder's Edge is a Good Thing (from our point of view) or not. My company does B2C auctions (e.g. we actually warehouse the products unlike eBay's C2C model). There's two schools of thought:

    1. Good: It potentially drives traffic to the site. The whole point of an auction is to allow as many people as possible to see it to find the one who is willing to pay the highest price.
    2. Bad: Bidder's Edge (AFAIK) aggregates auctions (e.g. it consolidates auctions from eBay, Yahoo, Amazon, etc.) and so if there are 2 of the same product, people will obviously bid on the one they can get cheaper. This essentially drives traffic away.

    Our current opinion is that 2 is the dominant factor. Plus, these sites train users to go to Bidder's Edge before going to a specific site.

    The real problem is that there is a significant difference between a person viewing a page and a bot "viewing" a page. It is reasonable to assume that a person might (however remotely) bid on the listing. Bots can't bid.

    I agree with your assessment of the ethics of the situation.

  • by geekatlrg ( 153223 ) on Thursday May 25, 2000 @09:36PM (#1047041) Homepage

    This is entirely short sighted, unrealistic, and fails to take into account any real world scenerios.

    The internet is a public place, no different from the local shopping mall, grocery store, public library, movie theater, whatever...

    When you go to the mall they expect that you will and won't do a number of things, commonly accepted criteria for activity in a public place (read: no shirt, no shoes, no service).

    On the internet nobody gives a damn if your in your shorts, but suddenly your not allowed to do any number of things that you could realisticly do physically.

    For example, lets say I'm doing market research and I send 50 people to the local mall to run around and look at what kind of (product) is being sold, and how many of them, and how much they cost. This is perfectly legitimate, and legal since that information is publicly available. I probably can't go into the stock room and see how many of (product) are not on the shelves, but I can certianly just have a look around the store (just like anyone else).

    If we apply the latest internet precidents to this senerio I would NOT be allowed to do this without breaking the law. Suddenly I would be using that stores resources and denying them use of that resource for whatever reason they deemed more important. So I'm not allowed to go the store if I'm just window shopping now?

    Publicly accessable resources are held up to a very high standard. Anyone can find out how much a store charges for , this is good for everyone, the store, the customers, the manufacturers.... The same applies (or should apply) to the internet.

    As far as purely internet related impact is concerned, can anyone who hosts a site look through their server logs and sue anyone who connects to their site to much? Or all those search engines that come through on a regular basis? Or anyone that pre-caches the site automatically (gee, this is even a feature in Internet Explore... more MS trials?).

    My opinion: The internet is a public place, and fair use of a public place is already governed by a certian set of rules and regulations (at least here in the US). Let these rules and regulations do their job and stop creating "special regulation" for a situation that isn't radically different from anything else we humans do on this planet. Just because you do something on the internet doesn't mean it requires special regulation. -Gentry

  • I am definitely not a lawyer/judge/etc. so this is at the risk of sounding stupid, but anyway:

    That way, cases like this one involving technology can be heard by judges familiar with it, thus limiting the likelyhood of a poor decision and a stifling precedent.

    I thought that judges were supposed to be impartial to the case. They would then base their decision on the two arguments presented by the prosecution and the defense. This, effectively, would have to rule out "technology-sympathetic" judges. Even though it would be nice, the practice could not be not compatible with our impartial justice system.

  • Perhaps what is needed here is something in the robots.txt file (does ebay have one of these?). But then again, not all spiders respect this file, so I don't know.

    robots.txt can have the opposite effect, also. Black hats and script kiddies often look at it to see what the site admin wants to hide.
  • Isn't this basically what Slashdot does? Slashdot steals bandwidth and CPU cycles by making us all /. a webserver. Isn't it the same thing?
  • eBay's main source of income is the "value fee" percentages that they make from each item sold. I seriously (very seriously) doubt that they make any noticable revenues from banner ads that Bidder's Edge "bypasses."

    Why? Because those who frequent eBay are looking for bargains, most often on previously owned/used items. They are not really the type to click on banner ads looking for the "newest thing."

    I'm not really that familiar with Bidder's Edge. I would imagine that they offer a service that helps buyers find items near the auction closing date that are within the buyer's desired price range. Maybe they do more than that. In any case, it doesn't matter. (See next paragraph).


    What eBay should do is to see what extra functionality that Bidder's Edge offers to buyers, and incorporate it into eBay. If it lets you find all items closing in the next 5 minutes, make eBay do so. If it does XYZ, make eBay do XYZ. There is nothing a seperate site such as Bidder's Edge can do by searching eBay's database that eBay can't do better by OWNING the database.

    However: Some may say that Bidder's Edge allows search of multiple auction sites, etc... It may. As I said, I am not that familiar with their service. This, of course, opens a whole new can of worms. The classic example is "what happens to the free market when computers search every online merchant to find the best price?"

    What DOES happen? Does everyone undercut everyone else, until every item is sold at cost? What about old/antique items with a small supply and limited demand? What about mass-produced commodity items? These questions, I can't so easily solve...
  • This was an injunction. You take out injunctions after you've told the guy to stop some other way. Ebay got no damages at all, all they did was assert their right to say "Don't do that again."

    Noone's going to be able to sue you because you visited a site, and then learned after the fact that you weren't allowed to. All they can do is tell you not to visit it again.

    Charles Miller
  • by breser ( 16790 ) on Thursday May 25, 2000 @09:50PM (#1047047) Homepage
    Not really... Take a look at some of the other robots.txt files they have on various other machines. For example []
    # go away

    User-agent: *
    Disallow: /
    I'd say that's everything.
  • Zebbers wrote:

    As far as ebay...I kind of side with them and kind of don't...if you are posting content, expect it to be viewed. I don't know if copyright can be involved in this or not? Repackaging information from places has been around for awhile.

    It's been around, yes. Several people here have pointed out search engines, but there's a big difference: AltaVista/Google/etc. honor robots.txt. According to some posters here BiddersEdge appears not to have honored it. That changes everything.

    In law there's a concept of intent. By not honoring robots.txt, BE demonstrated at best deplorable ignorance of the generally accepted responsibilities of spider-users, and at worst an intent to circumvent measures taken against services such as theirs.

    The article doesn't go into detail but from BE's stated position it's not hard to imagine that even if eBay blocked their address block, that BE would shell out the relatively nominal money to buy, say, a bunch of dial-up accounts.

    What eBay has done is analogous to me running, say, a playground and kicking off somebody who appears to be using it for purposes I don't agree with. I'm not required to do any more than ask that they leave (robots.txt) before I'm entitled to call the police to assist.

    A lot of people have called eBay a "public" service. That's a term that should be used with extreme caution. Not everything that's public is a "common carrier". "Public" services that aren't common carriers still have most of the property rights of a private entity. There are exceptions, such as anti-discrimination laws, but outside of those "protected" conditions, a business owner can kick you out over anything he wants. Or he can just kick you out for no reason at all. Abridging that right in the name of "freedom" will do more to hurt the Internet than any site denying access ever could.

    Government-franchised monopoly utilities/services (cable, phone, power, etc.) usually are classed as common carriers. Check out your state/locality's regulations regarding common carriers and then ask yourself if you'd want to operate a large website under those restrictions. I'm sure it's different in other places, but here in Virginia, state-franchised monopolies have to get approval from the state before they can raise their rates, not something most site owners want to (or should) be subject to.

  • According to the Judge:

    "Even if its searches use only a small amount of eBay's computer system capacity, Bidder's Edge has nonetheless deprived eBay of the ability to use that portion of its personal property for its own purposes. The law recognizes no such right to use another's personal property."

    I really don't see how this is different than any other web-crawling search engine. By the judges argument, he is denying all use of ebay since every transaction takes of a small amount of resources.

    From the technical side, Bidder's Edge may actually be reducing the load on the ebay servers by offloading some of the search load to their own servers. Without knowing how much they are caching and how many users they have it is hard to tell for sure.

    Now the interesting bit: would the ruling prevent me from writing software to do the same thing and then selling it? Yeah, you'd need a big/fast network pipe to use it, but that's not really the point here...


  • by gargle ( 97883 ) on Thursday May 25, 2000 @10:15PM (#1047055) Homepage
    For example, lets say I'm doing market research and I send 50 people to the local mall to run around and look at what kind of (product) is being sold, and how many of them, and how much they cost. This is perfectly legitimate, and legal since that information is publicly available.

    Not really. Try taking photographs at your local supermarket. You'll be stopped by security - I've had this happen to me before (and we were just taking a family photo). If you go around taking extensive notes on products and prices, I've no doubt you'll be stopped as well. They certainly have the right to prohibit certain activities on their premises, although I don't know that a general court ruling is needed or valid.
  • Imagine a sign in the future ... "only humans allowed in park. no robots, cyborgs, automated machinery. or semi-autonomous devices. Strict infobit-control zone. Violators will be sued and their datacores purged."

    Ebay wants to analyse the human activities (your click stream) and denying robots and other automated search sequences (unless you're willing to pay a hefty access fee) is probably a deliberate strategy. Now whether this is right or wrong is a moot point. Obviously as providers of a service, they can put arbitrary (even downright discriminatory as no League for artificial robot fair access rights exists :-) ) limits on who can access their information (much like a restaurant can say no thongs) but I suspect it is a little disingeneous to call it "stealing cycles" when their ulterior motives is to probably save them the extra hassle of stripping out electronic tracks and to discourage potential superior services. An analogy can be made as to who "owns" the final stock closing price which is used to create index and long-run analyses.

    The question down the line then arises whether this is a "fair" business tactic or anti-competitive behaviour. Much like certain countries have obstensibly low tarrifs but erect substantial hidden barriers to entry, it can be difficult to judge. If you look at the latest theories on infowar (IT applied to battlefield) you can see the importance of denying the enemy access to information. Much like you can forbid a competitor from wandering into your shop to browse the prices and observe your layouts/fitouts, e-commerce sites are putting in place technical and legal challenges to make it difficult for their peers to emulate their design and scan any price-sensitive movements. Of course in a technology arms race it's probably not too difficult to come up with a distributed random sampling system from multiple access points but it does make life a little bit more challenging for new entrants, especially those wishing to arbitrage between multiple auction sites.

    Too bad we don't have educational futures as in the coming techwars the only sure winners will be the overpriced salaries of consultants and MBAs.

  • To me, it seems that Bidder's Edge have been unethical, taking away from eBay's ad revenues. The question is what they did wrong that search engines didn't.

    I think the answer lies in compilation copyright. This is a special type of copyright granted for an aggregate of information, even if the individual bits of information have others' (or no-one's) copyright. This is what protects, for instance, an anthology or index.

    eBay could claim that even though their pages are freely viewable, they own compilation copyright on the whole of the work, i.e. the entire bidding collection and its indexing. There is merit to that claim, as they undoubtedly have to invest a lot of energy in collecting, maintaining and indexing all biddings.
    In this case, by essentially copying the entire collection and offering their own search facilities, Bidder's Edge have copied and violated eBay's compilation copyright on the work. Note that the violation lies not in downloading the entire archive (they are entitled to it as much as anyone is entitled to remember what she reads), but in offering that archive to other people.

    Other search engines do not violate compilation copyright, because they are essentially having their own compilation copyright: they index a much broaded range of works, and do not offer direct access to the individual compilation of eBay. They have the same rights as the 'keyword lookup' index books you occasionally find in libraries, which are an immense help when doing research.

  • ...Unlike Auctionwatch and other tracking sites, Bidder's Edge was tracking in realtime, making the act of "sniping" so much easier for bidders who would otherwise have to log onto eBay directly.

    In such a setup, a bid is cast (usually at an abnormally high rate) within the final hour of the auction, in many cases, within the last 2 minutes to 3 seconds. Anyone else who bid on the item early on in the auction (like a day or four) gets screwed. This also rasise suspicions of "shill" bids, bids cast by the seller & their associates to jack up the price (& hopefully demand) of the item.

    While eBay goes to some lengths in saying they disapprove of shilling, they absolutely love sniping. It's where much of their revenue comes from, as hot items like Pokemon, Voodoo X cards, etc. can attract a Dutch tulip-like frenzy.

    Meaning more customers, more items in demand for eBay's prersonalized (read: proprietary) site
    trackers to keep you informed of, more opportunities for folks to bring their crap to eBay, etc.

    Bidder's Edge, therefore, was doing two thirds of all their work for them. No wonder eBay was pissed.
  • Have you really never heard of a small thing called ROBOTS.TXT which most search engines honor? If a site doesn't want to be indexed on a search engine, most will let them opt-out -- it's simply common courtesy.
  • But eBay allows this because this "client-side gathering" places advertisements on retinas.
  • I don't see how at first blush (or even second, third, or last blush) this looks to be a dumb ruling. It's not even right for the wrong reasons. This injunction ruling is sound, and should probably be upheld by the final decision.

    First of all, he declined to address the copyright and trademark claims made by eBay, thus avoiding the whole troublesome can of worms that would open.

    Second of all, he bases his ruling on property rights. Read the ruling, Bidder's Edge is using eBay's servers and bandwidth to generate their content. That is server capacity and bandwidth that eBay is unable to use. As others have pointed out, a ruling in favor of BE could, if broadly interpreted, given a green light to DOS attacks.

    Finally, Bidder's Edge is violating the eBay User Agreement []. From Section 7: Access and Interference, "You agree that you will not take any action that imposes an unreasonable or disproportionately large load on our infrastructure." While eBay is a public site, there is a User Agreement which user (in theory) must abide by, if they don't, they can expect consequences such as what BE is getting. Not only is BE "stealing" cycles and bandwidth, they're doing it when they agreed to not via the TOS.

    This is as perfect of a judicial ruling as I've seen covered on Slashdot. The judge didn't give ownership of the content to eBay (taking it away from the sellers), he didn't forbid linking to eBay (chilling deep linking), he didn't give eBay a patent, he didn't pour hot grits down their shorts.

    This ruling doesn't prevent search engines from spidering. Why? Because most sites are completely open for spidering. They welcome the bandwidth usage. Bidder's Edge, however, was using a robot in violation of the TOS. In effect, they ignored a big huge "No Trespassing" sign (a very narrowly tailored "No Tresassing" sign.) As long as you follow the site's TOS, and respect the robots.txt, you're fine to spider it.

    While some of us probably consider eBay with the same disdain as we hold toward AOL, we should cheer this ruling.


  • Bidders edge isn't some ebay leech, its actually a meta-auction that categorizes and searches many different auction sites, not just ebay. I don't see why the consumer shouldn't be allowed to make cross-auction searches for products. Obviously, the reasons the judge gave would shut down every search engine.

    Instead of embracing a system that helps consumers and gives them choice, Ebay has decided to use its finacial muscle to eliminate easy to access competitor information. Essentially ebay is telling its customers, "Don't try for better deals just stick with the brand name you know, us."

    This is a ruling that is ignorant, anti-competitive, and needs to be overturned.
  • by Darchmare ( 5387 ) on Thursday May 25, 2000 @10:33PM (#1047079) Homepage
    This is just plain stupid; if you have a page on your website which is viewable by the public then it is available for the public to download.

    Hrm. How far do you want to extend this, though?

    Couldn't it be said that someone launching a Denial Of Service attack by simply requesting documents at an extremely high rate of speed is 'viewing the documents' as expected?

    I do think that the judge's excuse was a little suspect - if it were server resources alone, you'd think that this second company would be saving them load by diverting their customers (and advertising dollars) elsewhere.

    I do agree that they don't deserve to win, though, by virtue of how they're stealing someone else's content. It's not always certain what kind of precedent this will set though...

    - Jeff A. Campbell
    - VelociNews ( [])
  • by B'Trey ( 111263 ) on Friday May 26, 2000 @01:55AM (#1047084)
    It isn't quite that simple. ebay's servers are private property but they are also explicitly public access. That's their very purpose. If the public couldn't access them, ebay wouldn't have much of a business, would it?

    It isn't a question, really, of controlling access. It's a question of controlling how MUCH access, and to what use the info is put AFTER it's accessed.

    ebay intentionally and purposefully puts the information up on it's servers for public access. It seems to me that doing so rather negates the claim that it has "control" over that access. If ebay wants to maintain control over who accessed its sites, it should take steps such as requiring log-in to view the site, not just to place bids.

    Bidder's Edge is nothing more than a search engine of auction sites. ebay's real objection is that it hurts ebay's business by allowing users to compare prices with other sites.

  • I am rooting for them to lose because they are in effect *competing* with eBay for advertising dollars by *using* eBay's content.

    You mean the way Slashdot -- a technology news site / community / portal -- deep-links to articles on other news sources all over web for its own content? Admittedly /. links to the sites, but if the content didn't exist neither would the bulk of /.

    Wired and C|net in particular would have /. for breakfast if this set the kind of precedent that seems to be happening.

    • Yahoo, for example, will scan your site ...

    Why do you all use Yahoo for an example for a spider/search Engine?
    This is really interesting as Yahoo is not one of these. (And AFAIK never has been). It's just a directory.
    What you probably mean is Inktomi, which provides this service for Yahoo.
    What's the English word for Klugscheisser? ;-) "Wie immer sind alle Angaben ohne Gewähr"

  • I use Junkbuster [], which means ebay doesn't get any ad revenue from me in any event. So am I "stealing" cycles from ebay too? Is my conduct indefensible and really lousy?
  • by Virtex ( 2914 ) on Thursday May 25, 2000 @08:54PM (#1047095)
    This could have interesting consequences. Most sales-oriented sites use scripts which your average spider will avoid (due the the *.cgi URLs). On the other hand, to reduce the system load, some sites re-generate their pages at regular intervals (say, every minute) instead of every time a user loads the page. Looking at ebay's site, it appears their URLs end in *.html. Perhaps what is needed here is something in the robots.txt file (does ebay have one of these?). But then again, not all spiders respect this file, so I don't know. If the web site designers don't take the necessary precautions to protect the private information on their site from spiders, how can the spiders know not to catalog them?

  • Bah, a legal "solution" to a technical problem. What is ROBOTS.TXT for?!

    (Now, if the search engine provider doesn't respect the ROBOTS.TXT exclusions, I say go after them with legal means.)
  • by BJH ( 11355 ) on Thursday May 25, 2000 @08:55PM (#1047098)

    Well, seeing as hiw there are only "first posts" up so far (five of them, no less), I guess I'll take a shot at an on-topic post.

    The problem here is where do you draw the line at fair use? ebay is providing a publically-accessible database; why should a search of that database (by robot or not) be considered an abuse of their servers? Of course, if the search puts so much strain on their servers that no-one else is able to access them (effectively a DOS), then an injunction would be reasonable, but this search doesn't seem to have caused undue server load.

    The point that timothy brings up is not really relevant here, I think - your HD is not a database offered for public access, and should thus be protected from undesired searching, whether or not this injunction is upheld.
  • If you call used car dealers to get price comparissons on the '98 Crown Victoria, you are costing the "spied on" store money through the waste of sales-staff time... but car dealers do this to each other all the time. They don't sue each other over it.

    E-Bay is perfectly within their right to deny someone access to their computers just as a retail store has the right to throw out spies when caught (which they also do).

    If eBay had made a technology change that "broke" the crawler, that would be like throwing a spy out... what they did was criminalize the behavior in the courts, which is a Bad Thing, if you ask me.

  • ...server side? Like, adding a deny ... to their web server config file?

    Aside from that, this is really a problem. I mean, not the individual story, but the additude that a company can simply deny somebody like this the right to use a service that they provide. If would be like if you owned a software shop, but you kicked the reviewers out everytime they wanted to write up a story about what you have there. It's just not right. I mean, most everybody on Slashdot should realize that the amount of server time these requests are taking up is less than the amount of time their legal department has spent looking into this. Less than the amount of time needed to post this, even. I mean, eBay isn't running on a 486 anymore. They can handle thousands of people a day, so what's wrong with a web-bot or two? Actually, web-bots, in addition to bringing more people to the site than a normal person connecting to the server would, are also much faster in completing their communication with the server, giving eBay more customers faster and quite possibly easing the load on their servers, by (if I read the story right) mirroring product data on another site. This isn't like somebody is launching a DDOS against eBay. This is a web-bot, for goodness sake! Why do they even care?
  • by The Code Hog ( 79645 ) on Thursday May 25, 2000 @08:56PM (#1047101)
    At first blush, it seems like this is a stupid ruling, mainly for the reasons the judge gave for making it. He claims that they are essentially stealing cycles from eBay's servers and this could slow down ebay's service and have a negative impact on their customers' experience.

    This is just plain stupid; if you have a page on your website which is viewable by the public then it is available for the public to download. That's the point of having a public website. Hey, I'm a customer of eBay's, am I guilty of using server cycles and slowing down the eBay website for other customers? You bet. eBay should secure the entire site and require authentication if they really want to pick and choose who can view.

    On the other hand, I think what Bidder's Edge does is really indefensible from an ethical standpoint and I am rooting for them to lose because they are in effect *competing* with eBay for advertising dollars by *using* eBay's content. If you view content from ebay through Bidder's Edge, that's advertising revenue eBay doosn't get which BE does. Seems really lousy.

    So it seems like the right ruling, but for totally the wrong reasons. The way the judge worded things it sounds like you could make a case for suing Yahoo, AltaVista, Google etc., if they dare to spider your site.

    Whot crap!

  • I really don't see much similarity with search engines. Yahoo, for example, will scan your site (within parameters you can specify via robots.txt) once in a blue moon. That's VERY different than a robot that scans the same site repeatedly 224 hours a day. I'd get pretty miffed, too, if Yahoo, Altavista, Google, and all the rest of 'em hammered away on my site constantly just to see if I'd changed anything.

    On a lower-bandwidth site than eBay, you'd call it a DoS attack.

    Consult, v. t. To seek another's approval of a course already decided on.
  • Search engines spider your site and you get more traffic (and maybe more money).

    Bidder's Edge spiders your site and you get less traffic (and maybe less money). See the difference?

    I think this cuts to the heart of the matter.

    What does a spider (both of the above examples) do, from the user's point of view? It tells you what is out there (and where it is). Google tells you where to get a Star Trek episode guide, and Bidder's Edge tell you where to find some Star Trek steak knives for sale.

    If, after using Bidder's Edge, the user decides to buy the steak knives from an eBay auction, the transaction still goes through eBay, and eBay still gets their cut of the sale. In that manner, Bidder's Edge is increasing eBay's transactions (and money).

    The problem is that eBay isn't just in the auction business. Only part of their revenue comes from auctions. The other part is ad revenue, which they make as people spend time on their site, searching, browsing auctions that they don't participate in, etc. And that's the part that Bidder's Edge is cutting out.

    There has been a lot of web-related litigation (actually not just the web; other media, such as TV, has played a part too) that really isn't aimed at protecting IP, as much as it is aimed at protecting the advertising business model. I think that before we let this protection get any more entrenched, we need to think hard about whether we really want it.

    An ISP can't even run a web cache for its users anymore without risking the wrath of some web site that depends on ads. And to anyone who thinks that ads have given us a lot of good things (Slashdot, The Simpsons, etc) for free, I say this: TANSTAAFL.

  • That's one of the most interesting observations I have heard so far. If eBay were smart, they would go out of their way to encourage Buyer's Edge to crawl their site. After all, if somebody sees an eBay item via BE, they still gotta go to eBay to buy it, and they might not have seen it otherwise.

    This is like a photo-negative of all those companies trying to buy fake pattern matches on search engines like Lycos a couple years ago.

    How would ebay feel if all it's thousands of users turned their own robots loose on the the servers?

    That's an adroit question. Too bad you are not on the board at eBay.

  • by Phaid ( 938 ) on Thursday May 25, 2000 @11:41PM (#1047110) Homepage
    This is absolutely awful. Republicans in the US always like to use the term "legislating from the bench" to describe rulings by liberal judges which overstep the bounds of the case being argued. This is actually worse -- the judge is essentially making up technological terms as he goes along. Using someone's resources that they can never get back? Give me a break, the crawler that Bidder's Edge uses uses an infinitesimal amount of EBay's and other auctions' server capacity compared to the legions of "legitimate" EBay users. This judge is speaking from pure ignorance, and his ruling endangers everything the Web is based upon.

    Where do you draw the line? Are we only going to allow "manually" retrieving information from a Web site? What does that mean? Do I have to write code for each page I want to see? Are offline browser caches now going to be illegal since they automatically "drill down" into sites and grab several pages at a time for later viewing?

    When you create a Web site, you do it under the implicit assumption that people are going to connect to it and retrieve informaton. End of story. There is no "right" to only have your pages viewed by means you approve of. Every time _anyone_ connects to your site they use some of your resources, and doing it by automated means is no more onerous than by doing it "manually".
  • by Anonymous Coward on Friday May 26, 2000 @12:25AM (#1047116)
    $ telnet 80
    Connected to
    Escape character is '^]'.
    GET /robots.txt HTTP/1.0

    HTTP/1.1 404 Not Found
    Date: Fri, 26 May 2000 09:21:38 GMT
    Server: Apache/1.3.6 (Unix)
    Connection: close
    Content-Type: text/html

    404 Not Found

    Not Found
    The requested URL /robots.txt was not found on this server.

    Apache/1.3.6 Server at Port 80

    Connection closed by foreign host.

  • While IANAL, I have a friend that is and I think that giving out legal advice (which can consistitue answering legal questions as such) without charging for it can be considered a violation of the Bar (not originally used to make money but basically to prevent the legal profession from being blamed for giving out bad advice when casually asked for it.)
  • by aufait ( 45237 ) on Friday May 26, 2000 @02:50AM (#1047122) Homepage
    On the other hand, I think what Bidder's Edge does is really indefensible from an ethical standpoint and I am rooting for them to lose because they are in effect *competing* with eBay for advertising dollars by *using* eBay's content.

    What percentage of its income comes from advertisinge? And, what percentage of its income comes from the commission on sales?

    It is the old story of the leader in the market trying to keep users locked into their product. AOL does it with Instant Messaging. Microsoft does it with their file fomrates.

    Most other Auction Sites welcome Bidder's Edge's robot. Why? It gives a wider exposure to their auctions meaning that more bidders will see it and raise the price of the item under sale.

    Why does eBay object? Because they are the market leader and don't want bidders to see alternate auction sites. If eBay prevails in this case, then bidders will have to search eBay themselves. eBay is betting that rather than check two seperate sites, the bidders will stay on their site.

    What is the difference between Bidder's Edge automatically doing the searchs and my running a program for me to automatically do the searches?

  • by uebernewby ( 149493 ) on Friday May 26, 2000 @02:52AM (#1047123) Homepage
    If I understand this correctly, the reason the judge forbade Bidders Edge to crawl eBay is that eBay told Bidders Edge not to do it but they did anyway. So it seems fair to me: if I ask you not to keep following me around and you keep doing it, I take you to court to get a restraint order slapped on you.

    Whether it's a smart move by eBay to do this is an entirely different matter, however. Apparently all the experts agree that in the near future all transactions on the internet will be mediated by robots, which search out the best deal on the product you want to buy. So as far as the consumer is concerned, it makes no difference if a book comes from Amazon or Barnes and Noble, he'll just get it where his robot tells him it's cheapest. Most probably, he won't even know it's from a certain retailer until he looks at the box in which it was sent to him. Probably he won't care very much even then. At this moment, there's already a number of services that operate on this principle: Bidders Edge does it for auctions, and in NL there's a site called ElCheapo that lets you find the best deal on products ranging from airplane tickets to records.

    By banning a robot-service from their site, eBay is, in effect, shutting itself off from the way business will be transacted in the near future. They will not merely lose some ad revenue, as is the case now, but they will lose all their customers.

    If this really is what they want to do, I'd say let them.

  • it'd be interesting if such a ruling helped discourage hard-drive searching by software which searches for "undesirable" content without your consent or knowledge.

    Yeah right.
    eBay is a big company with money. Even more, they have a little 'e' at the begining of their name.

    Are we (as individules) big conpanies? No.
    So why on earth would you think the law would protect us?

  • I agree with the gist of this lawsuit -- no one can use my computers (or computing power that I pay for) without my consent. There's case law (in the US at least) on the books that backs this up (re: Kevin Mitnick, and the guy that got caught in the "Cuckoo's Egg" true story by Clifford Stoll).

    I do not object to individuals making use of my site in an interactive manner. I don't mind if people make use of the information on my site. What I do object to is people using a disproportional amount of resources that I am paying for. Many spidering and site sucking programs do exactly that.

    If all programs out there that act like "spiders" (not just the search engines) actually used and respected robots.txt and other mechanisms in place to tell spidering programs how to behave, we wouldn't have to use the courts to enforce our right to tell people if, when, and how they can use our resources.

    -- PhoneBoy

  • by Citrix ( 14447 ) on Thursday May 25, 2000 @09:04PM (#1047130) Homepage
    I would have some sympathy with Bidders Edge but they don't follow the robots.txt [] file on eBay.

    here is []:

    # robots.txt for eBay

    User-agent: *
    Disallow: /aw/listings/
    Disallow: /aw-cgi/
    Disallow: /aw-secure/
    Disallow: /cgi-bin/
    It isn't like eBay is disallowing access to everything, crawlers are allowed to index anything on (no robots.txt) and whatever is not excluded IMO whether the judge knows it he is upholding a standard and that is a good thing.
  • by Jim Tyre ( 100017 ) on Thursday May 25, 2000 @09:06PM (#1047131) Homepage
    As a lawyer, I am always leery of news reports, without reading the ruling itself. But *if* the ruling says what the report says, and if it stands, I can see how site owners could use it to prevent being scanned by censorware bots.

    No scan, no ban?

  • eBay has decided that it would be much more profitable to simply sue all of its customers for infringing upon its personal property.

    After a freak DNS outage Monday morning, executives internal to the network noticed that their servers performed thousands of times better than when all "those damned customers" were using them. CPU load dropped to .001%, and response time was instantaneous.

    It is this degradation of performance that enables eBay to lay claim to the pocketbooks of all its consumers, and auctioneers. eBay is currently debating lawsuits on volunteers helping to run eBay.

    When asked "Isn't this going to jeopardize the good PR that has made eBay so successful?", eBay's response was "Ah, fuck it.. Microsoft has been doing this to customers for years, and they appear to love it... Besides (excitedly) WE'RE FINALLY GOING TO GET OUT OF DEBT!!.

    Seriously though, this is a bad idea. If we can't expect our high-tech businesses to make more informed decisions of how to deal with a problem, then who can we count on?

    The problem itself could easily have been resolved with a robots.txt, and failing that, DENY host settings. Aside from negative PR, why would a company want to take this to their lawyers before taking it to the same techs that made the company to begin with?!?

    The next biggest question, how do we protect ourselves from this?

  • by DLG ( 14172 ) on Friday May 26, 2000 @07:28AM (#1047135)
    As someone who has written search engines, both as spiders and as meta search engines, I found the existence of the robots.txt standard to be a real blessing. It was a way for me to easily follow rules that kept my site from simply being BLOCKED at a firewall. By further doing things like not going to a specific site twice in a row and other wierd tricks, I could know that I was not abusing other peoples site with my code running automatically.

    In the case of a system that is essentially placing their own interface ontop of another system, what I don't understand is why EBay didn't simply refuse the packets. It should not have been THAT difficult. No user has a right to see a site. I have blocked parts of my site from certain users during times when they perhaps inadvertantly caused trouble. I have blocked use of my email system for the purposes of redirection. All of this I do with the technology available to me. The fact is that IF the improper usage of the site is impossible to detect by the owners of the site it isn't a technical problem. On the other hand, if the issue is that they are successfully using the site in a non intrusive manner to profit, it is a business problem, and the problem is NOT with the offendor but with the offended.

    The problem is the notion that a site can control how it is viewed.

    If you tell me that it is illegal to look at a site without seeing its ads then we are in for a long hard battle. Browsers are being built with the ability to recognize advertising banners and exclude them from being viewed. Text based browsers have always avoided banner ads. The use internet for advertising sales will someday be impossible unless the advertising is completely the same as content, which it IS to a large extent. Content can be filtered, and should be. I look at slashdot with a WAP enabled phone. No ads are sent. Sooner or later someone will fix that somehow but in truth the ad will be just a link to another site, which is fine with me.

    In the future it is considered likely that everyone will have a semiautomated search engine customized for them personally. Near future. In the long run, automated agents will be the primary conduit between a user and data, the web browser will be a thing of the past, and the idea that people won't filter out advertising when it bothers them/wastes their bandwidth is nonsensical.

    The Robot.txt file says, "Don't check these URL's for content" which basicly means "Don't read me". In the future sites that restrict use of their pages won't be read. If the only revenue a site gets is through advertising, then they will have to find another way to deliver it.

  • ebay is providing a publically-accessible database; why should a search of that database (by robot or not) be considered an abuse of their servers?

    I agree, a search of their database should not be considered abuse. But what about re-publishing the contents of that database? I think that this is the problem; Bidders Edge presents the ebay results on its own pages, in its own format. The NYTimes wouldn't allow this; hell, slashdot probably wouldn't allow it.

  • Nice theory, however, eBay's pages have no advertising. Go check for yourself.
    • On the other hand, I think what Bidder's Edge does is really indefensible from an ethical standpoint and I am rooting for them to lose because they are in effect *competing* with eBay for advertising dollars by *using* eBay's content. If you view content from ebay through Bidder's Edge, that's advertising revenue eBay doosn't get which BE does. Seems really lousy

    I really find it hard to think that a site the size of eBay makes the bulk of its revenue out of add banners. IIRC, I saw eBay's owner on TV saying that most of their money comes from their cut from sales price []. And this is precisely what they are loosing.

    eBay is the best known auction site, so it's where most people go to when looking for something, or at least it's the first chice of most people. That means anything auctioned on eBay will have a potentially higher demand, thus sell for a higher price. That's what draws most sellers there, turning this into a vicious circle. Quite a comfortable situation...

    What Bidders Edge is doing (and I think the name is quite appropriate here) is bringing competition to the game. Not competition on add revenues, but competition that comes from price comparison. And that's what eBay is pissed of with. This "resources" BS is nothing but a lawyers excuse.

  • You know, news like this is really depressing. I got on the internet in college before the web. Back then, (listen up you young whipersnappers!) if you tried to sell something on the internet you were spammed into oblivion. These days with $$$$,Lawyers, and Uncle Sam's long arm, the internet is fast becoming an environment owned by corporations that control how and what you can do. Pretty soon it will be easier to start a radio station than launch a web site.

The human mind ordinarily operates at only ten percent of its capacity -- the rest is overhead for the operating system.