Amazon Launches Full Text Book Search

m00nun1t writes "Amazon have launched a new service that allows you to search the full text of books. This sounds like an incredibly useful function as well as technically impressive at this scale. I wonder if a patent is in the works." Or if a patent is already owned.

Yeah, but..

can you do it with one click?

Re:Yeah, but..

Some web sites have 100's of A4 pages, but google still returns in a jiffy. I'm pretty sure their book collection is well indexed, if they're offering this service. Probably with the google engine, too.

Daniel

Re:Amazon...

It is really nice, I was using amazon right as they switched it one.
I was searching for books on Object Role Modeling(ORM), I had first done a search for ORM and did not find anything of interest. They then switched it on while I did a search of 'Object Role Modeling', this poped up a few books with the text where it was being used.

You can see whole pages

You can read the page it is on and +/- two pages.

This is equivalent of the facility you have in a physical bookstore to open a book and browse a few pages before purchasing. I can see it might be very useful, if they get the majority of books in a field accessible like this.

I wanted a PHP book the other day, and it is very difficult to decidew which one of the plethora available I wanted. So I went to my physoical bookstore. Smaller choice, but I could open each and get an impression of whther ther were slow, detail by detail, dummies books or the sort of high-speed summary I wanted.

abuse

I can almost hear the screams of joy from the underground book pirates.

How easy can this service be abused, with automatic webbots doing the searching?

I can imagine there might be filters, time limits, and max searchs/day limits for something of this scale, no?

Re:abuse

How easy can this service be abused, with automatic webbots doing the searching?
You can only browse two pages in either direction per search. You also have to be logged in. I suppose someone could script a system to create thousands of account, then use an army of zombie machines to OCR the pages from a variety of different IPs. That is assuming that Amazon has EVERY page of every book available to the service, which I doubt.

It would probably by easier to coble together a robot built around a laptop with an ocr equiped camera and book manipulation software and set it loose in a big library at night. For 50 years.

Re:abuse

You 'almost', but not quite, hear the book pirates, most probably because they don't formally exist. ebooks are widely available in unencrypted format, and the latest releases, while in secure formats such as Secure MS Reader or Adobe, are probably much easier to crack than creating a bot to collect a book online page by page.

ebooks are a pretty healthy alternative to normal books, but I don't see the publishers worrying too much about piracy. Perhaps it's because the average script kiddie who will spend 2 days downloading Matrix Reloaded from Usenet is just not the type to try and crack open a book, much less crack an ebook.

Re:abuse

How easy can this service be abused, with automatic webbots doing the searching?

Not so easily. It's easy to see why. The books will be scanned in using OCR. These days a fast and convenient and almost error-free process. But not entirely error-free. Good enough to find documents that are highly relevant to a particular keyword (if "hydraulics" occurs 9 times, what are the odds of OCR getting it wrong all 9 times?) but not good enough for entirely automated book-to-text.

If amazon would display highlighted portions of the books contents if would probably not exceed a few lines, just like google doesn't present entire webpages in it's result screen). If they did want to show more, they'd have to show an image of the scanned in page anyway, since OCR errors would not be very pretty. (A lot of digital archiving products use a similar approach; they index PDF files that contain the OCR'ed text, invisible to the end-user, and the scanned pages as content which the end-user looks at).

Besides, to search for each page of a book, you'd have to search for a keyword on each page of that book. Such keywords would most easily be extracted by scanning in the book via OCR anyway!

It works!!!

1) I typed 'porn'
2) It returned a lot of results

Conclusion: It works!!!

Fine grain searches take the adventure away

Back in the early days of the web, when Yahoo was still a catalog of links and not some super news/search/auction/ebusiness/do-it-all website that it is now, searches were much more fun.

You really never knew what would turn up as you traversed the Yahoo directory structure. You start searching for blues music and you'd end up with a list of 15 or so good links with .wav samples and more than likely an artist you'd never heard of before. That was the best part, getting introduced to things you hadn't even thought to look for.

As search techniques are becoming more refined, we are now able to do specific word searches on websites and now books. That's fine if you know exactly what you are looking for. For example if you want to get that book about 'replicants' you'll find Blade Runner, but you won't find anything else. You won't get any information except exactly the thing you are looking for.

And I think that that is where the problem with this kind of search lies for books/music/etc. If you want to find a song or a book, it most likely isn't going to be a specific word you remember, it will be the tune or the plot, both of which are not searchable.

I don't see this improvement in Amazon's search system as that much of an improvement. A better improvement could be made to the 'We thought you'd like' feature. Instead of finding only what I'm looking for, I'd like to find other things I might also be interested in.

Potential tool for discovering plagiarism?

I remember a teacher once telling a class I was in that our essays may be compared to other essays published online to check for plagiarism.

Granted, Amazon.com's feature will only (for now) include 150,000 books, but this may very well be another way to catch plagiarizers. Just type in a suspicious phrase and see if there are any 'hits'.

No Searching Inside O'Reilly Books

Even though he said he was 'blown away' by Amazon's new Search Inside the Book feature, Tim O'Reilly has decided not to participate in the program [wsj.com] for now. 'If they end up being a Google for published content...we need to think better about what publishers get out of it,' he said.

Re:No Searching Inside O'Reilly Books

As a Safari subscriber, I'd say it's probably because Full Text Search of online book content is also present at O'Reilly's own Safari [oreilly.com]online tech book site. You've been able to do the same thing Amazon is now crowing about, on every book Safari has, since launch quite some time ago (year or two perhaps?)

Safari is more of a "service" (i.e. renting access to book content) than a "feature" of a retail website, which is all Amazon's "innovation" seems to be.

Basically the only real different between the two (aside from what is cited above) is that Amazon just lets you know the content is mentioned, and shows you a page or two. Safari gives you the entire book. That and that Amazon has a much wider range of books in non-tech genres

No more out-of-print books

As the digital index builds up, we will rapidly come across the situation where the electronic book is searchable, but the printed form is out of print. If this service ultimately allows single copies to be printed for delivery, it will be an outstanding demonstration of print-on-demand technology as advocated by the Print On Demand Initiative [podi.org] and others.

I'd love to be able to browse a giant back catalog, knowing that an original or facsimile copy could definitely be delivered to me.

One click search.

In other news... Amazon announced that the USPTO has granted them a patent on their proprietary "one click search" technology.

When questioned for comment Google CEO Eric Schmidt said "ug".

New age youth

Youth in the old days: lookup 'vagina' in a dictionary.
Youth nowadays: lookup 'vagina' in all books on this planet.

Wow!

I'm impressed. A couple of days I want onto Amazon to find books about Singular Value Decompositions (a mathematical technique that can be used for efficient statistical analysis of large groups of documents, among other things). I wasn't particularly surprised when it returned 0 results, since anyone who puts the term "Singular Value Decomposition" in their book's title obviously doesn't know much about marketing. Of course I don't actually give a damn if the term is in the title or not; I just want to know if the books talks about this technique.

I tried the search again today and got nearly 5,000 results, with the capability to actually look inside the book and see if the reference is useful to me. Very impressive indeed, patent or no patent.

Various worthwhile uses

Bash Amazon all you want, but this is a very useful technology.

In five minutes I was able to find three books that talked about findings first listed in two of my own published scientific papers, yet these books did not cite me, or anyone else, as the source of that information. My lawyer is currently preparing three letters.

I also found two other books in which the author used verbatim quotes and original theories from various interviews I have given, yet both authors passed off the statements as their own. My lawyer is now preparing five letters.

Aside from being used to protect my own research rights, I have found the search system useful for finding topics of interest discussed in certain books which are not referenced in any of the descriptions about the books. I just ordered three books I would not otherwise have ever purchased.

While I don't think highly of all of Amazon's practices, I must hand it to them for whatever technical undertaking created this search feature.

ebooks vs CD/DVD

I warmly welcome any initiative that makes more and more books, or parts thereof, available online.

I used to think, like many people, that ebooks just didn't work because 'I like the feel of paper under my fingers'. Since I bought a PDA and discovered the joys of Fictionwise [fictionwise.com], I just can't go back to these clumsy wood pulp apparels.

Amazon is pretty progressive in this regard, making a great number of their collection available electronically. It was probably fairly easy from there to make their stock searchable. And how I wish the MPAA and RIAA could work like the publishing industry...

The existence of ebooks is NOT threatening traditional books, because people see more value in a printed book over an electronic copy. This is clearly not the case with a CD and a DVD, since most people couldn't care less about the jacket if they have the goods on the CD/DVD. I wish the MPAA and RIAA would understand how to make traditional CDs and DVDs "value-added", and make people less inclined to getting a computer file instead of shelling out the money.

Then again, I guess the case with ebooks is that your typical DVD or CD pirate is just not interested in swapping files to get the latest Stephen King and read it on screen. Not only that, but most of History's greatest books are available for free, and one could probably read free books for the rest of their lives if they chose so.

Anyone else notice this?

You have to have an account to view the pages. Fine, great. But then it brought up this screen:

By publishers' agreement, we are pleased to offer Amazon.com customers with a valid credit card the ability to view copyrighted pages.
Your account will not be charged.
This one-time process enables you to view limited copyrighted material through our Search Inside the Book feature.

So they'll let you browse the search pages, if you can prove your identity on record and provide them with financial information. No thanks.

Scanner problems

Neat idea, but some excerpts come out all wrong:
See this for example... [amazon.com]
Mass-OCR'ing has it's drawbacks..

This technology is called. . .

"grep."

I believe there is a body of prior art for scanning in books and greping them. Is that not one of the oft repeated benefits of ebooks?

Whether or not Amazon can get a patent on a shell script to serve up the results . . . on the web oooooooo, remains to be seen I suppose.

They managed to get one on "Give me one of those, put it on my account and drop it by my house" a "technology" my grocer has been offering over the phone for 40 years that I'm personally aware of.

However, since this sort of "technology" is exactly the sort of thing that the web, and the internet itself for that matter, was invented for I'd have to guess there's a lot of prior art. It's certainly obvious and trivial, but that doesn't seem to count for much these days.

The problem with things that are so obvious and trivial that "everyone" has been doing it for decades is that it's hard to demonstrate in court because no one bothers to document it.

Can you prove your grandfather put his pants on one leg at a time?

Common sense tells you he did, but common sense no longer applies in an age that grants patents to perpetual motion machines and peanut butter sandwiches.

KFG

Wired article: "The Great Library of Amazonia"

Article [wired.com] in December Wired talks about Amazon's book scanning, how they legally do it, who does it, how many books so far, and protections.

Now we just need...

A full text search of slashdot, so the editors can search for duplicate articles before they post.

Scott

How do we know _which_ books...

...are included in the search?

A check on "the clocks were striking thirteen" yields seventeen hits, including the Cliff's Notes to Nineteen Eighty-Four and a reference in the Oxford Dictionary of Modern Quotations...

but none to Orwell's Nineteen Eighty-Four itself.

We must conclude that the coverage is spotty.

Why..This would be like searching through the LOC!

What a feat of computing genius! Using computers to search through large bodies of text!!!! Has ANYONE ever done this before?!

Patent Bashing Du Jour

Or if a patent is already owned.

This type of editorializing is pathetic in that its only purpose is to stir up the masses. Gee...now let's take a look shall we? 20% of the comments are "patents suck" or "isn't this some example prior art"?

This story is about a new feature people...it's not about a patent. Wipe the froth from your mouths and comment on the merits (of lack of) the feature...not on a completely fabricated hypothetical comment meant to incite you into a frenzy.

Here's a quote relevant to the parent post

There's books about everything:

Encyclopedia of New Media : An Essential Reference to Communication and Technology -- Steve Jones (Editor); Hardcover

Excerpt from page 0: ". . . post-ranking system used by members the of Web message board Slashdot.org, began as a result of community self- restraint in the face of unrelenting trolls (pointlessly hostile posters). In addition, some cyberspace forums now require . . ."

See more references to slashdot troll in this book.

Re:Amazon have?

I don't think there is in the generall case a correct answer to whether collectives should be singular or plural - it depends upon the context.

"Congress have failed to agreee..." because you are talking about a lod of swuablling politicians who are definitely plural.

"Congress has past a bill..." because those politicians have managed to achiueve a consensus and act as as a single entity.

In this case the sungular is correct, because Amnazon as an entity is offering a new service. But you could use the term collectively for all employees of Amazon.

Re:Those crazy Brits

Well at least they don't refer to a liquid as 'gas' like the Americans do when talking about petrol.