Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
The Media

Computers Summarize the News 175

Posted by michael from the replaced-by-a-very-small-shell-script dept.
oily_ants writes "I get sick and tired of reading the same story on different web sites. That's why I like slashdot so much. Good (??) summaries of all of the stuff out there on the net. Now there is a project at Columbia University by the nlp group that attempts to generate computer summaries of all of those news articles on different web sites. The project is called Newsblaster and the summaries are excellent. You can read about the project on regular news sites like Online Journalism Review or USA Today."
This discussion has been archived. No new comments can be posted.

Computers Summarize the News More

Computers Summarize the News

Comments Filter:

  • Also try... (Score:5, Informative)

    by SlashChick ( 544252 ) <erica@noSpam.erica.biz> on Friday March 15, 2002 @12:45PM (#3168612) Homepage Journal
    news.google.com. [google.com] Just released yesterday. I haven't yet played around with it enough to say whether it's cool or not, but it does look promising.

    • Re:Also try... (Score:2, Interesting)

      by FinnishFlash ( 414045 )
      Yesterday ?

      Okay the timezone is different (I'm in europe). But two weeks...

      There you can also easily see how differently different newsagencies report the same story, for example few days ago there was this story about depleted uranium and it being dangerous:

      news Agency #1: new study finds connection between depleted uranium and hightened risk to get cancer.

      news Agency #2: Soldiers exposed to depleted uranium bullets in risk to get cancer.

      news Agency #3: Children in Yugoslavia might get cancer because of NATO's depleted uranium bullets

      Talk about reporting the facts objectively...

      It would have been intresting to see the summary generated from these three... Might have been a bit schizophrenic...

    • Re:Also try... (Score:4, Interesting)

      by elfkicker ( 162256 ) on Friday March 15, 2002 @01:04PM (#3168747)
      I've been using that for a couple of months now. (It has been available buried at http://www.google.com/news/newsheadlines.html) I find it very useful. I wish they'd explain exactly how it worked though. Is it all machine parsed? Are the articles listed in order of relevance, time posted, etc?

      I've see a couple occasions where it's had an article on a completely different subject under the header, but it's not the norm. It's always up to date. My only gripe is that it doesn't have an "Older Stories" link. I've gone back to try and find something I've seen before only to find that it had been pushed off.

      They also keep a list of links to news sourse and current relevant resources at http://www.google.com/news/ [google.com].

      • There is a little information available [google.com]; the headlines are selected automatically, though that's all the info that they're willing to give out at the moment.
      • I don't know how they work exactly, but it's certainly not hard to guess at some of the workings. Assuming they can determine what page hits are news articles and that they can strip the article text only from the body of the page, then one can simply grab words that aren't part of the common langauge, including proper nouns, legal terms, etc. Possibly donote some longer phases, such as "Space Shuttle" or "Artificial Intelligence" as phrases to watch for. This gives each page a set of keywords that make it 'unique' from other news articles on the same site. Then one can argue that articles that are written within the same timeframe (48hrs) and have similar sets of keywords are the same news story from difference sources, and they can group those together as they display now.

        Of course, they might have a completely different formula, or they may have a large body of optimizations they can do, but that's one possibility.

    • the google service seems to work pretty well - it certainly gets around the main weakness of google that its CRAP for finding information thats less than a week old.

      As they add new sources to the service I reckon it will start to truly rock.

      If I have a criticism its that I have to go to the original site to view the story - all I get at google is the headline. This means I have to wait about a MONTH for a CNN page to load when I want to know some more...
    • news.altavista.com [slashdot.org] is a similar service. Been around for a while.

  • One More Time! (Score:2, Funny)

    by Anonymous Coward
    "I get sick and tired of reading the same story on different web sites"

    So you read Slashdot, where they are happy to post the same story over and over, on different days?

  • Say what? (Score:5, Funny)

    by turbine216 ( 458014 ) <turbine216@NosPAm.gmail.com> on Friday March 15, 2002 @12:46PM (#3168624)
    I get sick and tired of reading the same story on different web sites. That's why I like slashdot so much.

    I'm sure most will agree with me when I say that this makes ABSOLUTELY NO SENSE.
  • Try Google News Headlines [google.com]. It automatically finds news stories on the different news sites and groups them togther. From the page you can see the most popular news topics and read several different views of the same story.
    • Wouldn't that kind of go against the part where he said "I get sick and tired of reading the same story on different web sites"?

      • Re:Google kind of does the same thing (Score:1)

        by Anonymous Coward
        My take on it is that the author doesn't like reading all the headlines on cnn.com, then going to abcnews.com and wading through the same headlines to try and find a new story. Then the same thing at the BBC and so on.

        I could be wrong though...

      • If it's the AP feed that they're all picking up, then sure, it's the same story.

        What's interesting is when the story is different - if CNN says one thing, the BBC says something a little different, and the Times of India has yet a third viewpoint, then it's interesting to speculate on the editorial biases that are leading to such divergent viewpoints.

  • Well, there you go (Score:3, Interesting)

    by Otter ( 3800 ) on Friday March 15, 2002 @12:48PM (#3168631) Journal
    Now there is a project at Columbia University by the nlp group that attempts to generate computer summaries of all of those news articles on different web sites.

    Well, there's the answer to the Ask Slashdot from a couple of days ago [slashdot.org].

  • From google's main page, click "news and resources" to see stories from all the world's major papers.

    As others have pointed out, they've also just launched a beta news summary service.

  • Now to ask... (Score:3, Insightful)

    by Sorthum ( 123064 ) on Friday March 15, 2002 @12:50PM (#3168646) Homepage
    ...whether this will include the obscure stories that are actually interesting, or whether it'll be just a rehash of the major stories that we can find in ten or twelve other places.

    • Re:Now to ask... (Score:3, Informative)

      by oily_ants ( 566702 )
      It's just a rehash of all of those other stories. But the nice part about it is it is in reader's digest condensed version. I only have to read one small paragraph to get the major points of the event instead of sifting through a long article that doesn't include much actual information. It is meant as a summary so the information is NOT the obsure stuff (which is interesting) but quick and dirty summaries of important events.
  • Our company has been running a similar service for a very long time. It's free, and you canget it here [newsscape.com]. It's called NewsScape.
    • No you provide a basic news grouping and ordering service, this sumarizes the articles based off of many different sources. This is sort of like Slate's [slate.com] Today's Papers feature except for articles and not just the days news.

  • Newsbot (Score:4, Funny)

    by TheGreenLantern ( 537864 ) <thegreenlntrn@yahoo.com> on Friday March 15, 2002 @12:51PM (#3168653) Homepage Journal
    Sounds like a good idea, but I'm worried about the "Newsbots" objectivity. If I wanted to read a bunch of stories about the latest NVidia GeForce 4 release, 10 reasons more RAM is better, and why you should upgrade your hard drive, I'd just watch TechTV.
    • So you think getting your news from a single source is somehow more objective than from a dozen competing sources?
      • I think that you two agree, actually -- I read the thread starter's last sentence as a criticism of TechTV given that what he listed (news meant to make you BUY BUY BUY) might instead be referred to as "marketing".
    • I'm not worried about that any more than I am about a meaty editor's objectivity.

      I'm more interested in whether the thing makes any humorous errors. That and whether it can eventually out-rotten Daily Rotten [rotten.com].
  • Looks like subscriptions can be axed, Slashdot won't need editors anymore!

    Although, it will only be possible to replace slashdot's editors with the newsblaster program if they can implement some sort of misspelling and false information algorithm.
  • I've been using my.yahoo.com for years as my start page. It gives me a good concise and highly configurable view of many news feeds and financial information (stock quotes, etc). Recently, however, it has become overwhelmed my annoying ads.

    Does anyone know of an alternative? The newsfeed in the article looks prommising, but not exactly what I'm looking for.
    • I'm a big fan of Yahoo Messenger for this. You can display the top X news items in any categories you want to follow, and present the categories in any order you like. I have top 8 stories by Reuters and AP, followed by science from Reuters and AP, tech by Reuters, AP, CNET, ZDnet, and Reuters Internet Report, entertainment from E! Online and oddly enough from Reuters.

      Also follow stocks, check the weather, see if I have Yahoo e-mail, and chat with the same application. And there's only one minor ad at the top, which I don't notice anymore. I'm not saying you should change from ICQ or AIM or whatever, but Yahoo Messenger is a pretty cool app for keeping up with the world outside your cube. ;)

      -FF
  • Does anybody know how I can get access to Newsblaster so I can read a synopsis of the article to determine whether Newsblaster is something that would interest me?

  • I think Newshub is better (Score:3, Interesting)

    by brunnock ( 18853 ) on Friday March 15, 2002 @12:53PM (#3168668) Homepage
    http://newshub.com/

  • Impressive (Score:5, Informative)

    by Reality Master 101 ( 179095 ) <RealityMaster101@gmail. c o m> on Friday March 15, 2002 @12:53PM (#3168670) Homepage Journal

    To tell you the truth, at first I thought the summaries were TOO good; I was suspicious that it wasn't really automated.

    But after looking at a few more stories, it looks like it just pulls sentences out of the stories that seem to have a different point to make, and strings them together.

    Sometimes you see some redundancy and some non-sequiturs, but I have to admit the illusion is pretty good.

    • Read the papers (Score:3, Informative)

      by mizhi ( 186984 )
      Reserach Papers [columbia.edu]

      I'm not sure if they've done anything really novel. I skimmed through one of the more recent papers, on sentence ordering; but that seem to only operate on the same event There's research like this going one at alot of major universities like CMU [cmu.edu] and MIT [mit.edu]. That said, it does look impressive.

      • Re:Read the papers (Score:5, Informative)

        by DavidKirkEvans ( 147595 ) on Friday March 15, 2002 @01:46PM (#3169002) Homepage
        We have a summarization strategy that selects from three summarizers: one that works over documents describing a "single event" which is novel, one that works over documents describing a person (so-called biography events) using sentence extraction, and one that is a general sentence extractor based on the biographical summarizer which does use more than just TFIDF weighting for the extraction. (It has a notion of semantic classes, and some other stuff.)

        The "single event" summarizer is novel though. It uses a clustering component to cluster the sentences, then for each cluster it takes the intersection of the sentences (yes, we need to parse the text to do this, and we do) and RE-GENERATES (does not extract) a sentence that synthesizes the information from the cluster.

        There's a lot of other stuff going on as well, we're using a text categorization system that we developed here, a text clustering system, our own system for categorizing the images that come with the articles (you'll be able to browse by image categories soon as well) and some other stuff.
    • for when super-intelligent machines take over the world. Then they`ll be making the news AND reporting it. Perhaps we should build in such an interface now - perhaps they`ll be easier on us then...

  • breadth vs depth (Score:5, Insightful)

    by Ubergrendle ( 531719 ) on Friday March 15, 2002 @12:54PM (#3168672) Journal
    This is a somewhat dangerous trend, IMHO. CNN Headline news gives us blurbs...soundbites...with no substance. "Israelis shot Palestinians" or vice versa on a daily basis. Little reporting of substance of negotiations; why there was a conflict in that location at that time for what reason. The great thing about the internet is that there is great reporting in depth. I like to check out the Drudge report, BBC, disinfo.com, etc on a regular basis to get a good blend of various points of view so that I can make my OWN opinion. I don't want to be served watered down sentence fragments by a corporate AOL/TimeWarner beheometh. Slashdot is one of a few exceptions to this rule, since they typically link to articles of substance and allow for dialogue and debate by (usually) intelligent users. The moderation system isn't perfect, but it helps dodge the trolls. My guess is that automated summaries will lose the flavour of good journalism/writing, and by taking an "average" will end up with a C+ "factual comprehension" review as opposed to multiple A+ "theory" and "syntehsis" editorials.
    • This is a somewhat dangerous trend, IMHO. CNN Headline news gives us blurbs...soundbites...with no substance. "Israelis shot Palestinians" or vice versa on a daily basis.

      Newsblaster doesn't do that. What it does is grab news stories from a selection of different sites, searches through the stories for certain words, phrases, or sentences, and then creates a summary of the story, puts it in a heading under the "hits" it made, and provides the link.

      This actually unseated /. from its throne as my home page. =]

    • I don't want to be served watered down sentence fragments by a corporate AOL/TimeWarner beheometh.

      Sure you do, when you're trying to figure out the gist of the story in overview mode. The problem you mention with CNN HN is because TV is a non-interactive medium, and you can't find out/they can't provide you with more information about the story in question. With this, you can.

      The summarizer has to get its information from somewhere, from the full news story...this is just a way of giving you the executive version (akin to browsing slashdot with a comment threshold to 5), so you can find out the basics before delving into the details.

    • don't care about "good journalism/writing". In fact, most of us don't care about most of what goes on in the world. Hell, most of us don't even know that there *IS* a world outside the United States, except when we go somewhere to kick some ass.

      This new averaged, filtered, genericized "news" is exactly the kind of crap suited to a society that spawned "Judge Judy" and "A Current Affair". Sure, it's a nice piece of technical wizardry, but all things clever are not useful or worthwhile.

    • Fair point, though Newsblaster does link to the original stories so you can read them and check what they actually say if, having read the summary, you're interested.

      Having said that, my preferred source for real-world news is Radio 4 [bbc.co.uk] (especially Today [bbc.co.uk] and PM [bbc.co.uk]), which is especially good for listening to as I'm waking up or going to sleep, apart from the moments when a politician says something so outrageous that my blood boils...

  • Still Some Work To Be Done... (Score:4, Funny)

    by po8 ( 187055 ) on Friday March 15, 2002 @12:55PM (#3168677)

    Check out this odd story about incarcerated Browns [columbia.edu]. The summarizer could apparently still use some manual supervision.

  • Seems nice enough... (Score:4, Insightful)

    by Davorama ( 11731 ) on Friday March 15, 2002 @12:55PM (#3168681) Journal
    So where's the slashbox for it?
  • The news. or newsblaster sites remind me of the days when the web was just starting to get going. No banners, pics, ads, frames, etc... Just lines and lines of text for one to sift through.

  • Copyright Infringement - Fair Use Doctrine -NOT! (Score:3, Interesting)

    by Anonymous Coward on Friday March 15, 2002 @12:58PM (#3168700)
    This is usually Copyright Infringement - Fair Use Doctrine is not applicable.

    Every one of these paraphrasers lift large chunks of syntax.

    I would maintain that this is still a plagiarsist or copyright violation unless it is done really well.

    And it never will be done really well unless NeuralNetwork chips are common and mankind has advances in Artificial Intelligence research. Five years away at best.

    I dare the commerical services to hit Enyclopedia Britannica. Or I dare them to routinely slurp New York Times and boast that they digest the New York Times..

    A massive Civil Suit is awaiting some of these early adapters planning on creating a business out of this.

    And they deserve it.

    It is just "Word Twiddling", however useful.

    If the twiddling is done live, once, per user client, then maybe its OK, but none of these business models are setup THAT way.
    • Umm, you seem to have a few problems:
      • Neural Nets need not be on chips to be used, indeed one can (as I have) do them on paper or mentally. Indeed I am unclear on what a NN chip would be, a very fast matrix solver???
      • The legality of summaries is always a vexing question. Since this system generally will take similiar lines of text, a lot of what it takes are quotes to begin with. Also it does cite the articles. It doesn't seem to take long enough stetches to be in violation, though it could.
      • As for a business plan, well it is a univeristy project, and probably not one taken too seriously. They aren't making money off this any way I can see

      I do think this service would be a lot more usefull if it were done so that one didn't have to go to a seperate page for the summary.
    • Why would fair use not be applicable?

      Slurping a sentence or two from an 5-25 paragraph article and quoting it with attribution is considered fair use, right?

      I'm not clear on if they're quoting and attributing it sufficiently to meet a legal challenge however. IANAL. But it's not the open and shut case you make it out to be as far as I can tell.

      --LP
      • The original poster is correct. This is essentially automated plagiarism. Here's why.

        The service claims to be a computerized summary. However, in terms of copyright, a summary is something that expresses the same idea using different words. Therefore, using exact quotes and labelling them as a summary is a textbook case of plagiarism.

        Nathan
        • I think you are confusing plagarism, and a violation of copyright. I am primarily concerned with the legal issue of Copyright violation raised by the previous poster, not an amorphous ethical one.

          As Bitlaw [bitlaw.com] points out, under the Copyright Act, four factors are to be considered in order to determine whether a specific action is to be considered a "fair use." These factors are as follows:

          1) the purpose and character of the use, including whether such use is of commercial nature or is for nonprofit educational purposes;
          2) the nature of the copyrighted work;
          3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
          4) the effect of the use upon the potential market for or value of the copyrighted work.
          • Dang, clicked the submit button by mistake.

            Attempting to apply the four factors there, while some could be argued either way, I can see that on balance, you both might be right. I could probably make a stronger case that it doesn't qualify as fair use, than that it does, based on those four factors. I think I was focusing over-much on the "amount taken" criteria and overlooking the others.

            --LP
          • I am pretty sure that plagiarism is a de facto copyright violation. You are using another author's words without proper credit. Even if the author is cited in the "summary," the definition of a summary is that the words are the authors, not the cited person's words. If they are going to use the author's exact words, it needs to be labeled as an "auto-quote" generator, rather than a summary.

            Nathan

  • copyright/legality? (Score:5, Interesting)

    by ddeboer ( 197882 ) on Friday March 15, 2002 @12:58PM (#3168703) Homepage
    What are the copyright or other legal issues to republishing news stories collected from web sites? The Newsblaster site clearly states where the information comes from - like every good college student is taught to cite information sources. On the other hand, on the bottom of many of the stories is the notice: "Copyright 2002 Associated Press. All rights reserved. This material may not be published, broadcast, rewritten, or redistributed." Is collecting and condensing news stories "republishing" - does this violate copyright stuff?
    • What are the copyright or other legal issues to republishing news stories collected from web sites?

      Paraphrasing is legal under copyright law, so long as the sources are cited and it's not just a cut-and-paste of the entire selection.

    • They copy the entire article text and redistribute it from their own server, without ads and even the "branding" from the original site. That's a very different animal that slashdot, which just links to the original material at the original site.

      Sure, their main page just briefly quotes, which is probably ok, but all the links point to local copies of the copyrighted news articles.

      (In the USA) there are four criteria for judging what is a fair use [benedict.com] of copyrighted material.

      The purpose and character of their use isn't academic or educational, it's a news service just like the original sites they got the text from. The fact that it's hosted from a .edu domain doesn't change anything.

      The amount and substantiality of the portion used in relation to the copyrighted work as a whole is darn close to 100% of the copyrighted material.

      The effect of the use upon the potential market for or the value of the copyrighted work is particularily bad... if people can easily get the news from this convienent summary site, why would they bother to visit the original site (and thus be an audience for their advertising, become "loyal" readers, etc).

      Now the nature of the copyrighted work is informational news, and not really expressive (like songs, movies, etc), so at least they've sort-of got one of the four criteria for fair-use.

  • Direct NewsBlaster link (Score:3, Informative)

    by Alien54 ( 180860 ) on Friday March 15, 2002 @12:59PM (#3168715) Journal
    The direct link is here:

    www.cs.columbia.edu/nlp/newsblaster/ [columbia.edu]

    although I found some of the summaries slightly shallow, they are not bad.

    The problem is that it becomes an average of opinion, when you sometimes need that longer insightful article. This easily could become the news of sheep everywhere.

    This could be bad when facts come in to contradict initial impressions.

    oops

  • Don't post the competition!!!!

  • I think it does an OK job for a computer, but nobody's going to accuse those summaries of being overly coherent (or well-typed).

    Occasionally, the summary will juxtapose two sentences (it's just ripping examplar sentences from different stories), that when put together create screw up the meaning [columbia.edu]:

    "Now that David Letterman is staying at CBS, ABC s corporate bosses took steps to mend fences with"Nightline"host Ted Koppel on Tuesday. And that ' s appealing to beer companies..."

    Doh!

    I think a more fruitful avenue of research is new methods of presenting information so that humans can decide what to read. Instead of using tricks to simulate a computer understanding the meaning of an article, this uses the same tricks to simply assist reading the article.

    Apple's research group did some interesting work [acm.org] in that area in the 90's.
  • I suspect that U.S.A. today has been using a similar technology for years now to generate their "McNews".
  • I already get this in my email from other people:

    Hi, I send you this news to ask for your advice!

    Along with all kinds of pertinent documents...
  • ...500 different opinions on every headline with a little insight here and there. It's already like getting news from different sources.

  • Grammar, even (Score:1, Flamebait)

    by micromoog ( 206608 )
    Wow, this bot's grammar is far better than the human editors at Slashdot!

  • So they grab news from the Washington Post, Reuters and the BBC (amongst others), but leave out the National Enquirer [nationalenquirer.com]? Why can't I have all my "Space Aliens Abducted Britney Spears" stories in one place?

  • How does it work? (Score:3, Interesting)

    by AndyChrist ( 161262 ) <andy_christ@yah[ ]com ['oo.' in gap]> on Friday March 15, 2002 @01:11PM (#3168793) Homepage
    I don't see any concrete information on what it does to summarize stories...is it using something like Cyc [cyc.com]? Does it just have some heuristics for picking out the important parts of paragraphs?

    Also, who else thought "neuro-linguistic programming" for at least a moment when they saw "nlp"?
  • ... that attempts to generate computer summaries of all of those news articles on different web sites. The project is called Newsblaster and the summaries are excellent Is it relly possible that a computer created an excellent summary of a news article? All the computer generated articles I've read were almost unreadable... Just look at babelfish [altavista.com] translation. Is that really english? Hope that NewsBlaster will be much better!
    • I read a few of the summaries and they aren't bad at all. Much better than other attempts in the past. Here is a description of one of their projects. Now I don't know if TIDES is used in newsblaster, but it's still interesting.

      In the TIDES project, we will develop a practical, multilingual and multidocument information tracking and summarization system. Our design features the integration of robust, statistical techniques, shallow linguistic approaches and machine learning to achieve scalability within languages and portability across languages. To realize these goals, we will develop methods for summarization across documents using information fusion and identification of key differences, summarization across languages relying on identification and translation of terms, and new methods for identification, expansion and translation of terms. Unlike most other approaches, rather than relying on sentence extraction, our work uses information fusion of similar information, merging together repetitive phrases into a single phrase allowing dramatic reduction of information across many articles. Our work will focus on characterizing types of differences to include in a summary, which is an unexplored direction in multi-document summarization. We will develop difference operators to identify new information, contradictions, trends, multiple perspectives, and different topics. Our approach will minimize reliance on full machine translation, instead using identification, expansion and translation of terms where possible. We will begin work with a language such as Spanish, but quickly expand to include Asian languages and other non Indo-European languages.

      From the look of it, NLP (natural language parsing) seems to be evolving nicely. It used to be that NLP required processing the entire document and understanding the sentences by mapping heirarchies of valence/word order.

  • Here is one borderline-incoherent Newsblaster summary [columbia.edu]:

    Will Hollywood's 'Tomb' be a box office 'Raider?'

    Summary:

    After the success of last year ' s Lara Croft: Tomb Raider and high expectations for Resident Evil, out Friday, studios are booting up for more. The game: Alice in Wonderland gets a twisted remake in American McGee ' s Alice, a gothic horror version of the classic tale; based on the game by Electronic Arts Studio: Dimension Status: Horror master Wes Craven directs. The game: A sunglasses-wearing all-American hero blows away bad guys with machine guns; 3D Realms Studio: Dimension Status: In limbo. Star Angelina Jolie was attached to the sequel before the original Tomb Raider opened. The game: Amateur taxi drivers take to the sidewalks and crowded streets, picking up customers and delivering them to their destinations unscathed; Sega Studio: No distributor yet. Brothers Jon and Erich Hoeber(Montana) currently are writing the screenplay.

  • Pretty soon summaries will be one word 'catch-phrases' which convey enough meaning for us to get the gist of the story. We can then 'think' to the computer let me learn more about this. And then we will.

    the future. god bless it.
  • Those. clowns. in. Congress. did. it. again.
    What. a. bunch. of. clowns.
  • i check www.newslinx.com on a daily basis. very good source of all-round news.

  • So I need to go to several regular newsites to read the stories about a site that is supposed to make such requirements unnecessary......ok

  • I want to read today's news today. I can pick yesterday's paper out of the trash can. They're still speculating on how Holy Cross will make out against Kansas. (Hello, the game was last night. Kansas won.)
  • The O'Reilly network already has a pretty good resource for news from multiple sites in the Meerkat Wire Service [oreillynet.com]
  • 1. yahoo's my start page. I get to see up to 4 possibly-relevant news articles.

    2. Fark. I get my good share of the weird news, and of course NewsFlash articles, which are just links to other news sites. I'm happy.
  • The article in the Online Journalism Review [usc.edu] says: "Newsblaster seems to make things somewhat generic or more conservative, especially when summarizing reports over several days. This can take away the editorial edge or nuance that a reporter or editor might use to make a lead or report powerful. Summarizing news over several days in this approach results in a certain staleness."

    I noticed the "blandness" of the summaries too, but I think that's a benefit-- reading CNN stories can get really tiring after a few minutes since everything has to have as much punch as possible.

  • Here are some papers (Score:2, Informative)

    by Anonymous Coward
    Here are some papers [columbia.edu] about Newsblaster and computer text summarization in general.
  • I get sick and tired of reading the same story on different web sites. That's why I like slashdot so much.


    I assume that means you only read the stories here, and not the posts. :^P
  • or just interleaving? [columbia.edu]

    Compare the summary with the opening sentences of the first and last articles. Maybe we should wait a bit before speculating on the business impact of this "technology".

  • i wonder how long until they have a subscription plan.

    someone should make line graphs.

  • Our product (plug: www.knowledgebooks.com) does much the same thing, except it finds short relevant phrases with some postpocessing performed (e.g., pronoun resolution).

    Our product works by first categorizing text articles, then identifying which phrases most effectly support the categorization of the article.

    subject of copyright infringement: several people have pointed out that the linked site may go beyond fair use of text on the original news sites.

    I bet that the university obtained permission like I did. I sent about 10 news web sites copious documentation on what my system does, and three gave me permission to use their sites. As is usual in life, it helps to ask politely!

    -Mark Watson www.markwatson.com

  • Try and have it summarize a Jon Katz story. :-)

    I have yet to figure out exactly what his point is in a story, so it would be really interesting to see software try and handle it.

    Of course, it did say it summarizes NEWS stories.

  • There is an easy way to sort out badly-formulated articles. Just let the reades decide if the article is:
    a) interesting
    b) well-written
    c) in the right column

    or similar.
    So the majority of people who visit these sites wont have to worry about "bad-news" :)
    Those articles that get downvoted get off the frontpage. This process repeats whenever an article is added and assures a high-quality.

    comp.sci
  • Human journalists make connections between facts and between events or stories that can add context to a current report. This kind of contextualization is something that Newsblaster cannot do.
    Now is this necessarily bad? I often times have to hack and slash through bias and opinion to get to the facts. It's time like that I wish I had a little robot to monotonously spit the facts to me. This is a great tool because it carries little opinion about the events, and forces the reader to do all the real thinking. Sure, it only cranks out blurbs, but I find these blurbs better than "3 Israelis Killed" [to 15 Palestinians]* headlines I keep seeing.

    *I'm not trying to take a side in that, but I've recently found American news sources *seem* to skew that kind of news a little.

  • A Few Kinks, A few Comments (Score:3, Interesting)

    by Irvu ( 248207 ) on Friday March 15, 2002 @01:46PM (#3169000)
    As the summary here [columbia.edu] shows there are still a few kinks in the system.

    While I have to agree with some people that this isn't in-depth reporting I do think that it is pretty interesting AI. When it comes down to it the problem is not that a computer might be summarizing our news. The problem is twofold.

    Firstly people are not always inclined to look beyond summaries. When faced with typical time constraints people prefer to look at summaries because they do not have time to search across a dozen sources and articles. This is why USA today became big in the first place. Nothing there is more than 1 column long. (Incidentally did anybody else find it hilarious that this system "summarizes" USA today who themselves summarize other news sources?)

    Secondly much of the news is the same. News is big business and most major news media tell the stories that sell. Because they are all targeting the same markets they tell the same stories and in the same ways. Therefore there is little difference between CNN, the NY Times, etc in terms of tone and "facts". Especially since much of "their news" comes from the same wire services such as Reuters [reuters.com]. Fox News is different but that is because they have abandoned the mantle of impartiality and become all conservative all the time.

    In essence this system is perfect for the internet news style. Breif summaries of facts followed by more "in-depth" leads that we may peruse as we wish. The real question is, when will this begin drawing on sites like Indymedia [indymedia.org], The Register [theregister.co.uk] and /. [slashdot.org]

  • www.headlinehaikus.com [headlinehaikus.com]

    Basically, it looks at the headlines on Yahoo/Reuters, and finds sentences that scan as 5/7/5, and uses Perl cleverness to present them as a little news haikus (or senryu, if you wanna be picky). It's great stuff:

    Today:
    Commonwealth Group Blasts Zimbabwe Poll

    but defended by
    separate observer groups
    from South Africa

    Also today:
    Amnesty Charges U.S. Violated Rights of Detainees

    possible suspects
    connected to the attacks
    including their right

    My last birthday (Feb 4):
    Saudi Proposed Saddam Overthrow to US - Prince:

    we agree upon
    the various issues that
    we agree upon

    Christmas, 2001:
    Deep emotion, little joy in Bethlehem Christmas:

    Palestinians
    without the special permits
    very bad this year

    Sept 10 2001:
    Belarus Opposition Demands New Vote, Plans Rally:

    We do not agree
    with the official result
    RE RUN UNLIKELY

    June 1 2001:
    Bridgestone says some Ford Explorers defective:

    I am just here
    to say what needs to be said
    I am not here

    I'm hooked :)

    They have archives going back to the beginning of 2001, with only a few holes (e.g. the days after September 11), and they talk about how they are doing everything. Bonus points: you can have the haiku headlines mailed to you automagically every day. I just hope they have the bandwidth (etc) to withstand Slashdot....

  • How wrong can you get? (Score:3, Insightful)

    by TheAwfulTruth ( 325623 ) on Friday March 15, 2002 @01:49PM (#3169025) Homepage
    Wrong on every count.

    Besides the fact that /. gets is news from other places and is always hours or days late with it. The worst thing you can do is get all your news from one source.

    Every news site has some kind of slant to it. CNN, NPR, /. (And my favorite of your list "USA Today") Sometimes you get more information than contained in a story merely by seeing how different people report the story! Reading one paragraph summaries of the days news will tell you nothing at all. Maybe worse, mislead you due to there not being enough information.

    I read news from about 10 sources a day and if I see multiple articles that I'm not interested in they're easy to skip. If I am intersted in them I read them on all sites. You get much much more information that way.

    Though you do need to pick your sites. If you look at CNN, MSNBC and Salon and all three are merely parroting Reuters then you know your not doing yourself any good.
  • I hope they've done some serious testing on all these [slashdot.org] news stories. My guess is the summarization of nothing is still nothing :)
  • Heh. I'm at Columbia now and taking a class [columbia.edu] in NLP taught by one of the people behind Newsblaster. We had a lecture about summarization (yes, it is just careful selection of source sentences) and got a tour of Newsblaster as a case study. Now it's on Slashdot. oily_ants, what did you think of yesterday's midterm? Happy spring break!

    This is doubly funny because last spring I took a class in computer graphics by the man behind augmented reality [slashdot.org], which is again on the front page today because of the street sign article. [slashdot.org]

    This means:
    • A lot of my classmates read Slashdot,
    • Sooner of later the whole university will be too slashdotted for me to be able to finish my classes and graduate, and
    • I've got to start submitting all of my professors' projects, no matter how obscure. I hear my Databases professor got X working under Linux.




  • www.newsnow.co.uk does similar stuff I guess, but is the summary builder the thing here?

  • Check out newsseer [newsseer.com] It was written by the same people who wrote citeseer [citeseer.com], the great research index.
  • What if we have two such automated news services and they scan each other? Wouldn't they get stuck in some sort of infinite loop where they repeatedly pass the same story back and forth, summarizing it over and over again?
  • It's still in beta but it's already pretty impressive:

    http://news.google.com/ [google.com]

    It indexes a huge array of news sites several times a day for fresh stories - enter a search term and it will bring up all the headlines it can find for that subject. Best of all, it uses an algorithm to identify alternative coverage of any one story and lists these links in a block beneath the main search results. That way you get links to several different accounts of the same story (although in practise they end up being pretty similar due to using the same news agencys) without having to hunt around for them yourselves.

    They're still working on the algorithm and are requesting as much feedback as possible - read more here [google.com].

  • Now it just needs SOAP (Score:3, Interesting)

    by jfsather ( 310648 ) on Friday March 15, 2002 @02:56PM (#3169417) Homepage
    I think this would be pretty cool if they could add some sort of a SOAP/XML-RPC type interface where you could query on sections, stories, whatever. It would be nice to allow content syncing like this.

    I was writing about this in response to a post in a user's journal the other day that even better would be to make a story content P2P system where you could allow story distribution. You might place a limit and only allow the summary to drive people to your site, but it could still help with bandwidth issues. This would basically be like an enhanced RDF/RSS type system but over a P2P type network you wouldn't even really have to host your own feeds for people. Add in some sort of DB persistance and you could just say "get new headlines and summaries from site x"--the system would bring in all the new content. Anyway, that is just a dream I have and probably will never happen the way some people feel about their content.
  • I've been using Newshub for 2 years now, does essentially the same thing.

    newshub.com [newshub.com]

  • ...but rather in identifying multiple documents that appear to be talking about the same thing. Summarization is a well-researched (but not well-perfected) NLP topic, but finding inter-document similarities is quite a bit more challenging. This is easy for me and you to do when we read something, but think about what it takes to get a machine to do this. Take a look at some of the examples--you'll find that although large chunks may be verbatim from document to document (especially ones that rehash standard news feeds like Reuters and AP), most articles have a different wording or spin on each idea.

Slashdot Top Deals

Get hold of portable property. -- Charles Dickens, "Great Expectations"

Close