Newsblaster and Robot Reporting

I remember back in the pre-Watergate days addressing a journalism class at our local college and telling students -- with all the certainty that only a brand new city editor could muster -- that the jobs in their chosen field would always be safe. "After all," I said smugly, "robots are never going to be able to cover a beat and write a story for tomorrow morning's newspaper. Only we humans can write a lead."

Well, at least that remark was correct for a longer span of time than some of my other pronouncements of the period. But that statement might not be correct much longer. The robot reporters are at the gate. Want to see how they're doing at trying to eat your lunch?

Columbia Newsblaster is an experiment in artificial intelligence at Columbia University that tries to create fresh news leads by analyzing the dispatches it finds from assorted news organizations on the Web. Newsblaster (http://www.cs.columbia.edu/nlp/newsblaster) looks at reports from nearly a dozen resources, including the Los Angeles Times, The Washington Post, USA Today, ABC News, CNN, Reuters, CBS News, Canadian Broadcasting Corp., Yahoo!, Virtual New York, and Wired. The site uses artificial intelligence techniques to sift through news reports published online, then sort and summarize these dispatches into five different news categories: U.S., world, finance, entertainment, and sports.

To organize the material it finds, the software bases the summaries on "reflecting factors," such as where a fact is mentioned in the published reports and how often it is repeated across reports dealing with the same event or subject. The summaries also are based on the news value of individual facts, such as how many were killed or injured in an incident or how much damage was done to property. The program then produces its leads for each story's display on the Web site along with links to the original source material it used in producing the leads.

Significantly, Newsblaster doesn't merely extract whole sentences from the original reports. Instead, it uses "natural language processing techniques" to produce its own copy. Kathy McKeown, the professor who heads the team of researchers at Columbia's Department of Computer Science in the School of Engineering and Applied Science, says journalists really don't have to be unsettled by this seeming encroachment on their turf. What Newsblaster does, she recently told The Christian Science Monitor, "is take what journalists have already written and provides a way to easily browse. We could not write the stories ourselves." At least, not yet. But some of the leads the software provides are remarkably concise and lucid.

Other considerations for using Newsblaster