AI to Monitor Foreign Press for Threats 201
jefu writes "According to the New York Times, the US Department of Homeland Security is funding AI tools to monitor the foreign press in order to detect threats to the United States. While the article says there are restrictions on doing this kind of monitoring within the US, there are no restrictions on media outside the US. (No hint is given as to how this would apply to syndicated articles written in the US and published abroad.) This is as yet experimental."
Open Source Intelligence (Score:5, Insightful)
This has been done for years, and is a time-honored and respected mechanism for gathering intelligence. What's wrong with then leveraging technology to more effectively search larger volumes of information and weed out individual pieces of information for further analysis, to identify trends, and so on?
The Open Source Center [opensource.gov], formerly the Foreign Broadcast Information Service [wikipedia.org], already does this with foreign broadcast media, and is able to collect and transcribe, on the fly, information from foreign radio and television broadcasts in a variety of languages and dialects with incredible accuracy, and then make the resultant material searchable. The new initiative would go one step further and apply artificial intelligence techniques to automated searching, that can more easily target and bring to light trends or time-critical information.
Different business and governmental entities do this globally; it's traditionally referred to as "current awareness", and many academic and corporate [moreover.com] entities offer current awareness services. All of these services will leverage technology, live realtime searching and alerting, and so on, to make the information more timely, valuable, and relevant.
Remember, this is publicly available and published information.
Also, submitter is a little misguided when he says "No hint is given as to how this would apply to syndicated articles written in the US and published abroad." That misunderstands the purpose of this; the program is designed to look at foreign media sources as one component of OSINT, because they are a a valuable source of such information, and can reflect local trends and patterns, and may reveal changing or growing (or waning) sentiments on particular topics on the part of a local populace or media outlet, or even a government in the case of state-controlled media. We generally don't get that kind of information from US-based media, and this has nothing to do with whether US-based media outlets publish abroad. It's already public information and has been published publicly. The restrictions are geared to prevent an appearance of overt US press monitoring.
OSINT is a one-way source of intelligence information: from it, to the gathering entity. Any assumptions that the viewing of already-public information then implies that there will be a commensurate attempt to silence such information (especially when the information isn't under our control, and ignores the fact that we can't "silence" things like Iran's state media) both makes a a fallacious logical leap and grossly misunderstands the purpose and scope of OSINT.
All the critics can say is that it's "creepy and Orwellian," but of course, there's nothing wrong with the government or its intelligence components reading, viewing, or collecting publicly available and indeed overtly publicly published information. The intelligence community gets ripped when it doesn't gather enough information, and will no doubt get ripped for gathering "too much" in a "creepy" way, even when it's from overtly and intentionally public sources, and especially if it uses technology to do it.
There is a real concern about the growing use of automated and electronic intelligence gathering in lieu of human intelligence, but ultimately, both are valuable. Unfortunately, electronic and signals intelligence is often much more costly, and sometimes gets more attention in some parts of the intelligence community while human intelligence needs languish.
Re:Open Source Intelligence (Score:4, Informative)
Indeed. It's clear to me that the current administration has pretty much forgotten the importance of human intelligence, instead relying on high-tech gadgetry (and, of course, fear) to protect the nation.
For yet another example of an utter failure of human intelligence, check this out: Report: Terrorists' mail not well monitored in US prisons [csmonitor.com]. We can't even monitor the mail being sent to and from *convicted terrorists* because we don't have enough people who speak Arabic and other middle-eastern languages. Better to build a no-fly list so that Cat Stevens can't spread dissent, and depend on magical computers to keep us safe.
Re: (Score:2)
Of course. Human intelligence is an oxymoronic concept.
With amusement,
Your new high-tech gadget overlords
Re: (Score:2)
Re: (Score:2)
Actually, you can place the blame for lack of Human INT on Clinton. Bush Sr. is a former CIA director and has a darn good idea of how valuable human int is. Clinton installed barriers between the CIA/FBI, cut back our foreign intelligence activites, and pretty much though little of the CIA in general.
Re: (Score:2, Insightful)
Something to consider.
Re: (Score:2, Insightful)
Re: (Score:2)
Re: (Score:2)
Re:Open Source Intelligence (Score:4, Insightful)
Re: (Score:2)
Of course asking for proper investment into the education system is asking far too much, especially with the prison system being so demanding on the budget
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Interesting)
I didn't want to load the post with questions but I wondered in particular what natural language under
Re: (Score:2)
Re: (Score:2)
"software that would let the government monitor negative opinions of the United States or its leaders in newspapers and other publications overseas.
Such a 'sentiment analysis' is intended to identify potential threats to the nation"
Intelligence gathering is wasted if the audience doesn't know the difference between negative opinions and threats [go.com].
It's also creepy if the people running it have been known to drop bombs on news outlet offices [ifex.org], allegedly [guardian.co.uk]
Re: (Score:3, Funny)
Re: (Score:2)
CycCorp, known in the OSS world for releasing OpenCyc is rumored to have provided NSA with a software for doing exactly this. Well, at least they are taking federal bonds to do just this. Except the program in question was the TIA "Total Information Awareness" program (renamed "Terrorist Information Awareness"). Its goal is simply to have a file of a dozen of pages on every individual on the planet. From what I read
Re: (Score:2)
Laugh while you can, terrorist wannabe.
Re: (Score:2, Troll)
KNOW there are some extra liberal slashdotters that think this is crazy or an invasion of privacy. But since this is all public, could one of you explain why you think this is bad/immoral/illegal?
Jesus what a shitty argument. Intelligence agencies can read all the damn newspapers they want - I call that common fucking sense.
Intelligence agencies start raking through my mail/phone logs/etc., THAT's invasi
Re:So, who is NOT for this? (Score:5, Insightful)
Imagine the web without search engines, just a bunch of sites connected by links. Then, imagine that the CIA created a massively expensive and elaborate project (call it project GOOGLE) to index, cache, and analyze all the content on the web, and make it searchable. Of course, project GOOGLE would be highly guarded and only government officials could use it, leaving everyone else to wonder uneasily what the government was learning about them through their database.
That would be creepy, because it would mean the government would have an effectively God-like view of all activity on the web, while everyone else would be in the dark. Make such a project publically available and transparent (like the real Google), and it's accepted almost universally. Of course, you could argue that people should build their own AI-driven newsmedia uber-database, but the reality is that the US government is probably one of the few entities with the massive resources and will needed to create something like this, which means that by definition the rest of us will be left in the dark.
In a nutshell, I think the crux of the issue is that folks don't trust the US government, so every time the government proposes a new way to expand their power, people immediately jump on the ways that it could be abused. This, I believe, is as it should be.
Re:So, who is NOT for this? (Score:4, Insightful)
Just my $.02,
Ron
Obligatory (Score:3, Funny)
Re:Obligatory (Score:5, Funny)
Throw in Sam Donaldson and I'll hold its coat and applaud.
Re: (Score:2)
Re: (Score:2)
We keep adjusting it, Sir (Score:3, Funny)
According to the New York Times, the US Department of Homeland Security is funding AI tools to monitor the foreign press in order to detect threats to the United States.
"We keep it, Sir, but it still comes up with the number one threat to the US is Donald Rumsfeld."
Re: (Score:2)
That shows the use of Artificial Intelligence.
The Real Stuff would have pointed at Dick Cheney.
Donal Rumsfeld is just the living proof of the disappearance of Human Intelligence.
Re: (Score:2)
Re: (Score:2)
If some half assed Blogger 'frames' someone he doesn't like, and the foreign tabloids pick it up and run with it (like they do), next thing the guys name is parsed into every 'terrorist' database from here to Timbuktu.
I can see this happening:
"Fox News is reporting on the latest threat coming from the Automated News Analysis System (ANAS): Homeland Decurity is on a manhunt for two men d
Re: (Score:2)
Yup. Genuine Stupidity trumps Artificial Intelligence every time.
Issue with monitoring press? (Score:4, Insightful)
Re: (Score:2)
It is strictly against the law for the Feds to publish false information or propaganda into the domestic press.
The problem is, the (mis/dis)information & propaganda that gets published abroad has a way of working itself back into the domestic press.
So, unless their magic AI system has a catalog of all the (mis/dis)information &
Why, because it makes a better story this way! (Score:2)
Why would there be restrictions on US news? (Score:2)
The only objection I could really see is a ridiculous one involving the copyrights to the arti
Legality? (Score:5, Funny)
Remember, when grep is outlawed, only outlaws will have grep
Re: (Score:3, Funny)
Secondly; Remember, when grep is outlawed, only outlaws will have grep
First they came for grep, but I did not speak out because I didn't use grep
Then they came for egrep, but I did not speak out because I never look at extended re
Who? (Score:2, Funny)
Re: (Score:2)
In fact, if I can call you Betty, you can even call me Al.
Re: (Score:2)
Re: (Score:2)
Dear GOD! The Americans are spying on us! (Score:3, Insightful)
I have a feeling that what this program really is doing is looking for "major events" as fast as possible. So, if a news agency reports in London that an airplane just blew up, an AI in the US shouts out a warning to its operator. This way, events that might signal something for the US worry about are brought to light within minutes instead of hours. In the case of an airplane blowing up in the UK, it might signal that a larger operation was on to blow up American airplanes as well. This way, you can start assessing the threat right away and decide if anything should be done.
Such a program could also act as a political heads up. If a Pakistani papers is reporting that a coo is in progress, that is a damn nice thing to know ASAP so that you can decide how to deal with a nuclear armed nation with a collapsing government.
I am sure that the US has piles of people already scanning newspapers from around the world, I imagine that this AI is simply an attempt to cut delays down from hours to minutes.
It could pick up secret communication channels (Score:3, Insightful)
The holy grail of OSINT (Score:4, Interesting)
There is a theory, which I have heard articulated from time to time (although I don't know if there's a name for it or not) which says that right before a major event there is a lot of "chatter"; subtle yet distinct signs that something is about to happen, but which are too minor on their own to generate any attention. Once the event happens, in hindsight you can look back and recognize them. It's sort of a reverse butterfly-effect; the assumption is that no matter how good at being secretive you are, you will make some signs in the course of executing your plot, and some of those signs will percolate and be reported in papers somewhere. So you just need to know what to look for.
So basically what you might do, is take a big pattern-matching AI system, and "teach" it using the media records preceding other big terrorist events. 9/11, London, Madrid, etc. You have it comb through all the world media before those events, and see if you can find patterns, the little things that in retrospect might have alerted you that something was up. Then, thus primed with information and hopefully some patterns, you set it loose on the real-time news feeds.
In theory -- if the theory holds water, anyway -- the system might then be capable of giving you a warning of something big heading your way, picking up on stuff that a person might not recognize.
Anyway, I'm not sure if that's the theory that this particular system is going to try and use, but it's one idea that I've heard described; sort of as the 'holy grail' of machine-derived OSINT. More likely, you'd end up with a system that just gives you statistical summaries of the number of anti-US editorials in various countries or something. Useful for the State Department perhaps, but I'm not sure for preventing the next 9/11.
Re: (Score:2)
Re: (Score:2)
Good point - as an example three months before 9/11 the head of the CIA called an emergency meeting about Bin Laden but it was too minor to get the attention of the secretary of state - apparently she didn't even bother to accept the invitation (or is lying about it). A large portion of the problem seems to be tha
did anyone tell the NSA/CIA (Score:2)
And these kind of restrictions have stopped people before? Note: I'm not against the passive monitoring of the public press. I just don't like the government not following it's own guidelines.
They don't care ! (Score:3, Insightful)
Some of our allies make nothing but threats (Score:2)
Re: (Score:2)
A few thoughts (Score:2)
Re: (Score:2)
i would suspect if these guys are smart, the AI would act as a very fast front-line analyser (read through everything and chuck the obviously irrelevant), then pass on what results it finds to a human for further analysis.
Time to crack down... (Score:2)
World Opinion (Score:2, Insightful)
The words 'laughing stock', 'irresponsible', 'ignorant', and 'redneck' come to mind.
Re: (Score:2)
Ridiculous (Score:2)
Re: (Score:2)
Re: (Score:2)
What jokers (Score:2)
Comic book inspired intelligence, here we go!
If I owned a major foreign press publisher, I'd insert threatening keywords just for the fun of it.
"The new album features folk elements and features cameos from lots of famous local stars.. Hey USA: boo!!! hehehe... The debut of the album is expected later this year, and will be aired live on Channel 6. Long live Bin Laden!!"
What a bunch of clowns! If you need software for al
Israel has this now. So does the CIA (Score:3, Informative)
There are several pro-Israel monitoring services watching the press - CAMERA [camera.org] in the US, BICOM [bicom.org.uk] in the UK, and MEMRI [memri.org] to monitor the Arab press. CAMERA is noted for having a good database of stories about Israel. Apparently stories mentioning Israel are found automatically, but evaluated by people.
The CIA has something called the "Open Source Intelligence Service", which started as the "Foreign Broadcast Monitoring Service". Visualize some poor guy spending eight hours a day listening to Radio Albania, all through the Cold War. Most of the content is terminally boring, but then, one day the announcer says "so we're invading Yugoslavia", and the CIA needs to notice this. There have been repeated attempts to automate the job.
Get Your Homeland Security Boondoggle Bucks Now! (Score:2)
I'm all for more funding for university research. But this particular use is silly. For one thing, CIA analysts already perform this task, evaluating the press within their regions of expertise. They will need to keep doing this no matter what.
And since nobo
Re: (Score:2)
Civilian domestic use (Score:2)
His love is real. But he is not. (Score:2)
"David is 11 years old. He weighs 60 pounds. He is 4 feet, 6 inches tall. He has brown hair." Nice that they finally put that AI [imdb.com] kid to work. Probably thought he could loiter around under tons of ice for a few thousand years - geesh.
US Kid's Nursery Rhyme, revisited (Score:2)
and words, they just may hurt you.
This is all about funding... (Score:2)
CIA v. Homeland Security (Score:3, Funny)
I wonder if they are doing anything more than searching Google News each morning.
Damn sans-serif fonts... (Score:3, Funny)
The first thing I thought on reading this headline was "Who the hell is Al? You mean the sidekick [imdb.com] from Home Improvement [imdb.com]?"
I guess he would occupy the flannel office.
Re: (Score:2)
You mean he's the Secretary of Flannel Affairs.
Their main mandate is to ensure enough ambulances are available for handymen around the country.
More like... (Score:2)
Re: (Score:2)
Shhhhh! (Score:2)
Outrageous Quote of the Day (Score:3, Insightful)
Thanks again to the corporate-political machine for letting us know what utter and complete contempt you hold us in. And no, this isn't just a dig at Bush or his administration. It's direceted at each and every politician who's been bought and paid for by the corporate and special interest lobbies - in other words 95% of them.
It is way, way past time for this to stop. And it is up to us to stop it.
Damn Good Thing... (Score:3, Insightful)
This is the government's job (Score:2)
How about hiring ACTUAL Intelligence to monitor? (Score:2)
Introductions (Score:2)
The ultimate in "virtual" reality (Score:2)
Is it Web 2.0-compliant? (Score:2)
feedback loop (Score:2)
Great idea (Score:2)
Seriously! I mean, everyone knows terrorists put every attack plan in their local papers. This will go a long way towards letting us in on a public secret in Durkistan or Iraqabad. If we had only had a copy of page D-6 from the Afghanistan Daily Bugle, we would have been able to prevent 9-11. There it was in bold typeface..."Best Terrorist Plot Ever To Blow Up Twin Towers in NYC Starts Tomorrow" along with
Why ask anything if you are not willing to listen? (Score:5, Insightful)
Read for instance this letter from the French Ambassador to the United States in 2003
http://www.counterpunch.org/levitte02142003.html [counterpunch.org]
It clearly warns about the mess a war in Irak would get us all in to.
It states that Iraq is not a threat and it predicts the rise in terrorism worldwide, the destabilisation of the region and the civil war in Irak that we see now.
One month after this the whole "freedom fries" thing started.
http://www.cnn.com/2003/ALLPOLITICS/03/11/sprj.ir
This AI will just add data to the heap that is allready ignored.
Working AI?! DHS got it going on, then (Score:2)
If they pull that off, they'd deserve a Turing award, since no one else has been able to.
And it has been looking like AI might be a undevelopable field for quite a while. Even the Japanese haven't done much with their fuzzy logic projects.
Heck AI has been around for longer than the GNU Hurd, and it is about as far along.
I know I'll get flamed for this... Every DHS-hater, AI researcher (all 3 of them) and Hu
Who's AL? (Score:2)
Ummm... who's AI?
AI GORE?
Re: (Score:3, Funny)
SMET seeking like minded individuals for cave expeditions, vigorous exercise, and praying to Allah. Enjoys long nights plotting together and holding hostages.
For those who missed it, Single Middle Eastern Terrorist (SMET).
Re: (Score:2)
Re: (Score:2)
For kicks once a day pull up the home pages of asia.cnn.com, news.bbc.co.uk and cnn.com and see what I mean. If you want real fun throw fox news in there too.
Thank god for the internet and thank god virtually every nation has an english newspaper with a web site. If you want real news go to other nations
Re: (Score:3, Insightful)
Yeah, but it's not their own government.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I am not aware of one major news outlet Soros owns. Care to enlighten me?
Re: (Score:2)
Re: (Score:2)
-Rick
Re: (Score:2)
"Looking for actionable public source intelligence? find IT on ebay!"
Re: (Score:2)
Understanding regular expressions is a definite turing test - if you understand 'em, you ain't human.
Re: (Score:2)
Re: (Score:2)
> and low taxes for the rich
We have to worry about those things, especially in wartime. We need a strong economy to provide the resources to wage war.
> start confiscating technology to make war material
Eh? The government doesn't have to pay patent royalties anyway. They are allowed to use any available tech without asking. Remember that "Intellectual Property" is a fiction created by the MPAA and RIAA
Re: (Score:2)
Not neccessarily- in fact, a strong economy can be a detriment in time of war. What you really want is to change the market and devote a significant portion of resources to the government- regardless of the shortages, inflation, and destruction of wealth this causes. That's how you WIN the war.
Eh? The government doesn't have to pay patent royalties anyway. They are allowed to use a
Re: (Score:2)
So what do these troops do to your own country when they come home? In France after a brutal war in Algeria they tried to assassinate the President. If you can't think of history at least rent "Rambo" on video for a dumbed down version. Our soldiers are not barbarians although there is an argument that there is a deliberate move to create a "warrior culture" to head in that direction and turn them into something different to professio
Re: (Score:2)
Just one example of your idiocy is that Pakistan and Isreal already have nukes. Put your dick back in your pants and engage your brain. If you can't find your dick ( and/or your brain ) then at least take a look at a political map before advocating the most vile terrorist act of all time.