Google Researchers Create TV Audio Analysis System 108
segphault writes "Ars Technica reports on a paper (PDF) about ambient audio analysis authored by Google researchers. The system described in the paper can effectively determine what television show a user is watching just by capturing a short audio clip. The paper explains how a regular computer microphone can be used to record an audio clip that is then converted into a statistical data summary and transmitted to a remote server which matches the clip against archived data in order to ascertain which TV show it is associated with. Apparently, the system is fully viable, and other kinds of ambient noise don't negatively impact its accuracy. The paper also describes how web services can provide contextually relevant information based on a consumer's television viewing activities."
I have just this to say... (Score:2, Funny)
Re:I have just this to say... (Score:2)
There is no way I'm letting Google or anyone else have an open mic. in my home.
Even if it were just turned on for brief periods at random intervals.
Why would anyone allow this in their livingroom? No, I didn't RTFM.
Re:I have just this to say... (Score:2)
Re:I have just this to say... (Score:1)
Re:I have just this to say... (Score:1)
From The Fine Paper;
...the system can easily be designed to use an explicit 'mute/un-mute' button, to give the viewer full control of when acoustic statistics are collected for transmission.
Again what incentive do I have to ever un-mute?
I see where the adv
This already exists? (Score:4, Interesting)
Re:This already exists? (Score:3, Informative)
This here matches a sound clip to a pattern to find the TV show, meaning it doesn't have all the current episodes of the program in its database, it just has statistical data and patterns which help it match the audio. The latter could successfully match new (live) episodes without having the database updated. Your tune system wouldn't.
Re:This already exists? (Score:3, Funny)
Re:This already exists? (Score:2)
It sounds like they are effectively building a database to compare the recordings to. (even if these are not the actual material but
Re:This already exists? (Score:2)
Anyway, besides obvious RIAA spyware being able to catalogu
Re:This already exists? (Score:4, Informative)
It's been done. Here's a system where you can hum a tune and it tells you the song: http://www.musipedia.org/ [musipedia.org]
Current systems are mostly based on pitch changes, so they aren't perfect (especially with the recycled slush turned out by low-grade high-visibility pop acts), and largely useless for rap, but they mostly work. There are numerous variations on the system, this is just one of the more significant ones that is publically availabel on the web.
I would think by making a hash based on values relative to sound signatures within the clip this might be possible, but I don't really know how this stuff works
What google is doing may or may not be related. They might instead be using a form of speech recognition technology, or a combination of both, or something else entirely.
Re:This already exists? (Score:2, Interesting)
Its called Shazam (Score:2)
Re:This already exists? (Score:2)
South Park on Paramount is about the worst viewing experience I know of... I could swear the show's volume decreases as it goes on, then:
HI! I'M BARRY SCOTT! *BANG* AND THE SHIT'S OFF THE WALLS!
Re:This already exists? (Score:1)
Re:This already exists? (Score:2)
South Park, this evening, Paramount UK. Visibly higher amplitude.
Re:This already exists? (Score:2)
I know the people from the UK have their own form of spelling, but "here" instead of "hear"? (I'll have to remember this one)
The big question is: will Google will apply for (and obtain) a patent akin to Microsoft's "How To Tell If A Baseball Game Is Interesting".
Re:Great... (Score:2)
A PVR that doesn't need to rely on blind luck and often incorrect listings to know if it's recording the right thing.
My Tivo often mischannels to PBS. I'm pretty sure this algorithm should be able to tell Family Guy from the "Boring ass old people talking about politics hour".
whereas (Score:3)
football head baby and big fat cartoon man talking about his ass gas hour...
Re:Great... (Score:2, Insightful)
Re:Great... (Score:2, Interesting)
Re:Great... (Score:1)
Re:Great... (Score:3, Insightful)
Re:Great... (Score:2)
if you want to check what tv show it is - it means that you like it, and want to watch later/tell your friends about it. Google can sell data about tv shows popularity to interested parties, that will know where to plant ads (or they could place ads themselves...). It also can be used to determine an ad price for a given show.
I'm not watching tv for 4 years now, it feels great. If I accidentally see some of it somewhere I'm shocked at how dumb it is.
Re:It's Google, what do you think? (Score:2)
If Google really wants to "do no evil", then they need to use this technology to recognize that a commercial just started, and turn the darn volume down! I'm a reasonable guy - they don't need to turn the volume off or skip over the commercial (although they are welcome to do this), just turn it down to the point where the overcompressed signal is not blasting me out of my brain. It's almost impossible to quietly watch any 10PM network TV show without getting blasted by the c
Re:Great... (Score:2)
Popup Television (Score:3)
Do you remember the MTV show Popup Video? They showed older music videos with popup balloons that gave extra information, like actors in the video that later became famous or mistakes made during production. If Google analyzed the sounds coming into your laptop and gave you a link to a site like the Internet Movie Database [imdb.com] then you could have Popup Television. Learn more about the specific episode you are watching, and even have the ability to edit that information yourself.
It'd make an interesting toy
Re:Popup Television (Score:1)
Re:Popup Television (Score:2)
uh, I mean, so I heard.
I thought something like this was up! (Score:3, Funny)
Re:I thought something like this was up! (Score:1)
implications of information (Score:2)
This exposes an example of how even sanitized information is not sanitized. How can you return information on what people are watching and not return at least some very personal information. Of course, some people might say "well, I don't watch that kind of stuff, so I don't mind", but the issue is subtle. First, that attitude leads quickly to an assumption that there are two classes of people: people who don't mind being tr
Uses & Motives? (Score:5, Insightful)
Since Google has more storage than you can imagine, they can most likely apply this fingerprinting technique to every episodes of every major show. Then they host the fingerprints in Google style and use their patented "Google Technology" to search it much the same way web content is searched.
Why would you want this? Well, there's the obvious marketing ploys. You know that people who watch Darma & Greg like to shop at Trader Joe's and like Odwalla brand food so you offer free episodes of Darma & Greg with only Trader Joe's & Odwalla episodes. You let the sponsors (Trader Joe's and Odwalla) foot the bill for the bandwidth/royalties or whatever.
The second useful implication would be cross suggesting shows to a user based on random sampling of the shows. You could allow users to watch old TV shows on the internet and then build a profile of them and their shows. Much how Amazon works, you could then suggest other shows, other DVDs of shows or perhaps build a site that randomly shows the user episodes that they might like based on prior viewings and statistics of other users.
The take away from this article for me was the fact that Google has vested interest in archiving and now television will be archived Google style.
I can't think of many other uses for this as the system isn't really "inferring" or "thinking" about data samples but is more so matching extracted features against a database. You know, voice recognition software allows for decent voice fingerprinting. You could most likely easily identify characters based on voices (but not actors due to stars like Hank Azaria who do multiple voices). Then you wouldn't need a database of all shows but more so just a database of character voice fingerprints. I would find this sort of approach more interesting but less specific and useful.
Aside from showing this off to your friends, it's not very useful. What I personally would like to see this new Google strategy applied to is all the tapes recorded of famous people like the United States Presidents. If you divided those up into sessions and I was listening to a particular tape of the Nixon set where he talked about the "new right", perhaps a database with references would then point me to some tapes or materials on Joe McCarthey's staunch views on the right.
Re:Uses & Motives? (Score:2)
A bizarrely useful search engine is a complete search engine.
Subpoena (Score:3, Insightful)
Designed to maximize user privacy while minimizing dependency on unique hardware, the system described in the paper seems interesting and feasible. In order to protect user privacy, the software uses "summary statistics" automatically generated from ambient audio rather than transmitting an actual recording. The actual audio cannot be extrapolated from the summary statistic data, so the system doesn't "overhear" or transmit user conversations.
Still, if the data reveals what show the person is watching, your President or anyone else who gets to see the data might start treating you differently depending on what you are watching latley.
Re:Subpoena (Score:2)
But, now that they'll be able to know that we own all 10 seasons of Friends..... they've gone too far!!!
TVDB (Score:2)
Google seem to go out of their way to freak us out (Score:1, Redundant)
This seems to be just asking for privacy concerns.
Re:Google seem to go out of their way to freak us (Score:2)
Figuring out TV ratings is an expensive things for corps. Google can get a huge market by automating the job of figuring out whose watching what at any given time---if they can only convince the users to let them (``hey, install this google audio-ad analyzer and get 10 gigs added to your gmail account---and it lets you see tv listings in real time, as well as clips from tv pr
What else does it work on? (Score:1)
What... (Score:1)
Oh... I guess that would have to be a dupe^H^H^H^Hseparate story in YRO.
What about p0rn?! (Score:1)
other possible uses? (Score:1)
TV equalizer (Score:2)
Re:TV equalizer (Score:2)
been there, seen that, now it's dead (Score:1)
remember cuecat? that funky little free barcode reader from radioshack?
http://yro.slashdot.org/article.pl?sid=01/06/10/04 10208 [slashdot.org]
in one incarnation i beleive they included a jack on the device and the end user was suppose to hook up an audio cable from one's tv to cuecat v.2. the computer would do all the heavy lifting, eventually finding a hidden tone that would magically pull up an advertiser's web page.
it was spam magic that never took off. gee, i wonder why.
Re:been there, seen that, now it's dead (Score:2)
Re:been there, seen that, now it's dead (Score:1)
I find it pleasingly fanciful to know... (Score:1)
Re:I find it pleasingly fanciful to know... (Score:1)
Re:I find it pleasingly fanciful to know... (Score:1)
Nielson (Score:2)
Actually - it appears they do the same thing Google's researchers talk about already:
Re:Nielsen (Score:3, Interesting)
Not sure about PPM's tech, but Nielsen's A/P meter does exactly what TFA describes. That's the only way Nielsen Media could roll out Time Shifted Viewing [slashdot.org] at all (disclosure: I work for them). To say that Google "created" it is an insult to the people I work with every day.
I see a patent suit in Google's future. As much as I hate patents and like Google, I'd like to at least see some full disclosure here. To (erroneously) state one one
In my house, they will be very disapointed. (Score:2)
Re:In my house, they will be very disapointed. (Score:1)
Re:In my house, they will be very disapointed. (Score:2)
Re:In my house, they will be very disapointed. (Score:2)
Privacy Maximization (Score:3, Interesting)
In the mean time, I avoid non free software and even have bad thoughts about my cell phone.
Re:Privacy Maximization (Score:2)
Bullshit, the mic is set to be off as default, and each
Re:Privacy Maximization (Score:2)
Similar tech (Score:2)
Dan East
Comment removed (Score:3, Funny)
Re:I have tested it (Score:2)
Joking aside: imagine this as a legit non-marketing use. While watching the news, GoogleSnoop detects some lies being spun by some pundits, and immediatley provides links to opposing viewpoints, definitions, articles of law, or any other silly "facts" the reporter hasn't bothered to research.
Even from a marketing point of view, imagine if a commercial for "Product X" came on, and suddenly Froogle launched and found the cheapest 10 spots online to go buy it.
I'm not trying to glaze over the privacy issues
recognizing sound samples (Score:4, Insightful)
Re:recognizing sound samples (Score:2, Informative)
http://en.wikipedia.org/wiki/MusicBrainz [wikipedia.org]
http://musicbrainz.org/ [musicbrainz.org]
New Concept? (Score:1)
Major points to the first person who... (Score:2)
Posts a screenshot with something like:
Re:Major points to the first person who... (Score:2)
:: underage goat detected ::
Report to NSA?
[yes] [no] [cancel] [baaah!]
Peoplemeter? (Score:2)
Can the same system be used to cut out ads? (Score:2)
Re:Can the same system be used to cut out ads? (Score:2)
Nice, but how about muting commercials? (Score:2)
I, for one, would gladly pay $10 extra per month to have a button on my remote that when pressed kills t
eyes wide shout (Score:3, Interesting)
This very statement presupposes that other noise is irrelevant, which seems bogus.
Snoring is background noise, and suggests non-watching.
Laughter is background noise, and suggests careful watching.
Of course, the laughter might not be about what's on TV...
It seems to me that watching is an activity involving the eyes and mental processing. It seems to me that audio of what is coming out of the TV is not a statement about either the eyes or about mental processing. This technology of Google's may be an advance in something, but I hope the advertisers paying for this data have their eyes open about the nature of what they are buying because (to re-mix a metaphor) to my eyes this sounds a bit suspect.
Sociologically, it sounds like a foot in the door to get harmless censors in place. Oops, Freudian slip there. That's sensors, I mean. Google would never involve itself with censorship.
Once the sensors are in place, when "we" realize that it's not getting "us" the data "we" want, we'll just do a few "harmless" downloads of "upgrades", perhaps causing a minor tweak to look at the video data rather than the audio, or perhaps doing language processing after all, and ... With user-friendly software like this, who needs spyware?
I also question the claim that because no information is transmitted back to Google that this is the definition of not invading privacy. How is this fundamentally different than the claim that if the police search your house but find nothing, they have not invaded your privacy because they've not placed any record of illegal activity on your permanent record?
It seems to me that once you place a Turing Machine into someone's environment, capable of doing arbitrary processing, and all it sends is a sanitized report, you have all the mechanism in place for abuse. What if the Turing Machine, capable of arbitrary processing, decides that it doesn't want to send a sanitized report. Who is auditing what is sanitized and what is not?
What if it turns out to later be possible to lift information from the supposedly cleansed records? Who will audit the use of that data?
There seem to me to be a lot of slippery slopes here.
Re:eyes wide shout (Score:1)
As said, so initially the mic's are exactly what google promises, there are too many concerns with putting in a device conveniently designed to listen, record, and analyze incoming conversation. This is a lot of power to give someone.
How much do other people -really- value your privacy? Would you listen in, just for a moment, if you could? It's tempting just for curiousity.
Add a payment for every person who says they like Happy Margarine. Intellectual interest: H
Where is that 'do not listen' button (Score:2, Insightful)
turn your laptop into a visual aid (Score:3, Interesting)
I can envision running a speech-to-text translator on my laptop mic and then piping that text into my beagle desktop searcher, or maybe even one of those google desktop search tools on windows. I'd rather not send this data to google, for privacy reasons, though.
I could see this being useful at work, or in a conference or class, too. I could stand to have relevant pieces of notes that I took from previous classes pulled up with my professor mentions a particular topic.
Anyone know of a tool or project like this?
Big brother...... if you agree to it (Score:2)
Saying that Google will use this to spy on people is like saying that the NSA will spy on people who email them all of their personal information, daily habits, etc.
Re:Big brother...... if you agree to it (Score:2)
And if all of a sudden they turned into a spyware corp (not that far off their business model), they may even do it covertly---but that would be the end of Google.
And what do we get for our involvement? (Score:2, Insightful)
I'm very skeptical this wouldn't be abused - if not by Google, then by someone else. And even if this is not abused, I run the risk for what?
I don't like ads now.
Everyone who loves the idea of personalized ads, put up your hand!
----------
From the other side, what will your friends think when that "random" ad for viagra pops up?
Cool! (Score:1)
What's more, fro
Nothing new here (Score:1)
Easy task (Score:2)
I didn't even RTFA, but from the summary I have an idea on how to implement this idea, it's fairly simple, although it's probably not as computationally efficient as what they came up with, no need to be a great engineer, if you have studied digital signal processing for a few monthes it will be enough.
So you take that audio clip, and you simply cross-correlate (reverse in the time-domain and convolve) it with your audio data base. The highest peak in your results denote a correlation between the audio clip
A very similar one published in Nov/05 (Score:1)
http://doi.acm.org/10.1145/1114223.1114238 [acm.org]
This paper describes a scalable real-time audio fingerprinting system developed by IBOPE Midia for radio and TV broadcast monitoring. A special temporal feature extraction strategy based on the Short-Time Fourier Transform has been designed. When given an input stream to analyse, the system matches it against the