How MythTV Detects and Flags Commercials 403
peterdaly writes "Automatic commercial detection is the "killer app" feature that none of the commercial DVR's dare to include. MythTV's automatic commercial detection does a great job of properly separating commercials from content. Here's how the commercial flagging works."
From the summary... (Score:5, Insightful)
A sentence that (I think) neatly points up the big problem with the USA's legal system...
Re:From the summary... (Score:5, Funny)
Ahh, the power of chee^H^H^H^Hopen source.
Re: (Score:2, Insightful)
Personally, I'm willing to pay for whatever I watch, like I do with HBO. I just don't want it interrupted all the time. Maybe some sort of a la carte system??
Sorry, I have lots of
Re:From the summary... (Score:4, Interesting)
Lost, 4400, Firefly... I've gotten these on DVD from Netflix, and enjoyed watching them all commercial-free, for the flat-rate price I pay to Netflix for monthly membership.
If I didn't have to wait so long for these shows to come out on DVD, then this would be even better.
The TV show makers need to abandon this silly idea of having to broadcast their shows on a weekly basis, and wait for the season to be over before releasing them on DVD. Let's just skip broadcast altogether and go straight to DVD.
TV on DVD is *the* best way to watch TV (Score:3, Insightful)
We got hooked, loved it, and sucked it all it over the summer. We couldn't wait for season 3 to start on, what is it, CW? So we sit down on opening night and it sucked! The pacing was all off, we couldn't rewind it to hear that line we
Re:From the summary... (Score:4, Interesting)
Re: (Score:2)
Because we put up with it.
Why not better commercials? (Score:2)
Bac
Re:Why not better commercials? (Score:4, Funny)
Step 1(Minutes 0-2). Useless introduction and program intro
Step 2(Minutes 3-5). Explain the situation
Step 3(Minutes 6-8). Commercials
Step 4(Minutes 9-10). Re-explain the situation
Step 5(Minutes 11-14). Try to create some drama out of thin air
Step 6(Minutes 15-17). Commercials
Step 7(Minutes 18-19). Re-explain the situation
Step 8(Minutes 20-23). Try to create some more drama out of thin air
Step 9(Minutes 24-26). Commercials
Step 10(Minutes 27-29). Restate what happened
Step 11(Minutes 29-30). Cue the music and intro the next program
As you can see, there's not too much real show in there.
Re: (Score:2)
Let's be honest here, in "free" (as in software, not beer, i.e. uncrypted) cable you get mostly reruns, silly talk shows, braindead "reality" shows and game shows. The real good movies or first time aired shows are few, far between and you've already seen them in chunks by the time they're finally aired 'cause they have been promoted for at least 2 weeks so you don't even want to see them anymore. And when they finally get aired, they're interrupted ever
Re:From the summary... (Score:5, Interesting)
Indeed, the promise of satellite/cable channels was that because you were paying for the subscription to see them, you wouldn't have to see commercials. Then, they got greedy and added commercials anyway.
If all my favorite shows were available on iTunes Store, I'd drop my DirecTV subscription entirely. I have to pay more for DirecTV (cable would cost even more....) for that content than I would to buy it via iTunes season passes. And if I just buy the DVDs used a year later, I'd pay half that. Cable TV is false economy unless you're a stay-at-home parent. For almost everyone else, direct purchasing costs less and you don't have to put up with the commercials.
Re: (Score:3, Insightful)
Cable TV is false economy unless you're a stay-at-home parent.
As the husband of a stay-at-home Mom (and home-schooled daughter), I can think of no worse "economy" than watching enough television (of *any* kind) to where the cost-per-hour (even just talking *actual* costs) becomes an important metric.
And that's measured in *any* way you'd like, from hard financial data all the way to the value of the brain cells wasted in the process...
Of course, if you equate /. with TV viewing, then I start to feel
Broader market... (Score:3, Insightful)
And that promise has been met - for channels where the programming and audience justifies it, like HBO. The thing is, when that promise was made, nobody envisioned 200 niche cable channels. There isn't enough subcriber revenue to pay for all those when the audience gets spread around like that, and commercials make up the difference.
You could have commerci
Re: (Score:2)
Re: (Score:3, Insightful)
Re: (Score:3, Funny)
Another check (Score:5, Interesting)
Tom.
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2)
You might have noticed that music in shows also tends to be "louder" than dialogue.
Re:Another check (Score:4, Informative)
No, they are louder. They are told what to submit in order to have the levels be equal. They submit clips with sound outside the bounds they know they should. It is purposeful and calculated. If you ever have sit in a TV station or cable head-end, you can watch the levels and tell when commercials come on. Even "loud" shows (like 24 and the constant explosions) are quieter than some guy talking about his low-priced furniture.
Re:Another check (Score:5, Interesting)
For those who run Windows... (Score:5, Informative)
It also has extremely sophisticated commercial detection (never failed me), based on blackouts, duration of blackouts, duration between blackouts, percentage of screen changed to black, etc.
http://www.videoredo.com/ [videoredo.com]
Re: (Score:2)
Similar to VideoRedo, I also use Womble MPEG [womble.com]. It also can cut sections without reencoding, and can do transitions, fades, and text overlays. It doesn't do the automatic detection, but its very fast at seeking and stream copying, so it works well for manual removal of commercials.
I'd sometimes use all three methods if I had an extra MythTV box around.
I wonder how many people will point this one out? (Score:5, Informative)
Ummm, no.... I'm sure many people here are already aware, but if not - check out Beyond TV (http://www.snapstream.com/). The guys over at Snapstream have been doing automatic commercial detection for a while now, and Beyond TV is in the category of a Commercial DVR. And, I'm pretty sure that other companies have been doing it too. This is nothing new - and hasn't yet been a 'killer app'.
Re:I wonder how many people will point this one ou (Score:2, Informative)
I'm not s
Same with MythTV then... (Score:2)
Even Tivo I think detects commercials. They just won't let you skip them outright, you have to fast forward through them.
Re: (Score:2, Insightful)
Re:I wonder how many people will point this one ou (Score:2)
Re:I wonder how many people will point this one ou (Score:2)
Yes they have, with code taken from MythTV.
DITTO! (Score:2)
What will happen... (Score:4, Interesting)
Re: (Score:2, Insightful)
Re: (Score:2)
Re:What will happen... (Score:5, Funny)
And You'll Get Sponsored Channels (Score:2)
Does it just skip them or cut them from the file? (Score:3, Interesting)
The detection couldn't be 100% accurate.
Re:Does it just skip them or cut them from the fil (Score:5, Informative)
Re: (Score:2)
And, of course, you can optionally edit the resulting cutlist, and then use it during the transcoding phase when producing an archival copy (for, say, burning to DVD).
A shame (Score:3, Interesting)
-Eric
Re: (Score:2)
Why? Are you willing to pay significantly more money for a PVR that skips commercials, or would you expect such a feature to be included for free? Of course, people will expect the feature for free. So why does it surprise you that PVR makers would shy away from a feature which is virtually guaranteed to invite expensive lawsuits, without a commensurate increase in unit price?
Re: (Score:2)
Re: (Score:2)
That's why I like putting in outtakes during the closing credits. Keeps those cheap bas^H^H^H^H^H^H^H^H^Hnetwork execs from squishing the content. :-D
Article Text (Score:4, Informative)
There are three key indicators that MythTV uses from recorded content to identify commercials.
A blank frame is many times sandwiched in-between the television show and the commercials. The most simple form of detecting commercials is to search for blank frames in the video feed. The problem with this is that it can be very misleading. There can be a blank frame anywhere. Just because there is a blank frame, doesn't mean it's a commercial break. You could easily end up with commercials marked as part of the show and parts of the show marked as commercial.
Scene transitions are another indicator. A scene transition is a cut between one video of something and a video of something else. A simple example would be in a newscast where someone is being interviewed. While the anchor is asking the question, you may see both the anchor and the person being interviewed. When the person being interviewed starts to answer the question, the scene "cuts" to a close-up of the face of the person answering the question. In regards to commercials, there is a scene transition "cut" between each commercial. Each commercial usually is unrelated to the next. The last frame of one commercial would be totally different from the first frame of the next. Looking for patterns in scene transitions is one way to identify commercials. Five groups of 30 second scenes all grouped together may be a good indication of a block of commercials. This method works better than the blank frame method, but also isn't foolproof. There's no reason scene changes in a show might not mimic commercials, and vis-versa.
The third indicator of commercials that MythTV uses I find rather ironic. Bugs, also referred to as DOGS (Digital On-Screen Graphics), or Watermarks. A Bug is that little TV station logo in usually the bottom right corner of your screen during a TV show. I find this ironic because one of the reasons or it being there is to build channel awareness in the world of digital video recorders like MythTV. Since DVR users usually find shows by name rather than by channel, they are less concerned with which station a show is on than are other viewers. MythTV watches for these things. Because the digital watermarks are generally not shown during commercials, identifying one and then watching for it is a good indication of when a commercial break starts or stops. While much more complicated to implement than watching for the blank frame or screen transition, in theory it's probably the most effective in some circumstances. Because in practice they are hard to identify on some stations, the actual implementation can be error prone.
MythTV looks for all three of these identifiers to locate commercials. It breaks each show up into scenes, and then applys a series of score for the scene based on looking at all three factors in relation to one another, especially taking timing and patterns into account. Based on the final score of a scene, it's either (essentially) dropped into the show bucket or the commercial bucket. It's not a black/white type thing. Because of the scoring, there are a whole range of grays in the middle. You end up with scenes that looks "more" like commercials or "more" like show content, and they are then flagged as such.
I've been quite impressed at the quality of the commercial flagger that MythTV has implemented. In my experience, the system does an excellent job.
Commercial flagging is set globally in:
Utilties/Setup -> Setup -> TV Settings-> General
Do you have ideas or talent that can help increase the quality of this great tool? Check out and contribute to the MythTV commercial flagging developers' wiki.
Re: (Score:2)
There are three key indicators that MythTV uses from recorded content to identify commercials.
A blank frame is many times sandwiched in-between the television show and the commercials. The most simple form of detecting commercials is to search for blank frames in the video feed. The problem with this is that it can be very misleading. There can be a blank frame anywhere. Just be
Re: (Score:2)
Adnix (Score:2)
Re: (Score:2)
So, is that Hypocritical or Ironic (or both)?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Could be possible to implement in Topfield boxes (Score:2)
Anyway, as I don't really fancy a full-blown PC to my living room, I'm wondering if someone has either already made a TAP for this purpose, or if the MythTV stuff could be ported to Topfield?
CDN (Score:3, Informative)
This was going to be a joke, but... (Score:4, Interesting)
I say this because, ultimately, the difference between commercials and "content" is entirely made up of the information they present. As advertisers and broadcaster get better at removing the "flag" type of marker (blank frames, scene cuts, predictable timing) from commercials, there will be incentive to develop more intelligent ad-blocking mechanisms. Obviously, we're not at that point yet, as the methods described as employed by MythTV are fairly naive flag detection mechanisms - but with growing incentive, the odds of working towards a truly intelligent ad-removal scheme increase.
I think it would be hilarious if the biggest mind-mushing technology of all time (television) turned out, indirectly, to contribute to the rise of alternate, machine, intelligence.
Re: (Score:2)
Imperfect (Score:2)
Re: (Score:2)
Re: (Score:2)
You know, they could really improve things by realizing that commercials are always in fifteen second increments, and almost always fifteen, thirty, or sixty seconds. If there is no apparent scene change within a second or so of the correct duration, it's not a commercial. Maybe add a little audio pop detection to catch badly timed inserts by cable providers that can be off by a couple of seconds. Either way, if it skips ahead for a dark scene by several minutes, it clearly is ignoring the consistency of
SHOCKING NEWS! MythTV linked to Al-Qaida, France (Score:5, Funny)
MythTV is a front for American's sworn enemy, Al-Qaida.
Also, some people who use MythTV have French accents, and many others have eaten French Bread.
Do the right thing. Install a wholesome, American operating system on your MythTV box and run a Advertising Ready (tm)** PVR solution.
It's the patriotic thing to do.
And we'll be watching.
* A product of the National Association of Broadcasters.
** Advertising Ready is a registered trademark of your friends and fellow consumers at AWMC (Americans Welcoming Mind Control) ***
*** A product of [REDACTED BY HOMELAND SECURITY]
France? (Score:2)
V-Chip rating signal present during commercials? (Score:2, Interesting)
Re: (Score:2, Informative)
Not gonna happen (Score:2, Funny)
the Industry (Score:2)
Sure television commercials are big business, but it's not the only avenue for the advertising industry. In-game ads, in-store displays, product placement in TV shows and movies, roadside billboards, newspaper and magazine ads, and... oh yeah, I heard that the Internet has some advertising in it, I think....
Advertising isn't going away any time soon. It'
Re: (Score:2, Insightful)
-1 Troll? Indeed. But the fact of the matter is that people are being forced out of their chosen industry all the time... It's happened before, and it will happen again in the future. "But people will lose jobs" should never be considered a valid
Volume? (Score:5, Insightful)
Loudness answer from Comcast (Score:3, Informative)
(I call B.S., of course. If I can do real-time "normalization" with MPC, AC3filter, etc. for free, you're telling me they can't figure it out on their budgets?)
Me: "Why is it that the commercials are about 50% louder than the regular programming content? It's bad enough that I have to pay a monthly bill to get so much advertising content, but it's even worse to have the ads shouted at me..."
Comcas
Commercials that play at bottom entire time? (Score:2)
I know some stations displa brief pop-up ads, but never seen any that displayed on the screen the entire time of a program.
Ron
Re: (Score:2)
a) It would piss people off. Ads are bad enough already without having distracting banners on-screen.
b) It would require fancy programming at all. Just crop the video. Voila, done.
Heck, you could probably automagically detect where the ad
Re: (Score:2)
"bad" when MythTV does it, OK for Comcast? (Score:3, Insightful)
So, I'm sure they're compensating the networks for the commercials they're overwriting, right? I mean, with network commercial time costing in the gigabuck$ and all. And when we viewers do the same thing, we're stealing the networks' life blood...
I'm going to need a little more convincing.
Re: (Score:2)
Umm, dude, that isn't anything new. Local avails have existed on cable television for a long time. Most major cable operators offer local avail insertion to companies, and insert those ads in slots marked by the network for local insertion. If they did anything else, they'd seriously piss off the networks, likely resulting in losing the rights to rebroadcast the content.
a good way to be sued (Score:3, Informative)
He wrote this on his own, and said it was pretty easy to figure out if you just watched the mpeg stream (though I've never done it...)
He had a discussion with the guys at Tivo once, and when they discovered that he had this feature, they told him NOT to make it public, and that if he did, "bad things" (involving lawyers) would happen.
So he kept the tools for his own personal use.
--Ray
Closed Captioning (Score:2)
Far from perfect (Score:2)
I want one (Score:2)
Actually, I think the best commercial remover at any point would simply look to see if a particular 30+ second sequence repeats itself. If so, it's probably a commercial (most commercials run multiple times throughout a show). This would also work nicely to not cut out superbowl commercials, which have far more value - as they aren't usually repeated.
Article is
Network bug removal (Score:2)
"Product placements" replace commercials... (Score:2)
As the technology like this improves, expect more and more product placements. I don't blame anyone, who wants to skip the commercials. But I don't blame the creators of entertainment for wanting to get paid either.
We refuse to watch the commercials, that would sponsor them. And we refuse to pay them directly by sharing their works with everyone, who can connect to our computer.
Can't do anything about product placements, so the phenomenon is here to stay and grow.
Two other methods (Score:2)
Another m
There should be an "Open" Commericial Identifier (Score:4, Interesting)
If there was one, we (AmericaFree.TV [americafree.tv]) would use it, and I suspect other Internet television broadcasters would too. Why ? Because in the long run commercials (as opposed to product placements, sponsored events, etc.) will only work if people want to receive them, and because people will just fast forward through them anyway.
fanless MythTV box? (Score:4, Interesting)
- is open: can run MythTV,Linux,Vlc,mplayer,... and no reverse engineering is required to use basic hardware (and hopefully mpeg decoders).
- has DVI output
- has S/PDIF out
- is fanless
Basically an open DVD player with DVI out. A DVD player costs $50, but an "open" DVD player costs $1500? $300-$500 would be ok.
Or basically, a fanless mac mini, fanless MiniPC,...
myth's commercial tagging is a great start .... (Score:3, Interesting)
(1) bug-blocking. ie: get rid of the channel identifier. It's annoying and gratuitous. Most of them are semi-transparent. There ought to be some way to xor them away.
(2) pop-up blocking. Those annoying animations that some networks are starting to put up on the bottom or right side of the screen right in the middle of a show, that are not related to the show.
(3) auto unsquish. When the network squishes the credits to the left 1/3 of the screen to put in some talking head telling you what's next. I want to squish the talking head.
(4) kill the talking head's overdubbed voice.
I know. I'm dreaming. Usual complaint applies: "I already pay through the nose for this. Stop making me get TV the way _I_ want it (from the torrent channel)."
Re: (Score:2)
Re: (Score:2)
Re:Article Text (Score:5, Funny)
Re: (Score:2)
Re: (Score:3, Insightful)
You seem to talk about this as though it's a bad thing.
Re: (Score:3, Insightful)
These days, it's almost impossible to watch TV, use the internet, turn on the radio, go to the cinema, read a newspaper or magazine, walk down the street, check your snailmail, answer the telephone, play a video game , or even go to school without being psychologically invaded and manipulated into buying on
Re: (Score:2, Insightful)
Of course, the MPAA and RIAA don't agree with me...
Re:So my question is.. (Score:5, Insightful)
Consumers are tired of the commercial bombardment. And many of us are already paying to watch TV -- through payments to the cable company or to the satellite company. From the consumer's point of view, we're paying for the show, and then the viewing experience is degraded with non-content.
TV networks are in dire need of a better marketing model that better serves the consummer of their product.
The availability of this technology is a threat to the existence of less innovative networks. And it should be. Creative destruction is a good thing.
Re: (Score:2)
Re: (Score:2)
Since a corporation is a pretend/legal version of a real being, let's not forget that like real live
Re: (Score:2)
Re: (Score:2)
Like it or not, that's what pays for the TV we watch
Re: (Score:2)
Re: (Score:3, Funny)
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:2)
There were a couple of VCRs that "looked" for the mark and would pause the tape if they saw it. They didn't work very well. More often than not, the VCR would miss the signal that indicated that the program was starting again and stay on pause.
Re: (Score:2)