Zap2It Labs Discontinuing Free TV Guide Service

QuijiboIsAWord writes "Zap2It Labs, which provides free TV listing data for personal use, has long been the main source of program guide information for users in the US and beyond. They've announced via their webpage that, due to abuse of the service, data will no longer be available after September 1st. There is no other direct source, and no option to pay for the service even if the users wanted to. Without a data feed of this type, users will be reduced to scraping websites at best. Is this going to be a killing blow for MythTV?"

This is troubling

I don't use MythTV, and so I was surprised to see that it relies on a private third-party source for TV listings. Isn't there any way to obtain this information in an "open-source" manner?

Re:This is troubling

They rely on advertising to make money and aren't going to freely help people with a device that most people are going to use to skip advertising.

They seem to forget that a substantial chunk of those viewers wouldn't be watching at ALL, if it weren't for DVRs. I like a show that's on when I'm usually playing with my kids. If I didn't have a DVR, I wouldn't watch that show period. Yes, I skip through many of the commericials during playback, but not all of them, and not if the commercial catches my eye, or is for a product I'm interested in. I even rewind to watch a commercial from the start (like if I skip into the middle of a Mac/PC ad I haven't already seen) etc.

Before I had a DVR I hit mute and/or pipped the commercials while I browsed the channel guide, or checked on the hockey game, or something. Its not like I was sitting there 'attentively watching' all the ads before.

I expect advertisers are probably losing eyeballs overall as people adopt DVRs, but its probably not nearly the issue they think it is.

Re:This is troubling

Not unless someone volunteers to compile the voluminous amount of information themselves, then set up a distribution method.

This was always MythTV's achilles heel, more than even HDTV. For all the talk about "Unlike Tivo, MythTV can NEVER be shut down or crippled," MythTV always had this dependency on a third party, for profit service. It's possible someone could replace them, but they're going to want SOME form of revenue (and since no one is going to tolerate ads on their MythTV, or pay for the service, this is unlikely).

Re:This is troubling

and since no one is going to tolerate ads on their MythTV, or pay for the service, this is unlikely

Given the choice between:

1. paying Comcast's fees, DVR service, etc
2. paying Tivo or
3. paying for a subscription to an XML TV Listing service, and keeping my MythTV box

I'll take #3.

People will pay for it as long as no free alternatives are out there.

TV stations are hardly interested in helping...

Isn't there any way to obtain this information in an "open-source" manner?

The best way to get the information is from the stations and cable operators.

Unfortunately, MythTV and other PVR users are in the game of cutting out ads; TV programming is purely to sell ad space, and always has been, save when programs were entirely paid for by one company and the show was branded in their name. What motivation do TV stations have to assist people who are purposefully going out of their way to cut out the ads?

Re:TV stations are hardly interested in helping...

Without a DVR, there is zero chance that I'll see any show that doesn't air between 7-10PM on weekdays. For that matter, the odds of me seeing shows that ARE between 7-10PM is slim, since I'm often doing other stuff too. From their POV, they'd probably be better off with me seeing bits of pieces of commercials at FF speed than with me seeing nothing at all.

Re:This is troubling

Isn't there any way to obtain this information in an "open-source" manner?

The best possible "open source" solution would be to create some sort of wiki for tv listing data.

Anybody know what the "abuse" was?

Their website does not explain. Is just using the data in MythTV, "abuse"?

Re:Anybody know what the "abuse" was?

Based on their previous complaints and this message, I think the problem was people were using the free data set, then redistributing it, probably for profit, possibly indirectly (say, selling devices that used Zap2It's free service). Zap2It makes money selling their data set and they were very generously offering it for free to individuals. But you weren't allow to redistribute it.

Re:Anybody know what the "abuse" was?

I think you're trying to suggest that Slashdot readers are, generally, hypocrites because they don't care if new technology or business models drive old ones out of business. Of course, that's utter nonsense. This isn't about Zap2It's business model being harmed by new technology or new business models. This is about Zap2It offering something to individuals for free out of simple generosity. This generosity has been abused and does threaten their business model, so they going to stop being generous. Zap2It doesn't suffer in the slightest, only those who benefited from their generosity do.

Now, if a third party started giving away television schedules without relying on Zap2It's data, harming Zap2It's business model, then we'd be on the same ground. Of course, if that happened and Zap2It whined about it, I don't think you'd see much sympathy here.

Re:Anybody know what the "abuse" was?

According to a posting on mythtv.org... [mythtv.org]

Some reasons and other comments given for the scheduled discontinuence, copied from the forum, copied from the mailing list:

• Continued use of the service to support commercial products, in violation of the agreement.
• Commercial products continued to grow despite previous appeals that this activity stop.
• There are significant changes to the supporting data structure forthcoming and we could not devote resources to the continued upkeep and enhancements of the service.
• Maintenance of the service is impacting our resource pool for other projects.
• We sought alternative options but were unable to find a solution.
• We recognize the hardship this creates for the user community. We are open to alternative solutions and would consider proposals that met the needs of the user community and our company.
• We looked into options to turn this into a paid service however we do not have the infrastructure at this time.

Re:Anybody know what the "abuse" was?

While no longer linked from the front page, this link [tribune.com] still works.

Basically, the content was provided free provided it remained for non-commercial use. After all, commercial ventures have to pay for those listings and if they could get it for free, nobody would pay.

I hope they at least tried to weed out the abusers before just cutting the cable.

How is MythTV dead?

... Without a data feed of this type, users will be reduced to scraping websites at best. Is this going to be a killing blow for MythTV?"

I fail to see how, unless the screenscrapers specifically mentioned don't work. I wrote a scraper for my MythTV box to pull PPV information from somewhere, since it wasn't coming from Zap2it. It's not that hard, really, and took me all of an hour to do.

Achilles Heel

I first became aware of MythTV some years ago from a developer that was excitedly working on the project. With all the effort that was going on at the time, nobody seemed to have a clear-cut idea of a long-term, stable way of getting TV listings. "Scraping web pages" was the only plan.

Looks like five years later, it's still the only plan.

This is quite bad :(

This was an invaluable service - makes me wonder who's putting the pressure on them :(

Re:This is quite bad :(

Tribune put new management in Zap2it. They have been unresponsiveand treated the Data Direct service like a redheaded stepchild for a year now. The data has been bad, with long outages on it from time to time for a while now. Many of us that have used myth and other xmltv systems have tried to pay for a subscription for a couple of years now and they refuse.

it's the new management, they hate that OSS people are getting access to the data and want to stop it.

Misuse?

How, exactly, does one go about "misusing" TV schedule listings? Is this really because Zap2It was making all other forms of TV listings obsolete while not making any money at it? The announcement is quite vague. Does anybody have details on what's going on?

TitanTV

I use a Windows based PVR that works with the Web site titantv.com Clicking on the 'record' button on the Web site sends a small file that a helper app converts to a Windows scheduler entry. I can't believe it would be that hard to adapt to Linux.

The Titan TV web site includes advertising and also does tracking. While personally, I don't care if anyone knows I watch StarGate and Myth Busters, privacy issues may be a concern for some.

Going, going, gone?

I've been a Zap2it subscriber for at least three years for my MythTV.

At first, they made me fill out a big online survey as "payment" for the service. The first time it was about 30 questions.

The third time (this is like every 3-6 months) they only asked one question.

For the last year, the survey has been "click here to renew."

What's with that? I'm willing to give up some personal time and info to pay for this service, and they can't even think of a way to leverage that?

The main listings

It doesn't appear that this will affect their main listings on their website, just the programs that tap into their database.
I've always just setup an easy shortcut to their web page to get listings, so I really don't think this will affect my usage in the least.

The killing blow? I think not.

This will be the killing blow for MythTV and other open source DVRs? I think not. MythTV predates Zap2It and managed to do okay. Yes, it relied on screen scraping, but it worked. Furthermore, I know I'm perfectly prepared to pay a small monthly fee for a good data source. Maybe $5 a month? Since a company offering such a service doesn't need to recoup costs for selling hardware below cost (as Tivo does), such a price should be feasible. Since Zap2It was free, there wasn't much incentive for someone to offer the service, but now there is. I'm hoping the free market will see the opportunity and we can work something else.

I don't see the problem

What's wrong with just "scraping" web pages (I assume that means writing code which automatically downloads the webpages and captures the data of interest, rather than requiring a human to do it. I do this all the time with Perl code.)?

There's multiple sites out there with TV listings: Yahoo TV, Zap2It, MeeTV (the one I use), etc. Just write perl scripts to capture the listing information from these sites, and modify MythTV to allow the user to choose any service he wishes. Of course, some of these sites may (stupidly) screw with their HTML in order to throw off these scripts, but that's easily worked around with regular updates. So we just need to have a "myth-scripts" distribution site where your Myth box automatically checks for updates to the perl scripts every day and downloads them if necessary, just like we already do with many other things.

No, it's not quite as reliable and efficient as a static interface to this data, but if these companies are stupid enough to remove static interfaces, thinking we're just going to go back to doing everything manually and looking at all the ads, this seems like a reasonable solution. There's no way of preventing automated scripts from downloading webpages.

Killing blow?

Me thinks that someone doesn't quite grasp the strength of OSS: That strength not just being free. Ultimately, it's the dissemination of data. Which is precisely why this problem is uniquely suited to being solved by OSS.

This will be a minor set back at worse. But, like any set back, it will make the overall product stronger.

That said..."NNOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO OO!!!!!!!!!!!!!". Zap2it was damn handy. Thank you Zap2it, for a great service.

Thanks

Thank you for being the first to thank them.

It sucks that it' shutting down (I use it for GB-PVR)- but they did do it for free for years. Let's not demonize them, no matter the reason, and start looking for alternatives.

MythTV devs are working on this...

http://www.gossamer-threads.com/lists/mythtv/users /275533#275533 [gossamer-threads.com]
And it isn't just MythTV that uses the guide data provided...

Google should provide a WebService

Shouldn't Google or Yahoo or Microsoft or any of the other big media / tech companies of the 21st century provide this information as a free WebService? Seems like something Google would jump on top of since their mission statement is to organize the worlds information. Well, TV listings is information.... get on that Google!

What if I was to write a web service?

What if I was to write a web service that exposed the data garnered from website-scraping? You could just write a standard XML request, wrap it in SOAP tags and send it to the web service, and you'd be returned whatever information you requested- by channel, time, or show name...

Any takers?

Rather than a Million Screen Scrapers...

Rather than a million screen scrapers each hitting various listing sites, better one screen scraper sharing the resulting data with a million video rebels. This solution would probably leave everyone happier.

Now there's just the question of who? Who is expert at spidering the web? Who likes to provide new cheap-to-free services in their quest to take over new markets. Who would love to put yet another spike into Microsoft's side by removing yet another possible revenue source for them? Who doesn't have to worry about financing such a small, cheap service alongside their already multitudes of underutilized servers and bandwidth?

Google?

We need GoogleTVListings (tm)

Google needs a TV Listings aggregator!

Of course it would be the best ever... With version numbers for each day and diffs available that only contain the changes. The whole thing would be available as an rss feed and would be free, of course.

Why not use Guide+?

My TV automatically downloads, somehow (over the air? cable?) channel lineup listings through the Guide+ system.

Could a computer not do the same thing?

Make it a paid service

Like many others, I'm a little surprised that they aren't moving to a subscription model. Clearly they know better what their available resources are, and what they are and are not capable of handling, but it seems like a missed opportunity to walk away from a situation where their servers are getting hammered, and start charging a small fee. Many, many of the MythTV users would happily pay a few dollars a month to have a steady stream of information. Sounds like it could be a million dollar annual income right there, and that's got to be very hard to walk away from for any company.

As for commercial abuse, if they know it's happening, they presumably are taking steps to quash it as well, without much luck. Probably like playing whack-a-mole.

Let's all hope Google comes to the rescue.

Re:Make it a paid service

I was happy to find your clear, concise, comment down here all by itself. It makes it easy for a clean response.

Let's say that MythTV implemented your paid service plan and began charging the princely sum of $2 per month for the data.

I would give it all of 7 days before that paid for data became available for free. Someone, somewhere, would buy the data for $2 per month and load it up for others to have free of charge. It would be a daily torrent that you could pull, or a streamed RSS feed, a static layout site with a downloadable screen scraper, or any one of a dozen other ways I can think.

So now instead of a million dollar revenue stream you'd get a thousand dollar revenue stream coming from the 500 users who would actually be wiling to pay when a free source is available.

If you can answer the question of how to prevent the above scenario from happening I can put you in touch with some content providers who will pay REAL money for your idea. The kind of money that allows people to retire for life...at the age of twenty.

What happened to Mythtv's paid service, TechnoVera

A year ago it was announced that an alternative paid service through TechnoVera [slashdot.org] was available to replace Zap2It with part of the funds going back to OSS and mythtv -- and no periodic surveys. Couldn't users switch to this service? -- or is it no longer available? (I've never used this service myself.. any users care to respond?)