Everyone's A Beta Tester 114
kukyfrope writes "Many people dream of being videogame beta testers but, in reality, a lot of us already are. GameDaily's Greg Atkinson discusses how developers are using the ability to patch games as a crutch for launching games ahead of schedule, using a 'we'll patch it later' mentality, as opposed to extensive play testing." From the article: "What's going on lately that so many games are being released unfinished? Why are the people now paying to essentially beta test the games rather than purchasing completed games? ... If you scan through the PC reviews, on this and any other site, you will notice an overabundance of games that lost points or otherwise hampered their players' experience by being unpolished, full of bugs, and sometimes downright annoying to play. Everything from controls and camera movements to balancing issues, broken quests, and of course graphical errors are abundant in probably half the titles on the shelf these days. It's become habit to look for any patches to a game while I'm installing it, and that's not right."
under construction??? (Score:1, Funny)
Re:under construction??? (Score:1)
Re:under construction??? (Score:2)
Not just games (Score:1)
Re:Not just games (Score:3, Insightful)
Re:Not just games (Score:1)
Developers have been using the "patch it later" mentality as a crutch, but it's only because of the explosion of the Internet has allowed developers to make improvements to games for years after they've released them (like Blizzard with Diablo II). Sometimes whole new features are implemente
Complete opposite of the point.. gg.... (Score:1)
Google is the opposite, keeping it in beta past the point it reaches stablility to absoloutely assure its completed quality.
Nothing new (Score:2)
Re:Nothing new (Score:1)
Re:Nothing new (Score:2)
Re:Nothing new (Score:2)
Re:Nothing new (Score:2)
The only apparent DRM on consoles is the need to have the disk in. Since it can only handle the one game at a time, unlike the computer, there's no loss in functionality.
Copy protection on games on the computer limits the usefullness of the computer. It does not limit it on consoles. I'm at the point that, unless it is
Alpha (Score:5, Funny)
Re:Alpha (Score:2)
Yet another reason to get a console (Score:3, Insightful)
Re:Yet another reason to get a console (Score:1)
Re:Yet another reason to get a console (Score:2)
Re:Yet another reason to get a console (Score:1)
Re:Yet another reason to get a console (Score:2)
Re:Yet another reason to get a console (Score:1)
It could in fact encourage just that. With an online service that everyone will have access to, they can sell the players "additional" content for their games... something along the lines of, oh I don't know, maybe a Horse Armor model for a couple bucks?
Instead of just putting these things in the game in the first place they will find it more profitable to leave them out and then char
Re:Yet another reason to get a console (Score:3, Insightful)
You say that, but... (Score:4, Insightful)
We've got either Direct3D or Direct3D for our graphics API. DirectSound or DirectSound for our audio API. DirectInput or DirectInput for our input API. (I do realize I'm discluding OpenGL and SDL, which are both great tools, but it's to make a point).
So with Direct3D, DirectInput, and DirectSound wrapping different hardware, what's the big deal? I can buy the argument that console games will be more optimized because you can get up close and personal with the hardware, since everyone will have the same damn thing.
No, the differences will be things like games crashing or the computer crashing due to bugs in the DirectX side of the driver, or the kernel-side of the driver.
So what about gameplay? With gameplay bugs, that's logic in the program. If I shoot you and you don't die, that's not because I have an ATI video card instead of an nVidia card, it's because the hit detection algorithm is broken.
Non-fixed platforms mean I have to deal with lower performance (often poorly tuned game design is the major factor here) and maybe crashes, but it doesn't mean I should put up with broken game logic.
Re:You say that, but... (Score:2)
Those are pretty valid bugs. (Score:2)
The boss at the end of stage 2 may be impossible to see, but that can be fixed without patching the game engine itself. Merely changing the video card will fix it. The OP was referring to problems caused by hardware that made a game unplayable, but those are fixable without patching the games, so I argue
We're already seeing patches for consoles now... (Score:1)
Also, obligatory "Oblivion is an awesome game anyways" that everyone always says. Because, really, it still is an awesome game.
My policy (Score:3, Insightful)
Re:My policy (Score:2)
So stop buying unfinished games (Score:3, Insightful)
As long as you keep buying beta-quality stuff, and the companies keep making their sales and profit goals, they'll keep doing it.
Duh.
Re:So stop buying unfinished games (Score:1)
Re:So stop buying unfinished games (Score:1)
Re:So stop buying unfinished games (Score:1)
Re:So stop buying unfinished games (Score:2)
Except your just told us not to buy any newly released game. If nobody buys it, who will be in the online forumns to tell us about it?
Re:So stop buying unfinished games (Score:2)
And once they have your money, don't expect much in the way of patches. They'll be expecting you to give them more money for the sequel, and will be too busy making that to fix any significant bugs in the game you just bought. The sequel will be full of bugs too.
Re:So stop buying unfinished games (Score:2)
Jaysyn
Re:So stop buying unfinished games (Score:3, Insightful)
At what point do we learn that a particular game is buggy and/or unfinished? The day the game is released? Do we wait for the game reviews at IGN / Gamerankings? How on earth do you find out if a game is buggy or unfinished unless someone, somewhere, buys it? If no one buys the game because th
Re:So stop buying unfinished games (Score:2)
The answer to that is a simple one: There are always sucke...people...who are willing to run out and buy a game on day 1 of its release. The advice in question is meant for people who actually can stand to wait an extra month or two for a game. The folks who go so far as to preorder (one of the worst develops in retail game sales) a game will continue to do so because they've already decided that t
Re:So stop buying unfinished games (Score:1)
Re:So stop buying unfinished games (Score:1)
Blaming the wrong people (Score:3, Insightful)
When the publisher isn't the developer, it's the publisher who sets the release date. Then everyone complains about bugs and the developers go back and fix everything they would have caught with sufficient testing..
Re:Blaming the wrong people (Score:2)
Publishers set the release date. Really? And HOW do they set the release date? By asking the developer, "when will this be ready?" So the developer's failing to deliver the product on in time is not the publisher's fault. The publisher, on the other hand, has the job of working out all the time-sensitive deals that make a product go: advertisements, baksheesh to the gaming rags, dup
Re:Blaming the wrong people (Score:2)
The problem, as you say, is with inexperienced developers:
Re:Blaming the wrong people (2nd try) (Score:3, Insightful)
[Crap, I knew I should have pressed "Preview"]
Publishers set the release date. Really? And HOW do they set the release date? By asking the developer, "when will this be ready?" So the developer's failing to deliver the product on in time is not the publisher's fault.
The problem, as you say, is with inexperienced developers:
Re:Blaming the wrong people (Score:2)
Sure, but this developer might soon be unemployed. Of course, within a company you should be firm towards managers. But in the game industry there are two parties involved. The publisher wants a game, say a first-person shooter. They contact a developer, who says he can make that game in 18 months. The publisher says 'OK, here's 20 million dollar, go make that game'.
The opposite happens too, that a g
Consoles (Score:5, Insightful)
Re:Consoles (Score:1)
Re:Consoles (Score:2)
Re:Consoles (Score:2)
When I first saw this article, I thought it WAS referring to consoles. I've been amazed by the number of patches to games via Xbox Live for the Xbox360. I don't have one myself, but I constantly hear about various patches for various games.
Sure, some of them add content, which are nice... but a lot of them are to fix problems. (For example, I've been hearing a lot about a patch to fix some baseball game some people are having pr
Re:Consoles (Score:2)
It's not as if there's a broad hardware base they have to test against. As someone said more recently in this thread, now that xbox 360 has a patching system, developers can rush games out the door and patch later.
Re:Consoles (Score:3, Informative)
But I've found bugs in every game I've played on the XBOX lately. Prince of Persia 3 has sound bugs (sound doesn't stop playing even after you've left behind the area that was 'generating' the sound - e.g. a water fountain), game-freeze bugs, and control bugs. Lego Star wars has a game-freeze bug - very reproducible, just hang out outside the Cantina long enough. Tomb Raider has "Lara gets stuck" bugs. I could go on.
Bugs, bugs, bugs. They're everywhere. You know how some sites bre
Ding! Wrong answer. (Score:5, Interesting)
That is precisely what I am NOT seeing. One if the biggest problems with reviews is that the reviewer rarely depends on the product (or even pretends to). For instance, take a new car review. The reviewer might mention (this happened in a long term study in a major car magazine recently) that they had to have it in the shop on average once every three weeks for eight months. For those of us in the real world, being without our vehicle for at LEAST 2-3 hours (often 2-3 days) every month would cause major problems. For them, it is a minor issue because they are not dependant on the product.
The same applies to gaming. If I purchase a product and it won't work correctly out of the box on my system (most recent example: galciv2), I can't play it. This is because I have one decent gaming system. Sure, for those out there that have more than one gaming system it is a minor inconvenience, but for the other 99%... Reviewers constantly ignore this. If they have that issue they simply use another review system and note it in the review. The game may lose 2/10 points for that. Unfortunately, saying a game is say 7/10 including that issue does not reveal the actual rating of the product, because many people will be unable to use it, making it a 0/10.
The response to this is often "just wait for a patch." Fantastic. I paid $50 for a game that by the time it is playable for me the retail price has dropped to $40 and I am just then getting to play. Can I take it back? Nope, not under current laws. Does the consumer get screwed? Yep.
Re:Ding! Wrong answer. (Score:2)
They all seem so taken by the fact that some no name company made a good game they don't want to point out it's many glaring problems.
Beta testing is a Marketing Function (Score:3, Informative)
Alpha testing is actual testing, and what game testers who work at places like EA do. Not Beta.
Re:Beta testing is a Marketing Function (Score:2)
No offense, but your definition of beta testing does not correspond to mine.
The label "beta" has been applied and misapplied as a marketing tool, but that does not mean that the stage, and the QA processes with it, mysteriously disappears by an act of newspeak.
Re:Beta testing is a Marketing Function (Score:1)
However, Beta is a marketing tool, NOT a QA tool, unless you believe in QA's role as a customer advocate to bring the customer the features they want. I did not say to stop testing after alpha, but if your company is relying on Beta to find bugs, your company is making a crappy non-qual
Re:Beta testing is a Marketing Function (Score:2)
Not exactly. For products, Beta testing is testing the software in a wide range of target production environments and real life conditions. The idea is to have many people in many different machines exercising the software without developer bias (ie doing to the software the kind of th
I dunno (Score:2)
Again? (Score:2, Funny)
duh (Score:1)
2) put it out in stores
3) ???
4) profit!
Re:duh (Score:1)
this has been true for a long time (Score:4, Insightful)
Inevitable (Score:1)
What about graphics card drivers? (Score:1)
There's actually an additional "benefit" of this (Score:3, Insightful)
When a game is released, the cracking groups jump it and try to beat each other in the attempt to release the 0day. But a crack for a patch? Has no priority. You will more often than not notice that the game is cracked the day, or before the day, it appears in the store. A crack for a patch can be delayed by days, sometimes even more than a week.
Of course, this also serves as an annoyance to those who use cracked games. They have to get the patch, and the corresponding (not any, but the corresponding) crack.
Let's face it. Game studios have many reasons to release buggy betas, and zero reason to provide us with finished games. So why should they? As long as we buy their buggy betas, this won't stop.
Re:There's actually an additional "benefit" of thi (Score:1)
Re:There's actually an additional "benefit" of thi (Score:2)
Re:There's actually an additional "benefit" of thi (Score:2)
Actually, they can release finished products.
They just need to find a trick so that the amateur cracks miss a copy-protection that occurrs at the 75% mark of the game. Even if the game is still possible at that point, it will be much harder to complete. (Just make sure that normal users aren't bitten.)
Or failing
Re:There's actually an additional "benefit" of thi (Score:2)
Penny arcade was talking about this (Score:3, Insightful)
The more things change the more they stay the same I suppose.
This is why I don't buy games (Score:4, Insightful)
A few years ago I downloaded the demo of Sim Golf and loved it. So I went out and bought the game. BIG MISTAKE. That game was horrendously buggy and most of the bugs were glaringly obvious too. Token example: golfer complain about having to walk up hills, so you get them golf carts. When driving a golf cart up hill... they would complain about walking up hills. They fixed this, but it's the kind of thing that struck you instantly and should have been caught. Add in bugs that prevented you from editing your courses, playing your courses, and other such things and the game became almost unplayable and lost all it's fun.
This is one of the reasons why I stick to console title now, but as I've said I rent them because this kind of stuff is starting to creep in (combined with just plain bad games). This is a real shame because if I find a game that I really like, I won't buy it because I beat it while it was rented and I have no reason to go buy it since I won't play it. It is a very rare exception that I buy the game (Frequency and Amplitude are about the only two).
There are only a few companies and game series that I will buy without playing first. Nintendo is probably 90% of that. Harmonix is another company that has achieved that status for me. Other companies that had it decided they didn't care and lost it due to blunders (bad games, buggy games, whatever).
If you buy a game and it's buggy... RETURN IT. COMPLAIN to the company and the retailer. It is DEFECTIVE. If you put up with that kind of treatment, it will only get worse (as history has shown).
I think a good test is the zero day patch for game-play. If there is a patch out when the game is released (or within any short time frame) that fixes game-play bugs (hardware compatibility stuff is OK) then that company just doesn't care. Don't give 'em your money.
Let's look at Nintendo. I can't remember experiencing any bugs in their games (I've seen them in plenty of others). Do you remember what happened when it was found out that Pokemon Gold & Silver wouldn't let you harvest berries after you had been playing for a year or two? Many companies would say "too bad" or "here's $5" or "send it it, but you'll lose all your data". Nintendo fixed it. For free. On a two year old game. And then even gave you a rare item or a rare Pokemon as an apology gift. Pure class.
And notice it took like two years of playing to find that bug. I realize that Pokemon Gold and Silver are less complex than Oblivion and other recent games... but the sheer number of bugs in such a large volume of games can't be blamed on complexity alone but hubris and an uncaring attitude towards the consumer.
Re:This is why I don't buy games (Score:1)
Nintendo has shipped plenty of games with bugs. How about the Minus World [wikipedia.org] in Super Mario Bros.?
Re: (Score:1)
One can hardly blame the game programmers... (Score:1)
Seriously though, I would put more blame on the game publishers forcing games out early (LucasArts is especially bad for this) than those who write them.
Is this the case anymore? (Score:3, Interesting)
It used to be that any game I bought had about 10 patches before the month its was released changed.
Now, it seems like game are in perpetual development, NOT being released to customers, ever (Duke Nukem Forever). Games like Half Life 2 and Quake 4 were in development for years and missed umpteen deadlines before they finally came out, and the developer's excuses were they wanted to make it perfect.
While I don't play the whole plethora of games available today, I still usually pick up the popular titles. I find very few times that any of the big releases have any issues that prevent enjoyable gameplay. When a patch is made available (months after release), it usually tweaks games settings or fixes (usually) any multiplayer networking problems. These kinds of issues can only be encountered once something enters wide release. No beta environment could every duplicate what the open market can find for bugs (re, beta testers are usually people in the know, real life is actually filled with morons and people without a clue).
I seldom ever find myself using a game that feels like a beta.
The problem I have is, I would rather a company release a game that might still have a few lingering issues, and then patch it later, rather then holding a game from release for 12 - 18 months to make it perfect. As long as those issues do not interrupt gameplay, or are only encountered infrequently or very randomly, then I won't mind playing the game. What I can't stand is the idea that developers spend 3 months tracking down a bug that maybe only 5% of the market would ever encounter, and that only being once in their life time.
HL2 and Duke Nukem Forever are extremes in this case, where people make it a career to hype about a game that takes years to develop, and then keep postponing it indefinitely. For the most part, I don't need a game to be perfect on release, just good enough to play without too many interruptions or problems. I would be hypercritical of a game that has bugs but was still delayed 12 - 18 months to make it perfect over a game that was released too soon but has a quick patch cycle.
Yet another reason not to buy new releases... (Score:1)
There are some games that don't even bother to fix the bugs, so a buyer could scout out whether the community is still dealing with bugs or not.
What Rock Are You Living Under? (Score:1)
He's pretty close. (Score:5, Insightful)
In some ways, Greg Atkinson is absolutely right, but he seems to be right for the wrong reasons.
There are three global problems in game development: marketing usually promises a date that cannot be met, throwing more people at a problem cannot fix it, and bugs found at the end of a project are hard to impossible to fix.
The marketing date is a huge issue because 90% of the time, the people making the game have no buy-in regarding it. They're working towards being done when it's done, and then when they get told that they have six months when they need a year, things get implemented too fast and half-assed.
Of course, here we are at six months out with no testing so far. In fact, the game is generally in an untestable state due to the huge influx of new, untested, unstable code and/or assets. Several major developers and publishers are now moving to the "monkey" model of testing: hire 100 temps for six weeks at the end, have them hammer on the game, and the end result is 5,000 bugs with little time to fix it.
So, the team gets the game to a basic level of functionality, throws it in a box, and gets to work on the patch while the box winds its way to retail.
Until the industry as a whole learns that QA is no longer just a line-item expense but a necessity, we're going to have issues like this.
Console developers are starting to get it, but mostly because the platform holders have a set of tests that every game released on the console must pass. Fail one test or a permutation of one test, and there is a high likelihood that you won't ship. Suddenly, spending an extra few dollars on testing early to find and fix the problem doesn't seem like a big expenditure compared to the nightmare that is missing your street date.
I'm happy that we've had testing on "SiN Episodes: Emergence" from day 1. Are there still going to be bugs? Always...there's nothing you can do to eliminate bugs entirely. It's the nature of software development. But by getting on the project early and testing through to the end, we're able to make sure that the game is stable, completable, and fun out of the box.
And for a game, that's all you can ask.
Re:He's pretty close. (Score:1)
I think that's a poor excuse. The date needs to be agreed on between marketing and development well ahead of time. A rough date should probably be part of the initial game design where the game is being defined.
Ev
Re:He's pretty close. Retreat, regroup, reduce. (Score:1, Funny)
Haha, you obvious don't work in the business world, do you? Am I right, Michael?
Re:He's pretty close-"/." Interview: Ask Michael. (Score:2)
With pre-existing engines, the engine-related bugs are more focused on usage scenarios that were never anticipated with the original engine (surface penetration, rotating geometry, gravity normals, etc.) or bugs that were there, but never found or were found, not documented, and as a result, never fixed. With new engines, you get some of that (design almost always goes beyond the originally decided-on limits of
Well, you could have no game at all. (Score:1)
Unfortunately, games cost money to make, and there isn't always enough money to make it to the finish line. Sometimes, they need to release SOMETHING, just so that the company won't go out of business. The
Re:Well, you could have no game at all. (Score:2)
That's no excuse.
At least put a warning label on it. Call it Beta, and promise to finish it or go bankrupt. Sometimes I want one of the fewer games of release quality, sometimes I want a beta. Let me choose, don't take my money and then tell me it's beta.
Re:Well, you could have no game at all. (Score:1)
Well, in most media, this is what reviewers are for. Roger Ebert, for example, is happy to point out that he watches bad movies, so you don't have to. If you just buy any old thing that comes in a pretty box, you've got nothing but yourself to blame. There's a lot of crap out there -- even finished crap.
Re:Well, you could have no game at all. (Score:1)
I suppose I'm easy to please
Re:Well, you could have no game at all. (Score:1)
Don't we call this... (Score:2)
To think this all started with Unreal (kind of) (Score:2)
only... (Score:2)
In the past QA for games was much more vital due to the media (cartridges). I don't doubt that developers are also pushing for online features so they can push more games out the door, rather than focusing on delivering a finished product.
It's all about getting the buck as soon as possible for them. Bugs... let's worry about those later.
I could see patching to balance the game... (Score:1)
1) It's nothing new. (For us old folks, remember Microprose?)
2) It's not ever going to go away.
Where there are some stupid bugs that have made it through to the release, I can see patching a game to help keep the balance.
I give credit to certain game makers for trying to make a RTS whos units have different strengths and weaknesses, but I can remember the tank rushes from the original Command and Conquer and the Zergling rushes of Starcraft that lost players s
Re:I could see patching to balance the game... (Score:2)
But I don't care so much about gameplay patches. The things that bother me are bugs in the code, which make a game unplayable until a patch is released.
Doesn't bother me so much, though.
Re:I could see patching to balance the game... (Score:1)
As for multiplayer, I'm all for periodic tweaks to the game to try to balance things out. How many times has Blizzard tweaked Warcraft 3 over the years? Change the health on this, the speed on that, hmm maybe this unit should have a different armor type... all in the name of making t
Complex games, cutthroat market (Score:2)
- Games are a lot more complex these days, with a lot more places that things can go wrong. More and better testing is required to assure the same level of product quality as in yesterday's games. This testing isn't happening; perhaps because schedules haven't changed with the games to reflect the additional required testing time.
- Games don't have as long of a shelf life these days, especially PC games (which, incidentally, are easier to release patches
I agree. (Score:2)
Developer's Dilemma (Score:2)
If you delay the game until it's ready, everyone bitches that it's late.
If you split the difference, everyone bitches that it's buggy and late.
If you manage to stop adding features early enough so that you actually ship a game that's on time and pretty much bug free, everyone bitches that the gameplay is lame, and they should have added feature x from other game y.
And if you hire enough programmers to put out a massive killer game on time w
Simple: (Score:2)
If you don't know whether you can make a release date, don't publish a release date. I applaud Duke Nukem Forever for that, at least. People laugh, but if it's good enough, they won't complain about lateness. Remember how long we waited for Half-Life 2? And remember how we didn't even know they were developing it for years?
The mistake made with HL2 was not the huge development time, it was the announcement of features that weren't included a
When will they learn? (Score:2)
If you're making a PC game, release it early and call it a beta. Charge no more for the beta (hopefully less) than for the finished product, and upgrade them to the final version for free. And pay attention to your beta testers, and patch every
Copy protection that makes pirates work harder (Score:2)
That m
Don't pay for betas? (Score:1)
Do like me, don't pay, download from P2P. I refuse to pay for an un-finished game, therefore I only pay for games when no patch has been released in years. I might buy Warcraft I now.
Patches are bad? You're missing the point... (Score:1)
But games always had bugs. Take perhaps the most successful game in history, Super Mario Bros. Walking through walls, getting stuck on a pixel...
It's not NEW that games are buggy. You have hundreds of different possible system configurations on PC that can lead to bugs. Games like any Elder Scrolls have an almost limitless number of player driven possibilities. Is it possible to find every single bug in them before release? Not unless t
Um, that's Alpha. (Score:2)
The code is supposed to be complete by Alpha, and stable by Beta.
But shouldn't that be true of the content as well? Or at least, complete by Beta, and stable by release -- that is, beta testing is saying, it's done, we know there are still bugs, but we can't find any more on our own.