Monday, March 22, 2010

A Bug's Life - Shipped Bugs

Before I was in Community Management, I had a two-year stint in QA as a tester on Def Jam Fight for New York (one of THE most underrated games to come out of EA Canada, if I do say so myself ;)  Things have changed since I left QA three years ago though, for the most part, game development and QA have remained somewhat inflexible when it comes to the tight schedules that teams have to keep.

Let's take a look at the cycle that most sports titles have to contend with.  There are some that have the belief that games are built start-to-finish within a year.  That would be quite the huge accomplishment but most games that I have worked on began 1.5-3 years before the actual ship date of that specific title.  Meaning that much of the planning and early pre-production phases are done even before the current year's game has shipped.  At this time a lot of the decisions regarding what goes into the game and what doesn't go into the game are made based on the scope: risk and time to implement the feature.

If it doesn't make it into the game, the decision is then whether or not to consider it for next year's (which is actually for two years away) planning stage.  In this way the team is always looking down the road at least a couple of years, while basing their decisions on improving and creating the best game, without compromising on the ship date.

Why is ship date so important? 

Simply put, there are many key partners in the publishing of a game that are affected when ship dates move, whether they be in or out.  Do you ever go to the credits of the game?  Not only are game teams made up of the developers such as artists, software engineers, testers, and animators; but as well the marketing folks, sales people, manufacturing and retail personnel all work with the specific date in mind.  In most sports titles, the biggest partner is obviously the league and the most the important thing for them is the ship date and ensuring that it's at or around the start of the actual sport's regular season.  I think the only exception in the EA SPORTS stable of games is Tiger Woods PGA TOUR.

Would it be better if games were then just developed every two years?

I don't know about you, but I can't imagine an August without Madden, a September without NHL, or an October without Live.  I personally need more than just a roster update every year.

When does QA really get into testing the game?

I won't go too deep into process, basically QA is involved 1.5 to 2 years prior to ship date finding bugs to everything that the the development team is working on.  At these early stages there are a lot of bugs from misaligned text issues to graphics with missing textures to jittery animations.  Everything is written up from a single spelling mistake to a hole in the world where players can leave the play area.  We're not just talking hundreds of bugs...we're talking thousands!

Bugs are categorized on many levels and then are distributed to one or a team of developers to prioritize among ever growing bug lists.  By the time a game is put on the shelves, its next iteration will already have bugs assigned based on the code already being designed for it.  Also at the time a game is shipped, a huge focus is put on the customer feedback as this critical period in the first couple of months help the team determine whether or not to stay on track with already made plans or make changes.

Keep in mind that this is also being done as feedback is also considered for current title updates and what can be done right away to improve the game without compromising it in any way.  It really does become a tricky time and risk-management balance that EA may not be perfect in by any means, but it's tough not to appreciate how well some of the teams manage their time and resources and are able to hit deadlines they way they do with the quality they have.

With that being said, once a game is shipped, there's a 6-7 month period where if there is a feature the teams wants to put in it better be during this time or else it mostly becomes something for the next year.  Once the team begins completing features during the early milestone phases, which are filled with bugs, they approach Alpha at which point every feature they want to have must be in the game, bugs and all.

During the alpha phase, QA digs in and writes up bug after bug which get assigned to devs who prioritize those.  Many times, fixes come in that take care of many bugs at a time.  Beta is declared once all issues are fixed and closed away or shipped.

What's a shipped bug?

A bug can be shipped for any number of reasons.  It could be a small graphical bug that is hardly noticeable.  It could be very difficult to reproduce requiring a user to go through multiple steps in order to make it happen.  Or fixing the issue could potentially cause other issues to occur that would make things worse.  There are very tough decisions that are made when it comes to determining what bugs get shipped and what don't.  With quality needing to be first in the minds of the developers, these decisions are sometimes the most difficult during the whole development process.

I've seen many a developer want so badly to fix a bug they knew they could fix only to be told they couldn't because it would break other areas of the game.  At the same time, I've seen the tough decision made to push a game out extra days to weeks much to the dismay of advertising and retail teams who have timed ads set in order to ensure quality is seen in a final product. 

Once a game goes Beta, it is sent through further rounds of testing internally along with third-party vendors such as Sony, Microsoft, and Nintendo who have their own strict guidelines that must be adhered to.  During this time, the team tests in parallel and if the decision for a title update is made the team would be ready to go with a build (specific version of the game) to submit.

While third-party testing goes on, manufacturers create the software and packaging, eventually shipping the final product to retail a few days before they are to be released.

All the while, planning for next year's game continues...

Here's some of the worst bugs of all time - Worst Videogame Bugs

And check out this list - Top 8 Bugs That Made the Game Better
...I'm especially a fan of #6 ;)

1 comment:

  1. This comment has been removed by a blog administrator.

    ReplyDelete

Blog Archive