Ask a QA Tester

[Ad_Astra_]

Hey guys,

I figured that since I've been testing for a new company over the past year that I might be able to help some folks either looking to know more about the industry from a production standpoint or those looking to become testers themselves.

Obviously I won't be able to answer every question thrown at me, but I promise to do my best.

Ask away!

 

[nesbitto]

Do you only get to test one level repeatedly, or do you get to experience whole games at a time?

Have you any knowledge of how they design and test multi-player maps (any genre)? Do they simply create something that looks good and then play test it to reveal flaws?

 

[Ad_Astra_]

Do you only get to test one level repeatedly, or do you get to experience whole games at a time?

Have you any knowledge of how they design and test multi-player maps (any genre)? Do they simply create something that looks good and then play test it to reveal flaws?

Well we test the game based on what is capable of being tested on the current build.

The devs send us builds whenever they have changed anything. So the first builds they send us tend to be VERY rough and unfinished. Lots of placeholder textures, no cutscenes, sometimes no models for certain characters. As a tester, you do everything you can to break the game. The devs will HATE you because they think what they have created is good, but you are there to show them the chinks in the armor.

Testing is VERY repetitious. Every time you get a new build, you have to wipe your save data and start all over. It can be frustrating. Especially if there are progression blockers that completely stop gameplay. Crashes are common, especially on certain engines. Expect a lot of reboots.

As for multiplayer maps, we have entire teams dedicated to that to simulate poor internet connectivity for desyncs, ethernet pulls, and server issues. However, we can only get a limited amount of data among a team of maybe 30 people (if we are lucky). That's why betas and alphas are so important. Servers need that stress to really point out the flaws that we simply can't find.

 

[Mutant1988]

Do you only get to test one level repeatedly, or do you get to experience whole games at a time?

Have you any knowledge of how they design and test multi-player maps (any genre)? Do they simply create something that looks good and then play test it to reveal flaws?

Any good level designer would design the layout out the map as it effects gameplay first and add the aesthetics afterwards.

That's what makes level designers different from 3D modellers.

Of course, there's always some kind of aesthetic consideration running through it that influence both how it plays and how it will ultimately look.

But bottom line is, it doesn't matter if it looks good, if it also doesn't play good.

Source: Having play tested two Half Life 2 mods and talking with the level designers. One which I know 100% is an industry professional. This guy - http://www.wesleytack.com/

 

[nesbitto]

Spoiler

Do you only get to test one level repeatedly, or do you get to experience whole games at a time?

Have you any knowledge of how they design and test multi-player maps (any genre)? Do they simply create something that looks good and then play test it to reveal flaws?

Any good level designer would design the layout out the map as it effects gameplay first and add the aesthetics afterwards.

That's what makes level designers different from 3D modellers.

Of course, there's always some kind of aesthetic consideration running through it that influence both how it plays and how it will ultimately look.

But bottom line is, it doesn't matter if it looks good, if it also doesn't play good.

Source: Having play tested two Half Life 2 mods and talking with the level designers. One which I know 100% is an industry professional. This guy - http://www.wesleytack.com/

Sorry, my question was poorly worded and easily misunderstood. I meant do they create something that looks like it will play well, as opposed to being aesthetically pleasing. And how do you learn about how a map will play without years of experience?

 

[Ad_Astra_]

Sorry, my question was poorly worded and easily misunderstood. I meant do they create something that looks like it will play well, as opposed to being aesthetically pleasing. And how do you learn about how a map will play without years of experience?

Most maps start out with basic geometry and whatever textures chosen by the designers. All assets are added in separately (hence why you will see a lot of things clipping into the ground). The first goal of any map is to lay out the field of play and contain the users within it with physics (invisible or represented by geometry or assets). Functionality always triumphs graphics in terms of importance.

Once a basic layout has been placed, testers can go in and try to break the physics and geometry. If there are bots, do they fall through the map? Can the user escape the map? Can they seen unintended areas? You take what controls you are given and do everything with them that you can within the environment. It helps to be knowledgeable about design and gaming in general, but you don't need years of experience to walk a camera into every wall or try to jump off cliffs, etc.

 

[Hectix777]

double post please ignore/delete

 

[Hectix777]

Most maps start out with basic geometry and whatever textures chosen by the designers. All assets are added in separately (hence why you will see a lot of things clipping into the ground). The first goal of any map is to lay out the field of play and contain the users within it with physics (invisible or represented by geometry or assets). Functionality always triumphs graphics in terms of importance.

Once a basic layout has been placed, testers can go in and try to break the physics and geometry. If there are bots, do they fall through the map? Can the user escape the map? Can they seen unintended areas? You take what controls you are given and do everything with them that you can within the environment. It helps to be knowledgeable about design and gaming in general, but you don't need years of experience to walk a camera into every wall or try to jump off cliffs, etc.

How exactly did you get to be a QA tester? Do you have to be apart of the company to begin with or is it a connection based thing?

 

[limmers]

Software dev here - not in the games industry, mind. If you're anything like my company, I assume you work reasonably closely with the devs - so it would be interesting to hear your spin on this.

Do you think gamers act unreasonably about bugs and optimization issues in games, given the difficulty and general chaos in developing a complex game?

 

[Ad_Astra_]

How exactly did you get to be a QA tester? Do you have to be apart of the company to begin with or is it a connection based thing?

You have to live near a tester office location for starters. A lot of people think that you can test from your home, but they forget that when dealing with sensitive IP, you can't just let it out into the open like that. You have to either already be living close by or be willing to move out to where they are testing their IP. Mind you, the testing offices will often be a separate division of the company if you work for one of the big publishing studios. I've never been part of a development QA team, but I imagine the process is somewhat similar.

There are the big offices like EA in Washington State, 2K has offices in CA and NV, ETC. There are also smaller non-publisher related groups like Babel (that is localization specific, but it counts) that you can join. When I joined, I was answering a Craigslist ad for testers for a non-affiliated tester group. From that first job, I had enough experience to get picked up by one of the bigger companies. However, the people I work with now come from various backgrounds that had little to nothing to do with the gaming industry (forklift operators, marines, baristas, and journalists, to name a few). Connections and experience help, but they are not necessary.

 

[Ad_Astra_]

Software dev here - not in the games industry, mind. If you're anything like my company, I assume you work reasonably closely with the devs - so it would be interesting to hear your spin on this.

Do you think gamers act unreasonably about bugs and optimization issues in games, given the difficulty and general chaos in developing a complex game?

I think that it depends on the game itself, but also what exactly they are getting upset over. Functionality bugs, progression blockers, save-deletes...all of those are totally reasonable in my opinion. Those should not happen in a finished project, but the fact of that matter is that both the devs and QA have such limited resources when it comes to testing that it's bound to happen on occasion. We simply cannot fix everything.

There are also many outside issues that gamers don't really take into consideration.

Just this Fall, I received a code to participate in the Evolve Big Alpha test on PS4. However, SONY was switching out their firmwave due to the recent hacking which delayed the alpha by several days. I saw a lot of people getting mad at the devs over this, acting like they should have been able to plan for SONY (a completely independent company) to change everything last minute.

Speaking of alphas and betas: people getting upset over how buggy an alpha is when they are told that it's not an accurate representation of the final product is INCREDIBLY frustrating. It reminds me of when I used to work in a movie theater and we had a GIANT sign above the icee maker telling people to put the lids on first...of course they never did.

I might have gone off on a bit of a tangent, but back to the original point: yes, I would say that gamers over-dramatize any flaws in the code. People love to focus on the negative rather than the positive. This does not excuse poorly made games, but I feel that gamers should be a bit more understanding of the thousands of man hours put into their favorite titles.

 

[Arnoxthe1]

What do you like most about QA Testing?

If anything...

 

[KyuubiNoKitsune-Hime]

How is it that very large obvious errors can sometimes slip through the cracks on large, big budget releases?

How often do developers demand that you run a specific game on a specific PC set up for PC games?

 

[Arnoxthe1]

How is it that very large obvious errors can sometimes slip through the cracks on large, big budget releases?

Usually the answer to that is time. Namely, not enough of it.

 

[Ad_Astra_]

What do you like most about QA Testing?

If anything...

I love my team an awful lot. I've been on 72 hour work weeks for the past 4 months (finally got a break since I switched projects) so it helps to like the people you work with. However, some of the bugs are just AMAZINGLY bad. Some of the placeholders are also cool if you have creative devs. One of our earlier bugs was that all NPCs barked like dogs. Somehow the sound files were replaced.

 

[Ad_Astra_]

How is it that very large obvious errors can sometimes slip through the cracks on large, big budget releases?

How often do developers demand that you run a specific game on a specific PC set up for PC games?

As someone else already answered, time is one of the biggest factors because time is money and the more time we have to spend on a game, the worse our producers feel about it.

It is not the only factor though.

QA is a team of editors. We only point out the flaws in the code. The devs choose what to fix. We can and will often fight them on certain issues, but we don't win every time. And in the case of multiplayer titles, you have to remember that most QA teams are not made of several thousand people. We can't stress the servers the way real users in the real world can. We cannot reproduce EVERY PC configuration either. It's simply impossible.

And numbers DO make a massive difference. Say I find a crash, but I am unable to figure out how to reproduce it. My team of 30 people and I might never see it again...but in the real world of millions, that bug will appear thousands of times.

As for specific PC set ups, we get a min-spec PC set up for any given title. That is the lowest grade PC we can run the game on. Any lower than that and we usually don't bother to test it. However within those min-specs, we will test a large variety of variables to make sure that we can cover as much ground as possible.

 

[KyuubiNoKitsune-Hime]

It really sounds like EA and iD Software could could use a team like yours. I understand that you guys are small teams and those answers were really informative. I'm still stumped on the fact that they only tested RAGE on one machine, and how Assassins Creed Unity had that really funny but annoying missing heads bug on launch.

If you're asked to test a game destined for console releases, specifically exclusive to a console, or not available on PC... Do you generally get a version for the console (or it's test/dev version) or do you get a PC version destined to be ported to it's native system(s)?

 

[Arnoxthe1]

In your experience, have the devs really been laying the hate down on the QA testers?

 

[Ad_Astra_]

It really sounds like EA and iD Software could could use a team like yours. I understand that you guys are small teams and those answers were really informative. I'm still stumped on the fact that they only tested RAGE on one machine, and how Assassins Creed Unity had that really funny but annoying missing heads bug on launch.

If you're asked to test a game destined for console releases, specifically exclusive to a console, or not available on PC... Do you generally get a version for the console (or it's test/dev version) or do you get a PC version destined to be ported to it's native system(s)?

It depends on what the devs think is more important for launch and what can be patched. With AC:U, the missing heads thing was weird, but outside of breaking immersion, it didn't break gameplay. Functionality is always the top tier for us. Find a functionality bug and the team lead starts making a lot of phone calls. Multiply that by 100 when you are a week away from submission.

As for RAGE, I can't say I ever had the chance to even look at it. However, my guess is that it was a more console focused game. Some teams prioritize the consoles over PCs because PCs can be patched easier than console editions. For every project I've been on, the PC version is usually the last to be tended to because most people who have decent PCs also have internet connection and can get a patch. However, lots of console owners don't have internet.

If we get a game that is exclusive, we only test it on the systems required. There is no porting during production. A PC build will be formatted entirely differently from a console build. We will often see problems exclusive to one console or to PC because of how this formatting affects the existing code and how the systems run the engine.

 

[Ad_Astra_]

In your experience, have the devs really been laying the hate down on the QA testers?

Some devs more than others. The more..."artistic" devs like to think of QA that is not in-house as just a bunch of corporate pencil-pushers. I have been cursed out in a database before. However, that usually ends badly for the dev because they are supposed to be professional toward us as we are to them.

Ultimately I think it's a hard place to be in for both the devs and QA. QA is trying to make the game better, but in order to do it, they have to be as honest and open about their criticism as possible. The devs have to smile and take it all while working like crazy with complicated code and heavy deadlines.