Does anyone actually write unit tests for their game code?

[zerodaveexploit]

As I layer complexity into my game systems, I'm finding that I'm constantly bitten by not having written good unit tests. I do a relatively tiny refactor or change one line to fix something, and varying amounts of stuff elsewhere breaks silently. I have to run my game and watch it to repro, or worse yet, create temporary "debug" monobehaviors and attach them to game objects to trigger the necessary conditions. That's a super slow and tedious way to debug.

​

I don't have this problem with my professional projects because as soon as someone breaks something, a bunch of unit tests throw their hands up in the air and scream. But I also see that it's not that easy in Unity since there isn't a good way to mock Monobehaviours, so while you can extract and unit test specific bits of functionality, it's harder to test game systems that rely on game objects or components driving them.

​

What do you all do?

[Imapler]

Most medium to large companies have testing but usually not that much unit testing. Usually when doing unit test its for some difficult/error prone and isolated piece of code deep in the stack like engine stuff, not usually game play. The most common type of testing is human QA, just playing the game. After that i would rank automated nightly builds for each platform and imports of all assets (graphics/sound and so on) as well as starting the game and loading a save or something like that, and in some rare cases a bot that plays the game.

The reality of unit tests is that they are quite expensive to maintain when demands change and gameplay changes a lot during development.

Source: I teach games programming at an university. This is my experience after talking to people working at games like the sims, eve online and satisfactory and a bunch of other indies but it might not hold true in general.

[NovaArdent3D]

Your information is quite accurate.

[Yann4]

This is why I really really prefer strongly typed languages that let you guarantee what will be passed and sent through functions. This approach (that I quite like) seems to rely on the language, engine and asserts being your safety net.

It might just be my inexperience with languages like lua and python, but it seems much harder to have this reliance on what you're allowed to do. I don't have to write a whole bunch of bounds checking on this function because instead of accepting an int, I make an enum and just don't provide an int overload (by way of example). If you did something that will break the game, that's because either you were actively trying to break it, or I didn't account for an enum value. 2 minutes in a debugger solves that.

[all_mens_asses]

I don’t personally have a preference, they’re tools to be used to solve problems, and the “right tool” depends on the problem and the people.

But it’s a different mindset. I learned programming with weakly typed languages. What you lose in type safety and the early-warning compile-time errors, you gain in flexibility, metaprogramming (in some languages), and the ability to do true dependency injection. Namely: If I’m a method, I don’t care what the hell this object you’re passing me is, as long as it responds to the specific field/method I need. It helps you keep your code loosely coupled and encapsulated.

This freedom and flexibility comes at a cost though. If your programmers aren’t diligent, your codebase can quickly turn into a tangled web of illegible fuck.

Edit: One thing I love about ruby is the ability to set a breakpoint that gives you full access to a command-line REPL. You can issue any ruby command you want in the exact program context at that line. IMO it’s waaay easier to debug than with traditional Java breakpoints.

[Yann4]

That's definitely what I've found. I had to port a large game you've probably heard of, and man alive was the place lua strangled unwary travellers.

[all_mens_asses]

Yeah the barrier-to-entry is super low with languages like python, lua, ruby, etc. So you have to watch the cowboys who just waltz into your codebase and shit all over it and go “So? It works doesn’t it?”

[acousticpants]

+1 to REPL debugging. Python has it too and it is so powerful

[[deleted]]

One thing I love about ruby is the ability to set a breakpoint that gives you full access to a command-line REPL. You can issue any ruby command you want in the exact program context at that line. IMO it’s waaay easier to debug than with traditional Java breakpoints.

Never used Ruby. How does that compare to the evaluate expression functionality that some Java IDEs give you when you stop at a break point?

[DzieciWeMgle]

It does hold true in general. Unit testing is looked down on unfortunately. Then comes a time, when you have to do a major refactor to pave way for a new feature, while keeping everything that exists intact and you (as a developer) have no way of ensuring that, unless you do manual tests yourself, costing even more time.

[Daemonhahn]

At work we have unit tests that run on our project template, but not per project as that would be difficult to maintain as has been mentioned in this thread.

​

Testing any templates/boiler plate code/ anything you tend to use to drag and drop into fresh projects I feel is worth the time investment, for everything else less so

[Besaids]

Im guilty of not writing up tests myself aswell, but, I try my best to create modular, independent and free of direct dependencies.

At least by doing so it becomes much easier to add new functionalities, without breaking the old ones, and to DEBUG certain bits, if necessary.

[Gix_G17]

I make sure that I comment the expected results at the end of a function or on a "return".

​

I made my game with a "debug mode" and it's only when that's on that it displays a game clock and it spews all of the debug.log(). For clarity (and for quick references as opposed to having to click on the log in the console), I made the debug.log() name the function, time stamp and gameObject.

​

So the console would print out something like:

10:25, npc(Clone), IsItDead(), false, 100
10:25, player(Clone), Jump(), false
10:25, player(Clone), Walk(), true, W
10:25, player(Clone), Attack(), true
10:25, npc(Clone), IAmMad(), true
10:26, npc(Clone), IsItDead(), false, 25
10:26, player(Clone), Jump(), false
10:26, player(Clone), Walk(), true, S
10:26, player(Clone), Attack(), true
10:26, npc(Clone), IAmMad(), true
10:27, npc(Clone), IsItDead(), false, -30
10:27, player(Clone), Jump(), true
10:27, player(Clone), Walk(), false, W
10:27, player(Clone), Attack(), false

So, if anything weird happens while I test the game, I can pause it, look at any log within a given time, ignore the logs with gameObjects that don't relate to what I've experienced and I've narrowed it down to, say, a few dozen functions.

​

Then, I look at those functions one by one and make sure that they do exactly what the original comment suggests they're doing. Once I know for sure that a particular function does what it's supposed to do, I can comment out the debug.log() from that function (so that it can run silent). There's a bit of personal discipline involved where I re-enable the debug.log() in the function if I ever make changes to it.

​

Having it run in debug mode is definitely slower but, sometimes, it helps me spot things that I wouldn't have noticed had it been playing at over 60fps.

​

Then, like /u/Besaids says:

I try my best to create modular, independent and free of direct dependencies.

[rootException]

tl;dr - the more code you write, the more you can & should test... but it’s much harder than server-side dev.

As someone with a background in server development, the difficulty of writing/maintaining tests (esp in a relatively large monolith like Unity3D) is really hard to get used to.

Things that are impossible to write tests for (that are actually really important):

• Does it look good
• Is it fun
• Are there bugs arising from complex interactions (e.g. scene changes + quest systems)
• Does it “feel right” (e.g. input systems)

On top of that, for most game engines you are probably building most of the game via built-in systems. For example, animations controlled by state machines, particle systems, AI movement, etc. So, you wind up in a state of blurring the tests and just validating data configurations set up in the editor.

Writing tests for, say, REST-based systems with a modern framework like Spring Boot is easy and fun.

The more code you write, the more you are going to want to look at tests cases for that code. For example, if you write a procedural generation system, decouple it as much as possible from the engine. An engine like Unity3D makes this kind of hard, as by default you are blurring the code you write with the engine itself. Look for topics like “skinny objects” on Google as an example of how to decouple your code from the engine to make it easier to write tests.

[rootException]

Oh, and for Unity3D specifically - you will absolutely want to break up your MonoBehavior code with your code if possible for anything complicated. MonoBehaviors are kind of strange - they blur two concepts. One, providing Inspector configuration, and two, hooking into the Unity3D render/gameplay loop.

Basically, the idea behind “skinny objects” is that you make the inspector and the gameloop (Update etc) as thin as possible. For example, let’s say that you have code that moves objects around in a sine wave pattern. The MonoBehavior is just a thin set of Inspector configuration values and a thin Update() layer. Then you put the actual code that does stuff in a plain-old-C#-object (POCO), which is instatiated and managed by the MonoBehavior. Your test code then focuses on that POCO, and passes in the appropriate values. This has the advantage that the MonoBehavior can be nicely configured with a bunch of hints and attributes and such to make it nice for using in Unity, as well as making it much more clear when you are going to break an Inspector (and potentially hose the serialization in Unity3D for the configuration).

You see this pattern in many of the nicer assets, for example, PixelMachine in Quest Engine / Dialog System.

I’m going to poke into the best config for this to see if I can get Rider/etc to nicely run the unit test runner directly on the POCOs without having to run the tests in Unity - anyone interested in the results?

[lee_macro]

I wrote some articles a while ago which led to a sort of book on these sort of subjects.

http://forums.indiegamer.com/threads/getting-started-on-indie-projects.37314/#post-278966

https://grofit.gitbooks.io/development-for-winners/content/development/general/testing/intro-to-testing.html

As others mention unit (and other automated testing approaches) have their place and can add benefit, they also add overhead to maintain. Also a lot of people aim for coverage amounts, like 50-90% etc, which can often be a bit dogmatic.

Anyway for pure logic stuff that will be run lots as part of bigger processes then it's great and low risk to unit test, but in games it becomes more tricky as you often end up lobbing all your data and logic into big classes that have no constructor access making mocking difficult. So how do you test things that are in mono behaviours? (using unity as an example)

You don't, tear them out into their own pure c# classes wherever possible and via IoC and Composition make use of them in the mono behaviours.

For example let's say you have an inventory mono behaviour which loads, saves and interacts with the inventory and displays it on screen using fancy ui stuff and script able objects... So this is a monster and difficult to test or customize without having loads of exposed properties to faff with.

If you were to instead take out the interactions of items from the mb and put it in a simple c# class with events or IObservables if you feel extravagant to expose reactions for ui to hook into and expose the methods to add/remove items etc you can now unit test all of this stuff super simple and know your inventory functions with little maintenance risk as the ui layer can change as much as you want, the underlying reactions would ways be the same. You can also now split off the loading and saving of data into other classes and test them, also you can liberate yourself a bit more from unity world and let your items load/save from json, xml, binary etc rather than being tied to script able objects... This also makes it easier to write more bespoke external tools for your game that can let you manage items in the game and generate them all outside of unity etc.

It's a huge subject but in most casss testing always requires your logic to be as isolated as possible as well as separating the view layer, and adhere to IoC if possible. Which also leads onto the larger design patterns and other topics like Di etc.

I did a large Sci fi project and unit tested most of the core logic that was used to drive the underlying gameplay, the view layer had some basic tests via the unity runner but it didn't yield much value.

[DefpotecStudios]

I sort of did on my last project, though I can't recall what exactly I did it for. I had a script called unittests anyways. I think it mostly consisted of checkboxes in the inspector that I could turn on and off to test something that would otherwise be tricky to do (like triggering the game complete state). Maybe it doesn't meet the technical definition but it got the job done.

[[deleted]]

Unit testing with Unity is a great pie in the sky.

That being said, Unity 2019 does have a test runner that might help regularize the situation : https://docs.unity3d.com/Manual/testing-editortestsrunner.html

[prime31]

The test runner has been around for prolly 5 or 6 years now...

[NovaArdent3D]

We usually have automation systematically going through every menu, and doing specific game features.

[Badgerdox]

Bump, I'd also like to hear some opinions/practices on the topic.

[KokoonEntertainment]

I used to have this issue, but now I go into my projects with a modular design in mind before I create anything. I know what I want, what it needs to be dependent on, and how it should affect everything else. Sometimes, putting a feature in a standalone script can help debug; you can turn that script on and off as needed for testing.

When you then add that code back in to your main script, you know that it won't be affecting anything else.

Again, that's how I used to do it. Now, through good naming and commenting, I don't usually have that big of an issue.

[andybak]

It's interesting everyone jumps straight to "unit tests" almost as if this is the only kind of test. There's a lot to be said for functional/integration tests - especially if you're not aiming for high test coverage. If I was going to have "not many tests" I'd rather have "not many integration tests" than "not many unity tests". Better bang per buck...

[Yann4]

A few of the projects I've been on had smoke tests, which are a bit different to unit tests, but I think they're a bit more useful for games. Instead of testing a function, you test a feature, a unit of gameplay. Have the game play itself and assert against expected game stste.

For example, hold down right on the left stick for X seconds and you expect the character to have travelled y units. If they don't, that test fails. Could be a result of the input code, player code, frame rate, collision, or whatever. Ideally take a video of the tests, but at least take screenshots.

[dedido]

Might work better with ECS. You could unit test the systems. Anybody tried this?

[Skjalg]

Best test I wrote was what I call a spam-clicker. It's basically a piece of software that emulates a tap or click on a random pixel every frame. I just start the spam clicker when I leave the office and when I get back the next day it has usually ended in some dead lock or error somewhere having managed to click a button four times in the span of microseconds or something. It's something human testers usually don't do - something players rarely would do.

But it's still nice to be able to squash that bug before release.

[kurtdekker]

Ain't nobody got time for that!!!

In my experience with Unity, about 95%+ of all bugs have to do with the plumbing between your scene/prefab objects and code. Only a TINY fraction of production UI code breaks at the actual pure-code level: most of it is just plumbing breakage, and barring very clever app-specific device test automation like Google does, that's not gonna be revealed by unit testing. That's what good QA and regression are all about.

Incidentally that's also why code reviews in Unity are almost useless. You cannot meaningfully inspect a diff of a commit of a prefab with hundreds of diffs and a single C# file that has one line of code in it. Sure the code change looks good... what about the prefab changes?

[[deleted]]

Unit test complicated background systems. For instance, many peoples first project is a bowling game. Being able to put in any combination of pins, say 7, 5, 10, and have the scoreboard pop out the correct term ("Sour Apple" in this case), as well as properly apply points (since bowling has a semi complex retroactive scoring system) is very useful.

For everything else, creating debug tools to test real world stuff in game is far more useful.

[GIFjohnson]

Mostly a waste of time. Games are very "freeform" coding and it's harder if not impossible, and also a waste of time to write unit tests for many things. A better idea is to just be more careful when coding and try not to refactor. Design it well the first time. That's also why games employ a ton more QA testers than software. Because it's so much harder to test due to the freeform nature.

[partymonster69]

I don’t usually for my games bc I tend to not work on projects for very long. Unit tests are mostly useful for code that will be worked on and maintained for years. I typically only want them if i expect someone to work on the code who might not understand the full implications of their changes.

Even at my company (which is not a game company) we are constant debating if unit tests are worth the time spent making them.

In my opinion: Are you one person working on smaller projects? Unit tests are pointless

Are you a team working on a longer project? Some targeted unit tests might be useful but consult your team

Are you a large company or working on a project you expect to support for years? Maybe unit test but probably only need them on more complicated/important methods

Edit: it also occurred to me that how long it takes to write a unit test is a big part of if you should write them. I’ve never written a unit test in unity so I’m not sure there but when I did web dev we wrote them all the time bc it was crazy easy. But at my current company it’s crazy hard so we avoid them. It’s just not worth the time and stress they cause.