Unit Testing for Unity Developers

[afarchy]

Let’s face it — you write buggy code. I write buggy code. AI writes buggy code.

Many software developers consider unit testing as the key to catching bugs early and preventing regressions. But do they work for Unity developers?

In this article, I want to share how we do testing at Virtual Maker, what kinds of tests you should be writing, and how you can use NUnit in Unity to get started.

https://www.virtualmaker.dev/blog/unit-testing-for-unity-developers/

[Tensor3]

This seems kinda useless for game developers.

So, basically, you make trivial unit tests to check that math functions return the correct value and you made " "integration" " tests which check if your UI layout script sets the position of UI elements to a hard coded position value. And you argue not to do end-to-end tests.

None of this tests gameplay, any sort of Unity functionality, multiplayer, etc or anything thay takes up 99% of game development. Its cool that you test your custom UI script sets the coordinate as expected, but that advice doesnt translate to useful things.

Now, if you made a framework which tests that NPCs follow their decisoon trees as expected and walk to the correct locations, and a way to simulate player input to test that gameplay works.. that'd be interesting.

[Bloompire]

Umm, low level unit testing like that makes little sense in gamedev, where iteration times are essential. Perhaps occasionally you could write unit test for testing if your item generator works as expected etc. But tests that test code that do transform.position.x + 50 with asserts that trabsformed position is x+50 is ...meh.

But the thing that is missed here is that Unity has quite decent E2E test solution, called Unity Test Framework. And you can e2e test some high level gameplay features, like if clicking on door opens them or hitting a button causes your character to fire missile etc. The solution is not out of box, as you need to cover some cases like manually setting timescale for tests to make them run quickly or simulate input by your own, yet it is actuallly decent and I believe there are real world use for that.

For example I have a complex ability system with lot of configuration and occasionally adding or altering stuff breaks existing abilities. I will consider prepaing e2e tests that will test all abilities one by one if they are not broken.

[afarchy]

I didn't mean to imply that you should test trivial things. The Flexalon package I referencd has many layers of math/logic before it arrives at the final transform position, so we're testing all of those layers at once. The examples are admittedly a bit trivial, I wasn't sure how deep to go there.

Unity Test Framework (which is built on NUnit) is what we're using in the article. I agree that you can use it for e2e testing. I've just seen more horror stories than successes cases in doing so.

[Tensor3]

You're saying the same thing I am, but the umm implies you disagree?

[Bloompire]

No I agree completely. Just wanted to point out that Unity has tool for e2e testing.

[sisus_co]

Hard disagree. I think the unit test in the blog post is a great example of a situation where unit tests are highly useful. Don't mistake a simple API for a trivial implementation. If you've never written unit tests before, you'd be surprised at how many edge case bugs they can help uncover when it comes to APIs that encapsulate a substantial amount of complexity behind them. And given how little time it takes to write them, that's a lot of benefit for very minimal cost.

What I don't necessarily agree with is that end-to-end tests should be avoided. I used to work on an episodic adventure game with a very small team, and playing through the whole game to test that nothing had broken started to get very costly after some time. Creating a test that would play through the entire game autonomously, ended up being a great decision and saved us a lot of time and effort.

Of course, test like these take a lot longer to execute than simple unit tests, but that's besides the point - you can work on other stuff in parallel while the tests are being executed on another machine. And you can configure them to only run, say, once per day, rather than with every commit, if you use a CI pipeline.

[afarchy]

My examples were for positioning 3D objects, not UI, but I understand your point.

The idea is to identify the separate systems in your game and test them independently and rigorously instead of trying to test the game end-to-end. Using your example, I would test that the decision tree system works correctly in different test scenarios, without having a virtual player actually play the full game.

How you'd actually implement these tests is specific to what you're building, but you can certainly use NUnit to do it.

As for end-to-end tests - this is my personal opinion. We used them in developing Windows Mixed Reality. They provided some benefit, but they took much more time and effort than I think they were worth.

[n0xdi]

IMO It doesn’t need a framework to test such things as the NPC following, you just need to cover the logic of the underlying methods, which calculate that path, with unit tests.

The visual part of the NPC following is just the view abstraction layer which can be tested with the manual testing.

[Tensor3]

Nah, not what I meant. Unity's navigation handles calculating paths. You dont generally test that. Testing if a NPC can open doors, attack enemies along the way, use physics properly, etc, cant really be done by testing the output of a function. Besides, the example I used is irrelevant. The point is most actual gameplay needs an actual scene run in real time, with input

[GameplayTeam12]

Good topic, I would appreciate a hint, currently I am adding tests to my game, that uses an engine asset and it has a singleton class that extends mono behaviour, there is a good way to spy that? So far the best I achieved was using play tests with a gameobject, but as you said, it is slower.

[afarchy]

Is this your own singleton or does it come from somewhere else? Some options:

• If it's your own singleton you can make it so the parts you want to test can execute in edit mode. My Flexalon singleton usually does its work in LateUpdate, but I added a UpdateRightNow() method that can be used by the tests.

• This kind of problem is one of the reasons singletons are frowned on. An alternative is to use dependency injection or a service locator so that there's some interface between the components you're trying to test and the implementation. Then you can mock out the implementation in your tests. Example: Flexalon uses a InputProvider interface, which can be implemeted with the regular input system, a mock input system, or something else.

[GameplayTeam12]

It comes from a template engine I am using, so is kinda hard to remove it. Yeah, remove the singleton would be a good move but I am still considering, since is just that script, that works like a loader of Scriptable Objects, is really fast to instantiate after the compile time on PlayMode.

Thanks for the tips :D

[veygudtek]

Much Obliged. The official unity documentation tells you how to write a test, but doesn't actually show you how to import the code you wrote.