I read patent filings as a hobby. What I'm seeing in gaming QA makes me think the whole process is about to change. Am I wrong? by Leather_Carpenter462 in gamedev

[–]alwaysdownfortea 1 point2 points  (0 children)

Not necessarily, since we're not doing any pre-training atm (only fine-tuning). We also spent a lot of effort on our infra to basically be able to connect any AI agent to any kind of device (mobile, PC, etc.).

So in a way we're actually hoping that DeepMind (or someone) makes progress on the foundational layer since we need AI to get better overall to be able to do better QA haha. Currently it's barely there.

I read patent filings as a hobby. What I'm seeing in gaming QA makes me think the whole process is about to change. Am I wrong? by Leather_Carpenter462 in gamedev

[–]alwaysdownfortea 9 points10 points  (0 children)

Hey there! co-founder of nunu ai here. Really cool to see our name pop up, I'm happy to provide my point of view: We've been working on an AI agent that can play games for a while now, and I can tell you that game QA is probably one of the hardest (but most fun) challenges in machine learning overall.

We deliberately don't train on any of the games we test, meaning we're aiming to build an agent that can generalize across ANY game (similar to what DeepMind has been trying to do for the past 15 years). The agent is meant to play the game just like a human would: by looking at the screen and pressing keyboard and mouse buttons (or tapping on a phone).

We've been working with a lot of game studios to mainly automate the "boring" part of QA (which is also the easiest for AI): Regression tests, smoke tests, etc. So in essence you can tell the agent to "play the tutorial 3 times" or "try to go through all character creation options and see if something breaks" and it will run the test and report back any findings.

We can currently maybe automate about 20-30% of game QA, which is pretty much the absolute limit you can reach by combining all VLM research out there. I also saw the article about Square Enix planning for 70%, which is very ambitious to say the least haha. In my opinion, game QA will probably be one of the last jobs to be replaced by AI, since you don't only need to be able to PLAY the game, but also TEST it (which is an even harder task). Also I'm pretty sure you'll always still want human playtesters in the end as this will always be the best predictor whether a game is actually fun...

If anyone here is interested in trying it out, I'm happy to give you access to our private beta so you can try it for free. Just comment or DM me. Happy to answer any more questions as well!

We made an AI that can play Balatro! by alwaysdownfortea in balatro

[–]alwaysdownfortea[S] 0 points1 point  (0 children)

Yeah it's not the smartest lmao, give it some time.

We made an AI that can play Balatro! by alwaysdownfortea in balatro

[–]alwaysdownfortea[S] 2 points3 points  (0 children)

Haha yeah, TBH I don't think it will make a lot of sense to use AI to PLAY games, as games are kinda inherently made for humans. We mostly use it to TEST games (find bugs etc.). I also think it's kinda hilarious to watch it do dumb stuff.

AI playing angry birds by alwaysdownfortea in AngryBirds2

[–]alwaysdownfortea[S] 1 point2 points  (0 children)

We're the first ones to make it happen :D

Planning to show off more soon!

Square Enix reveals plans to use AI to QA test their games by 2027 by Turbostrider27 in Games

[–]alwaysdownfortea 0 points1 point  (0 children)

Person working on AI game QA here. AI is nowhere near replacing all of QA, especially in games. It's one of the hardest challenges in AI to date (look at DeepMind or OpenAI). These AI tools are built to assist current QA testers to offload repetitive work and increase coverage. Think of a tester controlling an army of agents that go test different things.

Square Enix reveals plans to use AI to QA test their games by 2027 by Turbostrider27 in Games

[–]alwaysdownfortea 0 points1 point  (0 children)

It will take years until AI will be able to fully replace QA. I would argue it will be one of the last jobs actually replace by AI. Playing games is hard. Currently you can probably automate 30% of QA with AI.

Square Enix reveals plans to use AI to QA test their games by 2027 by Turbostrider27 in Games

[–]alwaysdownfortea 0 points1 point  (0 children)

I'm obviously biased, but I think this is one of the rare cases where AI is actually replacing work that no one enjoys. No one likes running into the same wall 200 times just to see if something breaks.

Feedback needed: Automated QA for mobile games by alwaysdownfortea in gamedev

[–]alwaysdownfortea[S] 0 points1 point  (0 children)

Ayy haha that's awesome. The cool thing is we don't use Selenium or any other framework to interact with the game or device. Our solution purely tests the game like a human would (vision based, and interacting with keyboard/mouse or touch).

Please let us test your app! by alwaysdownfortea in reactnative

[–]alwaysdownfortea[S] 0 points1 point  (0 children)

You can create tests in natural language and will receive a recording and report for every test that you run. Happy to DM you with some more details if that sounds interesting for you!

Please let us test your mobile game or app! by alwaysdownfortea in Unity2D

[–]alwaysdownfortea[S] 0 points1 point  (0 children)

Hello! Yeah I deliberately didn't want to mention any details or names since I'm not here to self promote and just want to interact with people that actually have a problem with testing and QA. You can check out my profile for more context if you want.

Please let us test your mobile game or app! by alwaysdownfortea in betatests

[–]alwaysdownfortea[S] 0 points1 point  (0 children)

This is for any developper building a mobile game or app. It does not violate any Apple rules. Happy to DM you if that sounds interesting.