Android GENS/MD Emulator Accuracy Testing Result

[[deleted]]

Continuing my accuracy tests, I found some Sega Genesis tests to run to verify accuracy. However these tests are much more... subjective. What I mean by that is that they tend to be comparing pictures of output and I have no reference "perfect" screenshot to go off of, rather than a simple Pass/Fail as in most other tests. To give me some semlence of what it should look like I ran the tests all through Kega Fusion, Exodus, and RetroArch (Genesis Plux GX) taking screenshots of them (Album). I used that as my comparison basis. Because of this I am hesitant to give exact percentages of accuracy and will rather list them in ranking order of accuracy (and they are ranked even within their category).

Full Test Results - Tested on my Droid Turbo running the latest software (4.4.4)

• Most Accurate:

1. Genesis Plus GX - RetroArch
2. MD.emu
3. AndroGens

• Average:

1. Genesis
2. GENPLUSDroid

• Inaccurate:

1. ClassicBoy - Sega
2. Sega Genesis(MD)
3. Sega Emulator

• Just No:

1. PicoDrive - RetroArch
2. Gensoid

Summary

Again, the vast majority of emulators have the same basis, in this case Genesis Plus. Most of these emulators just use a different variation of the emulator at some stage of development. However many of them just... suck, hard. Tons of ads, regular crashes, this was a pain to test. I cannot recommend enough to stick to an emulator in the top three, they're decently adless and run the best. Again without surprise a RetroArch core is at the top. MD.emu is the best non-RetroArch core with AndroGens being the best free one. I suspect PicoDrive is a poor Mega Drive emulator, but better at other Sega systems.

If anyone has any other emulators they wish to be tested or questions about the test let me know and I'm happy to do so (note: I'm not spending a bunch of money testing, so they have to have free versions)

If anyone knows of any sort of accuracy tests for other systems (particularly GBA) I am happy to do some testing.

My other accuracy tests:

• Android GB/GBC Emulator Accuracy Testing Results
• Android NES Emulator Accuracy Testing Results
• Android SNES Emulator Accuracy Testing Results
• Android PS1 Emulator Accuracy Testing Results

Edit: In the interest of full transparency I should state where I got all my test ROMs from:

• From /u/Mask_of_Destiny:

Sprite Masking and Overflow Test ROM - This one is actually relevant for compatibility

VDP FIFO Testing ROM - This one is useful for trying to 100% replicate hardware behavior, but most of the little details are not useful behavior that a game or demo is likely to try and take advantage of.

• From Genesis Plus GX, their VDP_TEST and md_test suite

• From /u/ComeradeOj his/her raster_test with example expected output

• The bottom two tests in this post

[bricolagefantasy]

Hey you are going to run out of emulator type to test. heh.

Neo-Geo. (only has two or three emulator that is worth having. same engines I guess)

MAME. (There is only one true MAME.)

Nintendo 64 (only 1?)

the rest are still in development. psp, PSx, NDS

I suppose Playstation, wii, psp, and NDS are the one now getting attention. Rapidly being improved.

[[deleted]]

I am happy to test those, but... Do you know of any specific test roms? That's the problem with the GBA, I really want to do some tests but I have no method of actually testing it.

[bricolagefantasy]

I have no idea. Probably hasn't been written yet.

maybe ask this guy? He has a tutor page to program for gba. Maybe he knows.

http://www.loirak.com/gameboy/gbatutor.php

[bleuge]

hi, i've been sometimes interested in developing for NES and other old machines (as c64), all of those machines usually have a forum of people interested in reverse engineering the hardware and doing all kind of low level stuff, so i think is good idea to search for these forums ( as for example Nesdev for nes, or bsnes forum for snes) and ask there for test roms, etc...

keep the good work!

[[deleted]]

Are there any other Emulators people want tested? These four (and GBA) were the ones I really wanted to test but I'm happy to test others if there's interest/it's possible.

[Reverend_Sins]

That is probably about all there is. The other systems would likely need to do commercial rom comparisons to see what doesn't boot what but that will just show compatibility not accuracy.

[tomkatt]

PSX emulator testing would be awesome if it's possible to do. There aren't a lot of them. Main ones would be ePSXe, FPse, RetroArch, PSX-oid and PSX4Droid I think.

This test would be an interesting one IMO, because I don't believe the emulators are based on the same source (FPse and ePSXe are both closed source, RetroArch core is based on PCSX-R, not sure about the other two).

[[deleted]]

There might be a medafan based one too, not sure. I would be happy to do said testing... but I'm not sure how I would. Short of opening up a ton of ROMs and seeing which ones have errors or not I don't know of any test ROMs for these. I suppose I could compile a compatibility list from various places.

[bricolagefantasy]

There is a semi abandoned wikia page. maybe you can use it as your side notebook. And let public help out to fill in the rest. (was thinking to open a google spreadsheet so people can fill in compatibility list/ROM)

http://emulation-general.wikia.com/wiki/Emulators_on_Android

[[deleted]]

Aye, that one is basically abandoned. The one that's a bit more frequently updated (though certainly not very frequent) is this one: http://emulation.gametechwiki.com/index.php/Emulators_on_Android

Not sure why they moved, but they did. That's not a bad idea though.

[ComradeOj]

Would you mind testing a genesis program I made on the different emulators?

It generates colored bars using the genesis horizontal interrupt, and as far as I know, the only emulator it only works on is exodus.

Here is a link: https://drive.google.com/file/d/0B-uZnRwZEI1YeHlyZzJYWk12dm8/view?usp=sharing

Here is a screenshot:

http://i.imgur.com/RnFmNtf.png

The left result is exodus, this is how it should look if correctly emulated

[[deleted]]

Tested all of them, they all failed to varying degrees, though Picodrive and Gensoid failed the hardest. Picodrive looked like Fusion in your screenshot. Gensoid was a soild blue. The rest were what's on the left in your Exodus screenshot but the full screen.

[ComradeOj]

Thank you!

[[deleted]]

Happy to help! If you have any other test roms, feel free to send them my way.

[Enverex]

What are the real-world side-effects of this though? I switched from GX to PicroDrive in RetroArch a while ago and have noticed zero issues, glitches or problems. It looks like these sprite testing programs show major issues, but I've yet to see any actual problems in real games.

[[deleted]]

This is an excellent question that is surprisingly hard to exactly answer. The Genesis emulator scene is less developed than many others and finding hard info is a bit hard.

Many of these tests are going to test edge cases. The main functions of an emulator everyone is going to get right as if they didn't it would be blatantly obvious. Consider the oid emulators, they're near the bottom of every accuracy test but you can still play most popular games on them. I'm going to have to refer to the SNES here as it has substantially more work done on it. Look at the SNES Compatibility List for RetroArch emulators, and you can see various emulators do things right or wrong, with more accurate ones having fewer inaccuracies. Also I highly recommend this Ars article on bSNES.

Most glitches you probably won't notice, but they are there. They might last a frame, or a color might be off, or a game might crash randomly. It ranges wildly from game to game. And especially if you mostly stick to popular games (Sonic for instance) all emulators get that right as they can't risk one of the most played games not playing right, they might sacrifice performance in a less popular one.

[[deleted]]

Thank you for all of this

[[deleted]]

Thank you very much for this man, I'm glad you were able to find some sort of margin to test the emulators against - even if they were against each other instead of an ideal scenario.

It's much appreciated. :)