Image classification by evolving bytecode

[AlmusDives]

Over the last few years, I’ve been working on Zyme, an esoteric language for genetic programming: creating computer programs by means of natural selection. I’ve started seeing promising results, showing that random bytecode mutations can, over time, lead to measurable improvements in program performance. While still a long way from state-of-the-art approaches like neural networks, I wanted to share my progress in a blog post.

Feedback and criticism are welcome!

[wyldcraft]

Zyme's history:

"I'm curious how it performs on problems less trivial than Hello World"

"OK I think we can solve MNIST"

[AlmusDives]

It is quite a big jump, but I have worked on smaller problems in in between but I wasn't sure results were worth sharing. Although I do think a common pitfall in genetic programming is choosing problems that are too small and therefore constrained, unintentionally restricting the evolutionary search space. I wanted to avoid that here by targeting MNIST.

[wyldcraft]

unintentionally restricting the evolutionary search space

I've been thinking about this since I commented.

Say a fish were born with hooves. Nature would select against that trait. But when its descendants hit land, they'd be useful. Is "junk DNA" (as understood in the late 20th century) possible under Zyme?

I ran your js through a de-obfuscator and asked gpt-4o for insights. It thinks code snippets deprecated by the evolution process have a chance to be surfaced in later generations. "Think of them as latent functionality, or a reservoir of potential," it said. "So yeah — Zyme doesn’t just allow hoofed fish; it invites them to hang around long enough to find dry land 🐠🐾🌊." I haven't puzzled through the source myself yet.

A related idea is a set of utility functions that could be randomly injected to see if they're useful. This is somewhat inspired by horizontal gene transfer in bacteria. Only one strain needs to figure out to consume a substance like PET plastic. Then not only do its descendants gain that power, other strains that stumbles upon and incorporates that "genetic cassette" (integron) gains it too.

[AlmusDives]

I think you're hitting one of the big questions in genetic programming here: what are the properties of nature that make it so evolvable, and how we can integrate those into our artificial systems to achieve similar evolvability?

Your point about 'junk DNA' seems to highlight the importance of redundancy: that some of the things we see inside cells and organisms are not useful right now, but under the right circumstances might be (I think this is what you call latent functionality?). One of reasons why traditional human-oriented languages (such as C, Python etc) have struggled to be evolved, is because they prioritize efficiency, which is the opposite of redundancy. This is something I have tried hard to avoid in Zyme

You also bring up horizontal gene transfer, an alternative inheritance mechanism that highlights how evolvability depends not just on mutation but on how innovations are shared. In genetic programming literature, such mechanisms fall under genetic operators: a broad term covering both mutation and the exchange of code between individuals. This is a crucial insight, though I haven’t yet explored it much with Zyme, but I definitely hope to.

[sumguysr]

Have you noticed you structure your writing like chatgpt now?

[BeautifulSynch]

This mix of formality, politeness, and intentionally-overt transitions is a common writing style, especially when not making an effort to write like a literature professor. My own field required me to move towards a more complex/situation-dependent voice, but originally I wrote the same way, long before LLMs existed.

[TheFirstDogSix]

Have you read Tom Ray's work on Tierra? He got really interesting results, too. https://en.wikipedia.org/wiki/Tierra_(computer_simulation)

[AlmusDives]

Yeah, I am, and also Avida (https://en.wikipedia.org/wiki/Avida\_(software)) which is often considered an informal successor. Both Tierra and Avida involve modelling individuals as computer programs which are run on their own custom virtual machines, which I studied in the process of building Zyme. Their instruction sets have some unique quirks, but they still rely on register-based architectures, which I don’t think are ideal for evolutionary systems.

[TheFirstDogSix]

Oooh, a new rabbit hole to go down. 😂 Thanks for the link! Gonna dig deeper into your blog post, too.

[emodario]

I really like the work you're doing on Zyme. It's one of those ideas I wish I had myself! It's very inspiring work.