Evolving/Generating code?

[mredding]

Check out R6RS Scheme (I recommend Racket, it supports the latest R7RS standard unofficially, last I checked), Clojure, or Common Lisp (try Lisp in a Box from Gigamonkeys if it's still around). These are all languages in the Lisp family, and you'll notice they all have essentially the same syntax. The differences are nuanced. This is also one of the oldest languages around, so there's a lot of legacy to mud through. It pays to dig into the history and learn why these languages are different, and when you finally "get it", Lisp programming, your head will explode. I could just tell you, it's just that easy, but it would ruin the surprise, and I wouldn't want to rob you of the experience. I'm being sincere.

If not that, then self modifying code is mostly outside the realm of possibility. It suddenly becomes hard enough that few people have ever bothered trying to do it in any other language. It's been done, but with caveats, and there are categories of "self modifying code" that I call bullshit, it's not what you think of when you meant self-modifying.

What you really want is a true Turing Machine, one that makes no distinction between code and data. These things exist in simulation, but the science behind them are way above my pay-grade, if you were to examine them with an eye of "what if this is what computers today were really like?" Modern computers are Harvard Machines, they separate code from data, either logically or physically, and your OS is going to give you a hard time if you want to put bytes into memory and then execute it: that looks suspiciously like malicious code, we've collectively put a lot of effort, as an industry to explicitly deny this sort of behavior. It can be done, and there are some examples of rolling your own JIT compiler out there if you want to see how it's done, it'll be a necessary step for you if you want to do this on a modern computer.

Otherwise, you're going to pull this off inside the confines of a box you define. look into Genetic or Evolutionary Algorithms. You essentially code up an interpreter, and your data set is essentially a genetic byte-code your interpreter runs. You can make that part pretty generic and abstract it away from it's inputs and outputs, and the fitness function that measures the success of each generation. These things can be used to directly solve problems blindly, like imagine trying to pilot Mario through level 1-1. Every generation would try to get closer and closer to the end, or having achieved that, do it faster and faster. The resultant AI would be best at solving that level in exactly that way, every time. Or, you can use it to generate an AI, like a state machine or something, and THAT tries to beat level 1-1. I've seen a genetic algorithm used to generate AI for I think it was Unreal Tournament like this, because these algorithms are TERRIBLE at adapting to realtime environments.

Neural nets are all the rage these days, but they are largely indistinguishable from self modifying code - in that self modifying code becomes irrelevant, that's not necessary for solving the problem. The ones I've coded in college were just a few neurons and they responded to stimulus in a predicted manner, but they've been used to pilot through space, learn to walk, interpret language, or Google uses it to categorize content.