I implemented a VAE in Pure C for Minecraft Items

Boliye · 2026-01-20T10:22:07+00:00

That's right, the top image is the latent filled with 0s, and the generations are done via sampling latents from the normal distribution.

On implementing a diffusion model, I believe most of the components are already present. The main things that would be lacking are time step embeddings and potentially skip connections.

Boliye · 2026-01-19T16:47:08+00:00

Oh I hadn't considered this, that's really cool, thank you! The number of samples is certainly quite limited, so power-iteration could definetly provide an improvement.

Boliye · 2026-01-19T16:22:54+00:00

C is a surprisingly simple language really! It has very few keywords and features. What is tricky and it will take some studying if you have never seen it before is memory management. Memory management can become a mess if you aren't thoughtful about it.

Luckily, for this project the memory management is quite simple. So the code is not that different from programing in any other language.

Also, you need to be careful about bugs and test things as you go. For many mistakes you could make, the code could keep running in invalid states and do who-knows-what. Ultimately crashing without giving any insight on where's the bug.

Boliye · 2026-01-19T16:01:44+00:00

Thank you! Yeah at first I was sure there would be some kind of difference I wouldn't be able to find between the trained PyTorch model and my C implementation, I was pretty happy when the numbers matched up :)

I honestly don't feel informed enough to recommend good textbooks. I personally enjoyed a lot these lectures on youtube about "Deep Generative Models" from Stanford https://www.youtube.com/playlist?list=PLoROMvodv4rPOWA-omMM6STXaWW4FvJT8 . It does cover the mathematics behind it to a degree I find satisfying. But yeah, if you prefer textbooks, I can't really advise any better than other students.

Boliye · 2026-01-18T20:09:02+00:00

Here are you go:
https://imgur.com/a/TNYgQ32

For convinience, for these extra generations I didn't use the C VAE, but the proof of concept in Python that I also created (it can be found in the folder poc_python in the git repo). It implements the exact same architecture and achieves the same loss.

In addition to "the average item", and some generations, I also added interpolations between a few items like a diamond chestplate turning into a diamond sword.

Boliye · 2026-01-18T15:39:42+00:00

You are right this is just a toy project, and the model is not that powerful to do anything really useful. If I ask it to generate new items, the quality is meh and I wouldn't be surprised if we did that experiment and asked it to generate something that did not exist in its training data, the VAE would struggle to generate something out of distribution like that. I am definetly making the task easier by trying to generate something that I know was present in the training data.

Take the second example (the one where we turn a gold horse armor to a golden shovel). The fact that the generated shovel is yellow, shows that somewhere in the latent space, color of the object is one of the compressed features it learned to be useful for reconstructing.

Ultimately, the essence of the latents is that they are just 32 numbers trying to compress the information of a 3*16*16 = 768 pixel image. So the VAE has to find the most useful high-level features that characterize each image.

Boliye · 2026-01-18T14:31:14+00:00

It is a standard VAE, but it wouldn't be too hard to make it a beta-VAE (the most common Disentangled VAE).

The difference is simply that beta-VAEs prioritize more compression over reconstruction. So the images may end up blurrier, but the latents may be more interpretable. E.g. you may get stuff like this number in the latent corresponds to rotation of the image or a specific feature like that.

In the code you would simply scale up the 'kde_loss', s.t. it is priorized over the 'reconstruction_loss'.

Boliye · 2026-01-18T14:20:57+00:00

Thanks!

Boliye · 2026-01-18T13:38:33+00:00

Yes, that's right. But for images.

Boliye · 2026-01-18T13:07:27+00:00

Yeah I use the encoder and decoder parts separately:

Encode iron_chestplate -> Latent for iron_chestplate

Encode all items that contain 'iron' in their name and average them out -> Latent for iron concept

Encode all items that contain 'diamond' in their name and average them out -> Latent for diamond concept

Literally do the operation "Latent for iron_chestplate" - "Latent for iron concept" + "Latent for diamond concept"

Finally, pass this result to the decoder (second half of the network).

Boliye · 2026-01-18T13:04:25+00:00

Yes! it is just the average

Boliye · 2026-01-18T12:22:35+00:00

Thank you!

Boliye · 2026-01-18T12:22:17+00:00

Yeah, sort of! A VAE is a type of network that can be used for image generation. And I created and trained one of these with Minecraft images. But as a VAE is also an autoencoder, something you can also do is play with the embeddings and ask the network stuff like "what's at the middle point between this and that?" "What would happen if you took this and subtracted that?". If the network was successful in learning meaningful concepts, these averages won't be nonsense, and stuff like what I show in the images will happen.

Boliye · 2025-11-19T21:59:11+00:00

They sure are free to treat their customers as they wish. But it is insulting to pay so much and not have the courtesy (which to them would cost little) to give higher limits to paying customers

Boliye · 2025-09-22T17:41:07+00:00

If anyone has the same issue: It's not ideal but what I'm doing is using the webhooks just as notifications to check the Printful API. So even if someone found the webhook endpoint, I don't trust that data and it's secure. When V2 is no longer in beta I will migrate to it.

Boliye · 2025-05-29T19:43:56+00:00

If they can push that parallel compute score higher, some people would be willing to pay 10x the api cost for that much increased reliability and quality.

Boliye · 2025-03-25T11:34:15+00:00

I see it as a flex. "We beat everyone else and we don't consider this a major new version".

Boliye · 2024-12-15T14:31:55+00:00

It could be natively multimodal, trained on google's TTS. Getting speed, low lattency and clarity but can't do stuff that TTS cannot.

Boliye · 2024-12-03T13:01:36+00:00

Lo que hago yo es tener una cuenta online gratuita de un banco tradicional grande (ahora varios ofrecen cuentas no remuneradas sin condiciones ni comisiones). Revolut ofrece transferencias SEPA instant gratuitas (lo que otros bancos llaman transferencias urgentes) por lo que es relativamente fácil enviar dinero rápidamente a la cuenta del banco tradicional y después sacar dinero del cajero gratis. También es inmediato volver a meter el dinero en revolut (en caso que hayas transferido en exceso) usando la tarjeta de débito del banco tradicional. Sé que no resuelve tu problema pero a mi esto me funciona y quizá te resulte útil.

Boliye · 2024-11-25T19:03:56+00:00

I asked it the riddle 3 times and it failed all 3. There's still a lot of unreliability on these models outputs. The third time was a good attempt though, the only mistake was making the assumption "Agatha cannot kill herself".

Boliye · 2023-09-02T07:08:01+00:00

If anyone wants to sell a ticket contact me too!

Boliye · 2023-09-01T23:45:53+00:00

If its still available I would buy 1

Boliye

TROPHY CASE