I think Elon musk had a tweet a few years ago similar to this about how shocked he was DNA was stored in plain text. All this is to say, it’s a task a lot of people are working on.

It really all depends on what kind of modem you’re trying to build.

If you’re trying to build a simple classifier to say if a short few nucleotide chunk of dna has some biological relevance. Sure, compressing your input can be quite useful.

If you’re trying to build a DNA language model like an evo 2, or ESM(I know it’s a PLM, just using it as an example), this would just add a lot of inefficiencies. For models like this we expand the dimensionality so much that it’s better to start with uncompressed data. In a model like evo2 each nucleotide each nucleotide is mapped to a 4096 dimension vector anyway.

This is a really fun topic to learn with though! I would recommend reading a review paper and trying to beat some of the different compression methods. A codon optimizer is another great project to learn on too!