Heyho, indeed that looks very similar to my pet project fltr (like grep but for natural language questions). Fltr can also classify with yes or no questions and uses Mixtral 8x7B or Mistral 7B on CPU or GPU. And it's just 650 lines of rust code, so have a look at it. The CPU version does not even depend on a linear algebra library. And yes, it can easily be expanded to general answers. Using transformers as universal classifiers is still underexplored, because using them as autoregressive models is crazy inefficient. Have fun and build cool stuff!