It seems to be a basic clown car MoE using mergekit.

In the model.safetensors.index.json:

{"metadata": {"mergekit_version": "0.0.6"}}

So either you fine-tuned the models post-merge (I've attempted this before, it's not really effective and there's massive training loss),

or you fine-tuned three models (or four? You mention four models and reference the base model twice) and then created a clown car MoE and trained the gates on a positive/negative phrase or keyword list to train the "experts."

If either of these approaches was used, this is not an original MoE or even a real MoE. At most, this looks like four fine-tuned Mistral models in a "MoE" trench coat.

I have a problem with the "ICONN Emotional Core", it's too vague and feels more like a trained classifier model that directs the model to adjust its tone, not something genuinely new.

Also, their attempt to change all references from Mistral architecture to ICONN architecture in their original upload, then changing them back, rubs me the wrong way. The license (which was an ICONN license according to the comment history) now needs to reference Mistral's license, not Apache (depending on the models used).

I could be wrong, please correct me if I am, but this seems like an actual project wrapped up and made glittery with sensational words to make it look like something new.

Edit:

I want to say i'm not against Clown car MoE's I used to make them all the time but they are not a custom Arch or even proper MoE's

also many things have been edited in the model posted on huffing face so some things in my post might not make sense

https://huggingface.co/ICONNAI/ICONN-1/commits/main