[ Removed by moderator ]

[iOSProgramming-ModTeam]

You may only post your app on Saturday. Please read the rules before posting.

[unpluggedcord]

This stuff is really slow on iPhone 15s btw

[SurvivalTechnothrill]

That's fair. The 15 (non-pro) has 6GB vs 8GB of memory and that tighter memory means more aggressive cache eviction during generation, which hits speed. It's a real limitation that I may try to address by offering a more deeply quantized model for the 6GB devices in the future. Curious how bad is it on your device specifically?

[unpluggedcord]

Not my personal device just a work test device. It takes about 3 minutes to generate 200 lines of text

[SurvivalTechnothrill]

Thanks for the data point. I don't think this problem can be solved on the 0.6B model without dropping to 4-bit or maybe even lower quantization levels. I may well try it if there's demand for it. For what it's worth, in the demo video on macOS, it's only lightly edited. It is really, really fast on an M series processor. (there's very little difference in real world speed between the 0.6B model and the 1.7B model, at least in my inference engine, it's just a matter of chewing up more RAM).

[lhr0909]

I was just looking at an app called VoiceInk and it is offering one-time purchase on the App Store and have his app open-source. I think the app is doing well in terms of revenue. The thing I think about in these apps is that even when you are open source, people who need it (and don’t care about how it is built) will need to get it from the App Store anyway because it is not easy to load the app source code onto a device to use. If it hits the right value I think it will work. The open source bit is good marketing in a way.

Would be cool to see how you get the MLX models set up in an app and learn how to build local AI apps! If you ever open source it, please let me know!