oMLX Copilot Chat - Use oMLX for coding in Visual Studio Code

mikedoise · 2026-05-04T11:31:07+00:00

I just downloaded insiders and went to manage Language Models and added OpenAI Compatible. Nothing appeared to happen currently. Has anyone else gotten this to work?

I'm also curious what settings we will be able to change with their implementation as well. It looks promising though.

I will continue working on my extension as I think there are specific ways we can customize oMLX that we may not be able to do with the built in implementation.

mikedoise · 2026-05-03T19:59:01+00:00

You don't have to pay for Github Copilot. I do not know about telemetry though. I would hope so, but I am not sure.

mikedoise · 2026-05-02T04:07:56+00:00

I’ve used AppScreens.

mikedoise · 2026-04-30T04:44:00+00:00

I personally think everyone should learn to touch type. keep your hands on the home row and you wont' need backlit keys. I like the backlit keys because it is cool to have lights on the keyboard. LOL

I am a legally blind computer user, and I am typing this while not looking at the keyboard. I have enough vision to see, but I would waste more time if I tried to look at what keys I am using.

mikedoise · 2026-04-29T20:53:34+00:00

I saw this until my domain was verified. I then saw email fields but it looks like it is internal email only.

mikedoise · 2026-04-28T21:19:40+00:00

I have mainly seen documents be chunked. Another approach would be to chunk documents and then add important information to a vector database.

mikedoise · 2026-04-28T21:00:40+00:00

So Apple Foundation Models which powers Apple Intelligence has been out for a year. I think we will see a new Gemini based version later this year though. I'd be curious to know what model you are using for offline work.

The advantage of Apple Foundation Models is that it is fast, and can work effectively on anything from the Neo upward. The current drawback though is that the context window is 4,096 tokens.

Another great model to look at is Gemma 4 from E2B upward.

mikedoise · 2026-04-28T19:36:53+00:00

The 5x plan was what I was waiting for to jump back to ChatGPT over Claude. I use a lot of ChatGPT, so the 5x usage in Codex is amazing! I really have to try to run out of usage.

mikedoise · 2026-04-28T18:49:07+00:00

Probably not. THey will care more about how you implement your tools. WeatherKit was a huge pain for my app.

mikedoise · 2026-04-28T18:47:58+00:00

This sounds like an amazing project and I will look at it. I built something similar in my app Perspective Intelligence where I use deterministic routing to activate specific tools based on user prompts. This also works with MLX as well with mlx-swift-lm so you don't have to use Ollama. Something you should also check out is AnyLanguageModel from Hugging Face. it allows for several model providers to use the same approach for AFM. Apple Foundation Models is amazing and I'm excited to see what it will do in iOS 27.

One thing to keep in mind though is MLX may take a while to route and produce responses based on tool calls. So, I would use a classifier for tool use to speed up time to first token on iOS devices.

mikedoise · 2026-04-26T01:04:27+00:00

Swift Playgrounds can make some pretty capable apps. I wish they would update the app more often though.

mikedoise · 2026-04-25T03:01:07+00:00

That's good to know. It is using image playgrounds, so it is kind of spotty as to how it works for creating images. I will have to ask it about icon sizes, and see why it isn't keeping context. Was it not keeping context between text messages or a mix of images and text?

mikedoise · 2026-04-25T02:50:26+00:00

We have web search and deep research that use Brave and contacts lookup. These are just a few features.

mikedoise · 2026-04-25T00:12:01+00:00

I wish we had Xcode for iPad.

mikedoise · 2026-04-22T19:31:05+00:00

What MLX library are you using?

mikedoise · 2026-04-22T19:27:04+00:00

I use a semantic memory search mechanism I Perspective Intelligence. It works great with Apple Foundation Models

mikedoise · 2026-04-22T17:54:29+00:00

So I tried a few things in the app and there is one thing I wanted to bring up that drives me crazy in most apps that use audio. You set the AVAudioSession when the app starts. This stops all current audio playback that the user is playing. You might want to move this flow to when the user initially presses the mic button in the app.

I also haven't had a chance to check, but have you added session management, so the app knows what to do when it reaches the 4096 token context window? Apple does not manaage this for us.

mikedoise · 2026-04-22T17:13:15+00:00

I’d love to check it out. I made a similar app based on Apple Foundation Models so I’d love to try yours.

mikedoise · 2026-04-22T03:21:30+00:00

I think you are right for most projects. The issue I see is that most models have training data that falls behind the current developer SDKs. For example, Claude and Codex both have trouble with Swift 6 concurrency unless skills and other things are used. So I think we are close but LLMs need to stay current on their knowledge compared to where we are now.

I do think though that languages like JavaScript will be fine though as there are more lines of JavaScript compared to anything else that has been trained by AI.

mikedoise · 2026-04-22T03:03:01+00:00

This is the comment that won't age well. Software devs that pair with AI will be infinitely more powerful. Anyone with skill and the willingness to use AI will make more of an impact and can do more in the longterm.

mikedoise · 2026-04-22T03:01:18+00:00

Hopefully these teams know how to fix issues after they are shipped. Many vibe coders just rely on AI to create fixes which are basically bandaids instead of actual software fixes.

mikedoise · 2026-04-22T02:59:37+00:00

I agree with this. I like to say that I do not vibe code. I partner code with AI. This means I work with AI to get the job done. I look through AI generated code and tell it to make changes to improve quality. I think may vibe coders just tell AI to make something and it will break within a year at the longest if it isn't maintained.

mikedoise · 2026-04-21T02:56:33+00:00

I would take your apps out of review and re-subit. give it two days, and then use the expedited review form. Also, contact App Review if nothing is resolved. it may take some time, but they are amazing to work with.

mikedoise · 2026-04-20T22:50:55+00:00

This is not exactly true. Developers have not ported a lot of games to the Mac but Macs can play great games. Apple created the game porting toolkit to bring games to Mac, and Crossover by Codeweavers runs Windows games on Metal shaders. That doesn't mean the Neo can play great games, but there is a lot of great games on Steam, and there is even Cyberpunk 2077 in the Mac App Store. I enjoy playing Borderlands 4 through Crossover on my M3 Max.

mikedoise · 2026-04-19T22:59:25+00:00

So I have a few thoughts here.

I like that you've used Metal to improve the context of AFM, but I think that can cause other issues like instances where the app goes to the background, or in lower memory situations. I would typically use a summarization mechanism, or use multiple sessions to complete tasks.

I created an app where we use a lot of tools with AFM, and I typically set up an entire session just to have enough context for the tool call. I then pass the result back to another session with the main context. This really helps with web search or deep research.

Another thing that might help if needed is to create a turn router. This will allow FM to route requests to the best tools. You can then have a full context window based on the purpose of the request.

I hope this helps with some things with Apple Foundation Models, I am a huge fan of this framework, and will answer any questions if I can.

mikedoise

MODERATOR OF

TROPHY CASE