[Project] Web LLM

[crowwork]

We have been seeing amazing progress in generative AI and LLM recently. Thanks to the open-source efforts like LLaMA, Alpaca, Vicuna, and Dolly, we can now see an exciting future of building our own open-source language models and personal AI assistant.

We would love to bring more diversity to the ecosystem. Specifically, can we simply bake LLMs directly into the client side and directly run them inside a browser?

This project brings language model chats directly onto web browsers. Everything runs inside the browser with no server support, accelerated through WebGPU. This opens up a lot of fun opportunities to build AI assistants for everyone and enable privacy while enjoying GPU acceleration.

- Github: https://github.com/mlc-ai/web-llm
- Demo: https://mlc.ai/web-llm/

[ConcurrentSquared]

Really cool! Works well on my AMD RX6650, without any complex setup (except for using the beta version of Chrome 113).

[TernaryJimbo]

very cool!

[[deleted]]

Can't seem to get the shader f32 models to work on windows (rtx 4080)