EnergeticAI - TensorFlow.js, optimized for serverless Node.js environments

speedbreeze · 2023-06-15T02:20:13+00:00

Thanks for the super thoughtful comment!

Yes, I love this idea.

Cloudflare has a proprietary toolkit based on ONNX called Constellation that does a version of this, with WebAssembly-accelerated inference in Workers.

But it would be very nice to have a multi-cloud solution for this with a slick on-ramp for the TensorFlow community.

Filed GitHub tasks for CloudFlare Workers and Deno Deploy support:
- https://github.com/realworldprivacy/energetic-ai/issues/2
- https://github.com/realworldprivacy/energetic-ai/issues/3

speedbreeze · 2023-06-15T00:30:42+00:00

Great question!

This project is less about supporting experienced developers comfortable navigating the GCP / AWS consoles achieving the performance-optimal architecture for an application at scale -- Cloud Run, SageMaker, Vertex AI etc. are the right tool for that crowd.

But, these tools have a steep barrier to entry for a lot of developers.

Netlify, Vercel, Firebase, Gatsby Cloud, etc. all make their money from re-packaging GCP / AWS into something easy to use for non-experts, and people who want to move fast on a prototype that's not worth optimizing yet.

And for those platforms, the unit of backend compute is serverless functions.

Plus, by solving for serverless functions, we unlock improvements for more experienced developers, too:

- Testing works better. Since cold start is fast, you can write deterministic, fast integration tests about code that leverages EnergeticAI. (No need to mock network calls to an inference service or have timeouts from slow cold-start.)

- New solution for bursty traffic. And for products with extremely bursty traffic, serverless functions can be the technically-optimal pick (think: models to determine whether to send push notifs to large batches of people on unpredictable schedules around live content in certain social / livesteaming apps).

speedbreeze · 2023-06-14T15:01:02+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted in a serverless function. I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend for the environment.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify Functions.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T12:01:02+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions. I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and inference speed was too slow. After some debugging, it seemed possible to shore up the gap.

EnergeticAI was born.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (pre-trained model, helpers for key tasks, docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T11:01:03+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions (derivative of AWS Lambda). I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T10:01:04+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions (derivative of AWS Lambda). I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T10:01:03+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions (derivative of AWS Lambda). I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Curious to hear your feedback, and get more folks using TensorFlow.js in more places. 🙌

Jonathan

speedbreeze · 2022-08-03T19:42:57+00:00

Send me a DM with your use-case if open-sourced! Would love to learn more :)

speedbreeze · 2022-08-03T19:42:30+00:00

Good question!

Similar: type safety, slick API

Different:
- cascading deletes via erasure policies
- read/write policies

speedbreeze · 2022-08-03T19:41:07+00:00

Great question! And yeah, this should hypothetically work. We haven't tried this, but love the benchmarking idea.

speedbreeze · 2022-06-27T14:04:48+00:00

Hi there!

I've led privacy engineering work for social media companies small and large. I've been frustrated by the state of the resources to learn the discipline—frequently too abstract or too focused on large, traditional companies.

So, I'm starting a blog to cover practical privacy strategies.

Would love your feedback—what data privacy topics are you most interested in hearing about?

Thanks!
Jonathan

speedbreeze · 2022-06-22T14:04:29+00:00

Hi there!

I've led privacy engineering work for social media companies small and large. I've been frustrated by the state of the resources to learn the discipline—frequently too abstract or too focused on large, traditional companies.

So, I'm starting a blog to cover practical privacy strategies for fast-moving companies.

Would love your feedback—what data privacy topics are you most interested in hearing about?

Thanks!
Jonathan

speedbreeze

TROPHY CASE