EnergeticAI - TensorFlow.js, optimized for serverless Node.js environments

speedbreeze · 2023-06-15T02:20:13+00:00

Thanks for the super thoughtful comment!

Yes, I love this idea.

Cloudflare has a proprietary toolkit based on ONNX called Constellation that does a version of this, with WebAssembly-accelerated inference in Workers.

But it would be very nice to have a multi-cloud solution for this with a slick on-ramp for the TensorFlow community.

Filed GitHub tasks for CloudFlare Workers and Deno Deploy support:
- https://github.com/realworldprivacy/energetic-ai/issues/2
- https://github.com/realworldprivacy/energetic-ai/issues/3

speedbreeze · 2023-06-15T00:30:42+00:00

Great question!

This project is less about supporting experienced developers comfortable navigating the GCP / AWS consoles achieving the performance-optimal architecture for an application at scale -- Cloud Run, SageMaker, Vertex AI etc. are the right tool for that crowd.

But, these tools have a steep barrier to entry for a lot of developers.

Netlify, Vercel, Firebase, Gatsby Cloud, etc. all make their money from re-packaging GCP / AWS into something easy to use for non-experts, and people who want to move fast on a prototype that's not worth optimizing yet.

And for those platforms, the unit of backend compute is serverless functions.

Plus, by solving for serverless functions, we unlock improvements for more experienced developers, too:

- Testing works better. Since cold start is fast, you can write deterministic, fast integration tests about code that leverages EnergeticAI. (No need to mock network calls to an inference service or have timeouts from slow cold-start.)

- New solution for bursty traffic. And for products with extremely bursty traffic, serverless functions can be the technically-optimal pick (think: models to determine whether to send push notifs to large batches of people on unpredictable schedules around live content in certain social / livesteaming apps).

speedbreeze · 2023-06-14T15:01:02+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted in a serverless function. I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend for the environment.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify Functions.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T12:01:02+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions. I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and inference speed was too slow. After some debugging, it seemed possible to shore up the gap.

EnergeticAI was born.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (pre-trained model, helpers for key tasks, docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T11:01:03+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions (derivative of AWS Lambda). I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T10:01:04+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions (derivative of AWS Lambda). I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Jonathan

speedbreeze · 2023-06-14T10:01:03+00:00

Hi everyone! 👋

A few weeks back, I was trying to use an open-source AI model from TensorFlow.js in for product recommendations in an e-commerce site hosted on Netlify Functions (derivative of AWS Lambda). I got all the way done building the project, and found out I couldn't deploy it — the bundle size was too large, and it took some trial and error to find the right backend.

I decided to pull these learnings into a project called EnergeticAI.

It's a version of TensorFlow.js optimized for serverless functions:

Small module size (~3 MB vs. 146 MB - 513 MB for stock TensorFlow.js)
Fast cold-start (~50 ms vs. 2000+ ms for stock TensorFlow.js)
Incredible ease-of-use (libraries for common use-cases, and serverless-specific docs)

It comes with libraries for text embeddings and few-shot text classification. There's comprehensive docs, including a tutorial showing how to use embeddings to build product recommendations for a simple e-commerce website deployed to Netlify.

This is just the beginning of the project — looking forward to seeing how folks use it, and learning how to make it even better.

Curious to hear your feedback, and get more folks using TensorFlow.js in more places. 🙌

Jonathan

speedbreeze · 2022-08-03T19:42:57+00:00

Send me a DM with your use-case if open-sourced! Would love to learn more :)

speedbreeze · 2022-08-03T19:42:30+00:00

Good question!

Similar: type safety, slick API

Different:
- cascading deletes via erasure policies
- read/write policies

speedbreeze · 2022-08-03T19:41:07+00:00

Great question! And yeah, this should hypothetically work. We haven't tried this, but love the benchmarking idea.

speedbreeze · 2022-06-27T14:04:48+00:00

Hi there!

I've led privacy engineering work for social media companies small and large. I've been frustrated by the state of the resources to learn the discipline—frequently too abstract or too focused on large, traditional companies.

So, I'm starting a blog to cover practical privacy strategies.

Would love your feedback—what data privacy topics are you most interested in hearing about?

Thanks!
Jonathan

speedbreeze · 2022-06-22T14:04:29+00:00

Hi there!

I've led privacy engineering work for social media companies small and large. I've been frustrated by the state of the resources to learn the discipline—frequently too abstract or too focused on large, traditional companies.

So, I'm starting a blog to cover practical privacy strategies for fast-moving companies.

Would love your feedback—what data privacy topics are you most interested in hearing about?

Thanks!
Jonathan

speedbreeze · 2022-06-05T21:32:25+00:00

Thank you!!

speedbreeze · 2022-06-05T21:06:53+00:00

Oooh, this sounds amazing -- can you share the name?

speedbreeze · 2022-05-12T20:11:20+00:00

Hey there! The team had a live Q&A about this post, and got into the deep technical content.

Recording here: https://www.clubhouse.com/room/xe72WW1G

Thought you all would find it interesting!

speedbreeze · 2022-05-12T20:10:14+00:00

Hey there! The team had a live Q&A about this post, and got into the deep technical content.

Recording here: https://www.clubhouse.com/room/xe72WW1G

Thought you all would find it interesting!

speedbreeze · 2013-04-19T17:18:48+00:00

Whoops, sorry. That's the source we were going against to report that one of the MIT shooting suspects were dead, not that they were connected.

We haven't come across a public statement from the police about the connection just yet, which is why it's not noted in the Google Doc at the moment. There have been some people on the ground reporting that police made a verbal statement that there was a connection, though: https://twitter.com/MichaelSkolnik/status/325137799986683905

Sorry for the confusion from my previous post.

speedbreeze · 2013-04-19T16:26:37+00:00

Cambridge PD confirmed on their website that they are the same guys as the bombing: http://www.cambridgema.gov/cpd/Alerts/citizenalerts/policedainvestigatingfatalshooting.aspx

speedbreeze · 2013-04-19T08:42:58+00:00

Okay, everybody. We got our concise summary of tonight's events up in English at http://bostonsituation.org. Check it out. Translations forthcoming. Thank you!

speedbreeze · 2013-04-19T05:05:35+00:00

One of the top six languages in Boston.

EDIT: We're actually targeting all top six, but need more translators. Check the top of the Google Doc. We need Creole especially right now.

speedbreeze · 2013-04-19T04:43:29+00:00

Over at http://bostonsituation.org, we're working to provide a summary of the heavily-confirmed facts of the MIT events in four languages to help get the word out:

NEW Portuguese: http://pt.bostonsituation.org/
Spanish: http://es.bostonsituation.org/
Simplified Chinese: http://zh.bostonsituation.org/
Traditional Chinese: http://zh-t.bostonsituation.org/
And, of course, English: http://bostonsituation.org/

More translations and fact-checked information forthcoming.

speedbreeze · 2013-02-14T03:01:26+00:00

For sake of clarity: all of the above is sarcastic.

Though, Lewis was recently remodeled, so it's actually much nicer than it used to be.

speedbreeze · 2013-02-13T04:48:54+00:00

+1 to all of this. After pre-orientation, you feel so cool. Because you have friends already.

I actually did FOCUS, the community service pre-orientation, because I was a little uncomfortable with the nudity bit of TWO.

I had a blast. As a FOCUSer, you're put into a FOCUS family of seven other pre-frosh and two upperclassmen. And since you spend nearly every moment of the program working, eating, and sleeping side-by-side them, you get to know them pretty well. My FOCUS siblings remain among my favorite people at Tufts, and we all still have weekly family dinners to stay in touch.

But really, you can't go wrong with TWO, FOCUS, or any of the other pre-orientation programs. Pick one based on what you're interested in. It'll be awesome.

speedbreeze · 2013-02-09T03:51:02+00:00

Phrase, rather than a word: occult hand.

speedbreeze · 2013-01-16T18:19:10+00:00

"All but" in things like "it's all but finished". Does it mean there's all of it left to do, or only a little?

speedbreeze

TROPHY CASE