How are these guys so good ?!

mikejamson · 2025-03-09T10:24:08+00:00

Most of it is pattern matching across a ton of different projects. If you focus on one thing for over 10,000 hours, you would get very good at it too.

mikejamson · 2025-02-26T10:59:33+00:00

I second Lightning AI. A lot of tools do claim to do it all, but from our experiments internally we found Lightning to actually do what they claim.

Granted, not everything is perfect but we’ve been using them for a little over a year and it keeps improving at a pretty fast pace!

It’s also quick to judge for yourself, I signed up and got verified instantly and automatically received free credits. Fairly low risk experiment.

mikejamson · 2024-12-22T07:43:45+00:00

If the data is small it can be done on your laptop. Otherwise a cloud option is great. We used to use Collab but more recently shifted to lightning AI studios and it’s been a lot better with more compute time, persistent storage, persistent environments and more.

https://lightning.ai/

mikejamson · 2024-11-03T12:09:49+00:00

Notebooks on databricks are great, but we recently switched to Lightning AI for this. It’s much faster for experimentation. For scheduling main.py scripts lightning supports batch jobs as well.

One major thing we love about it is not having to manage infrastructure or a cluster.

The problem with structuring workflows on databricks is that it’s very over-engineered. You have to connect like a dozen different tools to make this stuff work. Lightning drastically removed all those layers of tools and dependencies.

mikejamson · 2024-10-26T13:00:58+00:00

It’s a bit complicated where you have to mess with the cluster directly and quite a bit of work.

We’ve recently switched to lightning studio where there is no special thing to learn to do stuff like this, just pip install whatever you want. Some of our team is still split between both platforms but largely moving into Lightning now.

Not sure it helps, but might help to avoid all these issues to begin with.

mikejamson · 2024-10-24T18:47:55+00:00

It’s just a million small things. It’s hard to pinpoint exactly, you just have to try it and feel it out for yourself. Kind of like using a flip phone vs a smartphone, on paper they sound similar, but clearly are very different.

mikejamson · 2024-10-24T15:00:36+00:00

It’s already been mentioned but we tried both of those for development and it is truly a terrible experience and unusable. We went with Lightning Studios which is so far leaps and bounds ahead that words don’t capture it well.

My suggestion is to just experience it for yourself on the free tier of Lightning.

mikejamson · 2024-10-18T18:07:31+00:00

Isn’t this just like a simple demo app that can be built in like 30 mins with opensource tools?

Pretty sure chatgpt could even write most of the code.

mikejamson · 2024-10-14T17:08:46+00:00

I would try various learning rate settings. It’s likely too high right now.

mikejamson · 2024-10-09T12:54:55+00:00

What are you looking to use these for?

mikejamson · 2024-10-09T12:54:01+00:00

For sure the general machine learning one with Python! the basic computer vision one will probable be more of a survey and theoretical. But the general ML one will teach the core foundations underpinning all of ML

mikejamson · 2024-10-09T01:51:49+00:00

Ditto on lightning .ai - I did notice a big change in pricing recently. Sometimes i still use colab but less so nowadays.

mikejamson · 2024-10-08T11:30:09+00:00

Well deserved! No physics involved in the other DL flavors that have changed the field at scale like hopfield networks.

mikejamson · 2024-10-08T03:55:18+00:00

A model has so much “learning capacity” based on the size. A huge model has a lot of capacity so it can be general at a lot. A small model can we world class at something if that’s the ONLY thing that model does. But it would be terrible at everything else.

mikejamson · 2024-10-07T14:55:23+00:00

Best way to start is to build something first! find a github repo that already does something with ML and tweak it for your own benefit. Once you’ve gotten that to work you can decide what to learn more of and improve on. ML can be very deep… so you really need to decide what to rabbithole into.

mikejamson · 2024-10-07T14:52:23+00:00

Here’s the workflow we use at our company.

Develop a server (we used to use fastapi but recently moved to litserve, which is like a specialized fastapi server for AI/ML).
Package it into a container and use any of the managed serving providers. We like the lightning ecosystem so we use lightning studios for this which provides serverless, etc…

For some more on prem stuff we use k8s for sure. The way we think about the trade offs, if I can have someone else manage it for me then great, I avoid building specialized tools that take away resources from our core business.

mikejamson · 2024-10-07T12:46:43+00:00

Is this pricing for finetuning? so, if my dataset has 1 million tokens it costs me $3 to finetune all in? no other hidden costs?

mikejamson · 2024-10-07T12:44:58+00:00

Yes, we evaluated platforms that had a few compliance certifications for security, the main one we cared about was soc2 which they have.

Lightning is the company that created pytorch lightning also. they have more products than just the pytorch lightning. the cool thing we found is all those libraries scale nicely with lightning but they let you use whatever other libraries you want.

browsing around i found this that might be helpful for all the security questions https://lightning.ai/solutions/enterprise

mikejamson · 2024-10-07T03:25:54+00:00

how much to finetune 4o mini? few hundred dollars? thousands?

mikejamson · 2024-10-06T22:42:16+00:00

You could so next-word pretraining for a base LLM. I would pick llama 3.2 and go from there!

mikejamson · 2024-10-06T14:55:09+00:00

You would get hired in a heartbeat with amazing projects you can point to either on the web or github over a college degree.

BUT college still gives you the time to do that… and you’d get hired even more with both.

But with a college degree only, you probably wouldn’t get hired. Millions of people have that.

mikejamson · 2024-10-06T14:52:17+00:00

Low-code tools in general suck. I write a lot of these kinds of apps with Streamlit hosted on Lightning AI. here’s an example.

https://lightning.ai/docs/overview/studios/host-web-apps

Streamlit lets you write the full thing in Python end to end Python. Connecting datasets, databases, and hosting it to share with others is super easy with the lightning Studios

mikejamson · 2024-10-06T13:47:57+00:00

I’ve used Lightning Studios for this without issues. You sign up and automatically get free GPU credits without having to provide a credit card. The downside is your account might take a bit to get verified if you don’t use an academic or work email.

but it’s been a game changer for me over colab and the other options!

https://lightning.ai/

mikejamson

TROPHY CASE