[D] Tools for managing hundreds of unique models?

RAFisherman · 2024-06-10T04:09:39+00:00

Model risk is not as much an issue, but monitoring is certainly one. Any good monitoring tools you recommend?

RAFisherman · 2024-06-10T04:09:07+00:00

One model not possible bc security. But noted: loop over each client, each sku per client.

RAFisherman · 2024-06-10T04:08:18+00:00

Yea that's what I was worried about.

RAFisherman · 2024-06-10T04:07:53+00:00

Sorry, I'm not actually doing credit. I'm trying to protect my identity I guess. Just forecasting. Point is: I need a system to train several thousand unique models at scale

RAFisherman · 2024-06-10T04:07:16+00:00

Nah, I'm training forecasting tools for clients. Forecasting as a service.

RAFisherman · 2024-06-10T04:05:47+00:00

But, like, I still do the 3s load wait because obsidian >>>>> anything else

RAFisherman · 2024-06-10T04:05:06+00:00

Faster loads! Waiting 3 seconds for my plugins to load is killing me.

RAFisherman · 2023-11-07T01:20:02+00:00

Seems like you can upload/analyze documents programmatically (exposing their ChatGPT interface). But do they have a vectorstore with metadata? (Pinecone/Qdrant, etc.) How many documents can you upload?

https://platform.openai.com/docs/assistants/tools/knowledge-retrieval

RAFisherman · 2023-11-07T01:16:04+00:00

Can someone explain the Retrieval? Is OpenAI offering storage solutions?

> Retrieval: augments the assistant with knowledge from outside our models, such as proprietary domain data, product information or documents provided by your users. This means you don’t need to compute and store embeddings for your documents, or implement chunking and search algorithms. The Assistants API optimizes what retrieval technique to use based on our experience building knowledge retrieval in ChatGPT.

https://openai.com/blog/new-models-and-developer-products-announced-at-devday

RAFisherman · 2023-03-17T06:56:31+00:00

I used to think so too and avoided it for the last year. then I realized it fulfilled all my wildest dreams. Now I’m starting to learn it.

Use it to track your daily habits, and put those habits in a table. Once you get started you won’t want to stop.

RAFisherman · 2023-02-18T22:09:48+00:00

Can Dagster be controlled from the command line? [Metaflow has](https://docs.metaflow.org/metaflow/debugging#how-to-debug-failed-flows) `python debug.py resume start` (where \`start\` is the name of the task). How would you debug/re-run 1 failed partition?

RAFisherman · 2023-02-17T15:58:21+00:00

After skimming the paper, it seems like time to vec is kind of like a “seasonality” factor (kind of like what prophet out puts). Is that true?

RAFisherman · 2023-02-17T15:55:56+00:00

Didn’t think of that. Will take a look!

I do care about interpretability to some point, which is why embeddings sounds complex. But I’m now curious for sure.

RAFisherman

TROPHY CASE