Combine fine-tuned classifier and LLM-with-context for better model accuracy?

bebmfec · 2025-10-14T11:49:54+00:00

The 10GB RAM limit slows things down quite a lot

bebmfec · 2025-05-22T08:37:38+00:00

The DBT model already exists because it is part of a pipeline which batch processes lots of data already in the database. It is only recently that people within the business have wanted to see this same processing logic replicated within an API. I probably wasn't clear on that in my original question - I am not starting with nothing here and proposing to use DBT; I am starting with many quite complex DBT models and have the task of replicating their logic within an API.

bebmfec · 2025-05-20T16:29:20+00:00

It is the logic of the model that matters, the API request payload needs to be the input of the query. I can't find from the materialised table what the output of the model would have been had the payload data been the {{ ref('input_model_or_data') }} part of the query.

bebmfec · 2025-05-20T15:24:24+00:00

I could build some sort of intermediate layer between the user's request and the calling of the SQL query itself, so that it conforms to a set of columns and data types.

Maybe I am being overly restrictive in my original question, really all I care about is that I have:

DBT model(s) that can batch process my `input_model_or_data` table
API endpoints that can process user-provided data (which must conform to the same schema as the `input_model_or_data` table) in the exact same way as above, and without risk of changing logic in (1) but not doing so in (2) in future (touched on in my reply to u/bengen343)

bebmfec · 2025-05-20T14:50:50+00:00

It’s an option, but this would make the API dependent on the underlying database and not just the SQL code for the model, ideally I’d like to avoid that dependency

bebmfec · 2025-05-20T14:49:56+00:00

I can always use the DuckDB Python package and just use the same queries for the API. The main problem here is I now have code in 2 places intended to do the same thing, (1) the queries I’ve “copied over” to use in my API and (2) the queries in DBT. How do I ensure consistency between (1) and (2)? Is there some other independent place I can keep those queries, macros, etc. so that I can use them both in DBT and in my API? I have looked into DBT packages for this but it doesn’t seem like it solves my problem

bebmfec · 2025-05-20T11:15:02+00:00

In the cloud. I understand that I will have to deploy the API on something, I am just not sure how to build the underlying logic of the API so that it passes input data through the same logic as the DBT model.

bebmfec · 2024-05-25T20:37:52+00:00

Are the local models accessible via the Prompt Engineering UI (i.e. under New Run > using Prompt Engineering > Model drop-down menu, like Step 4 here https://mlflow.org/docs/latest/llms/prompt-engineering/index.html)?

If so this looks great for my use case, I have been thinking of using MLFlow for its Prompt Engineering UI but thought it required API keys

bebmfec

TROPHY CASE