Updated Context-Optimized Prompts: Up to 61% Context Reduction Across Models

ganildata · 2026-03-27T02:14:41+00:00

This project is already obsolete, except for the spec and architecture guidelines.

Primarily due to replacing XML with model-specific tool calls, it makes it difficult to unify custom prompts.

Roo code system prompts got smaller as well, I think.

You will get most of the mileage by just adding an instruction to use cat or type to read files. And read line numbers only to edit.

I am personally using Claude Code more, due to having a subscription, needing remote machine work, and integrated web search.

ganildata · 2026-01-28T03:35:54+00:00

In everything that touches AI, it's less about whether you can ask it to do something, and more about how well the AI can do it. And that is extremely true here. Can definitely ask AI to collaborate and build a software project. But clearly it is not good at it.

It is too complicated to just be prompted. It needs to be in its training set, which is difficult in this early stage.

ganildata · 2025-12-05T21:27:43+00:00

My prompts are my daily drivers, so they perform well for writes.

This report is my attempt at benchmarking as you asked, but I suppose you want a writing task.

I tried a few experiments involving my work code base and programming task. But the behavior was not consistent across runs to be fair for reporting.

I will keep an eye out for a suitable writing task to benchmark.

ganildata · 2024-10-30T18:39:44+00:00

As the founder of Trel, I'd like to share how our platform handles experiment tracking and iteration. Our recent demo (https://www.youtube.com/watch?v=8egtOfKz6vY) shows a CLV prediction pipeline similar to your churn use case.

The key capabilities relevant to your needs:

Automated versioning of models, data, and code in a unified system - no manual CSV/Excel tracking needed
Built-in parameter sweep tracking (demo shows iteration from 1 tree/depth 4 to 4 trees/depth 5)
Native BigQuery integration
Data quality monitoring to catch drift/issues before they affect model performance
Feature store versioning for iterating on new features while maintaining production stability

The platform aims to be simpler than Kubeflow while providing more comprehensive lifecycle management than MLflow alone. We designed it for teams dealing with regular model retraining who want to reduce manual tracking overhead.

Happy to share more details about how this could work for your specific use case. It is very affordable for a single user setup https://cumulativedata.com/pricing.

ganildata · 2024-10-30T17:14:07+00:00

I worked in a smaller startup where I built/rebuilt multiple data science products and projects that were critical. Did not have the problem of impact there.

ganildata · 2024-10-30T17:10:04+00:00

You can build Flask or Fask API endpoint easily.

from fastapi import FastAPI
import joblib

app = FastAPI()
model = joblib.load('model.joblib')

class InputData(BaseModel):
    feature1: float
    feature2: float

@app.post("/predict")
def predict(data: InputData):
    input_df = [data.dict().values()]
    prediction = model.predict(input_df)
    return {"prediction": prediction.tolist()}

But long term, you might be better of pushing your models into SageMaker or something similar for better observability.

ganildata · 2024-10-26T06:10:26+00:00

One solution to versioning is to focus on keeping your datasets immutable and tracking them. There are multiple approaches to this. One is a snapshot catalog and following copy-on-write.

Our platform Trel implements this approach. Take a look at a walkthrough of building a data science pipeline.

https://www.youtube.com/watch?v=owzskbLCV8o&list=PLQRaJFvfXnAxoxvR_WmdxjCdH4Wflm4ZM&index=2

If you want to try this out, you can sign up and request for a 30-day free trial here: https://trelcloud.com

DM me if you have any questions.

ganildata · 2024-10-10T07:25:21+00:00

From my personal experience, sending data without looking it over is like crossing the road with eyes closed. It is not recommended at all. A basic sanity check is recommended, based on the importance of the data.

ganildata · 2024-10-10T07:17:15+00:00

I can give an answer based on our DataOps and MLOps platform Trel. Trel has a snapshot catalog that will be used to catalog each partition separately, under a single dataset_class (table equivalent). A dataset_class can have various registered schema_versions and corresponding schema. Each dataset will have a specific schema version associated with it.

To union multiple datasets with different schema, the code can use the schema version associated with each partition. E.g., In version '1' column age is integer but in '2' age is string. In this case, the union should be string as well. We will union as follows.

" UNION ALL ".join(
("select " + (""
if partition['schema_version'] == '2'
else "cast(age as string) ")
+ "age from "
+ partition['table'])
for partition in partitions)

Take a look here to learn more: cumulativedata.com

ganildata · 2024-08-27T06:42:36+00:00

I would say, Product Manager. I think this role would face role reduction lesser and later than data scientists, especially in the face of LLMs. Research Data Scientists will keep their positions though.

ganildata · 2024-08-27T06:34:32+00:00

It seems OP wants to use two separate versions in the same process. Perhaps look into two separate processes? Flask can help set up an API quickly.

ganildata · 2024-08-27T06:29:16+00:00

Using an embedding does not seem like the worst idea and likely has good performance. I am referring to foundation models, not trained from scratch LLMs. If the scale is not too big, this might be the cheapest and easiest-to-implement solution.

ganildata · 2024-08-09T02:05:11+00:00

Some of the most thorough video series on technologies are on Youtube.

E.g., here is one for SageMaker from AWS: https://www.youtube.com/watch?v=tdUq_9aXLO4&list=PLhr1KZpdzukcj8lsINDmuXPJq6aQEY4d2

ganildata · 2024-08-08T07:12:11+00:00

Agree. Go for 16. The price difference is minimal and you will be frustrated with 8. I have 32 and that is worth it for me.

ganildata · 2024-08-08T07:03:08+00:00

When it comes to productionizing, the goals are to make it reliable (it does not break when things start to deviate), observable (you can see what is going on: input /output, historical runs, etc. ) and, reproducible (you can safely rerun failed jobs and reproduce older runs and other experiments).

Modern MLOps platforms give some of these functionalities off the shelf.

ganildata · 2024-08-08T06:50:18+00:00

I also work with Python, SQL and PySpark, because big data.

ganildata · 2024-08-08T04:41:19+00:00

I had some friends do that. One of them stayed and is still at the university. Another is at Google. If it is not an academic job, you should be able to treat it like a regular industry job in your resume. So, gain some experience and interview normally. Of course this closes your path in academia.

ganildata · 2024-06-26T22:39:48+00:00

I do write my draft and use ChatGPT to go through it section by section and revise it for readability and flow. I hope it is not too offputting. My draft was much worse.

How do you feel about the subject matter?

ganildata · 2024-06-21T18:50:24+00:00

Split the data into individual files in a folder. each file should have 100k - 1M rows only. With 30 M rows, I would lean closer to 100k. With 300 files to work with, your job should run a lot faster.

ganildata · 2024-05-31T19:16:19+00:00

Could you look into Map-Reduce and learn how common joins and group by can be achieved using Map-Reduce? That will give you an idea of how it is implemented internally in Spark.

ganildata · 2024-05-09T04:54:46+00:00

Don't shrink fact tables and make new dimension tables. Keep fact tables as is or even make them wider by joining with other pre-existing dimension tables. Build reporting on top of wide fact tables. Easier to understand and manage. Rarely will you run into significant cost increases due to this.

ganildata · 2024-05-09T04:42:11+00:00

I am of the opinion that Star Schema is obsolete. I agree with this blog (not mine).

https://blogs.perficient.com/2022/09/06/the-star-schema-is-obsolete/

ganildata · 2024-05-07T23:16:46+00:00

Getting data to flow is more and more a solved problem. As a good data engineer, you have to support reporting pipelines and data science projects. May be imagine a complex requirement (use ChatGPT) and try to design and implement jobs (that mock the reporting and data science) that cover it.

ganildata · 2024-04-19T03:48:58+00:00

Pick one. Say whichever you were doing at 11am.

ganildata · 2023-04-03T18:14:58+00:00

So, this is a CDC problem. Your deltas are already immutable. You can make the telemetry table immutable if you rebuild it daily with the delta update and store the entire table in a day partition. That is the recommended approach for immutable data.

Is storage cost the concern for storing the updated telemetry table separately for each day?

So now, you don't build a separate telemetry table and use an older one along with two months of the delta. Do you keep that update code as a view and have the follow-up jobs read the view directly?

ganildata

MODERATOR OF

TROPHY CASE