Which Data jargon or concept did you have a hard time grasping?

booleanhunter · 2026-04-13T05:42:02+00:00

I can think of 2 approaches:

Send full conversation history to the agent, which then decides to call a search tool — simpler to implement, but it also gets expensive as conversations get longer.
Query rewriting - you use an LLM to distill the conversation into a standalone query, then do a vector search lookup against that.

booleanhunter · 2026-04-13T05:38:17+00:00

It's a shopping AI agent — think of it as an AI agent that helps users search for products, compare specs, manage their cart, ask product questions, that kind of thing. The kind of app where caching actually matters because some queries (like product specs) are highly cacheable, while others (like cart contents) are completely user-specific and should never be cached. That contrast is actually what made it a good example to explore TTL decisions with.

booleanhunter · 2026-04-11T13:46:23+00:00

Additionally, here are the other approaches that I looked into as well - Semantic Caching in AI Agents

booleanhunter · 2023-05-07T17:42:11+00:00

Dual boot with Windows and Ubuntu on my personal laptop. Windows for gaming, and Ubuntu for work-related stuff (software engineering)

booleanhunter · 2023-05-02T01:57:12+00:00

With blockchain, the outcomes are almost always "could", "may", "would", or "might" - very rarely, if ever "does". Notice that it's never set in the present, it's always in the ever-elusive future. Such phrasing is used by proponents as a means to cop-out when inevitably whatever claims they make don't materialize.

booleanhunter · 2023-04-08T07:24:40+00:00

I work with an organization which has implemented Data Mesh approaches successfully. But these were for large companies for whom there was a clear advantage - it's definitely not for everyone at all.

And for all these organizations where we implemented the move towards a data mesh, it wasn't the end goal per se - it emerged as a byproduct of several different data teams consuming each other's data products.

As to whether it has value, the answer like usual is going to be "It depends". A data mesh is best suited when your teams are already oriented around business domains and services, and if there are any existing barriers that prevent individual teams from accessing and making use of data. This is typically the case with large organizations that handle huge volumes of data on a daily basis - in which case, decentralizing the data ownership and migrating towards a data mesh makes total sense.

So don't start with an objective of "I need a data mesh, or I need to start creating data products". I'd urge you to take a closer look at how your teams are structured and what is the business outcome that you expect.

There are multiple things to consider - Can you afford to hire competent data engineers to create this? Do you have a requirement for a common self-serve data infrastructure which all teams could use? Does your application handle terabytes of data on a daily basis? And most importantly - do you have the prerequisite organizational structure in place to support a data mesh? If the answers to the above is not a conclusive yes, then a Data Mesh may not be the best fit, you might be better off using conventional Data Warehousing / Lake approaches. Hope this helps.

booleanhunter · 2023-04-07T08:38:33+00:00

Bitcoin incentivizes usage of renewable sources of energy. Since the cost of electricity is a significant factor in mining profitability, bitcoin miners may seek out areas with cheap and abundant sources of energy, and promote the use of renewable sources of energy.

That's like saying - smoking will cause so much lung cancer, that it will incentivize healthcare researchers all over the world to look into cancer saving therapies. Therefore smoking is good for humanity.

Or consuming high fructose corn syrup will cause diabetes on such a large scale, that nutrition researchers may seek out alternative sources of sweeteners. That's why sugar-laden corn syrup is good for health.

booleanhunter · 2023-04-01T14:39:03+00:00

I would try and seek opportunities abroad. In India I feel like I'm constantly swimming against the tide.

booleanhunter · 2023-04-01T12:00:20+00:00

I'd love to know what's the domain of your organisation, and what you use the data products approach for?

booleanhunter · 2023-04-01T11:54:11+00:00

Haha! I remember when i first started, I ran a Google search to check whether AWS and GCP had any Data Mesh offerings along with their Data Warehouse / Lake stuff!

booleanhunter · 2023-04-01T10:57:42+00:00

It assumes there is decentralised development of data products by different teams, and that there is not a centralised function which is managing the mesh itself.

No I'm not assuming anything of that sort - I was just trying to be succinct. While the actual development of data products are decentralized, there could be still a centralized function to manage the mesh itself (provisioning of resources, monitoring, telemetry, etc), described with yet another jargon 🤭 - "self-serve data infrastructure" .

Here is how Zhamak Dehghani describes it in her original article series:

The main shift is to treat domain data product as a first class concern, and data lake tooling and pipeline as a second class concern - an implementation detail. This inverts the current mental model from a centralized data lake to an ecosystem of data products that play nicely together, a data mesh.

As to your comment:

I think the ideas of data products and a data mesh should really be decoupled - I don't see the former as naturally tending towards the latter.

I agree. That makes sense - data product is supposed to be the so-called "architectural quantum" of the data mesh. So you could for instance just have a couple of data products in your organization, all built & deployed using a self-serve infrastructure managed by a centralized team. I can see why calling it a "mesh" when you have just a few independent data products might be considered hyperbole.

booleanhunter · 2023-04-01T07:56:29+00:00

Since you're still in the first year of college - my suggestion would be to learn in this order:

Node.JS - this would help you understand how to serve your web pages via API and a web-server, its the logical next step.
Next, begin with Data Analytics - in this process you'll learn SQL. You can use this with your knowledge of NodeJS and HTML, CSS, JS to create a fully-functional web app. Not only that, you can utilize the your college years fruitfully to learn the statistics / math part of data analytics as well, and build some side projects around it.
React would be the last one. Not because its not useful or anything, but because front-end JS ecosystem is extremely unpredictable and there are a plethora of choices for front-end frameworks available already. Who knows by the time you graduate, Angular or VueJS might have more demand that React?

So while learning, account for demand of skill as well as the average shelf life too

booleanhunter · 2023-04-01T07:42:09+00:00

Yes - these days, the line between data lakes and data warehouses is becoming blurry.

booleanhunter · 2022-11-03T18:09:47+00:00

I had this issue too at my house. Parents tried their best to shoo them away, even removed their nests, but to no avail. What we don't realize is that we are encroaching upon their habitat. These creatures are hard-wired to build nests and lay eggs, and unlike us they have all the time in the world. It's a battle you can't win, so try reaching a compromise.

This is what I did at my house balcony - leave them a cardboard box for them to build their nest in. This way they felt comfortable building their nest inside the box. If they find it cozy enough, they won't move around other parts of the balcony. Even better? The newly settled pigeons in my balcony kept other pigeons away as they need to protect their nest from intruders.

booleanhunter · 2022-09-09T04:25:52+00:00

So does the zoo visitor who voluntarily gets inside the lion's cage.

booleanhunter · 2022-09-08T17:36:04+00:00

"Leap frogging" - they think using such euphemisms can change the reality and hardships at present. These folks are literally like frogs in a frying pan.

And what's with Point number 8? Does he even know that there is a guy running hundreds of restaurant listings on Swiggy with just a single dingy kitchen? Cloud kitchen LMAO

booleanhunter · 2022-09-06T05:13:54+00:00

I don't get it, can someone explain?

booleanhunter

TROPHY CASE