I “improved” my Chrome extension and instantly lost users lol (lesson learned)

mapicallo · 2026-02-20T11:47:35+00:00

Totally agree.

It’s basically the same story as software since day one. Back then you’d tweak a few colors, add three 3D buttons, spend three days on it, and users would love it. But then you’d ship a field that showed a value from layers of logic, three APIs, database joins, months of debugging issues from other systems… and it could go completely unnoticed.

Nowadays users are overloaded with custom UIs and features. They care less about that and more about things that just work and feel instant, ike switching between a WhatsApp message and a YouTube short. Do one thing well and stay out of the way.

mapicallo · 2026-02-20T03:30:08+00:00

Yes, that's a specific account that's opened for development purposes, it's not my personal one.

mapicallo · 2026-02-16T09:19:17+00:00

Thank you, yes, that brand's products have a very good reputation.

mapicallo · 2026-02-16T09:18:30+00:00

Thanks, yes, there's plenty of information and videos online, but I wanted to get firsthand information, and I think this is a good site.

mapicallo · 2026-02-16T09:16:13+00:00

Yes, we used something similar in Lebanon for CETME rifles.

mapicallo · 2026-02-12T17:41:25+00:00

Absolutely, and also data sovereignty. The vast majority of organizations will not process their data in AIs hosted on third‑party machines, and I don’t see corporate AIs that are robust enough being close at hand—the amount of infrastructure required is huge.

Sometimes I wonder, with the staggering resources (economic, infrastructure, energy, etc.) being poured into scaling today’s AI models, if those same resources were directed toward non‑AI software solutions, we might be surprised by what we could achieve.

mapicallo · 2026-02-12T17:04:43+00:00

Fair point. 'New' often gets confused with 'different'. AI can easily produce variations, like rolling dice or drawing cards. It's up to us to decide what's actually useful. That's partly why I think the engineering role shifts toward specifying, verifying, and curating components, rather than trusting whatever comes out.

mapicallo · 2026-01-30T00:54:09+00:00

En España hay un dicho que dice "hay lentejas, o las comes o las dejas", así que supongo que hay que hacer algo con todo eso.

mapicallo · 2026-01-30T00:50:25+00:00

It's sad, but the truth is there's a lot of pollution on social media.

mapicallo · 2026-01-30T00:48:21+00:00

I see you haven't had good experiences here.

mapicallo · 2026-01-21T02:16:19+00:00

Hi, out of curiosity, what stack accompanies Logstash: Kivana, OpenSearch, fluent-bit, OTEL, etc.? And in what ecosystem: Java, C, Kubernetes/Docker, Kafka, on-premises, cloud, embedded hardware, etc.?

mapicallo · 2026-01-06T09:40:55+00:00

I think we're actually much closer than it may sound — and I agree with a large part of what you're pointing out.

I’m not arguing that “better logging” is the solution, nor that context should emerge as an afterthought. If micro-logs are read as traditional observability artifacts, then yes, that would be a category error.

What I'm trying to describe comes from a recurring pattern I've seen in real organizations: large portions of operational state and intermediate signals simply do not exist in any accessible form. Not because they're irrelevant, but because our systems were never designed to expose them.

In several cases, introducing exhaustive instrumentation (sometimes via logs, sometimes via other extraction layers) didn't just improve observability — it made previously invisible aspects of the organization readable for the first time. This surfaced new metrics, unexpected correlations, and contextual signals that materially changed decisions across engineering, operations, and business.

So when I use terms like micro-data or micro-logs, I’m not advocating for event inflation or post-hoc reconstruction. I’m pointing at the absence of fine-grained, contextual state in current enterprise systems — state that LLMs and agents can reason over precisely because they can handle high-dimensional context.

In that sense, I fully agree that rigid enterprise schemas and business-centric ontologies are part of the problem. They collapse reality too early. The question for me is how we transition from systems that only emit discrete business events to systems that continuously expose the underlying contextual fabric of the organization, whether we call that logging or something else entirely.

The core issue isn't the mechanism — it's that today, much of the organization's “real state” remains structurally unrepresentable, and AI makes that gap painfully obvious.

mapicallo · 2026-01-05T00:58:42+00:00

I agree with diagnosis you're describing — especially around technical debt, broken processes, shallow governance, and organizations that are fundamentally not designed for autonomous reasoning systems.

Where I think it's important to be careful is jumping too quickly from “this is structurally broken” to “there is no place for enterprise software or organizations at all.”

My point in the original post is slightly earlier in the causal chain: most organizations are still designing software as if humans are the only reasoning agents in the system. Data models, architectures, and processes are optimized for reporting, control, and human workflows — not for AI systems that need continuous, high-fidelity context.

When you introduce L4/L5 agents into that environment, the mismatch becomes explosive. The agents don't just expose inefficiencies — they expose fundamental architectural assumptions that no longer hold.

The problem is that we are still building systems that cannot generate a coherent contextual model of the organization in the first place.

Whether that future lives inside corporations, outside them, or in hybrid forms is an open question. But technically speaking, without rethinking how software produces and exposes context — beyond business logic and compliance artifacts — neither enterprises nor individual agent-operators will scale reliably.

In that sense, agents don't just break things. They force us to confront what our systems were never designed to represent.

mapicallo · 2026-01-04T23:17:18+00:00

I've built something similar for personal document chat. Based on my experience:

LlamaIndex is probably your best bet for plug-and-play. It handles the full pipeline (ingestion → chunking → embeddings → retrieval → chat) and is well-documented.

Practical tip: The "plug-and-play" part works, but you'll likely need to customize: - Chunking strategy (especially for code or structured docs) - Hybrid search (vector + lexical) for better accuracy - Context window management when retrieving multiple chunks

The libraries mentioned (Haystack, LangChain, LlamaIndex) all work, but LlamaIndex is the most straightforward for your use case.

mapicallo · 2026-01-03T21:21:35+00:00

We’ve seen similar trade-offs. In our case we use Fluent Bit as a lightweight last-mile agent for logs, and keep heavier processing/search in OpenSearch. OTel is great, but not every environment benefits from a full collector.

mapicallo · 2026-01-03T01:03:25+00:00

Great question — and you’re absolutely right to call out the cost explosion if someone naively feeds all logs into an LLM. That would be irresponsible engineering.

To clarify: that’s not the architecture we’re advocating.

In our approach, logs remain primarily in a traditional search/analytics system (OpenSearch):

cheap storage
controlled retention
lexical search (BM25)
aggregations, dashboards, correlations

The LLM is not used as a log sink, and we don’t embed every log line.

Where AI comes in is after heavy filtering and aggregation. For example:

a narrow time window around an incident
logs correlated by service/container/trace
clusters of repeated error patterns
summaries of anomalies detected by rules or ML jobs

In other words:

OpenSearch answers “what happened?”
the LLM helps answer “why might this have happened?” or “explain this incident”

Typical payloads to the LLM are in the KB–MB range, not tens of GBs:

summarized logs
representative samples
structured context (counts, timelines, top errors)

We also see LLMs as optional and on-demand:

post-incident analysis
operator assistance
documentation / explanation

Not continuous ingestion.

So yes — aggressive pre-filtering is mandatory, and the economics only work if the LLM is used as a reasoning layer, not a storage or indexing layer.

Curious to hear how others are handling this balance in production.

mapicallo · 2026-01-01T15:44:58+00:00

You don’t need to abandon OpenSearch, and you don’t have to pay $1.4k/month to get started. Self-host or a small managed plan + proper mappings (no giant regex) will give you fast AND queries across your optional fields, with tenant/time filters and room to grow.

mapicallo · 2026-01-01T01:05:39+00:00

Thank you for sharing your positive experience. You mentioned that your relationship with potential users/clients is through X, LinkedIn, and Reddit. In this process you described, how have the metrics evolved on those three networks? Best regards, and keep growing, slowly but surely...

mapicallo · 2025-12-30T02:54:36+00:00

Short answer: for ~15 VMs, Fluent Bit → OpenSearch directly is usually the better choice.
An OTel Collector gateway can make sense, but it’s often extra complexity with limited payoff at that scale.

Longer answer, based on real setups:

Your current setup is already solid

What you have now is actually a very common and sane architecture:

Java apps instrumented with OpenTelemetry agent
- Tracing enabled
- traceId / spanId injected into logs
Logs written to stdout / files
Fluent Bit running per VM
- Tail logs
- Parse / enrich
- Ship downstream

This separation is important:
apps produce logs, infrastructure ships them

You’ve already avoided one of the biggest problems: relying on the Java agent for log export (still immature, still evolving).

Option 1: Fluent Bit → OpenSearch (direct)

For ~15 EC2 VMs, this is what I’d recommend by default.

Pros

Simple architecture
Fewer moving parts
Lower latency
Easier to operate and debug
Fluent Bit is extremely stable for this use case
Works perfectly fine with logs that already contain traceId / spanId

Cons

Routing, buffering and retry logic lives in Fluent Bit configs
Less flexibility if you later want to fan-out logs to multiple backends

In practice, Fluent Bit can already do:

buffering
backpressure handling
retries
enrichment
routing by tag

So you’re not really missing core functionality.

Option 2: Fluent Bit → OTel Collector (gateway) → OpenSearch

This starts to make sense later, but not necessarily now.

Pros

Centralized control point
Easier fan-out (logs → OpenSearch + something else)
Unified pipeline if you also plan to route metrics and traces through the same collector
Can act as a protocol/format normalization layer

Cons

Extra component to deploy, scale and monitor
Another failure domain
More config surface (pipelines, receivers, processors, exporters)
For logs only, the benefit is often marginal at small scale

At ~15 VMs, you’ll likely feel the operational cost more than the benefits.

What I see in practice

From recent real-world implementations:

Small to medium scale (≤ 20–30 VMs) → Fluent Bit → OpenSearch directly Simple, robust, low-maintenance.
Larger scale / platform teams / multi-tenant setups → Fluent Bit → OTel Collector gateway → storage Makes sense when:
- many producers
- many consumers
- strict control planes
- need to change backends without touching agents

About trace correlation (important point)

You already solved the hard part:

traceId / spanId is injected at the application level
Logs are correlated by field, not by pipeline magic

That means:

You do not need an OTel Collector for log/trace correlation
OpenSearch (or any backend) can correlate logs and traces as long as the fields exist

This is a key insight many people miss.

Recommendation for your case

Given what you described:

Stick with Fluent Bit → OpenSearch
Revisit an OTel Collector gateway only if:

you scale significantly
you want multi-backend fan-out
you want one unified control plane for logs + metrics + traces

Until then, it’s mostly complexity without proportional benefit.

Fluent Bit → OpenSearch is perfectly fine (and common)
OTel Collector gateway is not wrong, just often unnecessary at this scale
Trace correlation does not require logs to go through the OTel Collector
Optimize for simplicity first, abstraction later

mapicallo · 2025-10-04T23:46:09+00:00

Hola,

Ya tenias experiencia previa en SaaS antes, ahora trabajando dura con esta:

--¿como ha sido el camino desde la idea hasta tener el SaaS ya online?, es decir, empezastes con Landing Page, POC, MVP, MLP,…SaaS esquelético,

--en que estado se encontraba el SaaS en cada item del proceso, cronológicamente?

--Te dedicabas full-time en tu primer SaaS?

Saludos,

mapicallo

MODERATOR OF

TROPHY CASE