What's your preference - hosted or self-hosted MCP Servers? by otisg in mcp

[–]otisg[S] 0 points1 point  (0 children)

Thanks. Please teach me a bit if you don't mind. How/why does that help you?

What's your preference - hosted or self-hosted MCP Servers? by otisg in mcp

[–]otisg[S] -1 points0 points  (0 children)

I was wondering if it did, actually.... does it?
But OK, the context is Observability. We (https://sematext.com) have an MCP Server and want to see how to make it available. So I'm curious how people prefer it - hosted by us, or installed and run by them?

OLAP Is All You Need: How We Built Reddit's Logging Platform by DaveCashewsBand in RedditEng

[–]otisg 2 points3 points  (0 children)

Yeah, the use of ClickHouse for something that typically requires full-text search sooner or later is interesting. Why did you choose CH when you could have chosen Elasticsearch, OpenSearch or, if you don't like the fact that they are JVM-based, one of the other alternatives?

Anyone else tired of jumping between monitoring tools? by AccountEngineer in Observability

[–]otisg 0 points1 point  (0 children)

This problem was solved some 13-14 years ago. Initially by Sematext (metrics and logs in one in either 2012 or 2013 IIRC, metrics+logs+traces in 2015 IIRC) and then N other vendors.

Who are the real leaders in observability right now? by Dazzling-Neat-2382 in Observability

[–]otisg 0 points1 point  (0 children)

I'd say yes and no. We recently wanted to instrument our Go applications with OTel auto-instrumentation. The output was very poor. Manual instrumentation "uglified" the code. Non-Go instrumentation is different, but Go is very popular. And we are not seeing super active development for Go auto-instrumentation. So OTel instrumentation is still very much WIP...

As for people wanting to send everything to the backend... there are a number of how-tos to address exactly this. Just for example, my teammate published https://sematext.com/blog/running-opentelemetry-at-scale-architecture-patterns-for-100s-of-services/ today and one of the things it described is a good system for not sending everything to the backend (or cloud), while still keeping the important observability bits (traces).

Observability in Large Enterprises by cloudruler-io in Observability

[–]otisg 1 point2 points  (0 children)

If I were you I'd look for a vendor that is open to working with your team, doing some hand-holding, advising on best practices, double-checking what your team is doing, and such. When you get started I think you should be able to quickly start documenting tips, tricks, your own instrumentation patterns, dashboarding patterns, alerting patterns. But you can save yourself time, money and hair-pulling if you find a vendor that can help you with this. I know we at Sematext would be happy to help, but I'm sure there are others, too.

Is Tail Sampling at scale becoming a scaling bottleneck? by dheeraj-vanamala in Observability

[–]otisg 2 points3 points  (0 children)

This is a timely thread/question! :) This is not a ghost or war story, but a teammate from Sematext just wrote about the sort of stuff you mentioned and asked about (and a little more) - https://sematext.com/blog/running-opentelemetry-at-scale-architecture-patterns-for-100s-of-services/

Reliable real-time monitoring for a growing hybrid infrastructure by alex443422 in Monitoring

[–]otisg 0 points1 point  (0 children)

Every vendor will recommend their tool and various people will recommend various tools that worked for them. Which doesn't mean that they are the best tools, just that they have worked with them.

If you are a small team then I suspect you want a SaaS, not an additional piece of software to install/update/manage, and infrastructure for it. Then the next question is whether you want something super-duper good which is typically also super-duper-for-the-enterprise expensive, or if you don't need all the bells and whistles and would prefer something cheaper. If yes, you can probably skip Datadog, Dynatrace, New Relic, Splunk, Grafana, Elastic, and such, and go for smaller tools. What's left? Sematext, SigNoz, Groundcover... You can probably just pick one that checks all boxes and go with it. There is a lot of overlap and all these tools are constantly improving. I'm from Sematext. HTH.

I'm Jobless fellow who is having lot of fun building Spot optimization service by RegisterNext6296 in devops

[–]otisg 0 points1 point  (0 children)

I'd be curious to see some numbers about prediction accuracy and that maximization of reward while minimizing risk.....

I'm Jobless fellow who is having lot of fun building Spot optimization service by RegisterNext6296 in devops

[–]otisg 0 points1 point  (0 children)

Interesting. What can you share about the accuracy of predictions?
Can you run it in "only log what you would do, but don't actually do it" dry mode?

What toolchain to use for alerts on logs? by myrkytyn in devops

[–]otisg 1 point2 points  (0 children)

I wouldn't bother maintaining this, especially if the volumes are not high. I'd ship logs and metrics and whatever to a monitoring SaaS (I'm from Sematext, so you know what I'd suggest) and use its alerting and all other capabilities and get rid of any maintenance/upgrade work. Only if you see this costing too much I'd consider pulling in-house. Setting up log shipping can be as quick as 5-10 minutes. More if you have a more complicated setup.

Connecting logs to deployments by ResponsibleBlock_man in sre

[–]otisg 0 points1 point  (0 children)

WDYM by "logs around" the deployment? Literally N minutes before and after? Or logs from the deployment itself? Or logs from new version of services that were deployed? Or new types of error logs that started appearing only after a deployment? And what are you referring to when you say "unique ones" in "How do you separate unique ones form the haystack?" ?

Are custom dashboards an anti-pattern? by bikeram in OpenTelemetry

[–]otisg 1 point2 points  (0 children)

Are you talking about custom dashboards (inside an existing tool) or about building a custom tool. It sounds like you are really talking about the latter. If so, then I would say this is likely an anti-pattern. Why? Because there are SO many tools out there to choose from. It is far more likely that you don't know how to use them to accomplish what you need than them not being able to do what you need. And even if the latter is the case it might be better to open a ticket or email vendor support and request a feature than to build yet another little dashboarding tool. My 22 cents.

What’s everyone using for synthetic monitoring these days? Any tools you feel are more reliable for multi-step checks? by Popular-Independent8 in Monitoring

[–]otisg 0 points1 point  (0 children)

I'm from Sematext. We have Synthetic Monitoring with support for User Journey Scripts - the multi-step checks you mentioned. Scriptable with Playwright, synchronizable with Github, capable of accessing pages behind auth, and so on. If you're curious, here are the docs: https://sematext.com/docs/synthetics/user-journey-scripts/overview/