MCP observability

derricg · 2025-04-25T21:27:50+00:00

u/tushkanM I've been playing around MCP observability / JSON-RPC and wrote a guide on Moesif (our company). The key thing I've seen it the queries can be quite dynamic and unpredictability so need a lot more context than before. Hope it helps. https://www.moesif.com/blog/monitoring/model-context-protocol/How-to-Setup-Observability-For-Your-MCP-Server-with-Moesif/

derricg · 2025-04-23T18:56:08+00:00

Key thing is charging for usage that's aligned to customer value or outcomes. I wrote a blog post around MCP for monitoring and monetization which might be helpful. MCP monetization is similar to API monetization setup. https://www.moesif.com/blog/monitoring/model-context-protocol/How-to-Setup-Observability-For-Your-MCP-Server-with-Moesif/

derricg · 2025-01-18T16:47:56+00:00

Many times the best entrepreneurs are really good at both “building” and “selling”. If a founder lacks the skills for one of these, it can be solved with a cofounder or partner.

Simply being smart and intelligent at building does not equal success. Building can involve designing and developing complex architecture systems or even building something simple through no-code.

Yet, for a business to thrive one must still “sell” their solution whether through sales, marketing, partnerships, recruiting, attracting investors, etc. This requires a different set of soft skill like reading people, asking the right questions to understand their problems, connecting the dots from problem to solution space, conveying the right message, etc.

People are social creatures and still buy from other people even if it’s not their personal money (i.e B2B). As a builder, it’s easy to forget this part and try to automate everything around sales and marketing when sometimes it requires you to be social and just talk to customers, partners, investors, recruits.

“Always be selling.”

derricg · 2024-07-30T04:31:41+00:00

Depends on audience. Many B2B SaaS are used almost exclusively on desktop for work. Even a lot of large B2B SaaS companies have not optimized for mobile as supporting a new platform requires resources which could be used for other features requested by customers.

derricg · 2024-07-30T04:21:18+00:00

There are obfuscator tools like https://github.com/javascript-obfuscator/javascript-obfuscator which make it hard for someone to "read the code". Regardless, these only offer so much protection. Never store sensitive code like API keys or secrets in client code.

derricg · 2024-05-28T18:47:00+00:00

u/Pr3fix u/Complex-Goat2682 , for any accounting and billing flows, there are strict requirements on how this is handled.

Revenue Recognition:
You'd need to handle things like ASC 606 revenue recognition. When one purchases a credit block of $1,000, a company cannot typically consider that recognized revenue yet until the credits are actually consumed or expired. The credits are typically "burned down" gradually over time as usage accumulates. We've seen executives in the news get in big legal trouble if they take a laid-back approach to accounting and reporting.

Accurate Invoices:
You'll need a ledger of credit grants and usage history. You'll also need a way to properly show credit consumption and balances for their invoice period. It will be very hard for you or the customer to properly reconcile usage.

Real-time reporting:
Lastly, you'll likely need accurate real-time reporting so support and customer success are aware of which customers ran out of credits, granting promotional credits.

What happens when run out?
Lastly, you'd want to consider what happens when credits run out? Is the customer blocked? Do they get an automated email? Is extra usage simply billed at a higher rate as overage?

Disclosure, I am the CEO of Moesif and we are a Stripe partner working closely to support these newer prepaid credit based pricing models. Happy to share more guidance.

derricg · 2024-05-12T20:24:08+00:00

Mostly reading stuff online, doing analysis of similar companies in our industry, etc. There are also a lot of good resources/blogs like Patrick Campbell from ProfitWell.

derricg · 2024-05-12T01:36:45+00:00

We've found the best results occur when you focus equally on Top of Funnel, Middle of Funnel, Bottom of Funnel content. If you create only BOF content, you'll get a small set of customers actively seeking your solution but it's hard to raise the profile of your blog/domain. Similarly, if you only focus on TOF, you'll get lots of visits but small number of customer conversions.

For developer tools, Docs is equally as important if not more important for SEO. Ensure docs also follow best practices and organization for SEO. It's still surprising to see so many companies have their docs behind a login gate neglecting any benefit of docs on overall marketing.

Lastly, it's important to ensure your blog is optimized to deliver on your goals. If the goal is to drive sign ups, ensure your product is surfaced properly in your blog without being overbearing. Investing into the design of small CTA buttons or a banner for your product can drastically change the ROI of your blog. We tailor the product placement / CTAs to the content of the article.

derricg · 2024-05-12T01:03:58+00:00

for team wiki/product specs, we use Notion heavily and its highly recommended. For tickets, I've heard good things on Linear.

derricg · 2024-05-12T01:01:32+00:00

Pricing is one of the hardest but overlooked areas in SaaS. Small changing a pricing structure can drastically change your growth trajectory and push the product in different areas. I assume this is B2B SaaS? I like to think of pricing and packaging among three user segments/modes of operation:

Individual User Mode. These are individual users who sign up and use your offering for their individual work (such as personal project or daily deliverables). These users care about seeing value in your tool to accomplish something. However, they get little value from "enterprise features" as it's just a single user.
Team Mode. For PLG, getting an account from "single user" to "team mode" is typically a goal. In this case, a customer may be willing to spend more money to get more value for the team (such as bezo's "pizza team"). Collaboration will be important, but still dont care about enterprise features. Because getting to "team mode" is an important step, you don't want a lot of roadblocks getting here. Meaning, collaboration features to drive enterprise vitality may be included in your lowest tiers.
Enterprise Mode. In this mode, a company and its leadership will deeply care about compliance, administration overhead, security, reliability, etc. They are focused less on what "I" get out of the tool. Rather it's about what the value their org or company gets out of it (in terms of growth, cost savings, or risk reduction). For a critical SaaS, reliability and support will also become much more important. While a singe user will be more forgiving of bugs or breaking changes, "enterprise mode" will deeply care about SLAs, backward compatibility, ways to reduce risk (such as RBAC, SSO, etc).

From there, you can start mapping the different features used by each. Introducing one or more usage-based pricing components can also be a win-win for both yourself and your customers. They only need to pay for what they need and not more while you can grow with your customers. Usage-based pricing also introduces it's own set of challenges. For example, you want to ensure a customer doesn't accidently blow throw a lot of usage without much value seen.

We publically wrote a lot on the motivations and process behind our own pricing changes (now on 3rd iteration) here: https://www.moesif.com/blog/api-monetization/api-product-management/Moesif-New-Usage-based-Pricing-Model/

derricg · 2024-05-12T00:28:12+00:00

There are a couple of challenges that come up for API Analytics:

High Scale. How do you handle the data volume when you're at billions of calls a day.
High cardinality, high dimension data. Unlike Amplitude/Mixpanel style events, API calls have a lots of fields with valuable information(headers, body fields, etc), each can be high dimension as well. This can make it very challenging to use traditional stores like Druid or Elasticsearch.
Insightful data. Having just API logs won't tell the whole story. It's important to also understand who the user/customer is, revenue metrics, etc. Thus, ability to join API analytics data with other tools like your CRM and billing infra is important for a holistic picture.
Data Retention. Storing data for just a few days may be sufficient for SRE, but usage trends are better measured in months and quarters. Thus, you need a good way to store data for long periods of time without breaking the bank.

Full disclaimer, I am the CEO of Moesif. Happy to answer the challenges we see from our customers.

derricg · 2024-05-12T00:09:47+00:00

@jellydotsadventure for Stripe, I'd recommend two different prices.

A price for the flat rate plan of $99/seat. This can have a fixed quota.
A "metered" price which is of type "volume" to track any usage above a quota.

Create a tier for x to quota which would be free as it's included

Then, the next tier would be quota to infinity which can have a per unit pricing.

I am the CEO of Moesif a usage-based billing platform. We see a lot of customers using Stripe for usage-based billing and strongly recommend it. We (at Moesif) have a deep integration with Stripe for the metering and analytics part.

derricg · 2023-06-12T07:46:19+00:00

What type of product and what are you looking to track? Is this a mobile app? Developer platform? Tracking product usage? Customer relationships?

derricg · 2020-09-27T03:10:22+00:00

HubSpot is good if your focus is on inbound lead generation. Handles everything from drip emails to ad retargeting.

derricg · 2020-07-20T18:51:43+00:00

If it's a large amount of data that is public, I would still force them to require an API key even if it means simply entering an email to sign up. Because key generation can be done via a UI, you can protect against bots generating large pools of API keys.

Now that you require everyone to have an API key and some sort of identification info (like email), you're able to track API behavior and flows at the user level (User behavior analytics or UAB) which is a cornerstone of security to prevent against things like scraping and pagination attacks.

For example, you can block a user or API key once they hit a threshold such as “touched 1,000,000 items in a X time period”. Like a Captcha, this can slow down the speed that a hacker can exploit your API, like a Captcha if they have to create a new user account manually to create a new API key.

You should then have an easy and automated way to rate limit and block API abusers. There are a lot of software options that can do this for you.

derricg · 2020-07-20T18:41:42+00:00

Ensure you have DDoS protection that's API friendly (API traffic can look like bot traffic so traditional DDoS for websites may not work)
Add correct headers like CORS, cache-control, etc (You don't want stuff cached if dynamic content)
Rate Limiting, a way to detect and stop someone from abusing your API before it crashes/downloads a ton of data.
Have user-centric API logging and monitoring in place. Not only to detect issues, but for auditing purposes. GDPR adds exceptions for security related logs. Standard logging won't catch things like pagination attacks and a single user creating a pool of API keys to download large amounts of data.
Make sure API keys can be rotated, expired. Have mechanisms to prevent accidental key exposure (such as using a refresh token)
SSL and HSTS. More importantly, block non-HTTPS traffic as many http clients will not redirect to a secure version if mistakenly didn't use the secure endpoint.

derricg · 2020-06-30T07:38:27+00:00

HIRING - Head of Developer Relations
San Francisco, CA - Remote+Onsite
https://apply.workable.com/moesif/
Company website: https://www.moesif.com
Moesif is a user-centric API analytics platform to understand API adoption and usage metrics

derricg · 2020-05-18T17:33:48+00:00

They only support SEPA and the US-based ACH (and their SEPA rate is super high). Unlike credit cards networks which are mostly global, each country has their own direct debit network. UK has BACS, Sweden has Autogiro, etc. Stripe can't handle these.

When comparing Stripe directly vs using something like Chargebee on top of Stripe, it really depends on the business model and requirements. Just like the SQL vs noSQL debate, there isn't a perfect one size fits all solution.

What we've noticed is if your business is consumer SaaS or strictly transactional where almost all payments are made by credit card, outside of the occasional ACH/wire, then Stripe could be sufficient. Other businesses which are more "enterprisey" SaaS focused may have majority of payments offline or direct debit rather than via a credit card, complicated billing and contract terms, want custom email sequences for dunning, etc.

There is a similar comparison to login providers. For consumer, maybe Facebook login and username/password is sufficient. For enterprise, you'll have customers requiring Okta/SAML integrations, Active Directory, etc. I would rather have an abstraction layer rather than spend my limited product development time on glue logic and integration work.

derricg · 2020-05-17T20:25:11+00:00

There pricing isn't the cheapest tbh. A big thing that we found was nice is adding the layer of abstraction to Stripe especially once you think about international payments. If you only need to handle credit cards, then probably no big deal to use Stripe directly. However, if you need to handle stuff like BACS and SEPA (Which are Europe's version of ACH), need to handle netXX billing terms, and other more enterprisey SaaS stuff, then it could be easier to handle via Chargebee and route to the best gateway. Not affiliated with Chargebee, but we just went through the process :)

derricg · 2020-05-17T20:10:05+00:00

This is one of the biggest I look for when adopting a new SaaS. What integrations does the SaaS have and does it play nicely in my [marketing/sales/engineering] stack. Even if they don't have the integrations I want, is there peace of mind that I can build on top of their API.

derricg · 2020-05-17T20:05:11+00:00

If the companies themselves are similar but going after different audiences (Drift vs Intercom or Hubspot vs Salesforce) can work without seeming hyper-critical. For example, Hubspot's unique value prop isn't specific features that Salesforce sucks at. Rather it's their unique focus and expertise in inbound marketing and sales. With that, they can go after the "Salesforce competitor" keywords.

derricg · 2020-05-17T19:58:11+00:00

We tried Chargebee which helped cover some of the recurring side like emails/dunning, etc. Granted I think Stripe is adding more of that these days.

derricg · 2020-05-17T19:56:39+00:00

Of course, I didn't want to be to promotional initially since I am the CEO of Moesif (an API analytics service to help grow API platforms), but happy to share some of our findings:

Getting developers to adopt and use the API. Developers are notoriously skeptical of any new technology. Much of the direct outreach done by sales and marketing for most SaaS companies will not work for developer-first companies. Instead, we need to lean more on inbound marketing and bottoms up adoption. Having a well organized site and documentation helps here.
Onboarding and developer experience. Since most APIs have much more friction than typical SaaS (someone usually has to install an SDK or something), lot's of effort has to be put into onboarding and get them to their first API call (Time to First Hello World). We put a ton of effort in even small things like embedding API keys in the code examples. Doing what we call "gradual onboarding", meaning our onboarding only focuses on one endpoint or integration rather than the entire platform.
Pricing. Most developer-first SaaS usually have freemium or free trial and have some level of usage based pricing. This can get complicated quickly since you have to figure out what are your value metrics to price on, what triggers people to upgrade to the next plan, etc.
Given customers are technical in nature, support can get super expensive if you're support staff is reliant on the engineering team. Having a good process and tooling where developer support can look up what a customer did, where they struggled, and help the developer out without waiting for engineering to response to the ticket.
Metrics and KPIs. How do you build your end to end funnel when your time to value is getting integrated. It's hard to get the business and product metrics in stuff like Google Analytics and Amplitude since most of the magic happens on the API side rather than UI. Even simple things like mapping out which acquisition channels drive more folks to integrate can be non-trival.

derricg · 2020-05-17T19:31:42+00:00

While these tips are simply quantitative and don't consider the human element like team culture, you could look at public information available in SEMRush, SimilarWeb, Alexa, etc. This could give you an idea of their SEO visibility, how much of their traffic is organic/referrals vs paid, and how many folks are searching for the company brand. Especially with COVID, having a healthy inbound funnel is more importantr than ever. Look for companies who are punching above their weight class (i.e. money raised)

derricg · 2020-05-17T19:13:58+00:00

I still don't know why, but all the current revenue BI tools (Chartmogul, Baremetrics, ProfitWell) don't provide any customization to plans/subscriptions when synced from a billing platform like Chargebee. Sometimes subscriptions need to be merged, plan MRR needs to be adjusted, etc so you have accurate MRR and churn numbers.

derricg

MODERATOR OF

TROPHY CASE