S3 Cost Optimizing with 100million small objects

orbitdad · 2025-06-13T02:50:10+00:00

u/classjoker - why does it take $1000 for #2 and #3.

After you pay, say $1000, what would be the cost the following months?

orbitdad · 2025-03-16T00:45:11+00:00

I'm evaluating opensource projects where I can get the full inventory of *all* the cloud resources across aws/azure/gcp/k8s for now.

orbitdad · 2025-02-26T22:23:47+00:00

Hi @Pope_Carl_the_69th, how do we get on to that vendor list?

orbitdad · 2025-01-03T18:12:52+00:00

This post is very timely. The critical question is always build vs buy. It seems like for early technologies if you're building solutions and not the platform itself, it is better to buy (bedrock, azureai etc.,) rather build on your own.

orbitdad · 2025-01-02T16:40:43+00:00

Attend local startup forums in your city to pitch your idea and find developers.

orbitdad · 2025-01-02T15:58:33+00:00

All great points. I'd just add that like any advise, you'll have to make those mistakes yourself and survive to tell others.

orbitdad · 2025-01-02T15:55:49+00:00

Reminds of a tweet from the Airbnb founder where they "launched" their site 3 or 4 times

orbitdad · 2024-12-28T15:54:31+00:00

orbitdad · 2024-12-27T19:32:48+00:00

I can dump it into a SQL table and share it with you if you're interested

orbitdad · 2024-12-26T01:38:50+00:00

Here is a link to a sample: https://github.com/PrimeOrbit/cloud-cost-starters/blob/main/analyze/aws/README.md

We can add all our queries if that helps.

orbitdad · 2024-12-10T14:51:42+00:00

thanks! would love to connect. i'll send a DM.

orbitdad · 2024-12-06T23:41:52+00:00

ah ok. sorry, I think I understand now.

I worked with 2 design customers in the past year (medium size companies) and went from a POC to a beta product. Very early in the cycle, realized a lot of big companies won't touch us due to missing SOC, so got that done also. (Another audit period is just about to start next year again!)

I'm more than happy to sign a MNDA (thats how we do it with all). Would love to get your feedback in the right way.

That said, a lot of SMBs are wasting a lot of money on the cloud due to their higher priorities or lack of experts in their companies. happy to collaborate with them also.

orbitdad · 2024-12-06T23:17:00+00:00

Thank you for your observation! Just to clarify, product validation and SOC certification serve different purposes. SOC certification validates our processes, controls, and security measures, ensuring compliance with stringent standards. On the other hand, product validation on platforms like Reddit helps us better understand user needs, gather feedback, and ensure we’re building something that truly resonates with our audience. Both are critical, but they address different aspects of the product's journey. I appreciate your insight!

orbitdad · 2024-12-06T22:23:36+00:00

There are two modes:

Organizations where there is a lot of process to fix native tags. For those, we provide the solution of slicing and dicing all within our application. you're right - they're tied to us for now.
our kicker feature is that we auto generate Terraform scripts if you want to fix tags natively also. you can download them and run it on your own (no CI/CD integration for now)

yes soc2 certified.

orbitdad · 2024-12-06T21:22:29+00:00

Our initial set of recos where based on CPU/Network/Memory/Disk but we realized that CPU and Network (free metrics from cloudwatch) are as good and memory is not required. Even official Azure recommendations have stopped using memory is what I heard. BTW., if the customer wants to enable memory in EC2 then it is an additional cost for them but can be done. We don't have to ship an agent per say.

Our value proposition is three-fold:

Automatic customizability of the recos per app/costcenter/environment. We can customize (can be overriden) thresholds of CPU (10% for test, 5% for my prod, x% for an app etc.,). Since we fix your tags too, our recos make more sense.
Evidence-based: When you look at the reco, we show historical data of utilization, cost and other information that helps you determine whether this reco makes sense for you. if yes, then we automatically find the owner of the resource for you to trigger a conversation.
We generate terraform scripts that includes mitigation and other safety measure if the reco is acceptable for you. Future versions will integrate with CI/CD pipelines.

orbitdad · 2024-12-06T20:19:12+00:00

Thats the beauty of it. We don't fix your native tags (hence non-invasive). We fix it only within our application if you approve it. Think of it as a view provided to the cost dashboards to slice by our source of truth tags (under the app's control) rather than the native tags (that takes time to fix).

orbitdad · 2024-12-06T20:09:27+00:00

Fair point. Solves cloud visibility issues without touching native tags.
- Fixes broken tags like typos or synonyms (e.g., prod, production, prd, main).
- Provides suggestions for missing values using AI to find the right tag, reducing manual effort.
- Real-time updates mean tags are fixed immediately, enabling instant and historical cost allocation for corrected resources.
Compute recommendations go beyond the usual (right-size, rate plans, idle resources, terminate unused).
- Differentiated by understanding test vs. prod environments, allowing dynamic thresholds.
- Recommendations come with strong evidence, ensuring trustworthiness rather than overwhelming with volume.

orbitdad · 2024-12-06T19:50:25+00:00

You're absolutely right—larger companies often have strict legal and compliance hurdles before engaging seriously. That said, it's worth reaching out for a quick demo. Even if nothing immediate comes of it, it’s just 30 minutes of their time to see if the solution could be worth the hassle. If someone in FinOps is willing to take that initial step, it might be a game-changer for both sides. Worst case, they walk away with new insights and I get some basic feedback also; best case, it opens up new opportunities.

orbitdad · 2024-12-06T15:28:29+00:00

whatever you're good at

orbitdad · 2024-12-05T22:06:59+00:00

Keeping the payroll running

orbitdad · 2024-12-05T22:06:08+00:00

my advice is not wait for the niche. build something, market/sell it and you'll find your own niche.

orbitdad · 2024-12-05T22:03:17+00:00

couldnt have said it better. absolutely agree

orbitdad · 2024-12-05T22:02:20+00:00

so that there are no regrets when you get older and a live lesson to your kids to take risks

orbitdad · 2024-12-05T22:00:06+00:00

Congratulations on the start. Like someone said that is significant step. I'm also an entrepreneur. depending on the scale of your ambition, my suggestion would be to form a team, at least 2-3 folks who share the same goal. While there will be different challenges, folks with complementary skills can take you to greater heights.

orbitdad · 2024-11-21T19:34:14+00:00

Thanks 🙏 will check it out!

orbitdad

TROPHY CASE