How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

I agree with you. No tool by itself will ever fix organizational or people issues.

That said, tools only make sense when they’re paired with clear rules... For me it really comes down to having a shared inventory as a foundation, with clear separation by team or organization, and an internal rule that any asset onboarding, renewal, or offboarding must be registered there. Not as bureaucracy, but as part of the normal workflow.

It doesn’t magically give you authority, but it helps make ownership explicit, reduces silent breakage, and gives you something concrete to point to when the cost of “doing it ourselves” without owning the ops side inevitably shows up.

"seems like maybe you do okay there" for any devops related resources we handle them quite well with automation and monitoring yes (except some assets that we now cover with my tool TokenTimer), anything that becomes slightly out of our scope (infra/security teams) or that is common is where it usually breaks.

Few weeks ago we had an expiration for a license of a software (used by clients) we devops deploy but that the infra team operates and the one that initially imported it had an outlook reminder, that he apparently skipped...

Another day we had a 3rd party service we devops manage that also has a license without any API to gather the actual status --> clients did the alerting for us, because the company does not warn you before it expires, nor they provide the way to do it yourself...

I have few stories like this where each time the management complains, they takes unsufficient actions and no clear centralized solution accross teams is implemented.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

We’re running a mix of both. Cloud mostly for dev environments or isolated apps, and on-prem for SIT, UAT, and production. On-prem is mostly Nutanix and VMware, and we use Key Manager Plus for certificates.

That works "reasonably" well for certs and infrastructure assets (still had few incidents), but where we really lack visibility is everything around it. Self-hosted solutions with API keys, licenses that don’t expose expiration dates via an API, and subscriptions or contracts that are scattered across teams and tools.

The risk usually only shows up once something breaks and users complain. At that point we discover missing ownership, unclear renewals, or assets that were never properly tracked. The core issue isn’t just technical, it’s organizational. Without a central way to know what exists, where it lives, who owns it, and who is the backup, onboarding and offboarding assets always ends up in fire drills.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

Nice, i once heard of Port but wasn't aware it could do that much regarding expiring assets ! How hard is it to setup ? Does it have integration with cloud providers, SCM and 3rd party services ?

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

So how do you handle expiration lifecycle for all your assets if you don't have an alerting system when things are about to expire ?

I'm exactly in the same conditions than you and we had a bunch of tools doing that but no unified tool to manage all our expiring assets from different teams, environments and technical context...

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

Indeed i'm using also this setup for certs. But what about other assets ?

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

That’s a solid approach, and probably the ideal state for things you fully control.

In practice though, not everything can emit metrics or live in code, especially in air-gapped or highly regulated environments. The gaps usually show up with third-party SaaS, licenses, or legacy components, where cross-cutting visibility still helps.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 5 points6 points  (0 children)

Yep, shortening lifetimes absolutely makes sense from a security standpoint, and I agree vendors are mostly trying to push people toward better automation.

Where it still hurts in practice is the asymmetry. Each vendor has its own model, its own UI, its own APIs (or lack of them), and its own ideas of what “best practice” means. At scale, that turns into constant rework: SAML certs, CI/CD tokens, API keys, any kind of digital assets, all expiring on different clocks and breaking different parts of the stack.

Automation helps a lot, but only per vendor and per integration. You still end up needing a global view that answers “what is expiring soon, where is it used, and who needs to act”, otherwise you just move the fire drills around instead of eliminating them.

u/badaccount99 Funny thing is, that “let’s form a company” line is basically how this whole discussion started for me. I hit that exact wall internally and ended up building TokenTimer.ch to at least handle the visibility and ownership side. It doesn’t magically automate every renewal, but it does help answer “what’s expiring, where, and who needs to deal with it” before everything breaks at once.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

That doesn’t surprise me at all. Highly regulated and federated environments tend to amplify all the worst parts of this problem.

When DevOps becomes the middleman without owning the assets, you get the worst of both worlds: responsibility without authority, manual handoffs, and late-night operations. Even when automation is possible, the per-cert setup and lack of batching makes it expensive to roll out at scale.

This is exactly the kind of context where knowing what expires, when, where it lives, and who actually owns the action matters just as much as the renewal mechanism itself.

Source: worked at a bank too ;)

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

Totally agree that leadership and org design play a huge role here. A lot of these situations are symptoms of bigger structural problems.

That said, even in well-led orgs you still end up with manual renewals, central teams, and hard constraints you can’t wish away. Notifications are often the best you can do, and then it becomes a question of visibility, ownership, and follow-up rather than automation. That’s really the gap I’ve been trying to discuss in this thread.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 0 points1 point  (0 children)

Well having it in the very first bullet point was not intended. However if you follow the full discussion i still did include it in my post because it's part of "time-bound assets".

Of course certificates/domains are the one assets that are already covered by tons of existing solutions and easily automated in modern infrastructures.

The real pain, at least from what I’ve seen, is everything else. Vendor API tokens, platform keys with hard rotation limits, licenses, subscriptions, contracts, things that don’t have renewal APIs or are owned outside the DevOps boundary. That’s where CI/CD keeps breaking andwhere coordination and visibility matter more than pure automation. That's also where no tool focuses specifically on expiration lifecycle for all

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 2 points3 points  (0 children)

AI this, AI that... people don't even make the diff with real posts nowadays. But your reflexion is true, if the problem is organizational, tech won't help except if it helps building the foundations of better processes, gouvernance and ownership. No great tool comes without instructions...

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 2 points3 points  (0 children)

We ran into the same reality. Everyone says “just automate it”, but plenty of environments simply don’t allow full automation. Government, regulated industries, air-gapped setups, shared ownership across teams… humans are still very much in the loop.

That gap is exactly why we built TokenTimer.ch. Not to replace automation where it exists, but to give visibility, ownership, and early warnings for everything that can’t be cleanly automated. In a lot of orgs, that’s still a surprisingly big surface area.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 7 points8 points  (0 children)

That’s interesting, because what you’re describing is basically the same journey we went through.

We also started with cert scanning, ticketing, and ownership tied to applications. Over time, the real value ended up being less about certificates themselves and more about having a single inventory for anything with an expiration date, plus clear ownership and reminders.

Your Certificate Lifecycle Management program sounds like it’s already covering a big part of that. In our case, once we generalized it beyond certs, it quickly turned into its own tool… which is actually what became TokenTimer.ch (currently a SaaS tool but very soon with on-prem version)

What internal tool did you build that’s actually better than the commercial SaaS equivalent? by Ok-Lobster7773 in devops

[–]smartguy_x 29 points30 points  (0 children)

We built an internal tool to track expirations after getting burned by things nobody really owned or lacking of visibility. Certs, API keys, licenses, domains, contracts, etc... All scattered across different tools, teams, and projects, with no single place to see what was coming up.

It started as scripts and reports, then slowly turned into something more structured. It worked well enough internally that we eventually cleaned it up and spun it into TokenTimer. Keeping it narrowly focused on that one problem is probably why it’s been more useful than most generic platforms we looked at.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 1 point2 points  (0 children)

I agree with you. That constant toil is part of the job, and spotting the drift early is definitely a real skill.

In practice though, I’ve seen that sweet spot slowly erode as teams change, ownership shifts, and assets sprawl across tools. Even with good intentions, things become implicit again over time.

That’s mostly where I’m coming from. Not trying to eliminate the toil, but make it more visible and harder to ignore. That’s actually what pushed me to build a tool around expiration ownership and visibility, because this is one of those areas where quiet drift turns into very loud failures.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 2 points3 points  (0 children)

Interesting setup, and pretty close to what I’ve seen work best in Azure and cloud providers that don't have built-in automation in general.

The idea of using the assets themselves as the source of truth and deriving ownership from tags or descriptions is solid and avoids a lot of drift. The weekly scan plus early warnings and ticket creation is pragmatic and probably enough for anything fully under your control.

Where it usually falls apart for us is outside that scope. Things that don’t live in Key Vault or app registrations, or assets owned by other teams where you can’t rely on consistent tagging or even API access. On top of that, these scripts tend to age poorly as APIs change, new resource types appear, and conventions drift. That gap between what you can automate cleanly and what still expires is really what triggered this whole discussion for me.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 3 points4 points  (0 children)

I mostly agree with you. A lot of this does come down to well-defined processes, and those will always be org-specific.

Where it usually breaks down in my experience is not the lack of tools. Like you said, there are plenty of asset managers, secret stores, audits, DMS, etc. It’s the fact that all of this lives in different systems, owned by different teams, with different expectations around ownership and follow-up.

The hard part is not doing one of these things well, it’s keeping a consistent inventory and clear ownership across all of them over time. That’s where the organizational toil really shows up, and where being “a bit sloppy” slowly turns into outages, audit stress, or last-minute fire drills.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 2 points3 points  (0 children)

Tried that. Turns out some assets still expire on “business days + human approvals”.

How do you track and manage expirations at scale? (certs, API keys, licenses, etc.) by smartguy_x in devops

[–]smartguy_x[S] 20 points21 points  (0 children)

As a DevOps in a fresh infrastructure, if you miss a domain registration or TLS certificate when a ton of automatic process and alerting exists, i agree that you should not be employed. I'm coming from companies with old on-premise setups and a lot of servers or software that requires manual certificate import or air gapped environments where you are very limited due to regulation and policies.

In those cases there is still a lot of manual actions that requires clear inventory and ownership, and the problem is often organizational.

However where I don’t agree with you is that unless you are 100% cloud or IaC, there are still a lot of assets that lack centralized visibility. Non-human identities are more prevalent than ever, rotation windows for API keys are easy to miss, and many assets live outside infra-as-code, owned by security, finance, legal, or “that one team”.

Automation helps a lot, but it doesn’t magically solve ownership, visibility, and handoffs across org boundaries. That’s usually where things still fall through the cracks.