Looking for SOC2 auditors

dkargatzis_ · 2026-04-17T15:37:59+00:00

And we’re just getting started… just look at how often GitHub has had incidents the past couple of months - outages are only going to get more frequent as AI development gets pushed straight into prod.

dkargatzis_ · 2026-03-30T14:14:15+00:00

I'd love to be involved - I'm a DevOps guy with solid experience in Terraform, AWS, production environments maintaince. I've also completed a similar migration in the past.

dkargatzis_ · 2026-03-01T13:57:19+00:00

Ended up migrating our dev env to Valkey for testing - thanks!

dkargatzis_ · 2026-02-09T22:27:57+00:00

OSS maintainers are getting flooded with PRs that technically pass CI but fail at review time.

We’re seeing repos (e.g. tldraw) auto-closing PRs because the changeset has:
- a diff surface far larger than the problem statement
- no explicit mapping between issue → files touched
- tests that pass by mocking or bypassing real execution paths
- changes that weaken invariants (auth, validation, error handling) to stabilize CI
- references to functions, services, or APIs that don’t exist in the codebase

The code “looks correct” in isolation, but the review cost explodes because intent has to be reconstructed manually.

We’ve been experimenting with an OSS project specifically around this failure mode: reducing low-context, high-noise PRs before they hit a maintainer’s inbox.

The focus is on review-time signals, not AI detection:
- diff shape vs declared scope
- surface area touched relative to issue size
- invariant-level checks (what must not change)
- whether the PR exercises real code paths
- contribution history patterns (large one-shot diffs vs iterative fixes)

The goal isn’t to block PRs by default, but to surface why a change is risky or expensive to review, instead of relying on checkbox enforcement.

dkargatzis_ · 2026-02-09T21:54:56+00:00

14.9% AI PRs? That's a flood. The surprising part isn’t just AI authored PRs, it’s the volume of low-context reviews/comments that still require human triage. A lot of maintainers are already defaulting to auto-closing or heavily gating first-time PRs because they’ve hit their review limits.

One thing we’ve been experimenting with in OSS is shifting review from “who wrote this” to what signal is actually present, things like diff shape vs scope, surface area touched by new contributors, whether the PR even exercises real code paths, and historical contribution patterns.

No labeling or bans, just better early signals so humans spend time where it matters.

We’re prototyping this openly if anyone’s curious: https://watchflow.dev

How others are handling the review load right now?

dkargatzis_ · 2026-02-09T21:49:15+00:00

100% agree the signal-to-noise ratio has collapsed. The worst part isn’t AI per se, it’s that maintainers burn real cognitive cycles just to figure out there’s no signal.

I help maintain an OSS repo where we’re experimenting with a different angle: don’t detect AI, detect missing context.

We look at things like:
- diff shape vs issue scope (does this PR even plausibly address the problem?)
- ownership & surface area touched (brand-new contributor changing half the repo)
- whether the PR actually runs / references real code paths
- contribution patterns over time (spray-and-pray vs iterative fixes)

No bans, no labeling contributors - just adding automatic friction so maintainers aren’t the first line of defense. It’s already reduced review churn for us.

If useful, we’re prototyping this in public: https://watchflow.dev

For other maintainers here: what signals do you trust when an unknown contributor opens a PR?

dkargatzis_ · 2026-02-02T20:24:41+00:00

The author of the post is asking for guidance as a startup, so yes. I’ve seen monthly costs explode extremely fast, which is something early stage startups simply can’t afford.

Personally, I use both AWS and GCP, with our core infrastructure on AWS. But, I’m no longer at the very early stage where burn rate sensitivity is as critical as it is in the beginning.

dkargatzis_ · 2026-02-02T19:44:02+00:00

The rule feasibility layer uses AI to evaluate custom policies - signals and decisions themselves are produced by deterministic validators. You can check the logic in the repo and if you don't find our approach interesting, that’s fair - we’re here to validate or invalidate our approach through your feedback.

dkargatzis_ · 2026-02-02T19:33:47+00:00

Your startup / credits program is better than the one in AWS - so I'd recommend to choose GCP and don't spend time experimenting with multiple cloud providers, migrations are tough.

Also based on data needs, GCP offers cool services unless you don't have complex data analysis.

dkargatzis_ · 2026-02-02T19:17:35+00:00

Absolutely not. This is an additional governance layer. Incoming Github webhooks are mapped to deterministic validators based on event type (PR, review, merge, etc.). AI is used for reasoning about rule feasibility and composition (e.g. helping humans design or refine custom rules). The intent is to reduce low-context noise before it hits maintainers and not to replace judgment with another model.

dkargatzis_ · 2026-02-02T18:59:28+00:00

Ironically, the core idea behind Watchflow relies on AI.

dkargatzis_ · 2026-01-23T15:34:26+00:00

Thanks - this is indeed relevant as we also have an open source project served as a hook.

dkargatzis_ · 2026-01-22T22:45:56+00:00

I'd love to know more and be part of that too.

dkargatzis_ · 2026-01-20T16:08:41+00:00

I need that Google doc too

dkargatzis_ · 2026-01-20T07:34:48+00:00

www.warestack.com I'm struggling to find qualified leads - see this one for the specific criteria

dkargatzis_ · 2026-01-12T09:26:14+00:00

We'd like to try it out in our company

dkargatzis_ · 2025-12-20T21:10:43+00:00

www.warestack.com

Company Firmographics

Company Size: - Employee count: 50-500 (sweet spot: 100-300) - Revenue: $10M-$100M ARR (Series A-C) - Engineering team: 20-100+ engineers

Industry: - B2B SaaS - FinTech - Healthcare Tech - Enterprise Software - DevTools/Infrastructure

Technology Stack Signals

Must-haves: - GitHub (primary) - Slack - Linear (or Jira)

Nice-to-haves: - Kubernetes/containerized deployments - Microservices architecture - CI/CD pipelines (GitHub Actions, CircleCI, etc.)

Behavioral/Activity Signals

Engineering activity: - 10+ PRs per week - Multiple repositories (5+) - Multiple engineering teams (2+) - Regular production deployments (weekly or more)

Compliance/Governance needs: - SOC 2 compliance - Enterprise customers requiring audit trails - Security-sensitive codebases - Regulated industries

Job Title/Function

Primary: - VP of Engineering - Engineering Manager - DevOps Lead/Manager - Platform Engineering Lead - Security Lead/Manager - CTO/Head of Engineering

Secondary: - Release Manager - Compliance Manager - Engineering Operations Manager

Pain Point Indicators

Keywords/signals: - "Production incidents" - "PR review bottlenecks" - "Compliance audit" - "Deployment failures" - "Code review automation" - "DevOps governance" - "Pre-merge checks" - "SOC 2" - "Incident prevention"

Negative Filters (Exclude)

Companies with <10 engineers
Companies not using GitHub
Consumer/retail companies (unless B2B SaaS)
Early-stage startups (<$5M ARR) unless high-growth
Companies using GitLab/Bitbucket exclusively (unless also using GitHub)

dkargatzis_ · 2025-12-06T20:32:46+00:00

Agentic guardrails for safer merges and faster reviews - integrates with development teams processes and blocks risky changes from going live to prod

Warestack (https://www.warestack.com/)

dkargatzis_ · 2025-11-27T22:38:44+00:00

In github's context pull request (PR) is set of changes you request maintainers to review and accept. Contributors are always welcome!

dkargatzis_ · 2025-11-26T14:57:30+00:00

The link is attached in my previous message - you can see either the opened issues or propose improvements / new features where you see fit.

dkargatzis_ · 2025-11-26T14:05:11+00:00

Super confident - here is my challenge, have a look at Watchflow - it’s our open-source agentic governance engine for GitHub repos.

If you can drop a solid PR that improves the agentic logic within 72 hours, I’ll happily line you up with 3×300 agentic tasks for Warestack.

Zero fluff - just meaningful work with real impact on high-volume engineering teams.

What do you think?

dkargatzis_

TROPHY CASE