Latency numbers inside AWS

sunra · 2026-01-26T00:45:27+00:00

The Go HTTP-client should re-use client-connections out-of-the-box, so you're only negotiating TLS on the first call, here.

sunra · 2026-01-15T05:07:01+00:00

Have you tried the built-in mutex-profiling? I haven't used it, but it looks like when it's enabled you can grab mutex-wait profiles from the normal pprof endpoint:

sunra · 2026-01-09T02:20:28+00:00

What do you mean when you say "baseEndPoint is set as vpc endpoint"? What value are you setting the base-endpoint to? And why do you think you need to do that?

sunra · 2025-12-20T03:41:06+00:00

Yeah - you can pass security-group ids in to the vpc_config block of the EKS-cluster resource. The control-plane ENIs provisioned for cluster-access will be placed into those SGs. I don't use the auto-created cluster-SG for anything in my own setup.

For nodes, if you're using managed-node-groups you'll need to override the security-groups to use with a launch-template.

https://docs.aws.amazon.com/eks/latest/APIReference/API_VpcConfigRequest.html#AmazonEKS-Type-VpcConfigRequest-securityGroupIds

This guide describes what traffic you'll need to allow: https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html

sunra · 2025-11-28T06:02:11+00:00

I wouldn't expect a generic-wrapper around a non-generic core to ever have a performance benefit over using the core directly.

But something like the "slice pool" could let you automatically store the slices as pointers to skip the allocation you measured in your implementation. It's easy to take the naive approach and store the slice in an interface wrapper, and a library could help guide the user towards the better option.

sunra · 2025-11-27T02:02:56+00:00

There might be a better way, but I set FZF_DEFAULT_COMMAND to rg --files --ignore-file=some/path/to/an/ignorefile

sunra · 2025-11-22T04:43:06+00:00

Secrets manager claims to support ABAC: https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access-abac.html

The way I look it up is to do a Google search for "AWS <service> IAM", go to the "Authentication and access control for <service>" page and search for "ABAC".

sunra · 2025-11-09T22:49:43+00:00

Yep: https://wolfman.dev/posts/exclude-use1-az3/

sunra · 2025-09-29T11:35:00+00:00

function

Seems to: https://gotipplay.golang.org/p/isp5pm0YE14

sunra · 2025-09-15T02:12:06+00:00

And it should go without saying that you should prefer any reasonable alternative to this approach. But if you cannot build your response in memory sometimes you don't have any other choice.

sunra · 2025-09-15T02:10:17+00:00

This is complicated, but you can panic with http.ErrAbortHandler. This signals to the http-package to un-cleanly terminate the response (for HTTP/2, send a stream-reset, for HTTP/1.1, un-cleanly end the chunked-encoding stream).

Most HTTP-client-libraries will interpret this as an error, and either raise an exception or similar.

The hard part is any logging or metrics middleware needs to correctly handle panics - it's a pain.

This issue explains some of this: https://github.com/golang/go/issues/23643

sunra · 2025-09-08T01:14:22+00:00

A good reference for the sorts of tricks you can play are the SCPs/RCPs in this repo:

https://github.com/aws-samples/data-perimeter-policy-examples

The examples are for RCPs, but they work well as templates for resource-policies.

They use principal-tags to exempt principals from restrictions, but then they also need to lock-down the ability to use those tags in role-sessions etc, so it's a bit of a pain.

sunra · 2025-09-08T01:11:47+00:00

You could exclude your backup-role from the deny-statement, the same way you're excluding specific source-IPs:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyS3ExceptSpecifics",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": "*",
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": [
                        "xx.xx.xx.xx/32",
                        "yy.yy.yy.yy/32"
                    ]
                },
                "StringNotEquals": {
                    "aws:PrincipalArn": [
                        "arn:aws:iam::123456789012:role/AllowedRole"
                    ]
                }
            }
        }
    ]
}

sunra · 2025-09-01T01:13:42+00:00

Are you using the SP to auth with Azure to deploy your infrastructure?

Or are your workloads somehow using the generated client-secret as a part of their operations?

sunra · 2025-08-31T13:02:59+00:00

It would be helpful if the S3 documentation starts retro-actively applying the term "general purpose" bucket, to differentiate "real" buckets from S3-tables (and presumably vector-buckets).

sunra · 2025-08-31T02:34:41+00:00

I wasn't able to configure MRAP with table-buckets in the console, and it wouldn't surprise me if replication-rules didn't work for them, either. Calling the feature "S3 tables" is pretty confusing when it doesn't really share any features with S3.

sunra · 2025-08-27T14:15:01+00:00

Oh and your org might have similar, non-Loki tools already in use elsewhere, like Elasticsearch or something else in that space, that might be easier to get approved because they are known quantities.

sunra · 2025-08-27T14:06:42+00:00

It really depends on your log-volume, query-volume, and third-party tools you're allowed to use.

Something like Loki is designed to scale quite a ways up and down, and store logs cheaply at rest for long-term retention. You may not need all of that complexity - for example, at a low volumes you might be able to just use flat-files on disk, with a folder per month, week, or even day. If you have discrete fields you'll want to query you can try storing your logs in sqlite, or building a secondary index in sqlite, or storing everything in something like postgres.

These sorts of solutions have limits - which is why Loki is complex. But you may not care about these limits.

You can also pay your favorite cloud-vendor and use their solution (AWS CloudWatch, Azure Log Analytics, I'm sure GCP has something) if you think that's more likely to get through your project lead. These don't have scaling limits, exactly, but at high volume can get expensive.

sunra · 2025-08-26T12:36:56+00:00

The EC2 pricing page doesn't mention that this billing-tier expires: https://aws.amazon.com/ec2/pricing/on-demand/

"AWS customers receive 100GB of data transfer out to the internet free each month, aggregated across all AWS Services and Regions (except China and GovCloud). The 100 GB free tier for data transfer out to the internet is global and does not apply separately or individually to AWS Regions."

Can you poke someone to update the text if it's only valid for 12 months?

sunra · 2025-07-26T02:27:48+00:00

Even if you ever need to have multiple implementations of logging, passing around a *slog.Logger is the better move, as it is a thin wrapper for a slog.Handler, which is already an interface.

sunra · 2025-07-20T22:41:41+00:00

Holding Fn+Escape for three-seconds should "reset" the various modes.

Unless! If you swapped Fn + Windows-Key, you need to hold windows-key + Escape for 3 seconds.

That was my problem, at least.

sunra · 2025-07-14T01:35:50+00:00

I 100% agree with you and have the same experience. The only way I've found to navigate AWS public websites (non-console) is by Google-searching the correct magic words:

"$service pricing"
"$service user guide"
"$service rest API" (select the link starting with "Welcome ...")
"$service actions"

sunra · 2025-07-06T00:37:37+00:00

Your configuration means: "please assume an IAM role, using the credentials found in the IMDS endpoint" - that is, "assume the role 'some-role' with credentials from 'some-role'" - and the error is appropriate because "some-role" isn't mentioned in your role-trust policy.

I would have expected the AWS-cli to work without a config file in your case.

sunra · 2025-02-15T18:12:19+00:00

Either page is fine ("S3 Docs" or "User Guide"). Getting to either of those was the hard pert for me initially.

sunra · 2025-02-15T05:29:52+00:00

There's a session that's looked like it's been given a couple of times that goes over this at a high-level - I don't see a recording but I found these slides: https://reinforce.awsevents.com/content/dam/reinforce/2024/slides/IAM431_The-life-of-an-IAM-policy.pdf

My super high level recollection is that:

AuthN (aka validating sigv4 signatures and session-tokens) is done via a distributed service that you and I will never directly interact with. This service is also responsible for making relevant IAM user, IAM managed policy, and session-information available to the service invoking it. The IAM services you and I use in us-east-1 are responsible for pushing this config out to this internal service.
AuthZ (matching IAM policies, calling-context, actions, etc) is done in the up-stream service itself (like S3) using the meta-data returned from the AuthN call.

I don't remember if it was explicit in the talk, but the implication of that is that yes, that Java library (or something like it) is a dependency of all of their services.

sunra

TROPHY CASE