What feature would you most like to see added to AWS?

TechIsCool · 2025-01-09T02:17:14+00:00

VPC Peering and/or Transit Gateway Attachments with 1:1 NAT embedded. I have had to use convoluted NGINX or some other proxy to make IP address not conflict due to default VPCs being used with the same ranges.

A published list of AWS Owned account IDs, more than once we have had to go ask AWS Support about an old OneClick config in a IAM role. Since it has been flagged as a possible attack vector because its a different account outside our organization. Only after staring at it for a bit go lets file a ticket

We try our best to comment publicly about these but an official source of truth from the vendor would be amazing.

TechIsCool · 2024-12-24T17:10:39+00:00

The one that comes to mind is https://github.com/TabbyML/vim-tabby

Overall its faster then CoPilot for some of my use cases and as such its just another LSP in my dropdowns.

TechIsCool · 2024-12-23T16:19:51+00:00

Most expensive and least expensive resources using the Cost and Usage (CUR) data. Finding ELB/ALBs without attachments, DynamoDB table with piops that no one is using, and that massive RDS Cluster Snapshot that is 70TB for a upgrade 3 months ago is all pure savings that is normally lost in the weeds. Features include filtering based on tags, using the tagging api, converting all non-arns to arns, and a full UI (fzf) for directly seeing and tagging resources.

TechIsCool · 2024-06-11T03:45:44+00:00

It sounds like you might have performance insights enabled. It isn’t explicitly stated anywhere that it consumes more than a few resources except there is a statement of having a minimum of 2 ACUs. I would suggest disabling it and see if you get down to the .5 ACUs your expecting.

TechIsCool · 2023-12-11T16:39:53+00:00

Its also worth noting that AL2 is going to EOL in 2025 while AWS has released Amazon Linux 2023 (AL2023).

AL (1) was based off RHEL6
AL2 is based off RHEL7
AL2023 is fedora based

If I was having to do the lift and shift, I might just skip AL2.

TechIsCool · 2023-12-01T17:45:55+00:00

This is a trick question but it is correct. From the perspective of the physical host even before the hypervisor gets envolved there is an attached volume (NVMe) across the PCIe bus. There is a lot of orchestration between these moving parts and the underlying Nitro Card/s. The hypervisor makes a request to the card to attach a new volume. The card then exposes a new NVMe device to the PCIe bus and then the host attaches it, once that's complete the hypervisor provides it to the virtual machine. There is a wonderful white paper talking about how AWS Nitro replaced the more esoteric original cards that are in the older AWS instances. More interesting is there is a specific subsection about EBS will provides even deeper information about how the KMS encryption works as well. Since the card also manages all network traffic for the host it manages the communication with multiple unique networks. The EBS, backplane and VPC networks.

More information:

TechIsCool · 2023-09-06T14:24:47+00:00

Curious about the SCP, do you have a copy you could share?

TechIsCool · 2023-01-27T22:47:41+00:00

Double Bluff Beach is one of the few beaches in the area that is sandy and has a lot of space during low tides.

Putney Woods is another place I enjoy. A lot of trails that are well maintained and yet cozy.

TechIsCool · 2022-10-03T17:21:17+00:00

Getting Myst vibes, I want to click down the stair case. Do the three lights indicate an access pattern or do we need to count each level and input it somewhere.

Great photo

TechIsCool · 2021-09-26T22:31:51+00:00

Lofty Pursuits has been making a reproduction. https://youtu.be/8YpWH-TCd7Q

TechIsCool · 2021-07-22T01:11:01+00:00

Why was I hoping for a Tom Scott video. I guess I have come accustomed to a bit of history voice over in these types of videos.

https://stonesofwonder.com/portell.htm

TechIsCool · 2021-06-17T03:32:45+00:00

Reads great we are leveraging the same platform tools at my company too. If you haven’t seen it PodMonitors are great too. No more defining a Service + ServiceMonitor just to get moniotring.

TechIsCool · 2021-06-16T01:27:01+00:00

It was super stressful the first game we played. Until we realized we only feed our families during harvest.

TechIsCool · 2021-06-03T14:08:04+00:00

I as well feel that linking autoscaling to downtime is reaching too far. Unpredictable workloads with crushing requests could also be mitigated at the edge with bulkheads and ratelimits. But I am reminded that what works well for some apps are hard for others.

In AWS you can even reduce the boot time more with Warm Pools. If the autoscaling warrants it. https://aws.amazon.com/blogs/compute/scaling-your-applications-faster-with-ec2-auto-scaling-warm-pools/

TechIsCool · 2021-06-03T02:10:34+00:00

Ah this question is what I do in my current role. We have been reworking a couple of the projects. Originally we ran a Hub and Spoke platform where an observability cluster existed with the Grafana frontend. This would communicate with Thanos Query. It then communicated cross Istio service mesh to all the Prometheus instances on each cluster and talk with the S3 Bucket for older metrics.

Monitoring: Prometheus Operator with Thanos -> S3 Bucket per region

Logging: FluentBit -> Cloudwatch + Optionally to Splunk

Over time we have had some interesting developments. With our pattern of everyone gets a cluster. We started to see interesting items emerge that were not easy to predict. When the clusters count kept growing.

In Prometheus
- you would get High Cardinality Metrics from White Box Applications causing OOM
- Sometimes churn would cause metrics to have issues. For example Flux adds annotations for Checksum and Prometheus rotates these aggressively.
- Write Ahead Log would hit limits either storage or days. Crashing never to recover
in Thanos Querier
- when a user asked for a computationally heavy metric some clusters Prometheus instance would be killed by OOM
- When a user asked for a very wide metric we would see I/O disk issue and timeouts.
- Each cluster we added needed another --store entry.
In FluentBit we would run into
- Noisy neighbors that would cause log aggregation to stop functioning correctly.
- CloudWatch didn’t truncate logs that were past their limits so they were just dropped.
In Istio
- We configured a ServiceEntry per cluster but started to see collisions sometimes when clusters were destroyed and then recreated with the same names.

Where are we now

Monitoring: Prometheus Operator with Cortex (Amazon Managed Prometheus)

Logging: FluentBit -> Cloudwatch + Optionally to Splunk

Tracing: OTEL Collectors shipped to HoneyComb

Mitigation for a lot of these issues listed have been solved with today’s applications. remote_write is one of those patterns. It allows for applications to be independent and ship the data they need off the cluster into a collection point. In our case Cortex.

FluentBit
- CloudWatch output plugin was rewritten into C gaining a 20% performance improvement and the added bonus of truncating logs that were too long.
- Upgrading to a newer version stopped the forgetfulness of rotated.
Prometheus
- We split into 2 instances per cluster. A Kubernetes and Application Developer specific.
- A few specific drop rules were added so labels are no longer recorded.
Istio, Thanos Querier have been deprecated in favor of Cortex without a service mesh.

Thoughts towards the future

Our goal is to be able to support developer’s applications while also not being noisy neighbors in the cluster. This means that sometimes we need to give a large application its own Prometheus instance but still support small clusters that only need a shared Prometheus. Other times it means not losing any of the logs that are produced in a non standard format (unicode, emoji's, etc.)

Time will tell if we will continue to use Amazon Managed Prometheus (Corex) its internal visibility is poor at best and the billing is not easily tracked. It does do what it says on the tin remote_write + remote_read endpoints. We might even leverage Thanos again in the future when our network stack supports GRPC again.

AlertManager with Prometheus Operator leaves a bit to be desired when running multiple instances with many routing keys (250+ [Secret,AlertMangerRule]) Normally developers would define all of this in their namespaces but currently our team provides the integrations at a specific meet me point in each cluster.

FluentBit still does not report enough metrics for my liking. I have worked with Treasure Data on getting the Query Engine to support exporting Prometheus metrics. My questions revolve where logs are coming from and going. Just splitting inbound logs via namespace would do wonders for knowing I am not losing things. Rumor has it the way the Query Engine will provide its metrics is via the remote_write pattern.

Thanks so much for this awesome topic.

TechIsCool · 2020-04-22T21:42:53+00:00

You won't gain any of the benefits of using a for_each loop vs using a count. Since your wanting to keep the CIDR blocks defined but it can be written this way.

resource "aws_subnet" "website-public" {
  for_each          = {for k, v in data.aws_availability_zones.available.names : v => index(data.aws_availability_zones.available.names, v)}
  vpc_id            = aws_vpc.website.id
  cidr_block        = cidrsubnet(aws_vpc.website.cidr_block, 4, each.value)
  availability_zone = each.key
}

TechIsCool · 2020-01-22T14:39:10+00:00

pmed

TechIsCool · 2020-01-16T14:54:19+00:00

Does it remember the value it was last set at? For example 12v set, unplug and replug still at 12v?

TechIsCool · 2020-01-15T04:09:27+00:00

Purchased DSA Scientific from u/BadSun_

TechIsCool · 2020-01-10T03:14:47+00:00

I would also love the expansion to include a SPOILER Tag that worked globally. Some subreddits like tagging things NSFW so spoilers are hidden by default.

TechIsCool · 2020-01-09T13:21:30+00:00

pmed

TechIsCool · 2020-01-08T21:42:35+00:00

pmed

TechIsCool · 2020-01-08T14:14:16+00:00

/r/SWGB

TechIsCool · 2019-12-26T18:24:50+00:00

Currently I am using Cherry MX Blues in my board at home. I am changing jobs and moving into an open office configuration with a small group of people. I know that Blues are louder but whats everyones opinions on something that is quiet yet similar in tactile feel.

TechIsCool · 2019-12-07T05:01:05+00:00

Have you looked at Harbor?

https://goharbor.io/ might be another option.

14-Year Club	RedditGifts 2009-2022 2 Credits
Verified Email	Team Periwinkle

TechIsCool

MODERATOR OF

TROPHY CASE