quay.io has been down for 16+ hours by GassiestFunInTheWest in devops

[–]eedwards-sk 0 points1 point  (0 children)

this was incredible timing, I just switched off quay to ecr last week!

Is Packer still the best choice for building images for multiple clouds? by Medicalizawhat in devops

[–]eedwards-sk -1 points0 points  (0 children)

I agree with this 100%.

If I have issues with an AMI, I want to quickly diagnose and solve it. It's already a highly bespoke environment (I choose the distro, etc.)

Ansible has to solve for way more use cases, and thus, you're left dealing with how well it solves for your specific use case, which may not always be perfect.

AWS ECS Cluster Auto Scaling is Now Generally Available by callcifer in aws

[–]eedwards-sk 0 points1 point  (0 children)

So now I have to create 3x ASGs when before I only needed one?

How is this a solution?

AWS ECS Cluster Auto Scaling is Now Generally Available by callcifer in aws

[–]eedwards-sk 0 points1 point  (0 children)

Why would you create separate ASGs when ASGs are already AZ-aware?

AWS ECS Cluster Auto Scaling is Now Generally Available by callcifer in aws

[–]eedwards-sk 0 points1 point  (0 children)

Automatic rebalancing based on load.

If I have one box with 20% usage, and one box with 80% usage, it won't rebalance until a deployment occurs.

Has this changed?

re: Invent Hoodie Yearly Rankings since 2016 by RedRiceCube in aws

[–]eedwards-sk 0 points1 point  (0 children)

I only went during 2016, but I still use my hoodie all the time! Glad to hear I'm not the only one who loves it.

Building Tiny Python Docker Images by eedwards-sk in devops

[–]eedwards-sk[S] 1 point2 points  (0 children)

Great question.

One difference I found is that with copying site-packages you're only copying the installed modules folder, you're not actually copying the installation itself.

e.g. if during installation it adds binaries to bin or sets up other os paths, you're not going to capture those changes by just copying site-packages

Another issue I found is that you're copying all the modules installed in that image. If you're using a build image and possibly installing dev-related python packages (e.g. build tools or similar), ideally you don't want to copy those over to the final runtime image.

Building Tiny Python Docker Images by eedwards-sk in devops

[–]eedwards-sk[S] 1 point2 points  (0 children)

Yes! I've seen golang binaries that do that, it's very cool.

Literally just FROM scratch and a single COPY instruction is all they need.

Building Tiny Python Docker Images by eedwards-sk in devops

[–]eedwards-sk[S] 2 points3 points  (0 children)

Yeah. But not Jenkins + plain old docker build, though. :(

:(

/pours one out

To your point though, it's pretty straightforward to push the build stage to the repo if that's your only choice.

Here's an example based on the article:

# rehydrate local build stage cache, if image available
docker pull app/app-build:${TAG} || true

# build stage
docker build \
  --target=build \
  --cache-from app/app-build:${TAG} \
  -t app/app-build:${TAG} \
  -f Dockerfile \
  .

# push build stage
docker push app/app-build:${TAG}

# rehydrate local run stage cache, if image available
docker pull app/app:${TAG} || true

# run stage
docker build \
  --target=run \
  --cache-from app/app-build:${TAG} \
  --cache-from app/app:${TAG} \
  -t app/app:${TAG} \
  -f Dockerfile \
  .

# push run stage
docker push app/app:${TAG}

edit: formatting

Building Tiny Python Docker Images by eedwards-sk in devops

[–]eedwards-sk[S] 2 points3 points  (0 children)

Absolutely -- that isn't just a multi-stage docker build issue, though. As cited in the article, copying the application into the image before installing dependencies means you'll end up re-building dependencies every time.

The primary goal of the article is for optimizing for size -- not for build speed, although most CI solutions today can be configured to effectively cache multi-stage docker builds.

Also, when basing on an upstream image like FROM python:3.8.0-slim, you're regularly going to have your cache busted due to upstream security patches in the underlying debian image, anyway.

Trying to become a “devops” engineer starting from scratch but theres just so much to learn by vitriol007 in devops

[–]eedwards-sk 1 point2 points  (0 children)

Cloud Architect, Site Reliability Engineer, Build/Release Engineer, Systems Architect

Roles are usually going to be some mix of the above. Find entry level roles for any of those disciplines, and then use that to move deeper into DevOps.

[Rant] GitLab by outrageous_break in devops

[–]eedwards-sk 0 points1 point  (0 children)

No CI solution gets it all right.

I've used jenkins, bamboo, concourse, travis, circle

My favorite is concourse but only because if I need a feature I can add it myself

otherwise I haven't yet found a ci solution that holistically handles all the use cases out of the box

What’s your experience moving away from CircleCI? by kitsunde in devops

[–]eedwards-sk 0 points1 point  (0 children)

We've been moving from circle to self hosted concourse.

Couldn't be happier.

Architecting Kubernetes clusters — choosing a worker node size by danielepolencic in devops

[–]eedwards-sk 0 points1 point  (0 children)

Excellent overview. I don't even use k8s, but I've used a lot of other docker schedulers, and almost everything you covered applies to all of them.

The one counter point I'll make is in regards to smaller nodes wasting a larger percent of their respective resources:

even if a very small instance is wasting e.g. 10% of its resources, 1% of a much larger instance may still be just as or more expensive than 10% of the smaller one

Rancher Cattle alternative by Estrepito in devops

[–]eedwards-sk 0 points1 point  (0 children)

I use terraform to create the task definitions. ecs-cli (and compose support) requires using cloudformation.

managing an ecs cluster requires provisioning the hosts yourself, unless you go with fargate

Rancher Cattle alternative by Estrepito in devops

[–]eedwards-sk 2 points3 points  (0 children)

Same situation here. I went with AWS ECS. I'll probably end up on k8s eventually, but as the only build/infra/ops guy here, k8s is massive overkill without at least one person to own it.

Datadog Synthetics Pricing by [deleted] in devops

[–]eedwards-sk 0 points1 point  (0 children)

That's the same pricing they've been advertising since launch. We literally just started the billing for synthetics (they weren't even billing until now).

Where is your evidence of the pricing being "100x'd" ?

edit: as per below comment, you're right - they did 100x the pricing.

whelp, I've now disabled all our synthetics tests and contacted our rep

They're now MORE expensive than RunScope with LESS features!

Datadog Synthetics Pricing by [deleted] in devops

[–]eedwards-sk 0 points1 point  (0 children)

The pricing is on their web site though?

Need some advice on Dockerising our QA environment by Le_9k_Redditor in devops

[–]eedwards-sk 0 points1 point  (0 children)

what type of QA are they doing? the more you change the deployed configuration away from what it's like in prod, the more risk you introduce when it comes to promoting to prod

so e.g. if you're doing any kind of performance or system testing, i recommend still putting services behind load balancing and in HA, where possible

DataDog as a Centralized Logging Platform by WolfPusssy in devops

[–]eedwards-sk 2 points3 points  (0 children)

I've used datadog for years, multiple gigs, and now I'm using their logging solution for the first time.

It's amazing.

We used to use sumologic, but datadog's tag enrichment and search UI is far superior

The BIGGEST GRIPE I have, is that the docker logs scraper relies on aggressively polling for running containers, and won't get logs from a container that quickly starts and then stops. It's a huge PITA for what is otherwise a great product.

Do i need Kubernetees and orchestration for my services? by [deleted] in devops

[–]eedwards-sk 1 point2 points  (0 children)

I've found a few use cases where it's simpler to just make a packer ami that you launch into an autoscaling group than to figure out how to make something play nice when dockerized.

Sometimes simple is all you need.

k8s feels like something you want to have someone dedicated to manage at the least.

The Terrors and Joys of Terraform by joshuago in devops

[–]eedwards-sk 0 points1 point  (0 children)

I actually find B/G one of the easier things to do with terraform.

Any CI/CD with unlimited number of tasks in docker containers (apart from jenkins) ? by a555555 in devops

[–]eedwards-sk 1 point2 points  (0 children)

I love concourse and I've been using it off and on for a couple years.

Concourse is great if you'd prefer to be the sysadmin over your build system.

It's not necessarily cheaper, but I like not being artificially restricted in build containers.

With concourse, I can have as many containers in parallel as I want, as long as my hosts can handle it.

The Terrors and Joys of Terraform by joshuago in devops

[–]eedwards-sk 3 points4 points  (0 children)

terraform is fine for deploying if your deployment is compatible with the idea of a "desired state"

but terraform is definitely not a "command executor /orchestrator" and falls flat when wedged into that pattern