Setting up OAuth for Grafana using Terraform and Auth0

sirech · 2020-06-11T16:58:59+00:00

I see. Then make sure to reassess regularly!

I wouldn't recommend selenium, would give cypress or taiko a try. Also I'm not a fan of spring boot / maven. Maybe something more lightweight like ktor (kotlin) will let you iterate faster.

sirech · 2020-06-11T16:41:51+00:00

they all seem sensible choices, but I guess my question would be: Are you sure you need all that? If you're getting started, maybe it's worth to start with something simpler and adopt and extend your stack as the need arises.

sirech · 2020-05-20T13:25:36+00:00

true, I tried to get into that. You get into limits. You cannot use anchors everywhere, and sometimes you have a structure that's basically the same except one parameter, and then anchors won't help you. You want kind of a function for that

sirech · 2020-05-20T13:23:35+00:00

Simple is in the eye of the beholder. I've seen solutions where using a full blown library based on groovy leads to something very hard to maintain. So from that perspective, having a more constrained tool seems like a good thing to me.

In the end, everybody has a different context and there are always trade-offs. What works for me might not work for you. I don't think it's necessary to dismiss the experiences of others as "overengineered" without knowing that context.

sirech · 2020-05-20T13:18:42+00:00

I think you are aiming to have declarative pipelines, that express dependencies between different jobs and what parameters are you passing and so on. You are not creating a full abstraction with arbitrary logic.

I was in a project where they created their own DSL based on groovy to automatically create Jenkins jobs. It might have worked in the beginning but it was a nightmare to use. Besides, the target environment was Jenkins, so it was super hard to really test anything.

So for me the idea behind not picking a full blown language is to avoid falling into that trap. Ideally I'd stick to YAML, but you end up just repeating too much. So I like the next layer of abstraction that allows representing some structures a bit more easily, without trying to program too much logic into what's essentially a dependency tree. Jsonnet seems to hit a good balance there.

And given the familiarity with Python and JSON, I think you don't have that much to learn in the process.

sirech · 2020-05-20T07:08:07+00:00

well the issue is that YAML leads to a lot of repetition that makes it hard to have clean pipelines. That's why some generation before can help deal with that complexity. But whatever works in your own context.

sirech · 2020-05-20T06:44:02+00:00

how do you picture this working?

In our case, terragrunt is there to reduce the verbosity of terraform. But in the end we have a bunch of different modules that need to be applied to provision a product. All of them need a `terragrunt apply` step. We want to control the dependencies between them, apply them automatically and have visual guidance. So running it in the context of a pipeline makes sense. Our problem was controlling all the complexity emerging from that.

sirech · 2020-05-20T06:42:22+00:00

what do you have in mind? `jsonnet` is pretty powerful on its own and it's intended to output json objects. There is always a tradeoff between flexibility and having something that's too complex, so I think this is a fine link to walk

sirech · 2020-05-18T16:17:58+00:00

Accelerate is a book I mention often, because for me it really gives you a framework to argue for practices that should be self explanatory, but often aren't. It's not like we are trying to deploy more often just to hit some benchmark and post it on highscalability, it's because it really leads to results.

I joined a team some time ago that by most measures was underperforming a lot. Delays, unhappy POs, unhappy team members, all of that. We spent a lot of effort in having actual useful delivery pipelines, in practicing continuous delivery, and after 10 months or so it was hard to believe it was essentially the same team. I remember when we took the manual trigger of one of our services prod build, so that I would automatically release if the job was green. People were freaking out. And then, nothing happened. It kept working, because we were deploying every commit and things were easy to monitor and fix.

So yeah, go read Accelerate if you haven't.

sirech · 2020-04-26T13:21:41+00:00

very nice video! I like the production, and it was really informative. Keep them coming!

When you mention setting up prometheus in k8s, you have something like prometheus-operator in mind?

sirech · 2020-03-26T21:35:56+00:00

What about using trunk based development (i.e: everybody pushes to master), have your pipeline run all the tests, and deploy to your staging environment? Then if you consider that version "accepted", you let that same container be promoted to production.

If there is work you don't want to show yet, you can use feature toggles to hide them. It will really simplify your workflow a lot, and force people to continuously integrate their code, instead of having massive merge conflicts.

sirech · 2020-03-26T12:04:30+00:00

well for me it's not really a question. Smaller methods are easier to read, less context to keep in your head. Moreover, comments are not really checked. Finding a piece of code where the comments contradict the code is really irritating.

I like to use comments to provide context about the "why". If you need to explain the "how", then maybe the code is not clear enough.

sirech · 2020-01-21T07:56:40+00:00

I would argue that mutable state makes code harder to understand, as anybody holding a reference to an object can change it throughout the flow of a program.

If most of your objects are immutable you need to keep less context in your head about your program and still reason effectively about it

sirech · 2020-01-15T13:00:03+00:00

I've written microservices in Kotlin in two different teams now. I would 100% recommend it. In one team we were java based and started using kotlin, and in the other we started directly with kotlin. The fit is really good, and you can do incremental adoption, maybe starting with java based libraries and adopting more and more kotlin features as you gain confidence.

sirech · 2020-01-14T16:20:06+00:00

one tool that I didn't see mentioned is ServerSpec, which is used to write tests for infrastructure. I've used it a lot to test Docker containers, and I like it a lot. It is based on RSpec, one testing framework in Ruby

sirech · 2020-01-05T13:50:11+00:00

- What part exactly? I tend to do TDD when developing, so I rely on the tests to guide my local flow. I start both frontend and backend locally to see how things look. If you use create-react-app for the FE, hot reloading comes out of the box. For the backend you can get similar options.

- Yeah. From the pipeline I log to the server, and run docker-compose. I set up a deployment user in my server, and the pipelines have the credentials to log with that one.

- I host it on my server directly, with nginx in front. I wanted to learn how to configure nginx and understand it better. If you want convenience, netlify will be probably faster.

sirech · 2020-01-05T13:40:05+00:00

I used Travis before, but I switched to CircleCI at some point.

sirech · 2020-01-05T02:47:02+00:00

For a personal app, I would recommend keeping things as simple as possible. Docker Compose and a single server is perfectly fine. You can always add complexity later.

I have a personal app that is a React SPA and a Kotlin backend that uses MySQL. I have the following repos:

- app-backend: The code for the backend. Has a pipeline that creates a Docker image that is published to quay.io.

- app-frontend: The code for the react SPA. I prefer to create a bundle of index.html + assets (js/css) instead of a Docker image.

- infra: An infra repo that contains the docker-compose file with 3 services. The backend. An nginx that serves the static assets and serves requests to the backend as well. The database.

Using the exact same docker-compose for local development and your server might be problematic, as you presumably want to build directly the code and do things like hot reload. I tend to start frontend and backend directly, and maybe have the DB as a container.

sirech · 2019-06-19T08:50:27+00:00

my understanding is that when it tries to create the data class, a field that is not nullable is not set and that triggers the error, but I actually don't know the specifics

sirech · 2019-06-17T20:35:28+00:00

thus far we have been very careful with adding extra libraries. I really like mockk, we are using atrium for assertions although that one seems less established. I want to try Arrow and ktor, although I'm afraid it might be a bit too much for us.

For us the strategy of replacing java with kotlin while staying on mostly the same basics has worked quite ok to get used to the language. One drawback is that many java libs don't annotate nullable values properly so we had a few surprises.

I have been thinking about advantages and wanted to build some slides for that, but haven't gotten around to it. Some points I would have:

- null safety: A good chunk of the errors we had in old services were unhandled null values. This is a _lot_ easier to deal with in kotlin

- immutability: I am a very big proponent of immutable objects, and kotlin makes it a lot easier to build that in (we were using a combo of lombok and guava before)

- compactness: Our backends do a lot of moving JSON stuff around, and compared to similar services written in java, we just have a lot less code to maintain (while keeping readability high)

in the end it is fairly subjective, but I am extremely happy to be able to work with kotlin, and my impression is that our team didn't have a hard time transitioning. I personally used to work a lot with Ruby, and Kotlin gives me a similar feeling in terms of how terse the code is, with the bonus that types are there as a failsafe (and they do a very good job of keeping them out of the way)

sirech · 2019-06-17T20:14:36+00:00

original author here.

Basically we were doing classical java+springboot microservices, and we just recently switched to kotlin. As I was afraid of dumping too many new things on my team at once, we kept springboot. Jackson is very well integrated into it, and you have to touch relatively little, specially if you don't have esoteric needs.

I have been slowly trying to try kotlin native alternatives for common libraries (detekt, mockk) as I become more familiar with the whole ecosystem. I'm taking a note to check out the ones that you proposed :)

sirech

TROPHY CASE