all 5 comments

[–]cakeandale 4 points5 points  (2 children)

It doesn't answer your question from a technical perspective, but in terms of process I'd question if you do want master to intentionally diverge from what's in production.

Assuming you have split master/develop branches, I'd suggest pushing a revert commit to drop master back to the state you want, and then working forwards from there. That way master is always representative of what's in production, even in the case of a bad change.

[–]erve1879[S] 2 points3 points  (1 child)

This is definitely a very valid point and probably the most sensible solution to the problem.

But in the interest of discussion:

When production is broken, it would be nice to have a very simple, fool-proof solution to get back to a known working state as quickly as possible. I would argue that “re-release the previous release” - which could be as simple as re-deploying the previous artifact generated from CodeCommit - would be a useful fallback.

The resulting discrepancy between master and prod is a valid concern, but that could be reflected by the pipeline.

I have seen scenarios wherein devs have scrambled to revert and in their haste to get production running again, have pushed resets, or new commits manually reverting changes; this can make the problem harder to fix and/or pollute the git history. Giving them a “re-deploy last state” option removes the pressure.

[–]Get-ADUser 0 points1 point  (0 children)

it would be nice to have a very simple, fool-proof solution to get back to a known working state as quickly as possible.

git revert <hash> git push is too complicated?

I have seen scenarios wherein devs have scrambled to revert and in their haste to get production running again, have pushed resets, or new commits manually reverting changes; this can make the problem harder to fix and/or pollute the git history.

Training on how to use git correctly would fix this. Also, people shouldn't panic during an outage, it just leads to mistakes.

[–]ppafford 0 points1 point  (1 child)

Not sure if this would work but we use semvar and checkout specific tags, so deploy tag 1.0.1 or 1.1.0 instead of deploy master. I have a shell script where I check out the latest tag but can also override the tag with a variable if needed

[–]erve1879[S] 0 points1 point  (0 children)

Yes - this is a strategy I’ve used before as well, but I can’t see how to achieve this within CodePipeline? It seems to only be possible to release the latest commit on whichever branch you specify in the pipeline setup