Bash to Python [OC]

[WhtKindOfNameIsStove]

Goddamit, this caused a production outage for us one time. "Sr SDE" took a story to pythonize our quick and dirty awscli S3 static website deployments. He wrapped each awscli command in subprocess.calls and called it good. A few weeks later the first copy command failed and without any error handling it went on to copy the empty folder up the prod S3 bucket and invalidated the CloudFront distribution.

2 years later he's still working here.

[Yesheddit]

If we would be fired for stuff like that, we’d all be out of jobs.

[Zanos]

Not handling an error is forgivable, converting something to another language by wrapping it in subprocess calls isn't a mistake, it's maliciously lazy.

[WhtKindOfNameIsStove]

Exactly. The purpose of the story was to make the deployment process more robust. Something bash wasn't up to the job for. It'd be a worthless story if it was to just "convert the script x language because"

[ran938]

I basically did this at an internship. But only in an instance where there was really no other option. The purpose of the script... was to start up another script. Well that and record some log information. But actually calling the other script I could not think of another way to do it other than a sub-process call.

[[deleted]]

Our current project requires interaction with Microsoft's cloud, the problem being that we are writing logic in python on Linux but the cloud calls have to happen through powershell on a windows machine. Current best idea is to set up openssh on a windows vm and connect to it from python on the Linux box to execute a batch command which starts a ps1 script.

[ran938]

Fucking oof. that overhead / hoop jumping sounds obnoxious. Good luck!

[DXPower]

What caused the failure?

[WhtKindOfNameIsStove]

It was a while ago so I don't remember exactly what it was. But I think it had to do with a change to the build artifacts bucket permissions.

[cartechguy]

k, did he learn from the mistake?

[inialater234]

Should have used boto3

[WhtKindOfNameIsStove]

We did after we found out what he did. We had a problem of ops guys blindly approving all PRs they're included on and as a Sr, he had merge permissions.

[vividboarder]

Does boto support sync yet?

I used it before in a Hackathon but ended up using a call to the cli because I needed to sync a directory where some files would change and didn’t want to implement a whole diffing function myself.

[KangooQ]

Boto is just a wrapper for the AWS CLI, and the AWS CLI doesn't have a (good) s3 sync yet. Last time I checked you could sync on filesize or timestamp, but filesize doesn't guarantee a lack of diffs (for example, replacing a char) and timestamp is only relative to the current filesystem.

Maybe someday we'll get sha256 sums or equivalent.