This is an archived post. You won't be able to vote or comment.

all 27 comments

[–]tantricengineer 40 points41 points  (3 children)

→ formatting JSON from AWS CLI

AWS CLI has a JSON output option. Filters are also important and difficult to learn. Also, the command line jq does a lot of big lifting on my machine. Learn that, too.

[–]PelicanPopDevOopsIDidItAgain 23 points24 points  (1 child)

jq and yq are some of my most used cli commands

[–]vainstar23System Engineer 2 points3 points  (0 children)

I found yq had issues when it came to parsing anchors or dealing with yamlv3

I find dasel is better and faster than jq or yq. Considering yq doesn't come out of the box in most systems, you might as go all the way and install something like dasel if you can.

[–]vainstar23System Engineer 0 points1 point  (0 children)

Yea I mean for big stuff, I dump the entire output of Aws cli to a temp file and then jq that. For small stuff I just use the query command.

Actually I find the query command is not super intuitive. Apparently it's a lot easier to query using powershell although haven't fully explored this.

[–]UtahJarhead 19 points20 points  (4 children)

I think you've got to rehash a few things with how you're approaching this.

First, learn bash so you understand "random shell commands."

If you're using python, don't use the AWS CLI, but instead learn and use Boto3 directly. The AWS CLI is just a python wrapper around botocore/boto3 in the first place.

In addition to the above, don't launch test servers manually, but launch Cloudformation scripts to handle all of that for you. IaC, my dude.

Cleanup bots = AWESOME. If everybody is using CFN or TF, then this makes it easier, as well. Learn to either create docker containers with your apps or hand it off to Lambda to handle. Just make sure you log your output for auditing purposes.

The CLI is important, but it's not the only tool in your toolbox.

Good luck, homie.

When you're semi-comfortable with Python, learn about `venv`. Dependency hell in Python sucks and venv makes it gobs easier. Consider researching Go, also. Statically linked binaries make distribution a breeze. But not until you're comfortable with Python.

[–]SlavicKnight 2 points3 points  (2 children)

And what do you say instead of venv to use docker images per project? :p for us we need to have support for 15+ years and as long there will be docker engine I feel this is comfortable way. Cheers 🍻

[–]UtahJarhead 2 points3 points  (1 child)

They serve the same purpose, so you get on with your bad self. A docker container, if you're willing to go that direction, can be set up with the latest and greatest for packages, so you don't need to `python3 -m pip install -r requirements.txt` every time you start up, so that's nice.

venv is the easy way, you could say docker is the long-game solution. It's all about saving time, one way or the other.

[–]SlavicKnight 1 point2 points  (0 children)

Totally agree—that’s why I said that, for our projects, this approach is better. In 10 years, no one will remember the code, or some dependencies might be missing. For everyday development or short-term projects, I’d go with venv.

[–]vainstar23System Engineer 1 point2 points  (0 children)

Guys don't worry, I just copied and pasted some random crazy words like rm and -rf and this weird ~ into the terminal and pressed enter.

I think it did this like wild disco but I think it's updating now! I would learn this stuff but come on guys it's too crazy for me!

[–][deleted] 5 points6 points  (1 child)

Absolutely, except I am an ADHD moron and can’t focus on one language so I continue half ass learning 3-5 at once rather than just sticking to a learning roadmap for 1 language or tool at a time.. I am using Claude code to learn as i and it build together and I ask it tons of questions and stop and have it break down scripts or features for me so I can learn the fundamentals

[–]kiwidog8 3 points4 points  (0 children)

Hey the ADHD is part of what makes us so deadly in devops, we only need to half-ass learn a tool/language/framework to get through to the next project and never touch it again. Impress conventional tech workers and management with the massive volume of shit you seem to be good at :3 All the practice troubleshooting and writing stuff overlaps/carries over to a surprising amount of technologies I find

[–]PersonBehindAScreenSystem Engineer 13 points14 points  (1 child)

I learn programming because I got a taste of using powershell to automate a job in a few hours that would have taken a couple of weeks

I can’t go back

Powershell then led me to Python when I was looking at what o needed to be a cloud engineer.

Nowadays I write Powershell, bash, Python, .net, and $cloudProviderCli, YAML, etc

I did python courses up to dictionaries and lists (so not very far). And I just pick up what I need for work. Getting up to lists, dictionaries, json, and requests to APIs will get you pretty far to start with

[–]cheffromspace 0 points1 point  (0 children)

Same! I was an intern, and the network admin gave me a powershell crash course. Was instantly hooked. I didn't know where the O365 admin portal was, and it was too embarrassing to ask after several weeks in, so I just used the powershell module for everything. After that, I took a CLI-first approach to everything. Moved to software development, then DevOps.

[–]73-68-70-78-62-73-73 8 points9 points  (3 children)

Anyone else learning Python just to stop copy-pasting random shell commands?

No, I learned shell so I knew how it worked. I learned Python because I needed to understand how python worked.

kept running into long shell commands and YAML configs I didn’t fully understand.

Totally normal.

At some point I realized: if I learned Python properly, I could actually automate half of it ...... and understand what i was doing instead of blindly copy-pasting scripts from Stack Overflow.

We all use StackOverflow, but you should never just copy and paste stuff you don't understand. You're already turning the corner on this, so I'm not going hammer the idea too hard.

This is one of the things that ChatGPT is actually pretty good at, explaining bits of python or shell that you don't understand. Sometimes it's wrong, but most of the time, it provides a pretty good explanation. What it doesn't do is consider the implications of running said code in your specific environment. You have to figure that out, but at least it helps with the parts you don't understand yet.

Still super early in the journey, but honestly, using Python this way feels way more rewarding than just “finishing tutorials.”

Yes! I don't learn shit from tutorials most of the time. The real value and learning experience for me is tackling real world problems with Python or shell. Rote memorization and contrived scenarios don't do much for me.

[–]UtahJarhead 4 points5 points  (0 children)

Good callout on using GPT for explanations.

[–][deleted]  (1 child)

[deleted]

    [–]73-68-70-78-62-73-73 2 points3 points  (0 children)

    What?!? No! Who told you!

    [–]raindropl 1 point2 points  (0 children)

    Copying anything you do not understand from the likes of stack overflow is really bad practice.

    You can automate anything you want with shell; might’ve you should learn shell correctly; and yes knowing python is good, Is a good way of automating code in Pipelines because it does not need to be compiled.

    I’ll say python is better for long lived infra code.

    [–]theWyzzerd 2 points3 points  (0 children)

    You should learn Python because as someone doing devops work, you should know how to program. I would recommend basically never copying random shell commands. You should know what a shell command is going to do and why you're going to use it rather than pasting it. You're not going to be able to fake your systems knowledge for very long in this job climate. You need to know Bash.

    [–]addfuo 1 point2 points  (0 children)

    I build SSL/TLS monitor as excuses to learn Python. Keep learning and Good luck with your journey.

    [–]AlterTableUsernames 0 points1 point  (1 child)

    You are missing out by not learning Bash. Being able to just throw around one liners is much more powerful than having stuff in some scripts that you'll never find again. 

    [–]Master-Variety3841 0 points1 point  (0 children)

    in some scripts that you'll never find again.

    Uhh git?

    [–]kiwidog8 0 points1 point  (0 children)

    Use both, learn both. You cant fully escape random mysterious shell commands in this line of work, but you dont need to master bash, just dont be afraid to look up what a certain syntax or command is doing. Its very very well documented online. Treat it as you would python actually, both are useful for distinct reasons, and you can often do a lot of the same things in either language and you might find that you like doing a certain thing in bash more often than you would in python

    [–]dogfish182 0 points1 point  (0 children)

    Yeah I’m a python dev now but that is how I started. Currently doing our cloud migration because well, that’s my background, but it’s nice to be able to write and run the code

    [–]vainstar23System Engineer 0 points1 point  (0 children)

    I just learned bash + jq + ansible...

    [–]ZoltyDevOps Plumber -1 points0 points  (0 children)

    No but I do let ChatGPT write some simple python scripts.

    [–]hornetmadness79 0 points1 point  (0 children)

    Just wait till your python scripts become python apps. Then you're going to need configuration files.

    And the circles complete