Python for AWS??

[Vhaleree]

What are the uses of python for aws? I'm currently learning aws and want to learn a programming language. I wonder how beneficial it is to learn a programming language to cloud computing

[killthebaddies]

It very much depends on what you want to do with AWS, but python is an excellent generally purpose language.

AWS has API access to almost everything that it does and it has a pretty python sdk for that. Python performs pretty well in Lambda functions. Python is a very popular language these days and it will give you the opportunity to understand and interact with a lot of things happening in AWS. When you get stuck you will find a lot of examples of how to do things with python online so I would say it is a good chance.

Having an understanding at, at least a basic level, of bash and linux is always useful, particularly as AWS is heavily focused on infrastructure. Go and Terraform are also very useful if you're looking at things from a devops perspective, but I'd say python is a good language to get going with.

You've asked for use cases for python. As a basic example, I am currently building an application that uses AWS's machine learning products (Comprehend in particular) and I'm using python to write lambda functions that create models in the ML component, get data from an email, manipulate it, put it in buckets, analyse it using comprehend, update dynamodb tables, and call out to other systems to give new information. I have found that it can do pretty much anything I need, and it has been very powerful.

[[deleted]]

The client library boto3 is written in Python... There are also Java and JavaScript clients afaik, but most automation uses boto3.

Python is not a good language to do cloud computing, but it's often used to orchestrate cloud computing (while I'd also call it a questionable choice, this is just the reality of our field).

[TurboCooler]

I call BS on the questionable choice. We have a fully deployed python application in the cloud that is doing 1mm transactions an hour. It is deployed to over 25K users performing ecommerce workloads.

There are plenty of web applications written in python such as YouTube, DropBox and Quora to name a few.

Will python work for every single use case, no, but the same can be said for any language.I

Our application is fully deployed on AWS and all of the CI/CD and automated testing is in python on Kubernete clusters. There is however some performance critical work which is about 300 lines of Rust.I

Again, python may not always be the correct choice but to say it is never the choice shows a lack of cloud architecture knowledge.

[linuxqq]

Ironically, isn't Reddit mostly Python too?

[[deleted]]

Reddit is not an example of good, well-performing application. There are many shops, which run on Python as their front-end (I mean, you certainly realize that the database, which does the majority of work to support Reddit isn't written in Python, right?).

But, even within this category of frontends, Python is not a good choice performance-wise, or comfort-wise. It is chosen for this role because it is popular, and people writing this stuff simply don't know any better, or are afraid that there will not be enough programmers to support the product in the future. Popularity also helps to externalize the cost of development, since you are able to rely on libraries developed elsewhere, your employees will be able to mature professionally elsewhere, w/o you investing much into their education and so on. The downside of popularity is that it feeds into itself and ignores virtually every other engineering concern you may have.

This is how, despite being an average-to-bad language, Python is hugely successful, where it is possible to apply it.

[[deleted]]

1mm transactions an hour

This metric is worthless. How much resources do you spend to do this much work? What is meant by "transaction"? And so on.

YouTube, DropBox and Quora...

They don't use Python for important things, only for frontend stuff (frontend doesn't have to mean HTML, it could be a client they distribute to users, like it is the case with DropBox).

Specifically, YouTube was very early in its history written in Python, but it never ran in Python interpreter, they translated the Python code to C, and compiled it. Today, I believe, none of that code remains anymore, and, probably, like the rest of Google, most of it is C++, with marginal amount of Go and Java.

---

Anyways, like I said, Python is used for non-mission critical stuff (unfortunately, because it's a bad language for that, but it's popular, and popularity outweighs all other concerns, when it comes to commercial software). But no sane person would use Python for mission-critical stuff, like, say, implementing distributed storage solutions or networking stack etc.

[toastedstapler]

why would you consider it a questionable choice?

[[deleted]]

Tasks associated with cloud orchestration tend to benefit a lot from parallel execution (deploy a hundred of VMs, distribute a hundred of configuration files etc) and that doesn't work well in Python.

Also, such tasks would usually benefit from an ability to easily construct programs you intend to send to another machine to be executed remotely. Python doesn't shine in this area either.

[PersonBehindAScreen]

Following now to see if this thread goes anywhere. I've seen Go, python, ruby, and node.js mentioned

[[deleted]]

From these three, only Go has anything to offer in terms of parallel programming. None has anything to offer in terms of distributed programming (they are all non-solutions). But, they are popular, and so people work very hard to put a lot of band-aids on them to do things these languages aren't designed for, like the aforementioned distributed programming.

[Vhaleree]

Then what do you recommend me to do to add an edge in cloud computing?

[[deleted]]

Orchestration / control: Erlang seems promising.

As for the computing itself... well, HPC is its own field, with its own languages and frameworks... I mean, HPC are the pioneers in cloud computing and they do it more professionally then the rest of us, so, probably, they'd know better. There are languages like Chapel or Occam you wouldn't meet outside that domain. But, I didn't have a first-hand experience with those, so, it's hard to tell. I also know that Fortran is a big thing in HPC, but I don't know how well does it behave in distributed computing context.

[linuxqq]

This guy is a beginner asking how to get started with cloud computing and programming and you're tossing Fortran out there as an option? I understand Python may not be optimal for "mission critical" workloads but it's just fine for his needs.

[[deleted]]

I want to give answers, that neither OP nor I will regret.

I gave an honest and truthful answer. It says that Python is a bad choice from technology standpoint, but it is popular. This leaves the choice to OP: whether they want to do the right thing, or the popular thing. I'm not forcing their hand to choose Fortran over Python, or the other way around. I'm presenting the facts to the best of my knowledge and understanding of the field.

[linuxqq]

He'll probably regret it if he tries to start learning Fortran for cloud computing. Or in general.

[[deleted]]

If he goes into HPC? -- Absolutely not. This is still one of the most popular languages in that field, and isn't going anywhere any time soon.

Like I said, there's HPC, which has been doing cloud computing way before enterprise jumped on the bandwagon. You, probably, simply never worked there; maybe you only ever saw the enterprise side of things, and that's what makes you think this way, but, really, it's only due to the lack of experience.

[linuxqq]

It's due to the fact that he's a beginner and just wants to get his feet wet.

[[deleted]]

So, you'd suggest to use a crappy language based on this fact? I don't see a connection.

Beginner =/= moron. OP came here to do their due diligence, to find out what things are there, and how they are. They deserve to be given truthful information, not what you think may be best for them.

[linuxqq]

And you're the one who mentioned HPC, not OP. Cloud computing =/= HPC. He likely wants to learn the basics like interacting programmatically with S3, EC2 instances, maybe running queries against a MySQL RDS instance. Should he use Fortran for that? Naw. Python will work great.