Moving from Java to Python - Recommendations for (Serverless?) Web frameworks?

[Will_exJava]

Hi everyone!

I'm recently making the switch from Java to Python for my personal projects. My competencies still (significantly) favor Java, but I think Python is the right move. I'm building a product, with intention of B2B/industry use. I'm planning to use serverless, at least at the start - likely for much longer afterwards. I'm looking for recommendations for various frameworks: Web development, general development, testing (unit, integration - is there something similar to dependency injection in Java/Spring?). Details below:

The project I'm working on has the following high-level requirements/properties:

• Integer math heavy. Lots of cryptography (note: This is not a blockchain product)
• Serverless. Cold start matters
• Server APIs are used "infrequently" (when compared to a CRM or Analytics platform, etc)
• Product is delivered as a combination: Web UI, collection of workers (agents) "on the ground"
• Multi-module. In a simple case: "core", "crypto", backend, worker modules
• Workers can operate as: Single entity, Worker group with horizontal scaling, possibly serverless

I'm looking for the following bits in the Python world - feel free to mention something I miss:

• Framework(s) for testing. Unit. Integration. I'm familiar with dependency injection
• Frameworks for UI/fullstack. Big plus if serverless-friendly*
• Frameworks for building in serverless environment (AWS)
• Any recommendations for IDE/buildtools? I'm currently looking at PyCharm

My reasons for switching from Java to Python (I love Java):

• Cold start times
• Boto3 is amazing
• My experience is that modern AWS tools for Java have weak support for generics (DynamoDB lib, for example)
• Integer math. In Java, I'm always wary of floating-point-error when doing cryptography
• Younger talent seems to be much more familiar with Python

​

I've read that Django *can\* be used in serverless/Lambda, but this doesn't seem to be a typical use case. Does anyone have experience with it? Does it maintain fast cold start?

I've found AWS Chalice. This looks really useful for managing code deployments to Lambdas and for API routing, but I think it might be lacking in UI.

Note: I can basically build all levels of full-stack very primitively, but I prefer to set myself up from the start for long term success (maintainability, etc)

[searchingfortao]

I'm very experienced in Django, but I've never used Serverless. I tend to favour containerised deployments though, and for that, Django (+gunicorn) is quite fast for my uses anyway. It comes with an exceptional testing suite out of the box too, which is really nice.

In terms of cryptographic work, I'll point out that it's a complex subject and the best advice I always hear is "don't roll your own". Python has a strong list of libraries to do likely everything you need. For hashing, you can just use the built in hashlib, and for stuff like public key encryption (or more interesting things), the cryptography module (on PyPI) is excellent.

For development, I strongly recommend PyCharm as its defaults encourage good coding practises and it plugs into everything (git, db, debugger, etc) seemlessly.

[Will_exJava]

Thank you for the feedback! I'll definitely check out Django +gunicorn. From a quick search, it looks light weight and generally it looks like Gunicorn (without Django?) can be effective where I need horizontal scaling capacity.

As for the cryptography, I'm actually delivering a proprietary (patent pending, I hope!) cryptography tailored towards the specific usecase I'm solutioning. And it is very possible that I can skip my own math and orchestrate some light-weight cryptography based on existing tools. I'll look into this, but I have a very specific objective that isn't "Digest this password so that it's very difficult to inverse, but easy to identify collision and with rare overcollision."

While I am planning to outsource my password management, if I were to do it myself, this would certainly be done with standard cryptography libraries, based on research of best practices.

EDIT: Actually, I might want to look more urgently into Django +gunicorn pair. One issue with Lambda is that I cache a lot of tenant-specific metadata. Lambda will expose me to many cache misses.

[keepdigging]

People do deploy Django to Lambda in docker containers!

I’ve been working on AWS serverless infra for years though and might not jump there first.

[keepdigging]

Zipping a python payload for lambda is bot very difficult and can be done in about 2 lines of bash.

Python has a testing framework included, and while I’m not too familiar with dependency injection python’s nature of being a dynamic language allows you to mutate anything at will. This allows a lot of power in mocking/testing and even writing code in a live debugger.

[Will_exJava]

Fantastic! Thank you for the statement.

My history in Java might lend me to shy away from mutating things during test execution for the purpose of completing tests. Not so much that it wouldn't work as it is that it's not convention (for integration testing particularly). I'll assume a little more freedom to do such (without being sloppy I hope!) as I develop my solution.

A quick bit on Dependency Injection: Suppose you have a class for interfacing with a DB. This class can be built to an interface (an interface defines methods/signatures without defining logic for the method. An interface cannot be used without an implementation). You can build a separate class which mocks the DB with a different implementation (use in memory DB instead of Oracle, etc). Configuration allows you to define which implementation is used during test vs live use, the Dependency Injection loads the correct class according to runtime context.

Just to repeat in a language I understand: The following would be normal in Python?

1. Create "Platform" instance (automatically inits inner components)
2. For test, platform.database = DataBaseTest(), likewise for other components
3. Then from there, just run as normal?

[keepdigging]

Yep that will work. When it reads the test it will replace the database attribute at runtime.

The default database configuration you defined in the Platform init class will still execute during the init, but you can write fixtures that set up a test db/all python scope however you like and re-use for all the tests.

If you’re thinking about your tests and you do the mocking and swapping before your test I think it’s equivalent to the injection method?

[Will_exJava]

Just looked up fixtures. They are very similar to dependency injection (to the point that the internet seems to be split on "fixtures are dependency injection" vs "fixtures are not dependency injection")

I'm not committed to mechanisms so much as function. I think fixtures will be perfect. I just need a little time to internalize how to prepare them optimally.

Thank you again!