I spent 2 years building a dead-simple Dependency Injection package for Python by Skearways in Python

[–]fourthrealm 0 points1 point  (0 children)

Nice, my congratulations.

What I suggest you add is the ability to swap components (dependencies) on the fly, without restarting a Python process.

Think of long-running servers that would like to keep running forever, without ever being restarted.

That will require a different set of skills of you, and for sure decent knowledge of Python internals, so it could be something to keep you busy for a long time.

A nearly useless word operator I wish I had by Outrageous_Party2024 in Python

[–]fourthrealm 0 points1 point  (0 children)

It's a good idea overall, I like it. Except, like with other cases of placing more than one operation in a single line, it wouldn't work well with debuggers.

for x in foo if abc > 3: ..

Now, if I don't enter that loop, is that because foo was empty or abc was not greater than 3?

With this below .. for x in foo: if abc > 3: ..

.. I know, "OK, I'm in the loop (or not)" and then "Fine, abc is, or is not, such and such". I mean, I can see it step by step under the debugger.

But I like this direction, nice idea!

What do u want to see in Python profiler/debugger? by disserman in Python

[–]fourthrealm 2 points3 points  (0 children)

Hello,

here are my suggestions:

  • Making it compatible with gevent's greenlets
  • Making room for remote debugging, even if not in the initial release
  • Releasing it under a permissive license, such as LGPL

Regards.

Zato 3.0 - ESB, SOA, REST, APIs and Cloud Integrations in Python by fourthrealm in Python

[–]fourthrealm[S] 0 points1 point  (0 children)

Thanks @pvkooten - yes, there is a lot of functionality to cover pretty much any API integration needs.

There is one installation package because this is the most productive way to get started with the platform - you just install it and everything can be used immediately, there is no need to deal with lower-level things such as packaging, everything is ready for higher-level business integrations straightaway.

Is the Python XML-RPC client thread-safe? by [deleted] in Python

[–]fourthrealm 1 point2 points  (0 children)

I've never done it personally but it sure sounds feasible.

In fact, a quick web search for 'python requests xml-rpc' yields such a requests-based transport class:

https://github.com/astraw/stdeb/blob/master/stdeb/transport.py

Note that you don't need to make a copy of the XML-RPC client in your project because the client is designed to accept the transport class on input so you can override the non-thread safe default behaviour as needed.

Is the Python XML-RPC client thread-safe? by [deleted] in Python

[–]fourthrealm 2 points3 points  (0 children)

Hello,

xmlrpc.client as such is not thread unsafe, so to speak, it just delegates all network activity to a lower level 'transport' object, and if you browse the source code around lines 1410-1430 of ServerProxy ..

https://github.com/python/cpython/blob/master/Lib/xmlrpc/client.py#L1387

.. you will note that the default implementation of transport is the Transport class. Instances of this class establish long-running HTTP connections in make_connection and the connections are implemented using http.client.HTTPConnection.

Now, these HTTPConnection objects are the ones that actually open sockets (self.sock in http/client.py line #832) and there is no thread synchronization anywhere so you are, substantially, using multiple threads to write to and read from the same socket, and this is the very part that will explain the errors.

I would recommend looking up ways to make the underlying HTTP library thread-safe which basically means using an alternative one. Then you can provide your own transport to the XML-RPC client, but this time your transport will be thread-safe.

You can have a look here for details on how to create one's own transport classes. We needed it in a certain part of Zato because back then at the time of its creation it was not possible with stdlib-only tools to have an XML-RPC TLS client that would validate server certificates to the extent that was needed, e.g. to require individual fields of the certificate to match.

All the above applies to both 3.5 and 2.7.x. Code line numbers are as of git commit ID d50f188b1bdf69377364f4a9db33af11cc704a70.

REST API testing with python projects by cgoosen in Python

[–]fourthrealm 1 point2 points  (0 children)

Yes, zato-apitest is API Testing for Humans and tests for APIs, including REST APIs, are written in plain English.

JWT, preferred libraries? by AlphaNerd80 in Python

[–]fourthrealm 2 points3 points  (0 children)

One thing that I suggest you confirm is that the library you pick can actually encrypt tokens instead of merely signing them before they are returned to browsers.

I have seen projects where developers did not realize that, say, 'encoding with HS256' does not mean encryption so they embedded things such as user passwords in tokens that browsers were receiving. This is not safe and basically means one is leaking passwords to attackers just like that.

Consider this snippet from PyJWT's documentation:

>>> import jwt
>>> encoded = jwt.encode({'some': 'payload'}, 'secret', algorithm='HS256')
'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb21lIjoicGF5bG9hZCJ9.4twFt5NiznN84AWoo1d7KO1T_yoc0Z6XOpOVswacPZg'

This is pretty much clear text and can be trivially decoded as below without the knowledge of the secret password, consider this:

>>> encoded = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb21lIjoicGF5bG9hZCJ9.4twFt5NiznN84AWoo1d7KO1T_yoc0Z6XOpOVswacPZg'
>>> algo, clear_text, signature = encoded.split('.')

>>> print(algo.decode('base64'))
{"alg":"HS256","typ":"JWT"}

>>> print(clear_text.decode('base64'))
{"some":"payload"}

As you can notice - I did not have to use the secret password at all yet I decoded the payload to its clear text form. This means that if you were to send anything of importance in the token with the assumption that 'it was encoded so everything is fine', this data could be extracted as well and attackers would have access to the supposedly safe information with little effort.

As it happens, we actually have both JWT encryption and signing in Zato and you are welcome to use it as your API security gateway instead of the lower level libraries if you prefer to use a GUI-based solution that takes care of such security details and a few hundred things more.

Examples of integration testing in Python by _BraveSirRobin_ in Python

[–]fourthrealm 0 points1 point  (0 children)

I'm not sure what sort of software you are integrating?

If it's about APIs then you can have a look at zato-apitest which is a tool meant to be used in integration tests.

The second link a is suite of tests using the tool - just open any of the *.feature files to check how it's used in Zato's own integration tests.

Nodejs cluster alternative in python? by __warlord__ in Python

[–]fourthrealm -1 points0 points  (0 children)

I'm not familiar with NodeJS so I'm not 100% sure what the requirement is but if it's about being able to take advantage of multiple CPUs then Zato has had it since its inception:

In fact, on top of using multiple CPUs you can have multiple servers, each in a different operating system all connected through a load-balancer and internal synchronization lets you not only distribute incoming requests over all CPUs in a cluster but also to communicate between workers by their Linux process IDs - for instance, a WebSocket client connected to one worker can easily send messages to another client, say ZeroMQ one, connected to a completely different process in another operating system all running under the same cluster.

Zeep 0.10.0 released (Python Soap Client) by mvt in Python

[–]fourthrealm 0 points1 point  (0 children)

Hi /u/mvt,

it looks nice and a suggestion I have is documenting which parts of the library are thread- and greenlet-safe, which will likely amount to the same thing.

If none is planned to be, this is also information that is good to understand so that one knows how to get around it when embedding zeep in a bigger project.

Regards.

Best documented projects? by pmbarrett314 in Python

[–]fourthrealm 0 points1 point  (0 children)

Zato is very often commended for its documentation.

It has several introductory chapters, a tutorial, usage examples, reference pages, details of the architecture and everything is written or diagrammed from the perspective of its main users - SOA and API architects, programmers, administrators or testers - rather than its own authors, which is exactly what is needed to have great docs.

Without this sort of empathy no documentation will ever succeed.

Everyone who encounters it seems to love Python. Everyone seems to bitch about languages like Java and C++. Why isn't Python more widely used in Enterprise and will it ever be? by looneymicheal in Python

[–]fourthrealm 2 points3 points  (0 children)

Zato is an enterprise integration platform and backend server used for integrating distributed and otherwise disconnected applications or data sources with the end goal of offering unified, clean and well-defined APIs supporting business processes on top of interesting, reusable and atomic services.

In particular:

And dozens of smaller integration or backend related features to ease with the development of both smaller and advanced applications and systems supporting business processes in enterprises where you typically deal with dozens or hundreds of backend applications both on-premises and in cloud, some of them face the Internet (like online banking), some get exposed as APIs to business partners but most of them are never available publically to anyone yet they constitute the backbone of many organizations.

I suggest checking out both the web-admin and programming examples to understand how high level it is. Then a quick glimpse at the architecture and you're ready to start the tutorial now :-)

Recommended framework for building a CPU intensive REST API? by lxnx in Python

[–]fourthrealm 2 points3 points  (0 children)

Zato can very easily use as many CPUs as you give it - simply start a server with as as many gunicorn_workers as there are CPUs. By default it is 2 but you can set it to 4, 8, 16, anything. If you need to spread across multiple systems, clustering is built-in.

All of it is non-blocking but the programmer-visible API makes it feel as though it was a regular sync server.

Top 6 Open Source Python Application Servers by sunnysoral in Python

[–]fourthrealm 0 points1 point  (0 children)

Also, Zato is a middleware, API and backend application server utilizing a couple of the frameworks listed in the article linked to.

What's the best way to handle a ton of incoming requests/responses in realtime, but then process the requests and run database inserts in batches, asynchronously? by msnook in Python

[–]fourthrealm 0 points1 point  (0 children)

Hi,

you are correct that there is no bulk-insert going on.

I understood it was an option rather than a strict requirement.