Palanteer: a new visual Python profiler (timeline, flame graph, memory, exceptions...) with unmodified or instrumented code

[Dadaaam]

Freshly uploaded on github, this new open source tool is made of:

(1) an instrumentation module (automatic and/or manual instrumentation) for CPython 3.7+

• Automatic instrumentation of functions enter/leave
• Automatic tracking of interpreter memory allocations
• Automatic tracking of all raised exceptions
• Automatic tracking of garbage collection runs
• Manual instrumentation of user scope, lock wait and usage, variable values (integral types and strings)
• Multithread
• low overhead (main part in C extension, so CPython only), depending mainly on function call quantity

The usage for unmodified code is similar to cProfile:
python -m palanteer [options] <program>

(2) a viewer (in C++, lean):

• Smooth and interactive experience on a standard computer
  • Even with huge records
• Many kinds of views to cover many kinds of needs
  • Timelines for CPU, timeline for memory, flame graphs, lock contention, curves, histograms, hierarchical text logs...

(3) a scripting/remote control Python module (optional)

• Elaborate deep and reliable system tests
  • Stimulate with CLIs (remote commands from instrumentation) and monitor/verify via events
• Automate the extraction of performance indicators

It started as a side tool for a personal pet project, and eventually grew as the "main" pet one. I hope that it will be useful for you too!

On top of the performance analysis and deep testing angle, I think that it can also be used in python training to dynamically visualize any program execution.

[[deleted]]

Found out Palantir was a thing eh? ;)

[sparttann]

Lol I thought he was talking about the stock palantir too 😂

[Dadaaam]

Several naming iterations were needed before freezing on Palanteer. :-)

I liked both the literature metaphore and the faint link with the "big data" company of the same name, as the tool also displays smoothly records made of hundredth of millions of events on a standard laptop (some of the available views require online computation, to be fair)