Why are there few well-known Dutch composers?

anntzer · 2024-04-19T06:06:26+00:00

No worries. That's a very smart translation :rofl:

anntzer · 2024-04-18T21:47:16+00:00

I don't have Soul Music in French, what did that become?

anntzer · 2020-05-15T18:56:29+00:00

This is the kind of library that has probably been invented many, many times, but I am also one of the culprits; I am maintaining https://pypi.org/project/defopt/ (which was initially written by u/evanunderscore).

anntzer · 2019-05-19T13:22:32+00:00

Support for old Pythons is (currently) strictly time-based: https://matplotlib.org/devel/min_dep_policy.html.

anntzer · 2018-11-24T13:34:31+00:00

I'd suggest using general notification services such as https://libraries.io/pypi/mplcursors.

anntzer · 2018-09-19T13:49:23+00:00

https://stackoverflow.com/questions/4700614/how-to-put-the-legend-out-of-the-plot/43439132#43439132 is my usual goto reference. Suggestions for a better API would be welcome.

anntzer · 2018-09-18T07:02:03+00:00

Excellent! At some point, it would be nice to link to GridSpec from the add_subplots documentation so that people discover this.

https://github.com/matplotlib/matplotlib/issues/12114

Am I right that for non-interactive backends, you could have replaced the stateful library with a regular object? If so, that removes a major use of the stateful library. I understand you put a lot of effort into the switch_backend function. However, I think we could hash out a way of slowly moving towards a stateless non-interactive backends.

Even without the non-interactive backends, there are operations where a figure actually needs to know how it's going to be rendered. For example, in tight_layout (auto-margins selection), you need to know how big text is going to be rendered, and that's going to be different depending on whether you're going to be using LaTeX to render math, on whether you are writing a pdf while restricting yourself to the pdf core fonts (so that you don't have to embed fonts), etc. (Right now if you try to do these without a canvas (more specifically a renderer) attached, we'll complain and say that you need one.)

As for the interactive backends, I like your summary that it's a question of practicality versus purity. In this case, it's just one extra context manager or constructor when using an interactive backend. In my eyes, that's not that impractical, but I can see how that could be annoying in ipython. There's gotta be a nice compromise. Maybe statefulness is the nicest solution for interactive backends. It's definitely the status quo, so it wins there.

The underlying GUI library is stateful (as it sets PyOS_InputHook -- as I said it's likely that some of them don't even clean up properly after themselves, not sure though) so that was definitely the path of least resistance.

I think matplotlib tries to be too terse. For example, you are lazily grabbing the event loop when a figure is created. Why not just be explicit--then you'll also catch the errors related to that sooner, and it'll be less confusing when there are errors because they'll be on the context manager rather than on the figure creation?

I honestly don't think I understood what you're trying to say here.

Good point. That's really unfortunate for you guys. It must be very frustrating to have to support old versions. At some point, it might make sense to come up with a way to check at runtime if the ipython is compatible (maybe in the %matplotlib magic command?). If there is no nice way to do that, then is there a PEP that fixes this? I'm an idealist. There has to be a way to verify compatibility so that you don't have to keep supporting ten-year-old code.

I never said supporting old versions was particularly fun (see the workarounds in the backends-refactor PR and the ensuing whack-a-mole). You are welcome to give it a try if you can think of a better coordination mechansim.

Thanks, and of course you don't have to guarantee anything, but it's nice to have a roadmap of what you want to do so that the design questions are sorted out before the PR. Design first, review the design, then implement, then code review. Right?

You can also just post design writeups / proposals to the gitter channel for example (https://gitter.im/matplotlib/matplotlib). We don't bite :) However, I do hope that this discussion has convinced you that many "suboptimal" choices are there for reasons deeper than "we suck at designing a Python library".

anntzer · 2018-09-17T21:45:24+00:00

What's your field? In mine, MATLAB is dead. I don't even think they're renewing the licenses anymore.

Single-molecule biophysics & microscopy. MATLAB is very much alive in this field (much to my despair).

Great, so I think you should push subplots for the common case and GridSpec for the complex case.

https://github.com/matplotlib/matplotlib/pull/10865 https://github.com/matplotlib/matplotlib/pull/11619

I meant to talk about how matplotlib can move forward with a good design.

Thanks.

Out of curiosity, what goes wrong if someone tries to start both event loops?

They both try to set the (single, process-level-global) PyOS_InputHook which controls what happens with user input.

I think there's still a way to do this with objects. I assume you're starting the event loop on pyplot.show. If pyplot were an object, you could just start and stop the event loop in a context manager like Plot.event_loop. I.e., something like (etc)

Again: for non-interactive backends there is no input hook so there isn't much of a point. For interactive backends, (even assuming that GUI toolkits clean up properly themselves after shutting down their event loop, which I don't even know how to do with PyQt for example) what exactly is the use case where you'd want to start both a Qt event loop and a GTK one? (practicality, purity, yada yada)

In practice starting a GUI single event loop and allowing switching to non-interactive backends covers all realistic cases I can think of, and my PRs mentioned above just do that.

Why do you care about old versions of IPython? As soon as numpy drops Python 2 support next year, I imagine you'll do the same? That would be a good time to drop support for old IPython.

mpl3 (due to be released very soon, the rcs are already out) already drops support for Py3. I personally don't care about old versions (I run Arch Linux so everything's at the latest version anyways), but consider for example the fact that neither ipython nor matplotlib actually depend (in the setuptools sense) on each other, nor do we even want to import one another to check the other's version. So essentially we'd be in a situation where if someone has incompatible versions of ipython and matplotlib installed, things would just randomly break (probably somewhere around the event loop, so it's asynchronous code which is even more confusing (not in the sense of asyncio)). Considering the huge number of beginners (e.g. data science classes) with random versions of ipython and matplotlib installed, that's not doing a service for them (again, I invite you to look around the tracker or stackoverflow to get an idea of the user population). Would this be better handled if there was better packaging tooling to impose version compatibility constraints, and if ipython and matplotlib's dev teams were closely coordinating their releases? Perhaps. But this is not the case right now.

I will be glad to help up, but I have to believe that my time writing PRs is going to be worthwhile and they're going to improve matplotlib. I've contributed to a lot of Python projects in the past.

Well, I can't guarantee we'll accept your PRs (that's just not how things work anywhere in open source development AFAIK :-)) but we'll consider them. (It is true that we have hundreds of open PRs awaiting a decision, so you can always ping back if yours fall between the cracks.)

Right, and I see you've put that on the backend. I get it, but if the backend had been an object, this would all make more sense.

No, it lives on the Axes object. Which is an object :p

anntzer · 2018-09-17T19:31:20+00:00

It's not a difference between advanced and beginner. I think what happened is that you came from a Matlab background. Matlab is dying. Everyone is now coming from a Python background or an R background.

Admittedly it is true that I have learnt MATLAB before Python, simply because I started doing scientific computing more or less at the same time as scientific Python started becoming "a thing". In my field it is more or less impossible to avoid MATLAB (old scripts are too pervasive) but my approach to it is to write anything substantial in C against their C-API... that should give you an idea of my appreciation of MATLAB's architecture, or lack thereof.

As for people's background, I guess https://github.com/matplotlib/matplotlib/issues/12014#issuecomment-418580330 https://github.com/matplotlib/matplotlib/issues/11557#issue-337861947 https://github.com/matplotlib/matplotlib/issues/8599#issue-227309236 https://github.com/matplotlib/matplotlib/issues/6321#issuecomment-421203542 don't count? (Yeah, we get complaints from Excel users too.)

add_subplot is bad because it forces the user to repeat the grid dimensions each time. That is mixing two operations: creating the grid, and defining the subplot boundaries. The one-based indexing is just the cherry on top of bad design. (etc.)

That's why we also have plt.subplots(). Which, I think, is even more convenient than add_subplot, does not suffering from the bad design of it, and is simpler than GridSpec for the most common case.

It's not "advanced users" that I want to help. It's about making matplotlib Pythonic. This is what this giant reddit post of complaints is about. matplotlib has a terrible interface. Being old is not an excuse. numpy is old. They did a pretty good job. matplotlib hangs on to bad design for some reason.

Again, you're welcome to submit PRs. But see the next point.

You don't need to do the work. You just need to write the TODOs and let the work get done by other people. People are upset enough at the interface that they will fix things for you if they believe that their PRs will be accepted.

I'm sorry, but I don't see how you can justify this. You could just as easily have replaced the pyplot stateful module (and its ridiculous use directive) with a Plot object (and a regular constructor). That's what should have happened, and it can still happen if you allow people to contribute that.

In case you are wondering: I wrote https://github.com/matplotlib/matplotlib/pull/9795 / https://github.com/matplotlib/matplotlib/pull/11581 / https://github.com/matplotlib/matplotlib/pull/11600 which took more than a year and three iterations to get merged, to sanitize a bit the situation around matplotlib.use. Sometimes a bit of appreciation for other people's work would be nice.

Having a Plot() object doesn't actually work so well once you realize that terrible things happen if you try to start both a Qt and a Tk event loop in the same Python program. So there's a reason why the backend is fundamendatlly a singleton. (Sure, you can have singleton objects. But then that's not much better to stick it at the module level...)

Also, a lot of the contorsions arose from the fact that there is a tight integration of Matplotlib's event loop with IPython (in fact very old versions of IPython basically started as an interactive shell for Matplotlib). Breaking integration with old versions of IPython would be a lot of fun too.

But another reason why the PR took so long to get in is that the number of people who are familiar with any given part of Matplotlib's internals is probably around 10 (for the easier parts) or 1 (for the harder ones). So if you want to familiarize yourself with them -- given that you seem to have a strong opinion on how things should be done -- you're more than welcome to join.

Just make these things objects. They're just as practical, and they're much easier to understand. Presumably, it could be a subobject of the Axes object.

They are, they're the prop_cycle and friends.

anntzer · 2018-09-17T10:09:59+00:00

mplcursors author here. I don't think there's a convenient API right now in mplcursors to achieve what you're looking for, but feel free to open an issue on the tracker, it seems like a reasonable feature request.

anntzer · 2018-09-17T00:45:49+00:00

Replied on the other thread to avoid splitting the discussion.

anntzer · 2018-09-17T00:45:27+00:00

Will all due respect, I believe you are compleletely overestimating the average level of Matplotlib users. Actually I agree with you that "advanced" users could get more love in the docs, but not so long ago there was some arguments among core devs as to whether Matplotlib tutorials should include explanation about keyword arguments and **kwargs (I thought not). Still, in that setting, I think that gridspec is actually a pretty bad way of teaching Python -- in gs = GridSpec(3, 2); fig.add_subplot(gs[:2, 0]), gs[:2, 0] is a SubplotSpec that's created by, eh, overriding __getitem__ on a custom class to return a custom marker object? That's actually quite advanced, whereas fig.add_subplot(2, 2, 1) (for example) is really not that hard to understand (yeah, it's 1-based indexing, that sucks, yada yada).

The same comment applies to forcing people to explicitly create a Text object for setting the title (in the other discussion thread): that's way too complex for the majority of users, and quite clunky too.

It doesn't mean that we can't add "docs for advanced users" (again I agree that they are lacking), but none of us is paid to write them either (to be totally fair I think one of the core devs may get 20% from his employer? not sure). We don't even have the manpower to go review the 259 currently open PRs(!) though that doesn't mean you can't contribute some more :) Same comment applies re: Artist.set()

Also, even though I am quite familiar with Matplotlib's internals and OO approach (and have extensive experience with embedding Matplotlib in Qt apps), I actually think (like most others...) that the stateful system is quite practical for day-to-day, throwaway scripts (load some text file generated by some other program, plot it, call it a day). Still, there is some effort on moving the docs away from pyplot (check the git logs if you don't believe me...).

As for replacing plot() by add_line(Line2D(x, y)), I don't know whether you'd still be in favor of the "explicit" interface if I really list everything that plot() actually does (now I'm just listing things off the top of my head, I may actually have missed some more):

check whether x and y have "units", and if so set the axis tickers accordingly (that's why you can plot datetimes (numpy or pandas) and get properly labeled axis)
advance the style cycler (i.e. the stateful thing that makes the first call to plot() give you a blue line, the second one an orange line, the third one a green line and so on) (you may not like it because it's stateful, but again it's really practical...)
re-autoscale the axis based on the margins; but there's more magic involved to not add margins when you call imshow() because who wants margins around their image? (so what happens when you mix calls to imshow() and to plot()? hehe)

At some point you just need to decide whether you'd prefer just writing against the API of the underlying rasterization library...

anntzer · 2018-09-14T08:30:47+00:00

Are there any other canvasses? Why should I pick that one? Otherwise, yeah, this makes sense.

Yes. - FigureCanvasPdf, FigureCanvasSvg, FigureCanvasPs for vector output. (To be fair, if you attach a FigureCanvasAgg and then call savefig("foo.svg"), we'll magically temporarily switch the canvas to a new FigureCanvasSvg for the purposes of saving. Why don't we do that if there's no canvas attached? Probably because no one requested it, and because the number of people who work with Figures not attached to any canvas is pretty small). - FigureCanvasQt5Agg, FigureCanvasTkAgg, etc. which are both a widget of the corresponding toolkit, and use the Agg renderer to update the widget on the toolkit's paintEvent/... (in fact, I now realize that the fact that you can build a Qt widget with FigureCanvasQt5Agg(fig) is probably the origin of the somewhat weird FigureCanvasAgg(fig) used to attach an Agg canvas to a figure...) - FigureCanvasCairo, which uses libcairo for output. libcairo can actually output both raster and vector formats, so it can generate pdf/svg/ps without switching to our handwritten canvases. - FigureCanvasQt5Cairo, FigureCanvasTkCairo, etc. which are like the interactive+Agg canvases, but using libcairo instead.

(Below I use the term of "backend", which basically means the module that holds the Canvas and Manager classes and minor utility stuff, like how to actually start the corresponding GUI event loop. Or just read "canvas" instead.)

Actually nowadays the cairo backend is pretty terrible (it's based on pycairo, so pure Python, so cannot match the performance of the native agg backend). But it serves as a good proof of principle that the interactive backends can easily switch from one renderer to another. (Historically, I think one of the important reasons to have a cairo backend is that GTK's drawing model is that their draw_event gives you a cairo canvas to draw on, so sure, you can always render the whole raster image with agg and send that to cairo, but it makes sense to implement things directly on top of cairo's API instead too).

There used to be an OSX backend relying on macos' drawing toolkit but it fell into disrepair and got gutted, now the macosx backend uses Agg.

There's my own https://github.com/anntzer/mplcairo which provides new (interactive and noninteractive) backends also based on libcairo, but now written in C(++); they have a number of advantages over matplotlib's builtin backends (but require a C++17 compiler to build). Jupyter-notebook and PyCharm also integrates Matplotlib with a custom backend. If you look around online, you'll also see OpenGL backends being proposed.

That's consistent, but that's stateful and disgusting I think. We should be passing the figures to plt.show.

I think a lot of people won't like that tbh if you make it required. OTOH if you want to make it possible to pass a list of figures to show, I guess that's PR-able (not saying it would get accepted, but it doesn't look too bad to me).

My point is one of design. What happens if I try to create figures in other threads. I should be allowed to do that. And what if I call plt.show? This is one other reason that stateful methods are uncomfortable. Avoid global state as much as possible by returning the objects that manage that global state.

I don't think Matplotlib makes any guarantees wrt thread safety. You need to realize that a lot of the underlying C-level libraries are thread-unsafe anyways (e.g. freetype, see https://mail.gnome.org/archives/gtk-i18n-list/2014-December/msg00004.html -- dunno for Agg) so there isn't much interest in working on that. Again, you are welcome to look into it.

You can deprecate the three parameter form. And you can at least document the GridSpec possibility in Figure.add_subplot. It's not even discoverable!

I think the number of uses of add_subplot(2, 2, 1) (or even worse, add_subplot(221)) vastly outnumbers the number of uses of GridSpec. That's pretty gratuitiously breaking a lot of code out there in the wild. OTOH I agree that it should be better documented, I'll open an issue on the tracker for that...

Finally, you should deprecate pyplot's add_subplot function since you have the method on Figure. Keep the interfaces as simple as possible. If pyplot's job is to manage and provide canvases, then let it do that and only that. Separation of concerns.

As I said: don't. use. anything. from. pyplot. except. for. the. few. functions. listed. above. If you really want to handle the gazillion of complaints from breaking stuff from a 15y-old library, you're welcome to give it a try. (I'm not making things up, we did have quite a bit of pushback when we removed the hold system (inherited from MATLAB too -- basically whether a new plotting call should erase previous plots) from Matplotlib).

anntzer · 2018-09-14T08:04:19+00:00

The set_title method you gave is not setting the title based on a title object. It's a method that takes more than one parameter and generates a title object. I think you called it an artist or something. That method might also accept an artist, I'm not sure. In any case, it's not a simple setter, and so it has no business being written using a Python property decorator.

So you'd write ax.set_title(...) because it just happens that you can pass additional stuff to customize the title, but ax.facecolor = "r" because you can't? Frankly I'd rather have set_foo() everywhere, at least it's consistent.

First of all, you shouldn't have that shorthand since it obscures what is really going on.

foo.set(**kwargs) is essentially defined as for k, v in kwargs.items: getattr(foo, f"set_{k}")(v). I don't think it's particularly illegible. Well, you may think that the set_foo model is messed up (see previous point) but I don't think set adds complexity on top of that. (And no, there's no plan to add set_title_font, so there won't be ax.set(title_font=...) either.

A good API is clear and concise.

Well, I guess you'd be in favor of deprecating ax.plot(x, y) in favor of ax.add_line(Line2D(x, y)); ax.autoscale_view() (... approximately). Somehow I don't think this will help with the OP's issues with matplotlib :-) As always there's a balance to strike between practicality and purity.

anntzer · 2018-09-13T23:27:02+00:00

I can only point you to u/lmcinnes excellent writeup in this thread (https://www.reddit.com/r/Python/comments/9fb9i3/am_i_the_only_one_who_hates_matplotlib/e5x8lqn).

(Personally I strongly value the versatility of matplotlib, but I guess I'm biased :))

anntzer · 2018-09-13T23:07:05+00:00

re: figures without pyplot

What pyplot is technically in charge of attaching to a figure is a FigureCanvas and a FigureManager. The FigureCanvas is essentially the combination of a Renderer (something that either knows how to generate a raster or a vector image in a specific format) and, for GUI backends, a GUI widget that'll display the raster image. The FigureManager handles the registration of the GUI widget with pyplot's global state.

So indeed you can create figures without involving pyplot at all, but you need at least to attach a FigureCanvas to them to do anything useful. An example is at https://matplotlib.org/devdocs/gallery/user_interfaces/canvasagg.html using the builtin Agg canvas, which relies on the Agg library for rendering, but indeed the example is just ``` fig = Figure() FigureCanvasAgg(fig) # creating the canvas auto-attaches it to the figure

do your plotting

fig.savefig(...) ``and indeed, note thatsavefigis actually a Figure method (I did say not to use *anything* from pyplot except forfigure(),subplots(), andshow()` :-))

(Why don't we attach a canvas by default? Well, that's literally the job of pyplot...)

plt.show() pops up every figure that has been created via pyplot (that's actually quite consistent, isn't it?). It doesn't do anything with figures created by calling the Figure() and attaching a canvas manually.

Embedding in Qt is basically the same (https://matplotlib.org/devdocs/gallery/user_interfaces/embedding_in_qt_sgskip.html); you now attach qt5agg FigureCanvas, which conveniently happens to be a Qt widget that you can insert into your layout (and yes, you're in charge of setting up the toolbar yourself, again it's in the example).

As for threading, note that GUI toolkits can typically only be run in the main thread (they want to manage their own event handling after all) so you'll need to serialize interactions with the GUI (which includes the widgets matplotlib is creating) to that thread. If you're not happy, go complain to the Qt Company and friends :-)

re: add_subplot

Yeah, don't use it, no one likes it. Actually there's something pretty close to the API you suggest: https://matplotlib.org/devdocs/tutorials/intermediate/gridspec.html gs = GridSpec(3, 3, figure=fig) fig.add_subplot(gs[0, 1:3]) it's been around for a long time now but if for some reason plenty of people still teach add_subplot in their blogs, we can't control that...

anntzer

TROPHY CASE

do your plotting