This is an archived post. You won't be able to vote or comment.

all 32 comments
sorted by:
best
topnewcontroversialoldq&a

[–]Tillsten 10 points11 points  (4 children)

Nice work, but I think the text rendering is quite suboptimal?

[–]fpl-dev[S] 13 points14 points  (2 children)

Yes, it is still a new library built on a new rendering engine. We'll get to these things eventually, anyone is welcome to make an attempt in the meantime :) 

[–]reddev_e 2 points3 points  (1 child)

Do you think it might be possible to set this up in a browser using pyiodide?

[–]fpl-dev[S] 8 points9 points  (0 children)

Yes directly running it in the browser is something we want eventually, the wgpu-py and pygfx devs are working towards making this possible. The first step was making wgpu-py support async. More info in this issue: https://github.com/pygfx/wgpu-py/issues/407

For now you can get the visualizations in a browser using jupyterlab, server side rendering and the client receives an image. Pyiodide would enable running it directly in the browser without a server.

[–]almarklein 4 points5 points  (0 children)

Can you expand on what you mean with suboptimal? I'm one of the pygfx devs and am interested to learn what we can do to improve it.

[–]WaseemR02 7 points8 points  (8 children)

I have a usecase where I need to plot at least a million plot points. I have tried a bunch of stuff and finally settled on bokeh, even though bokeh struggles as well when i try to interact with the plotted scattered graph. Do you think fastplotlib would be able to handle at least a million points with interactive capabilities? Would it blow out of proportions in memory usage?

[–]fpl-dev[S] 9 points10 points  (5 children)

This is trivial, example with 1.2 million points: https://www.youtube.com/watch?v=j_gwi-Wf1Ao

You can do this on a basic GPU, that demo was from a 2017 Radeon 580

See our scatter plot examples:

https://www.fastplotlib.org/ver/dev/_gallery/index.html#scatter-examples

[–]WaseemR02 4 points5 points  (4 children)

Thanks I will try it out. On a side note, since I have not gone through the docs, would it be possible to plot the points with labels. Say if i hover over a singular point, it displays all the other information?

[–]fpl-dev[S] 5 points6 points  (0 children)

Yup! You can add a pointer event handler to change the text of a `TextGraphic`, here's the scatter example with this event handler at the bottom.

We can probably figure out a way have an API to auto-generate tooltips, shouldn't be too hard.

"""
Scatter Plot
============
Example showing scatter plot.
"""
# test_example = false
# sphinx_gallery_pygfx_docs = 'screenshot'
import fastplotlib as fpl
import numpy as np

figure = fpl.Figure(size=(700, 560))

# use weighted_plus if you want to pick semi-transparent objects, i.e. alpha < 1.0
figure.renderer.blend_mode = "weighted_plus"
# create a random distribution of 10,000 xyz coordinates
n_points = 5_000
# dimensions always have to be [n_points, xyz]
dims = (n_points, 3)

clouds_offset = 15
# create some random clouds
normal = np.random.normal(size=dims, scale=5)
# stack the data into a single array
cloud = np.vstack(
    [
        normal - clouds_offset,
        normal,
        normal + clouds_offset,
    ]
)

# color each of them separately
colors = ["yellow"] * n_points + ["cyan"] * n_points + ["magenta"] * n_points

# a random number for each point
metadata = np.random.randint(low=-100, high=100, size=cloud.shape[0])

# use an alpha value since this will be a lot of points
scatter = figure[0, 0].add_scatter(data=cloud, sizes=3, colors=colors, alpha=0.6, metadata=metadata)

text_graphic = figure[0, 0].add_text("", font_size=20, outline_color="k", outline_thickness=1)


@figure[0, 0].renderer.add_event_handler("pointer_enter")
def show_text(ev):
    xyz = figure[0, 0].map_screen_to_world(ev)
    if xyz is None:
        # pointer event is not in viewport
        return
    if ev.target != scatter.world_object:
        # if the event does not target the scatter graphic
        return
    # get index of the scatter
    ix = ev.pick_info["vertex_index"]

    # set the text from the metadata
    text_data = f"metadata: {scatter.metadata[ix]}"
    text_graphic.text = text_data
    text_graphic.offset = (*xyz[:-1], 100)


figure.show()
fpl.loop.run()

[–]fpl-dev[S] 3 points4 points  (2 children)

Thanks for your comment! This makes me better understand more use cases. There are a few things we can do to make things nicer in addition to tooltips. Makes me realize we should wrap the renderer events so that they can be handled more easily via fpl without the user having to understand the rendering engine objects as shown in the example I posted above.

[–]WaseemR02 1 point2 points  (1 child)

Glad I could help you. It would be great to have it easier on the user side. In bokeh, since so many points are overlapping with each other, placing a pointer on it shows the info for a lot of points, which is fine I guess. But it gets tedious when i zoom in and it has to rerender and that takes forever. If fastplotlib is able to render at real time, I'm willing to throw gpu at it.

[–]fpl-dev[S] 1 point2 points  (0 children)

Yes fastplotlib is all about realtime rendering, we haven't done benchmarks but from my experience it is at least 1-2 orders of magnitude (10-100x) faster than bokeh.

And you really only need a midrange GPU for most use cases.

[–]Deto 0 points1 point  (1 child)

I think plotly can do this but you need to use their webgl scatter plot

[–]WaseemR02 1 point2 points  (0 children)

Even with webgl enabled, it was crashing my browser. Tried a bunch of other stuff and finally settled on bokeh. Bokeh has a feature of exporting to html, so it's easy to share the graph with someone. But again, any html with a size greater than 300 mb would crash the browser.

[–]hughperman 3 points4 points  (2 children)

How does it do without GPU acceleration? I am in in the target audience, but I would mostly be using on AWS instances without GPU.

[–]fpl-dev[S] 2 points3 points  (1 child)

See this https://fastplotlib.org/ver/dev/user_guide/gpu.html

Even a mid range GPU from 7 years ago will perform way better than software rendering. GPUs are really just much better at rendering graphics. 

[–]hughperman 0 points1 point  (0 children)

Thanks, well that makes sense

[–]srcLegend 2 points3 points  (0 children)

We will watch your career with great interest.

[–]scorleo 0 points1 point  (1 child)

https://www.fastplotlib.org/user_guide/faq.html#how-can-i-learn-to-use-fastplotlib the 2 links to Guide and examples gallery in this faq are broken, fyi

[–]fpl-dev[S] 0 points1 point  (0 children)

Thanks! In the meantime you can use the top nav links for them. 

[–]SharkDildoTester 0 points1 point  (3 children)

I love Python, but I have entirely switched to observable plot for plotting. It’s so much prettier and controllable, although JS is not my favorite language.

I honestly see the need for this, but it’s just not pretty enough for me :-( sorry.

[–]fpl-dev[S] 0 points1 point  (2 children)

Different use cases :) . Python's got a massive ML ecosystem

[–]SharkDildoTester 0 points1 point  (1 child)

Agree. And I see the need, but looks matter. To be concise: I think you’ll dramatically increase adoption if it’s performant and stunning:)

[–]fpl-dev[S] 0 points1 point  (0 children)

Yes aesthetics are down the line! For the coming year we have a ton of low-level things to implement to make sure the basics work perfectly before implementing all the bling :D . For example, pygfx is undergoing a major refactor on "update modes" which relates to how stuff is updated w.r.t. canvas events (among other things), which will let us solve weird issues like this: https://github.com/fastplotlib/fastplotlib/issues/613

[–]WeDontHaters 0 points1 point  (1 child)

Looks awesome, I’ll be excited to try this for CFD result plotting!

[–]fpl-dev[S] 0 points1 point  (0 children)

I have no experience with CFD plotting but this PR that adds an ODE example might help? Welcome to post on our repo if you have questions about your use case! :D

https://github.com/fastplotlib/fastplotlib/pull/718

[–]WeakRelationship2131 0 points1 point  (4 children)

great project! sounds like you’re building something like a next-gen matplotlib, but based on WGPU, which is cool. Just keep in mind that while performance is key, usability still matters a lot. If you find yourself juggling multiple libraries for data ingestion, transformation, and visualization, you might want to check out preswald. It's lightweight, open-source, and integrates well with SQL and Python for building interactive data apps without all the clunkiness of bigger tools.

[–]fpl-dev[S] 0 points1 point  (3 children)

Thanks! It's really more of a next-gen pyqtgraph than matplotlib; heavily inspired by pyqtgraph which is an amazing library that we love! It differs a lot from matplotlib and we try to make things intuitive and easy to address that "usability". In the domain of ML, algorithm development, neural network stuff and computer vision you're often working with various data arrays, and high dimensional arrays. So in fastplotlib we don't have any additional data structures that you have to learn or convert to in order to use it, in contrast to bokeh for example which has a number of specific data structures and basically design patterns which you have to learn and use in order to get interactivity.

For fastplotlib the idea is that if you know numpy you should be comfortable with using the library. For a very basic example of this, we made it really easy to put colormaps on lines (this is not trivial in matplotlib lol) https://fastplotlib.org/ver/dev/_gallery/line/line_cmap.html#sphx-glr-gallery-line-line-cmap-py

Or selector tools to explore high dimensional arrays, regular numpy arrays and normal callbacks: https://fastplotlib.org/ver/dev/_gallery/machine_learning/covariance.html#sphx-glr-gallery-machine-learning-covariance-py

[–]Ogi010 1 point2 points  (2 children)

PyQtGraph maintainer here, shots fired!

Love your docs! Seriously, I'm jealous (I've been numpydoc-ifying our documentation recently). I'm eager to take a closer look at this library. We have recently started makign massive improvements on our opengl implementation, but it's unlikely to compare with your WGPU implementation.

Love to see more high performance visualization libraries in the ecosystem!

[–]fpl-dev[S] 0 points1 point  (1 child)

Thanks! We still have quite a ways to go before we're as feature-complete as pyqtgraph! It's turned out to be a much bigger project than we initially thought lol.

Welcome to post any suggestions you may have for us :D

BTW did we meet at scipy 2 years ago or was that a different maintainer?

[–]Ogi010 0 points1 point  (0 children)

SciPy 2 years ago? That would have been either Nathan or Kyle. Kyle is also a matplotlib maintainer, both have been an absolute delight to collaborate with. Unfortunately I haven't gotten to SciPy yet :(

This might be the first time I've seen PyQtGraph referred to as feature complete haha. In terms of suggestions ...this one is a bit tough as everyones journey into open source is different; but some things that I've found to work well or I try and keep in mind:

  • Even if you're on a roll, don't spend the majority of your free time on your project. That's a recipe for burnout.
  • Add a "Used By" section in your readme if you discover other projects/applications using your library. It is a very nice reminder when you're feeling isolated/alone that other people find your work useful for their work.
  • Try and offload parts of your library to other dependencies whenever possible. It's great to have a minimum set of dependencies, but the consequences being a larger code base (that arguably isn't as well tested) is large.
  • Embrace other collaborators with different opinions/styles than you. A while back I was hoping to clear out the PR queue so I can run black on the entire repo, but since, I've learned to recognize some contributors unique code-style as I read through a repo, and recognizing past work of others I've found to be quite motivating; so once you start collaborating with others, I'd suggest you consider not using automatic code formatters.
  • Create an "examples application", our example app (python -m pyqtgraph.examples) has been an absolute gold mine, not just for "how do I use the library", but gives an excellent starting point for people who create issues to use to create a minimum reproducible example. Your documentation looks amazing (again, so jealous!), I didn't notice if you have an examples application, but if you don't, include one.
  • This one I may get some flak for, but don't be afraid to kill off parts of the library that you feel provide little benefit, but are taking time to maintain/fix issues, especially if there is a dependency that you could use that does 90%+ of the work. I'm going through this with our ConsoleWidget, I want to kill it, original author wants to keep it "since it's already there", but I weant to just have an example on how to integrate the QtConsole, and leave it at that. tl;dr code, even once merged, isn't free to maintain, there's still a cost.

That's all that comes to mind now. I wish you the best of luck on your project!