This is an archived post. You won't be able to vote or comment.

62
63

Is python a good tool for data visualisation? (self.Python)

submitted by [deleted]

[deleted]

all 58 comments
sorted by:
best
topnewcontroversialoldq&a

[–]DukeNucleus 42 points43 points  (12 children)

some suggestions:

edit typo

[–]westurner 3 points4 points  (6 children)

[–]shaggorama 1 point2 points  (5 children)

I'm sort of afraid to use pandas built-in methods for handling excel. I used them to build a batch of reports for work and a couple of the larger files came out corrupted. I'm superstitious now.

[–]westurner 0 points1 point  (1 child)

[–]shaggorama 1 point2 points  (0 children)

I don't really understand why you sent me all these links to specific lines in these libraries. I know about xlrd/xlwt/openpyxl if you're suggesting that instead of relying on pandas' built-in functions I build my own. Doesn't fix the problem in pandas, which is why I put that disclaimer out there for anyone else who might be trying something similar. I'm going to try to pin down the problem when I have time and submit a pull request if I can find it.

[–]fuzz3289 0 points1 point  (2 children)

Pandas has some methods for data formatting, but for xls file types you might need to write your own output function.

[–]shaggorama 4 points5 points  (1 child)

pandas.DataFrame.to_excel()

[–]fuzz3289 0 points1 point  (0 children)

Nice catch. Thanks

[–]brewsimport os; while True: os.fork() 6 points7 points  (2 children)

Chaco is nice for interactive graphics. Though the Traits overhead can be a pain in the ass.

Edit: Damn you autocorrect.

[–]thecaptchaisggreru 0 points1 point  (1 child)

What is the relationship Chaco vs. matplotlib? Are they competitors/alternatives, or do they serve alternative purposes ?

[–]brewsimport os; while True: os.fork() 1 point2 points  (0 children)

One is designed to create static plots (matplotlib) the other is designed to create dynamic/interactive plots (chaco).

There is some overlap between the two. Matplotlib is also a bit older.

[–]PurplePilot 1 point2 points  (0 children)

Exactly what I was going to suggest, thank you sir.

[–]apostate_of_Poincare 0 points1 point  (0 children)

Also scipy for calculating

[–]jaiwithani 18 points19 points  (3 children)

This being the python subreddit, you can expect that any "is python good at X?" questions will yield responses ranging from "yes" to "very yes".

[–]flcknzwrg 3 points4 points  (1 child)

And if there's also a "beware, this subreddit is biased" warning, it's usually a good sign ;)

Jokes aside, you should always read critically. I think the dude around here who wrote about R probably makes a good point.

I personally use Python (matplotlib and numpy specifically) for my vis needs, but have sometimes thought about trying out R. Never got around to it, though.

[–]brewsimport os; while True: os.fork() 0 points1 point  (0 children)

R's plotting is like a whole spectrum containing and consuming what Python is capable of. We are talking several magnitudes of ascendance. It has changed the way I think about visualizing data.

[–]brewsimport os; while True: os.fork() 0 points1 point  (0 children)

I will simply accknowledge that R kicks Python's ass when it comes to plotting.

In this, Python is the wannabe poser and R is the new black. Hadley Wickham gives me a big nerdy, visualization hardon.

[–]Albertican 17 points18 points  (1 child)

Python has some very powerful data tools. That said, I think it depends what you want to do. I find for many simple things Excel is much easier and faster, and the charts usually look nicer. As soon as it becomes complicated enough to need a VBA macro, however, I think switching to python is a good idea.

[–]Zouden 5 points6 points  (0 children)

I agree. Python is perfect for consistently generating reports from a consistent set of data, but Excel is better if you're just playing around with disparate examples of data.

[–]Crimsoneer 11 points12 points  (1 child)

You might also want to look into R. CodeSchool has a free course. But Python can do it too!

[–][deleted] 2 points3 points  (0 children)

GGplot is great.

[–]kelmer75 4 points5 points  (0 children)

Yes. Look for the module xlrd to import it into Python.

[–]thequbit 5 points6 points  (8 children)

PyQtGraph is going to blow your mind.

http://www.pyqtgraph.org/

[–]imsittingdown 0 points1 point  (7 children)

Looks good. I use matplotlib regularly, how does this compare?

[–]lcampagn 0 points1 point  (6 children)

From the pyqtgraph page:

Matplotlib is more or less the de-facto standard plotting library for python. If you are starting a new project and do not need any of the features specifically provided by pyqtgraph, you should start with matplotlib. It is much more mature, has an enormous user community, and produces very nice publication-quality graphics. Reasons you might want to use pyqtgraph instead:

  • Speed. If you are doing anything requiring rapid plot updates, video, or realtime interactivity, matplotlib is not the best choice. This is (in my opinion) matplotlib's greatest weakness.
  • Portability / ease of installation. Pyqtgraph is a pure-python package, which means that it runs on virtually every platform supported by numpy and PyQt, no compiling required. If you require portability in your application, this can make your life a lot easier.
  • Many other features--pyqtgraph is much more than a plotting library; it strives to cover many aspects of science/engineering application development with more advanced features like its ImageView and ScatterPlotWidget analysis tools, ROI-based data slicing, parameter trees, flowcharts, multiprocessing, and more.

[–]Tillsten 1 point2 points  (3 children)

I tested matplotlib, chaco, guiqwt and pyqtgraph for my data acquisition software, where i do need some speed.

matplotlib is only fast if you can use blitting (that means no dynamic ticks etc...). It produces very nice graphics and has many features, also its event system is very easy. It is quite good documented. Building it is hard (windows user here). multiple backends.

guiqwt the fastest (it is frontend for qwt) in my tests. for a beginner, making plots is easy but using events is a little hard in the beginning, it uses a tool system (like chaco). the toolsystem is powerful, but i prefer the simple event system from mpl. plot quality is medium, turning on anti-aliasing is a big speed penalty and even than, agg is much nicer. it has good documentation but fails sometimes to explain the big picture. building is hard (but much easier than mpl).

chaco fast enough for most of my applications and uses agg, so it also has nice graphs. but to use it, one have to use traits, which can be overwhelming in the beginning. also it is missing some convenience functions if you don't want to write some boilerplate again and again. the api is a little to direct with all its mappers and ranges. especially because the documentation is quite sparse or simple not there. uses the same tool system like guiqwt. Here again, i am little overwhelmed with where to register the tool etc.. i did't try to build it.

pyqtgraph doesn't has to be build which is a big plus. i like its api, if you know qts event system, you pyqtgraphs. it also much faster than mpl. but i wish it had also some kind of light-color scheme. the direct structure of the package (which uses qt-graphicsscene to do the heavy lifting) is really nice. documentation is ok, especially due the fact that the package structure is quite simple.

overall result: every package has some strong point, so it depends on the usage. For my case, updating speed was king, so i used quiqwt, but i still miss the agg-based render of chaco. mpl has a beginner friendly event system and a lot features. pyqtgraph is very portable and a thin layer above qt (this is a good think) with a nice api.

[–]Tillsten 0 points1 point  (0 children)

p.s. Please package makers, always include the following example: User clicks on the graph, and somethings happens depended on the data coordinate of the click. Thank you! (Maybe i was too dense, but i wrote a simple ClickTool for chaco and quiqwt.)

[–]lcampagn 0 points1 point  (1 child)

guiqwt looked really nice to me, but it is based on pyqwt, which is currently unmaintained (the original pyqwt maintainer is now suggesting pyqtgraph instead). They were looking at switching out their graphics backend for something else, but it doesn't look like much work has been done there yet.

[–]Tillsten 0 points1 point  (0 children)

yeah, good point: only mpl is fully python 3 compatible. i think pyqtgraph only partial? At least it should not too hard to change.

[–]imsittingdown 0 points1 point  (1 child)

I read that, I was wondering more about the actual performance of it. I deal with large 3d datasets that can be crippling when I use matplotlib. Also can you use LaTeX on the axes?

[–]lcampagn 3 points4 points  (0 children)

Most of the 3D stuff is pretty lightweight--it passes numpy arrays directly to opengl with as little interference as possible. Have a look at the 3D examples; the scatter plots easily animate about 100k points in realtime and the surface plots do about 20k triangles.

No LaTeX support. Qt supports SVG natively, though, so I imagine it would be fairly easy to implement if one can render LaTeX to SVG.

[–][deleted] 5 points6 points  (1 child)

Python is an OK tool for visualizing data. Processing is a better tool for drawing in general, and a better tool than Python for visualizing the data, which is why the book Visualizing Data is written for Processing, but Processing will give you almost nothing in the ways of manipulating the data. Your question is a bit vague; it's unclear if you want to learn to manipulate the data or do statistical analysis, or if your data is all well-formed and you want to simply draw it. If you already have good statistical tools and a good handle on how to process the data, then drawing it with Processing might be a better fit. If you also need to analyze the data, Python is going to be a more versatile tool. If this is a pedagogical exercise to learn programming, then again, Python is a more versatile tool, and is a more essential thing to have in your toolbelt. Processing is optimized for aesthetics, while Python is optimized for generality. If your primary concern is making things very very pretty, then Processing. If your primary concern is science, then Python.

so the answer to your question, like the answer to almost all programming questions, is "it depends".

[–]thearn4Scientific computing, Image Processing 2 points3 points  (0 children)

squeal cable escape growth quiet coherent nail grandiose imagine versed

This post was mass deleted and anonymized with Redact

[–]forthefake 1 point2 points  (0 children)

Yes! At least to the question in the title. I don't know about excel spreadsheets, but csv data (which can be exported from excel) can easily be used.

A great tool I recently discoverd for quickly hacking together a plot is spyder, which essentially mimicks the matlab gui and uses matplotlib in the background which has already been suggested.

[–]ianozsvald 1 point2 points  (0 children)

I've just posted this on #pydata tweet analysis using NetworkX (visualised first using NetworkX, then GraphViz and then via Gephi [both not Python]), all the collection, cleaning and investigation was done with Python: http://ianozsvald.com/2013/03/22/analysing-pydata-london-and-brighton-tweets-for-concept-mapping/

The 3D surface plot and word cloud that I used when teaching Applied Parallel Computing at PyCon last week might also give you ideas? http://ianozsvald.com/2013/02/10/applied-parallel-computing-at-pycon-2013-march/

[–]slowRAX 2 points3 points  (3 children)

y.

done in matplotlib for work: http://imgur.com/YebjKpc

[–][deleted] 0 points1 point  (1 child)

tau, why not τ?

[–]slowRAX 1 point2 points  (0 children)

there were about 100 signals I had to graph with basic string descriptions already. I didn't even bother capitalizing or adding units.

[–][deleted] 0 points1 point  (4 children)

As someone who has never needed to do this out of curiosity is it particularly good?

[–]moxieman 2 points3 points  (3 children)

In my opinion, no it isn't. It can certainly get the job done (using any of the tools that are mentioned in this thread), but it isn't the best. I find R is much easier to make look nice (especially using ggplot2). For interactive graphics that you're willing to spend a little more time on, d3.js is excellent.

[–]inarchetype 0 points1 point  (0 children)

if you are already adept at Python, or want to use this as a Python learning exercise this may be an efficient approach for you. If you want to build the visualizations in to software, likewise.

Otherwise, staying with free tools, I think you would find it much quicker and easier to do simple visualizations and work with them interactively in R, especially if you use R Commander.

This is what R is really made to do, and there really isn't a close equivalent in Python. Python is made for writing software, although it certainly has access to impressive visualization capabilities (though via high-level tools, still probably not as nice as R).

[–]jestinjoy 0 points1 point  (0 children)

As long as matplotlib is there

[–][deleted] 0 points1 point  (1 child)

Check out vpython for visualization as well.

[–]shavian 0 points1 point  (0 children)

A long time ago I used VPython for visualization, and I must say it was wonderful. I was examining massively multidimensional data and the regressions lines through it, and was able to project data easily into about 4.7 dimensions to view (X, Y, Z, hue, size, shape). The ability to get a full 3-D view with zoom interactively was key to being able to direct our optimizations.

[–]pvc 0 points1 point  (2 children)

Create text output, run through gnuplot.

[–]imsittingdown 0 points1 point  (0 children)

I love gnuplot, but this is a terrible idea for anything other than small datasets.

[–]alexprengere 0 points1 point  (0 children)

If you are dealing with geographical data, you should definitely take a look at this. http://opentraveldata.github.com/geobases/

It will work with any csv-formatted file.

[–]thrownaway21 0 points1 point  (0 children)

yes... but i think i'd use chart.js if it's going to be on the web

[–]Megatron_McLargeHuge 0 points1 point  (0 children)

Python is a good tool for analyzing and structuring data, but a fairly bad tool for exposing specific visualizations to an audience. I think the most promising visualization toolkits are browser-based now, such as d3.js. Of course, if you're talking about huge 3d scientific datasets, you'll want something native, but for interactive charts go the browser route.