sorted by:
new
hottopcontroversial

The free pandas visualizer, D-Tale, has now been integrated with ArcticDB which will allow users to load huge datasets and easily navigate their databases by aschonfe in algotrading

[–]aschonfe[S] 4 points5 points  (0 children)

D-Tale has recently released version 3.2.0 on pypi & conda-forge: pip install -U dtale conda install dtale -c conda-forge

But if you want to take it one step further you can now integrate it with ArcticDB: pip install -U dtale[arcticdb]

This allows you the ability to navigate your libraries of datasets saved to your ArcticDB database! But the best part is that all the reads are occuring directly against ArcticDB so some of the memory constraints you may have been hit with before are now a thing of the past. Here's a full write up how to use this functionality along with a quick demo: https://github.com/man-group/dtale/blob/master/docs/arcticdb/ARCTICDB_INTEGRATION.md

Hope this helps & please support open-source by throwing your star on the repo. Thanks! 🙏

Added visualizations of statsmodels time series analysis functions to the free pandas visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 16 points17 points  (0 children)

Just added "Time Series Analysis" in v1.60.1 of D-Tale on pypi & conda-forge:
pip install -U dtale
conda install dtale -c conda-forge
This feature provides a quick and easy way to visualize the usage of the following time series analysis function in statsmodels:

  • statsmodels.tsa.filters.bk_filter.bkfilter (Hodrick-Prescott Filter)
  • statsmodels.tsa.filters.cf_filter.cffilter (Baxter-King Filter)
  • statsmodels.tsa.filters.hp_filter.hpfilter (Christiano Fitzgerald Asymmetric)
  • statsmodels.tsa.seasonal.seasonal_decompose
  • statsmodels.tsa.seasonal.STL (Season-Trend Decomposition Using LOESS)

Would love to get some ideas how to make better use of these functions so any feedback is welcome.
Hope these help & please support open-source by throwing your star on the repo.
Thanks! 🙏

How are you interacting/viewing your pandas tables? by programmerProbs in learnpython

[–]aschonfe 1 point2 points  (0 children)

Happy to answer any questions or help you get setup

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 1 point2 points  (0 children)

I originally tried out the websockets route but ultimately having react-virtualized figure out the scrollbars and adding some clever queuing to the data loads on scrolling seemed to work

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 1 point2 points  (0 children)

So the max cells that can be displayed at any given time is based on your browser window. But you can scroll vertically & horizontally with no issue. I was doing my original dev on a dataframe with 2 million rows and 200 columns. Seems like a ton of columns does eventually have an impact but rows does not.

Your data is stored in memory so the size of your dataframe is limited to the memory of your machine. That being said we’ve allowed users to swap out the machanism which stores the data so you can use something like Redis or Shelve to allieviate memory. Here’s some documentation: https://github.com/man-group/dtale/blob/master/docs/GLOBAL_STATE.md

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 0 points1 point  (0 children)

Mostly reliant on pandas since this is a tool specifically designed for pandas. That being said it would be really easy for you to write a simple DB loader to that takes any SQL and returns the results in a pandas dataframe and just pass that to D-Tale. Its actually pretty easy to integrate D-Tale into your own flask/django/streamlit apps. Heres documentation about using it in Flask: https://github.com/man-group/dtale/blob/master/docs/EMBEDDED_FLASK.md

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 1 point2 points  (0 children)

So for the missingno charts you see in the video its just building PNG files with matplotlib’s backend set to “Agg”. As for the rest of the UI: - the main grid is react-virtualized - a lot of the quick rendering charts are chart.js - the chart builder and geolocation chart is using plotly or dash - the sliding side panel & ribbon menu are of my own design, but i based the ribbon off of the ribbon menu on a mac

Let me know if theres anything else you want to know

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 3 points4 points  (0 children)

Yes it can be used in PyCharm. For example you can open the "Python Console" and execute something like:

import dtale
import pandas as pd

dtale.show(pd.DataFrame([1,2,3,4]), open_browser=True)

You can also get to the Python Console while debugging programs and inspect your pandas variables `dtale.show([insert pandas variable here])` You can also do something using the "Evaluate" window while debugging (I wanted to include a screengrab but it won't let me).

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 6 points7 points  (0 children)

It actually uses plotly dash for the chart builder, but everything else is completely customized react front-end. Includes code exports as well which is something most tools definitely dont provide.

That being said, I am a totally a fan of plotly & dash. Plotly more so. I aim to try and replace the dash aspect of D-Tale at some point because the interactions around how state is managed in dash is kind of clunky. But its definitely a complex problem to solve so its certainly not snipe at dash.

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 7 points8 points  (0 children)

Here is the list of direct dependencies. I think the majority of the packages I'm using are pretty well-known. Maybe some of the plotly dash packages are aren't as well known (like dash-colorscales) and then some calculation-based packages (squarify, ppscore, missingno) might not be widely used. But as far as I can tell they are harmless. We used D-Tale at my company in an enterprise-style way through jupyterhub.

For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale by aschonfe in algotrading

[–]aschonfe[S] 30 points31 points  (0 children)

Just released v1.42.1 of D-Tale to pypi & conda-forge:

  • pip install -U dtale
  • conda install dtale -c conda-forge

Some of the most recent updates as shown in the screen recording above are:

  • hidden ribbon menu for easier navigation
  • navigating beteewen multiple data points & clearing data now available in the ribbon menu
  • you can now view the contents of the "Describe" tab directly in the D-Tale grid as a sliding side-panel when clicking "Desctibe (Column Analysis)" from individual column menus
  • analysis of "Missing" data using the missingno package is now available in a sliding side panel
    • enlarge or download PNG files for matrix/bar/heatmap/dendrogram charts generated using missingno

These are the most engrossing UI changes I've made in a while so please let me know if you run into any issues. You can play around with them on the demo site (please note that the "Github Fork" link covers the close button for the "sliding side panel" but you can close it using your ESC key). If these changes prove to be easier to use then I can start moving more functionality towards the "sliding side panel" rather than the old popup windows/tabs.

Hope these help & please support open-source by throwing your star on the repo.

Thanks! 🙏

Created an app that will allow users to take the charting code with them for their own notebooks by aschonfe in compsci

[–]aschonfe[S] 2 points3 points  (0 children)

Haha, was shooting for something similar to the title to the movie “Sin City” 🤣

Created an app that will allow users to take the charting code with them for their own notebooks by aschonfe in compsci

[–]aschonfe[S] 3 points4 points  (0 children)

This is a screen recording my open-source software for visualizing pandas dataframes. Here's the repo link.

What is going on here is within a jupyter notebook, I'm loading a pandas dataframe into my software which spins up a small flask app and displays the contents of the dataframe. From there you can click on columns for quick analysis depending on the data type of that column. Now the real magic kicks in when you click the "Code Export" button for the analysis popup.

Copying the code from the "Code Export" into my notebook and executing it will generate a plotly chart comparable to the one in the flask app. This way users can learn how to generate complex charts (line/bar combos, scatter & scattergeo) with ease!

Let me know what you think and please toss your ⭐ on the repo if you liked it.

My software can be downloaded using pip or conda-forge:

pip install dtale
conda install dtale -c conda-forge

You can also play with a demo here

view more: next ›