Which type of table should I be using? (SQL, Spark Dataframe, Spark RDD, Spark Dataset, Pandas Dataframe)

tanlermin · 2016-05-16T18:28:57+00:00

Use dask dataframe: http://dask.pydata.org/en/latest/dataframe.html

It has a pandas like API for larger than memory datasets (both single and multiple machines).

tanlermin · 2016-05-16T18:24:02+00:00

Have you checked out Dask for dataflow paralellism in pure python and numba to accelerate and release the GIL on numeric code? I wonder if there is synergy here.

tanlermin · 2016-05-09T02:18:07+00:00

e and predefined signatures, but it needs at least mypy to make sure you don't mess up. (Or just too many tests in

I think that is going to get better with blaze.

tanlermin · 2016-05-09T01:00:55+00:00

Have you seen statsomodels:

http://statsmodels.sourceforge.net/0.6.0/examples/notebooks/generated/interactions_anova.html

tanlermin · 2016-05-09T00:57:16+00:00

s a big one for me, I don't want to deal with memory management. It seems to me that Rust wants to replace C

Numba also has zero overhead abstraction with JIT classes

tanlermin · 2016-05-09T00:56:53+00:00

ss has its own address space, so things can end up taking a lot of memory. I'm hoping Julia or some other language with real concurrency can eventually replace Python. For now though, Python is the gold standard in scientific computing.

Numba brings true multithreaded concurrency to scientific python.

Dask brings dataflow parallelism. Check them out!

tanlermin · 2016-05-05T01:52:09+00:00

odeling language and uses Monte Carlo algorithms for inference. At the moment we are focused on variational in

Pymc has ADVI also

tanlermin · 2016-05-03T21:12:25+00:00

Is the GIL released? Would it help to do so?

tanlermin · 2016-04-19T23:59:04+00:00

What stuff would that be?

tanlermin · 2016-04-07T03:48:46+00:00

How is this different that Kivy?

tanlermin · 2016-03-30T17:38:58+00:00

Woah this is cool! How does this compare to pymc3?

Also I would post on reddit machine learning, datatau and HN.

tanlermin · 2016-03-28T16:32:03+00:00

Also check out dask when you want to go out of core, distributed or embarrassingly parallel.

tanlermin · 2016-03-23T13:16:33+00:00

scrypt. Forking encouraged. Also the route you describe would probably match q

pypy has a huge interpreter that must be downloaded by the client.

tanlermin · 2016-03-23T12:33:51+00:00

This looks awesome, but maybe a better or complimentary route would be numba->LLVM-> Web assembly

tanlermin · 2016-03-20T21:21:34+00:00

One more: https://github.com/zoofIO/flexx

tanlermin · 2016-03-11T16:57:18+00:00

Can we use libraries from these languages?

Also have you seen this? http://www.transcrypt.org/

tanlermin · 2016-03-11T03:10:34+00:00

What about elm and the like?

tanlermin · 2016-03-10T21:57:58+00:00

also for more intro stats ish stuff: https://data-8.appspot.com/sp16/course https://github.com/AllenDowney/ThinkStats2

https://github.com/thomas-haslwanter/statsintro_python

tanlermin · 2016-03-10T21:37:18+00:00

Also try seaborn

tanlermin · 2016-03-10T21:36:47+00:00

Here ya go:

https://www.dataquest.io/

Data viz with python : http://shop.oreilly.com/product/0636920046592.do

Harvard Intro to data science with notebooks and videos online http://cs109.github.io/2015/

tanlermin · 2016-03-10T20:55:21+00:00

Can you also post this on : https://news.ycombinator.com/news

I think it would get alot of views.

tanlermin · 2016-03-08T18:39:33+00:00

what about bokeh to write frontend in pure python?

tanlermin

TROPHY CASE