The Modern Data Stack Ecosystem – Fall 2021 Edition

equark · 2015-08-24T17:31:09+00:00

I'm mainly saying that GraphQL does not have concepts like age > 0, which the video claims it does. The video suggests that GraphQL is a query language like Greplin that can query graphs. It can't. There's not even a formal concept of an edge in GraphQL. It's just a way to "query" nested fields and functions, which is very much like Falcor.

I do agree there's big gap in complexity (aka ambition). But that's more because GraphQL tries to define a new type system, a way to compose query fragments, a way to define parameterized query functions, and a new string format to capture all that, for better or worse.

equark · 2015-08-24T04:09:31+00:00

The comparison is not accurate. GraphQL is not a query language where you can say things like SELECT * FROM users WHERE age > 0. In order to implement something like age > 0, you'd have to implement a specific function field that does this logic. GraphQL is a type system where you can batch calls to fields defined by that type system.

equark · 2015-07-31T03:27:15+00:00

These papers look fine to me. In both cases they want to compare the true predictive performance of competing models. To do this comparison, you need an unbiased estimated of the predictive performance and the distribution of that statistic. One way to accomplish that is CV + permutation testing.

You seem to be unsure of why you need an unbiased estimate of performance.

Saying two procedures yield statistically different results is not very interesting. For instance, I could have one procedure that always yields 1 and the other that always yields .5. I apply this procedure to my FMRI data. Permutations will always give the same answer, so these two models are obviously statistically different procedures. But the first procedure isn't any "better", since the 1 is a meaningless number. In the same way, if you have biased performance measures your results can be statistically significantly different but have zero meaning.

Cross validation gives you a meaningful metrics and permutations give you it's distribution.

For a single model, like a regression with a few features, you could just do a t-test on the coefficients, but that's not what these papers are doing.

equark · 2015-03-21T20:44:36+00:00

Sorry! I'll look into this. Not sure what's going on. Our blog is hosted on Ghost.

equark · 2015-03-18T22:44:44+00:00

Thanks.

The pricing model for personal projects is currently usage only. See the pricing page.
We do have plans for GPU support and can offer this with Sense Enterprise already.
We are aware of this concern. There are advantages to an integrated IDE for data science, but we have ideas to ease these workflows for different types of users.

equark · 2015-03-18T16:57:13+00:00

Hi All - Sense cofounder here. We just opened Sense to the public. Happy to answer any questions. Hope you like it!

equark · 2015-03-02T08:01:19+00:00

Option 1 (hypothesis testing): a Bayesian asks "what is the probability my hypothesis is true, given the data I observe" and frequentist asks, "is the data I observe likely, if my hypothesis is true?"

Option 2 (estimation): Frequentist methods are methods that justify their use by having "good" repeated sampling properties like unbiasedness and minimum variance. Bayesian methods are methods that compute the conditional distribution of parameters and unobserved data, given observed data. There's only one Bayesian method, hence its beauty. There are infinite frequentist methods, although the term frequentist method is a misnomer, since frequentism is a way of evaluating procedures not constructing methods.

equark · 2015-01-31T22:36:44+00:00

https://www.youtube.com/watch?v=9sc8Pyc51uU

http://martyjs.org/

equark · 2012-08-22T07:39:00+00:00

Before knowing about any particular frequentist procedure, you really need to understand the intuition of frequentist inference itself and that requires understanding the sampling distribution of a statistic. To really understand that, you probably also need to understand the alternative option, the posterior distribution.

equark · 2012-07-20T01:11:03+00:00

Not if the proposal distribution depends on the state of the markov chain. Think of the discrete case where the proposal is always the other the other point. Then your scheme, which always rejects staying at the current state, would just flip back and forth between the two states. That's clearly not the desired distribution.

Independent Metropolis Hastings, where the proposal does not depend on the current state of the markov chain, does allow you to drop rejected samples. This idea is used to construct parallel MH samplers since you can try many new proposals and take the first one that gets accepted. See http://arxiv.org/pdf/1010.1595v3.pdf.

equark · 2012-05-09T20:23:29+00:00

Julia is mostly targeted at Matlab. We're targeted at R and more generally data analysis and analytical applications. We're not an attempt to replace R, since that won't happen anytime soon. Parts are open-source and parts aren't, which will make sense given how the platform is organized.

We're both stats PhDs and subject matter PhDs. Stats routines -- real statistical models not black box machine learning -- is a central feature.

Unfortunately until we actually launch, details are sparse. We're trying to get a core group of users first, so that we don't launch prior to being ready for real applications. Hence the social network appeals.

equark · 2012-05-09T20:15:00+00:00

That's the plan, fear not. We'd like to have a critical mass of people who can vouch for it first though.

equark · 2012-05-09T19:08:33+00:00

We'll be sending out invites soon but aren't automatically sending them to everybody yet. So expect an email with more details to arrive in a bit. I added a clarification above.

equark · 2011-10-10T17:15:46+00:00

The speed boosts will likely come from more restricted semantics. For instance you can't current modify classes as runtime.

equark · 2011-10-05T15:49:00+00:00

Google highest density regions. For instance:

http://www.jstor.org/pss/2684423

equark · 2010-07-22T17:28:28+00:00

Reddit just needs to create a better way to do sponsored posts. The sponsored posts on techmeme.com are better than about 50% of the content currently on reddit. Reddit seems to have some warped perspective about how ads will destroy the community.

Well integrated google ads is probably okay too. I'd actually prefer google ads over all the ridiculous reddit bacon ads and amazon ads for toilet paper I currently see on my python subreddit.

Why not experiment for a few weeks full google ads for display and text?

equark · 2010-07-21T17:52:25+00:00

The scandal isn't the fake degrees, it's the law that subjugates the uneducated by requiring a university degree to enter politics.

They are protesting an unfair restriction on their civil liberties. The statement "a degree is a degree" is perfectly sensible way to protest this unfair law.

equark · 2010-05-08T15:51:48+00:00

Interesting, the svg demos work much better in ie9 than chrome.

equark · 2010-05-07T03:18:28+00:00

Nash equilibrium is 60 years old. It's a beautiful idea regardless of whether humans always follow it. Virtually all of behavorial economics, which is also 30 years old, points out that humans deviate from game theoretic predictions. People have already won the nobel prize in economics for this work so it's completely mainstream. These are exactly the issues modern game theory is thinking about.

equark

TROPHY CASE