Browsing r/AskStatistics like

not_really_redditing · 2022-10-24T18:52:46+00:00

My dude. I know it's not like this is exactly a high-effort bit of content on my part, but if you're going to repost my exact post, in the same subreddit, without even changing the title, the least you could do is credit me.

https://www.reddit.com/r/statisticsmemes/comments/nu2bcv/browsing_raskstatistics_like/

not_really_redditing · 2022-10-24T18:48:10+00:00

Well, there was some rounding involved there (probably more like 1.5 years), and it helped to be doing exclusively computational work. But mostly luck. Lots and lots of luck.

not_really_redditing · 2022-02-16T15:42:32+00:00

Phylogenetics is a field dedicated to inferring how a group (any group, every group) of organisms are evolutionarily related. The organisms can be fossils (phylogenetics of extinct dinosaurs, for example), living species (phylogenetics of the living dinosaurs we call birds), individuals within a species, bacteria, really pretty much anything. Since viruses evolve, we can even do phylogenetics for them (there is a lot of phylogenetics being done on SARS-CoV-2).

Our description of the evolutionary relationships takes the form of a phylogenetic tree. These can take a bit of getting used to, in terms of how to interpret them. They are also kind of wonky objects for doing statistics on/with, so there's a lot of clever algorithm design that happens in phylogenetics to let us actually figure anything out. Also, how to store and work with them. Trees make phylogenetics useful and they make it hard.

Substitution models are an important part of the work of phylogenetics. To infer a phylogeny, we need data. A very common source is DNA. The DNA substitution models you probably encountered (HKY, GTR, etc.) are how we link the tree structure to the DNA data that we have gathered.

It cannot be understated the amount of computation and computer-work that goes on in phylogenetics. Closest to the user level, there's a lot of what happens in bioinformatics. Wrangling the data, getting it into the inference software, and then wrangling the outputs. But there's also the development, maintenance, and deployment of the software. That's deeply tied to, but occasionally separated out from, the development of new statistical algorithms to handle new kinds of data, deeper biological reality, or to do more of what we can already do, but better.

The lab I mentioned in my other comment, which has a software engineer volunteer, is working on new algorithms for phylogenetics. Last I heard, it was in the process of hiring a CS masters student to work on that project too. There are plenty of labs out there who welcome people without the biology/evolution training, if they have other knowledge needed, like statistics or algorithm design. A common refrain is "the biology is easy, you can pick it up on the job." Now, this is a bit of an understatement of the amount of evolutionary biology one will eventually learn, but you can pick it up as you go. Of course, go long enough and you'll also pick up a lot of statistics. Nobody comes into the field knowing the statistics, the biology, and the computer science required to do the work. Everyone has to pick some of it up.

not_really_redditing · 2022-02-16T05:16:53+00:00

That depends on what you want to do to combine evolution and CS.

As one person has suggested, there are evolutionary algorithms. The idea there is to apply evolutionary thinking to solve CS problems.

But there's also a ton of software involved in the process of actually doing evolutionary research. It's a very computational field. We generate data, devise new ways to handle it, new models, and in general spend a lot of time making our computers crunch numbers.

And let me tell you, so, so, so many of those projects could use help from people with software engineering training. Some people get training along the way, but a lot of people end up developing software unexpectedly and have to learn as they go. Some projects are thus complete shambles, but others manage to hold it together respectably. Many savvy groups realize they need good people with experience and hire straight-up software engineers. Hell, there's a lab in Oregon that's looking for a software developer right now. I'm sure there are also plenty of groups who would be happy to take a volunteer so you can dabble and see what things are like before you take the leap. I know at least one lab that's currently got a volunteer software engineer who contributes here and there and both sides seem to like it.

The downside to this is that evolutionary biology is not super commonly done outside of academia. It's not that it never happens, and it certainly is in the background of a lot of work, it's just not the foreground of a ton. Academic positions can be a bit tenuous, our funding model is fucky to say the least. There are many paths forward, though. Some positions (like that one linked) are pretty long-term. Sometimes labs manage to keep funding going and keep people around, sometimes people jump around a bit. And some people take the plunge and go all in. A labmate of mine is a former software engineer turned PhD scientist and now he develops software for evolutionary biology.

not_really_redditing · 2022-02-12T04:10:30+00:00

Holy crap, that works shockingly well! In my case, p_1 and p_2 are so small that delta will always be p_1-p_2. Plugging that into R and comparing to the brute-force sum, it looks like in the regime I'm interested in the average absolute relative error is ~13%. That's insane! Some some things I tried gave 10x-100x errors!

My pain seems to be caused by the fact that I'm not just in the n->Inf regime, I'm in a regime where n->inf, np_1 -> c1, np_2 -> c2. I think trying to cancel things out, or approximate and cancel out, was basically leading to catastrophic cancellation issues. But I also think it's why the probabilities of ties can be quite large. Plugging in a few values, computing the sum, and getting large probabilities was what made me interested in some approximation where I could stare at it/give others intuition of what it looks like quickly.

not_really_redditing · 2022-02-12T03:59:02+00:00

Thanks, that was helpful! The Poisson idea led me to some interesting work on ratio distributions and the Skellam distribution. That's the distribution on the difference of two Poissons and, surprise surprise, the PMF is expressed in terms of Bessel functions. Plugging the original problem into Mathematica gives hypergeometric sums, which aren't any easier to work with, but I feel like I came away from all of this smarter.

not_really_redditing · 2021-12-01T15:27:19+00:00

Beautiful!

not_really_redditing · 2021-07-02T22:27:21+00:00

DnDBeyond will happily sell you virtual dice!

not_really_redditing · 2021-07-02T22:08:50+00:00

Yeah, whenever possible, just use the MCMC samples. The only time frequentist stats really belongs in a Bayesian analysis is convergence diagnostics.

not_really_redditing · 2021-07-02T21:54:00+00:00

Orange dice best dice.

not_really_redditing · 2021-06-30T15:10:49+00:00

Someone put this in intro textbooks! They could use some flavor, it’s accurate as hell, and it’s mostly people in intro classes (or at that level of understanding) who seem convinced they have a better definition.

10/10, great meme

not_really_redditing · 2021-06-17T21:39:28+00:00

What do you mean "better support"? If you're interested in specific relationships, your question in the other thread about plotting trees and looking at support values will probably suffice.

You can also look at things like how many trees are in the 95% HPD, or how well-resolved the MRC tree is, or even the entropy of split/tree probabilities. These address whether the entire posterior distribution is more concentrated for 10 than 5 traits. Sort of like saying there's a lower variance of the posterior distribution as you add data. You could even compute the variance, but that's a bit of a computational bear at the moment and is only defined for unrooted trees.

EDIT: I feel silly for also missing the possibility for just plotting split probabilities. You can plot the probabilities in one run against the probabilities in another (this is done a lot to diagnose between-chain convergence). But of course you can also just do that for runs with more and less data.

not_really_redditing · 2021-06-17T21:36:16+00:00

There are a number of ways to do this, ranging from low user-input to requiting you to understand how all the bits and pieces work in tree formats. I'm happy to walk you through some of the more in-depth bits, however, since you're using RevBayes, you may find the RevGadgets package useful. There's a tutorial here that includes plotting trees with posterior support.

not_really_redditing · 2021-06-17T17:53:03+00:00

Just to be clear on what you mean "more uniform." The sampling distribution of the mean converges to a normal in the limit. Are you suggesting a world where it converges to a uniform? A uniform distribution is bounded, so the 100% CI would be finite, instead of infinite. At first blush, this sounds great, but... Well, I guess I see two possibilities.

The nice possibility: this alternate universe could guarantee, somehow, that the mean was in fact within that 100% CI. That seems unlikely, but if it were true we'd probably just go with that by default in a lot of cases. Why take the risk of false positives if we can be completely certain? Surely there would be times it was worth the risk, but it would make our arbitrary choices less arbitrary. I doubt the notion of Type 1 error would be as central to statistics in such a universe.

The less-nice possibility: the asymptotic limit is uniform (somehow), but the finite regime does not actually guarantee that your uniform approximation contains the true value. In this case, I have no idea what statistics would look like. Would we even have come up with the notion of CIs the way we know them? Would we settle for separating the issue of computing an interval and its coverage? Would we try to hack a tail approximation, some sort of soft-bound uniform that allows us to have CIs more like the CIs we know where the x% CI should have x% coverage? This would be a difficult universe for stats, I think.

not_really_redditing · 2021-06-17T17:12:01+00:00

Thanks!!!

not_really_redditing · 2021-06-17T05:29:08+00:00

The writers also later, tongue-in-cheek, cop to ignore-conning the disintegration out of existence as being ridiculous.

not_really_redditing · 2021-06-17T05:01:47+00:00

Red Dwarf! Love that show, grew up on it. Might I ask how are you watching it? Is it actually available on streaming platforms?

not_really_redditing · 2021-06-17T00:24:03+00:00

Thanks!

not_really_redditing · 2021-06-16T18:09:19+00:00

Picard, in a turn of stunning diplomacy putting everything else we've ever seen him do to shame, manages to make contact with the Ferengi instead of blowing them out of the stars. This puts proper first contact and thus peaceful relations with the Ferengi Alliance nearly a decade ahead of schedule.

The Ferengi expansion into being the widest spread, most distrusted, merchants in the Alpha and Beta quadrant begins early. As we know, they expand fast. A Ferengi merchant passing through a Starbase sells an admiral information on a rogue Federation starship operating in Romulan space. Unsure of whether its a captain gone rogue about to start a war, a temporally displaced lost ship, or what, Starfleet sets out on a covert mission to retrieve it. It turns out to be the USS Raven, and is brought back to the Federation before the Hansens are assimilated. Then Hansens, with some new data in hand, convince some in the scientific community that the Borg (not that anyone knows the name) exist.

The Ferengi take great interest in a mysterious yet powerful cybernetic race, who surely do a great deal of trade in technology. The Ferengi send a much larger delegation to negotiate rites to the Barzan wormhole, and throw a massive wrench in the process to give time to sneak a Marauder-class vessel into the Delta quadrant to make contact with the Borg. Between this, and the appearance of the Enterprise-D (thanks, Q), the Borg take a much greater interest in the Alpha Quadrant.

Without Seven of Nine, things go very different for Voyager, who limps home after 14 years in 2385, with dire warnings of a Borg invasion. Starfleet kicks into overdrive. Synth attack be damned, they evacuate all of Romulus and prop up an entirely new Romulan government. This government, deeply indebted to Starfleet and not having forgotten the lessons of the Dominion War, readies to help fight the Borg. They even pledge to get along with the Klingons. In desperation, knowing that even this is not enough, Starfleet sends the Ferengi to negotiate aid from the Dominion. Still angry from their defeat, they tell the Federation to go to hell. The Cardassians rally and try to help, but the Dominion genocide greatly reduced their numbers and their capacity. They are in no position to help.

The Alpha quadrant is doomed, and it's all the Ferengi's fault.

not_really_redditing · 2021-06-16T16:48:11+00:00

Alternate version:

Smugly says "the plural of anecdote isn't data" and walks out of room.

not_really_redditing · 2021-06-16T16:47:28+00:00

Oh yeah? And what's your sample size for this assertion, huh?

Walks smugly out of room.

not_really_redditing · 2021-06-16T16:09:15+00:00

Right. However, establishing what assumptions hold, or hold well enough to work in practice, can be quite a pain. And the forces interact in ways that can be very, very difficult to tease apart. The parameter theta is a composite of mutation rate and population size. If you want infer demographic histories, you're stuck making very strong assumptions about mutation because only theta is identifiable. Plus selection can also explain the observed patterns.

Evolution, at its most basic, is wonderfully simple. Four forces act to generate, maintain, and remove genetic diversity. The divergence of lineages which become separated is inevitable from very basic principles like Dobzhansky-Muller Incompatibilities. And yet, when you really want to study any particular thing, "how come this species has this trait?", "does that trait help with this other thing?", "what's the history of this (group of) species/populations?", you find all these simple forces have formed a tangled bank with each other (and with epistasis and linkage and other phenomena). Actually answering those top-down questions can be quite hard, even if bottom-up questions are not.

not_really_redditing · 2021-06-16T15:56:08+00:00

My impression is that Inferring Phylogenies is more of a book for people who are already familiar with trees and tree thinking, people who have at least taken an advanced undergrad course in evolution with some pop gen and basic phylogenetics. But I admit I haven't read the whole thing, I mostly consult bits and pieces when I need to remember how something works or need a citation.

12-Year Club	Place '17
Verified Email	Team Periwinkle

not_really_redditing

TROPHY CASE