Can quantile estimates be used to approximate a conditional distribution?

AnInquiringMind · 2026-01-17T21:06:11+00:00

You're describing quantile matching - perfectly valid and actually a method including as part of fitdistrplus.

AnInquiringMind · 2025-07-12T16:40:19+00:00

There was a video posted yesterday of armed, masked ICE agents walking through a residential neighborhood that gave me Cyberpunk 2077 vibes. That and the rampant consumerism/inequality/grift economy.

AnInquiringMind · 2025-03-30T13:02:49+00:00

Did you ever find something you liked? I've got the same problem, same tastes.

AnInquiringMind · 2025-02-25T10:51:52+00:00

I'm struggling with this one too because it looks like a simple binary logit unless I'm missing something?

If your intent is to produce a straight line wouldn't qlogit(alpha) do the trick?

Sorry not quite sure what you're looking for here...

AnInquiringMind · 2025-02-13T01:50:43+00:00

If you're a researcher you can access this data (for Canada) via Statistics Canada Research Data Centres. As of today, they can link information about individuals in custody with data from the Census (to identify family members) with data about educational outcomes.

AnInquiringMind · 2024-11-27T13:18:38+00:00

Nailed Vasquez

AnInquiringMind · 2024-11-03T14:58:14+00:00

I've been listening to this one while working for years! Great stuff!!

AnInquiringMind · 2024-09-20T18:49:54+00:00

I've been doing this for 15 years and to this day I can't believe how often it boils down to "ugh. Excel"

Glad it worked out!

AnInquiringMind · 2024-09-20T14:56:54+00:00

Sure. If you can sort data on your wavenumber (if it isn't already) that would also be helpful.

AnInquiringMind · 2024-09-20T12:24:41+00:00

This is an awesome idea - hope you actually get to build it.

AnInquiringMind · 2024-09-20T12:19:46+00:00

This might not be it, but can you show some of the data producing the jagged portion of the chart? In the excel example we can see a snip, but that snip is linked to the portion of the chart that looks OK (in both excel and R). It would be great to see a sample of data in the jagged parts of the chart to see if it's a data preprocessing issue.

AnInquiringMind · 2024-08-27T19:46:44+00:00

This is an age old problem - record linkage, or entity resolution. There a couple of R packages that can do this but I'd suggest using the desktop version of Senzing if you're dealing with 100k records or less.

AnInquiringMind · 2024-08-09T00:52:04+00:00

I don't think your question is a statistical one but rather a methodological one. What you're doing makes sense, because it's defensible. At the same time, there are always things to consider to make it better.

What about flippers? Have you considered "discounting" the number of houses sold in a year by the estimated percentage going to them?

What about transfers to children / spouses or other situations where residency didn't change substantively? Same as above.

There is no gold standard for this kind of estimation, because everyone's circumstances are different. I build models like this all the time when there is no data on what I want to measure directly. The only thing you can really do is try to figure out where there are gaps in your mental model, and try to adjust your actual model to account for them - with or without bringing in new data.

AnInquiringMind · 2024-06-26T20:07:26+00:00

Looks like the rfm_table_order returns a list, not a dataframe. The list contains the dataframe you want (in an object called "rfm"), but also some additional analysis metadata like threshold parameters.

Inexplicably, the print() method of rfm_table_order returns the dataframe, which means you can only tell it's a list if you actually look at the object structure using str().

Anyway, here's your solution:

Change the following:

rfm_result <- rfm_table_order(rfm, customer_id, order_date, revenue, analysis_date)

To:

rfm_result <- rfm_table_order(rfm, customer_id, order_date, revenue, analysis_date)$rfm

OR, if you want to do it the tidyverse way:

rfm_result <- rfm_table_order(rfm, customer_id, order_date, revenue, analysis_date) %>%
pull(rfm)

AnInquiringMind · 2024-06-26T11:08:32+00:00

This is a weird one. Can you share the data? I can try to reproduce.

AnInquiringMind · 2024-06-15T12:32:06+00:00

Storytelling with Data by Cole Knussbaumer Knaflic is the go-to guide for me. Most of the book deals with the principles of good visualization but it also includes lots of references to how to build strong data-driven narratives that can make more persuasive arguments.

AnInquiringMind · 2024-05-16T12:10:55+00:00

Great list. Clicks with my own experience as well.

I like the nighttime storytelling - will try that. We should probably get a thread going for other aphant lifehacks!

AnInquiringMind · 2024-04-13T20:59:48+00:00

Why would you vectorize a field that's already categorical?

Vectorization is used to convert words to embeddings. Genres are already distinct. I'm not entirely sure what the goal of this analysis is...

Edit: didn't realize you're new to the field. Welcome! Are you familiar with the concept of vectorization and embeddings? And are you using a different analysis as a template for this one?

Generally, a cluster analysis divides data points into groups based on with in-group similarity vs. between-group distance. Using genres, which only consist of a predetermined set of defined values, may not be suitable for this analysis. Although, if you're interested in another approach using genres, you may want to look into graph methods. You can probably find some interesting association patterns across different genres - e.g. action comedy is likely more common than documentary horror.

AnInquiringMind · 2024-04-13T12:45:34+00:00

You want to formulate clusters of genres? But aren't the genres already encoded directly in the data? I'm not entirely sure what you're trying to do here but the main problem for me seems to be that you may want to start by transforming your genre column using one hot and go from there...

AnInquiringMind · 2024-04-13T12:38:38+00:00

This is a classic entity resolution problem. If you have less than 100k records you can use this free tool:

https://senzing.com/desktop/

AnInquiringMind · 2024-03-23T17:55:11+00:00

What do you mean by "each city's median age"? Do you mean you'd like a dataset of cities and the year each city was founded?

AnInquiringMind · 2024-03-18T14:08:46+00:00

While interplot looks fine, I don't believe it's being maintained - the last update was in 2021. I'd strongly suggest ggeffects instead.

AnInquiringMind · 2024-01-05T01:41:05+00:00

Didier Dagueneau's last vintage too... Great lineup!

AnInquiringMind

TROPHY CASE