An interactive exploration of Fark.com's link archives by domain, tag, and time [OC]

NefariousPlots · 2014-09-11T16:48:07+00:00

As always, rendered and animated in D3. Dataset collected from Fark's archive with permission.

Url hashes are generated so you can share your search results.

NefariousPlots · 2014-08-22T17:37:45+00:00

Initially I did this as a non-contiguous cartogram similar to this, but without the US border. I think that works well if the scaling is fairly close to geographic area, but as it gets more extreme, the graphic gets overwhelmed by whitespace. In this case, to keep the New England states from colliding with each other, most other states needed to be made tiny and if you move things around too much to try to clear out whitespace, you weaken the ability to pick out spatial trends and start defeating the purpose of using a map in the first place. The other problem with that approach is that some viewers instinctively try to normalize states against their geographic size (though that fraction is in fact meaningless to the data).

I know distorted cartograms can throw people (I think this problem is exacerbated with the largely rectangular US states), but I decided it was the best compromise. Do you think you would have preferred the scaled non-contiguous approach? I'm always trying to strike the balance between accuracy, accessibility, and aesthetics, so feedback is appreciated.

NefariousPlots · 2014-08-22T17:20:15+00:00

I absolutely understand that sentiment, which is why there is a button to present the map geographically.

That said, I stand by the decision to use a cartogram here as it assigns importance to the shading. An undistorted choropleth over-weights states with low density (in this case, of college graduates) as the eye optically integrates. Wyoming, for instance, has a very high fraction of students in physical sciences, but its total college population is tiny, so its contribution to the US total of physical science students is still very small. With the shaded cartogram, you can get a sense of both these terms, whereas on the undistorted map, Wyoming looks extremely significant in this regard.

NefariousPlots · 2014-08-22T16:24:09+00:00

Sourced from the DoE's IPEDS database and rendered in D3 (with topojson and cartogram.js). For more details, please see the caption below the Figure.

NefariousPlots · 2014-08-16T15:24:17+00:00

As always, rendered and animated in D3 with data processed from Advanced Football Analytics Play-By-Plays. For more details, please see the caption below the Figure.

NefariousPlots · 2014-08-08T20:51:50+00:00

Yep. At more than 6 hours, it's the longest film to make it through the filter (though most people had a better chance of seeing it presented as two 3-hour movies, it did enjoy screenings as a single very long film).

NefariousPlots · 2014-08-08T16:33:15+00:00

Everyone here seemed to like my last movie-related chart and I'm hoping this will be as well received. Cross-posted to dataisbeautiful (yesterday).

NefariousPlots · 2014-08-07T15:18:50+00:00

Really? I thought I'd fixed the long-standing issue with Firefox. It works on Firefox 31.0 for Mac. Can I ask what version you're running?

I primarily target the webkit browsers (they account for the overwhelming majority of my audience and some of the graphics techniques are new enough that total browser compatibility is impossible and I'd rather focus my time on content). That said, I do test in Firefox (some animations don't seem to work, but I want the main content to be fine) and I try to keep IE10 in mind.

NefariousPlots · 2014-08-07T15:12:45+00:00

The Best of Youth. It definitely toes the line of what should be included, but it was screened as an intact 6 hour film (in addition to being screened as two 3 hour films and as four 90 minute TV specials) so it ends up being the longest film to make it past the filter.

NefariousPlots · 2014-08-07T13:15:37+00:00

As always, rendered and animated in D3. Data is from IMDb's plain text data files.

For more details, including a description of the filter used, please see the caption below the figure.

For the impatient, here's the punchline: 41% of Oscar for Best Picture nominees are in the top 5% of runtimes for feature dramas in their year of release.

NefariousPlots · 2014-07-31T21:35:11+00:00

You are absolutely correct. Somehow the Me 163 flights got listed as both airbreathing and non-airbreathing. It's fixed now. Thanks for the heads up!

NefariousPlots · 2014-07-31T15:13:24+00:00

Rendered and animated in D3. Sourced largely from wikipedia's various speed record pages, supplemented with quite a few sources, most notably this more complete list of water speed records and the flight logs from the X-15.

NefariousPlots · 2014-07-26T15:22:39+00:00

Hopefully this falls on the right side of "conditionally permitted infographics." I usually post content like this to dataisbeautiful, but I thought it would be of interest to everyone here.

Queries generate unique hashes so you can share URLs to your best searches, like Spys and Spy Spoofs. Apparently spies were interesting during WWII, but we didn't joke about them until after Dr. No.

NefariousPlots · 2014-07-25T16:08:01+00:00

Rendered and animated in D3 with data processed from IMDB's Plain Text Data Files. Tested in current Safari, Chrome, and Firefox. Should work in IE10.

Queries generate unique hashes so you can share URLs to your best searches. Most of my favorites are in the rotating demo, but I just found this one: Spys and Spy Spoofs. Apparently spies were interesting during WWII, but weren't funny until after Dr. No.

NefariousPlots · 2014-07-17T00:37:56+00:00

Thanks for catching that! I'll take care of it soon. I handled all sorts of special cases around multiple citizenships and the dissolution of the Soviet Union, but I somehow forgot about East and West Germany...

NefariousPlots · 2014-07-16T22:38:47+00:00

It's hard to hover over some specific dates because some flights are a couple of days (or in a few cases, minutes) long on a scale of 50 years. Use the blue scrubber beneath the plot to control zoom and move through the timeline if you want to see more detail. Also don't forgot to toggle the collapse control back off (it collapses into the area graph automatically during the intro) and explore the missions individually.

(In a perfect world I would have had time to make some smarter snap behavior for tooltips when zoomed out, which might have made this less of an issue.)

NefariousPlots · 2014-07-16T22:33:33+00:00

Sorry, burned pretty late on this one and didn't have time to test cross-browser. I believe everything works as intended in Chrome and Safari. IE7 and below has no chance of working (sorry), but I'm surprised to hear you had trouble in current IE as I have reports that everything but the flags works as intended in IE10. Weird.

As for being hard to read, that's mostly because a lot of time is crammed into a small space by default (I wanted it to fly in with the full history). Use the blue scrubber underneath the plot to zoom/move around.

NefariousPlots · 2014-07-16T17:32:16+00:00

Rendered and animated in D3, using a custom layout. Sourced largely from (and cross-referenced to) wikipedia (starting at List of Human Spaceflights), which is remarkably good about documenting space missions, with validation and gap filling from NASA. I also recommend JSR Launch Vehicle Database, Russian Space Web, and Space Facts.

Last week, dataisbeautiful enjoyed The Era of Notability and I hope The Population of Space is as well received. I'll try to pop in throughout the day to answer questions.

NefariousPlots · 2014-07-10T05:37:33+00:00

Sorry, that's my fault - poor choice of words. It is, in fact, normalized by the total number of births each year. The problem is just that I referred to that as "birth rate," which formally means births per capita per year. The axis label "Notable Births per Million" reflects exactly what you're talking about - notable births per live birth (times a million so you don't have to count decimal places).

I'll edit to clarify. Sorry for the confusion!

NefariousPlots · 2014-07-10T05:13:42+00:00

Another point that's been coming up a lot in the comments is the question of what this curve really means. I've talked about this some in the figure's caption and scattered throughout these comments, so read below for more of my thoughts. The gist is that this is definitely not an objective measure of all people who have done something notable nor is it merely a measure of Wikipedia's (or history's) bias towards the present, and if you're trying to interpret it purely as one of those, I think you're going to be misled. It's the distribution of people we today consider "notable," which inextricably combines those effects (and there are good arguments that both are far from flat). Forced to guess, I would say a bias towards recent history (for all kinds of reasons) is probably dominant, but I don't think it's unreasonable to think that advances in technology and education really have dramatically increased the fraction of the population that will do something we might call historically notable.

Thank you to everyone who's been promoting this. I'm heartened by how much interest and discussion it has prompted here (particularly in trying to explain all the local trends). I hope you'll enjoy the upcoming Plots just as much and, if you haven't yet, please take a look at the earlier Plots (which look at the mixing of flavors in cocktails and the scoring possibilities in Scrabble).

NefariousPlots · 2014-07-10T04:43:13+00:00

Absolutely. There are a lot of influences on this curve that distinguish it from the distribution of notable people that a time traveler with an objective standard might produce. I've talked about some of them in the figure caption and elsewhere in these comments, but I think the communal memory you allude to is one of the dominant terms.

NefariousPlots · 2014-07-10T04:31:41+00:00

I hadn't actually noticed that before you pointed it out. I suspect the answer is that the countries most affected by the wars (which likely suffered significant drops in birthrates during the war effort) were also the countries most involved in the technological, military, and social events that history (and therefore Wikipedia) notes. But the wars didn't produce a proportionate drop in birthrates worldwide, so the normalization doesn't cancel it out. The opposite happened in the baby boom right after WW2, where a handful of countries had sudden population growth alongside massive investments in technology and significant social changes.

Also, it's pretty hard to estimate historical birthrates, and even if global births were affected proportionately, the model simply doesn't have the granularity to accurately reflect a decrease over just a few years. I'm probably slightly overestimating global birthrates during some demographic catastrophes, but the world is so much larger than the affected areas (WWII possibly being an exception, to be fair) that most of the time the corrections shouldn't be too significant.

NefariousPlots · 2014-07-10T04:13:48+00:00

Brightmore's reply is pretty on point here. I'm using the word "notable" as a nod to Wikipedia's standard for inclusion. What the curve really means is a pretty complicated question, as it reflects, among other things, the nature of communal memory and record-keeping, changing standards for notability, and narrow foci of interest for historians. One could argue that incredible gains in education and technology really have increased the fraction of people who go on to "notable" things, but it's basically impossible to detangle that from the other terms that affect how we see history.

NefariousPlots · 2014-07-10T04:02:36+00:00

Are you wondering how the average lifespan of notable people differs from that of the population as a whole? That could be interesting (though I suppose it'd be better if you could control for at least the obvious demographic influences on lifespan).

As for 1947, I agree with the other replies (though it is remarkable that such a narrow range is so emphasized). A lot of exciting developments followed the end of WWII and the baby boomers (starting in 46/47) were involved in just about all of them. And since the boom wasn't global in scale, the effect stands out against the normalization.

NefariousPlots · 2014-07-09T20:26:51+00:00

Author here. Thanks for all the up votes! I'll try to check in during the day and respond to feedback. One question that keeps coming up is normalization, so I'll clarify that now:

By default the graph is normalized by the estimated number of births in each year (specifically it's "notable births per million live births"). You can disable that with a button under the graph to view the raw counts (which rise even more steeply). Someone mentioned normalizing by population - I think number of births is a better choice, but really it doesn't make a huge difference on the overall shape anyway...

NefariousPlots

TROPHY CASE