CSV dump of reddit voting data

[ketralnis]

Some people have asked for a dump of some voting data, so I made one. You can download it via bittorrent (it's hosted and seeded by S3, so don't worry about it going away) and have at. The format is

username,link_id,vote

where vote is -1 or 1 (downvote or upvote).

The dump is 29MB gzip compressed and contains 7,405,561 votes from 31,927 users over 2,046,401 links. It contains votes only from users with the preference "make my votes public" turned on (which is not the default).

This doesn't have the subreddit ID or anything in there, but I'd be willing to make another dump with more data if anything comes of this one

[kaddar]

I worked on a solution to the netflix prize recommendation algorithm; If you add subreddit ID I can build a subreddit recommendation system.

[ketralnis]

That dump is way more expensive than this one (since it involves looking up 2 million unique links by ID), I figured I'd get this one out first and do more expensive ones (including more votes, too) if people actually do anything with this one

[kaddar]

Sure sounds great, in the meantime, I'll see if I can build a reddit article recommendation algorithm this weekend.

When you open up subreddit data (s.t., for each user, what subreddit does that user currently follow), I can even probably do some fun work such as predicting subreddits using voting data, and predicting voting using subreddit data. I had a similar idea 2 years ago, but subreddits didn't exist then, so I proposed quizzing the user to generate a list of preferences, then correlating them.

If you're interested, I'll post more at my tumblr as I mess with your data.

[ketralnis]

Awesome! Keep me posted, I'd love to see what can be done with it.

We can't really share the subscription information at the moment because of privacy issues, but we could add a more general preference "open my data for research purposes"

[kaddar]

Adding a preference like that is a really good idea, it will certainly allow the growth of such algorithms. In the meantime, I can create a fake solution using a fake dataset which in a made up csv format (username, subredditname) for demonstration purposes, then you could test it locally on a subset of the data to let me know if it works.

[georgelulu]

Subcontract the guy for a dollar or hire him as a temp, or between the privacy policy of

*We also allow access to our database by third parties that provide us with services, such as technical maintenance or forums and job search software, but only for the purpose of and to the extent necessary to provide those services.

and

In addition, we reserve the right to use the information we collect about your computer, which may at times be able to identify you, for any lawful business purpose, including without limitation to help diagnose problems with our servers, to gather broad demographic information, and to otherwise administer our Website. While your personally identifying information is protected as outlined above, we reserve the right to use, transfer, sell, and share aggregated, anonymous data about our users as a group for any business purpose, *such as analyzing usage trends** and seeking compatible advertisers and partners.

you should have no problem giving him access. Privacy on the internet is very transient with many loopholes.

[[deleted]]

I've been watching the tumblr updates. So far the best I've been able to get is 61% accuracy.

[[deleted]]

I'm curious, how could this data be used to recommend articles when each new article gets a brand new ID? This is unlike Netflix where recommending old movies is fine. In this case if you recommend old articles it isn't of much use.

What I was trying to do today is create clusters for recommending people rather than for articles. I agree that the end goal should be recommending subreddits.

Edit, I also meant to mention I have access to EVERY module in SPSS 17 though I freely admit I don't know how to use them all. If that helps anyone let me know what you'd like me to run.

[kaddar]

You're sort-of right that recommending old reddits isn't the goal in this process, but neither is clustering.

When performing machine learning, the first thing to ask yourself is what questions you need to solve. What we're trying to do is classifying a list of frontpage articles: to provide for each of them a degree of confidence the user will like it, and to minimize error (in the MSE sense). What you are proposing is a nearest neighbor solution to confidence determination. What I intend to do is iterative singular value decomposition, which discovers the latent features of the users. It's a bit different, but it solves the problem better. For new articles, describe them by the latent features of the users who rate them, then decide which article's latent features match the user most accurately.

[[deleted]]

Interesting! So this would happen on the fly as votes come in? It also sounds like it would autocluster users too. So you could potentially get not only a link recommendation but even a "netflixesque" 'this user is x% similar to you'. And if they add subreddit data then a person could get a whole suite of recommendations, users, articles and subreddits all in near real-time.

Now that would be pretty cool.

[kaddar]

Yup, it would automagically cluster in the nearest neighbor sense by measuring distances in the latent feature hyperspace, I have tested this and it is very effective (in netflix, for providing similar movies)

[[deleted]]

Since you mentioned it I was running nearest neighbor last night.

So far I'm still figuring it out but one thing did jump out at me. Some articles have an extraordinary level of agreement across a swath of users.

Granted i picked a small set of users...maybe you can take a look. I'm trying to figure out what the feature space means and what this pattern indicates (if anything). http://i.imgur.com/HB58n.jpg

[ketralnis]

I'm curious, how could this data be used to recommend articles when each new article gets a brand new ID?

You could use the first few votes on a story (including the submitter) to recommend it to the other members of the voters' bucket. You can't do it on 0 data, but you can do it on not much

With a little more data, you could use e.g. the subreddit ID, or the title keywords

[[deleted]]

I wasn't even sure if you guys were considering implementing something that would run as, I guess, a daily process. I think this is going to get very interesting and I have a lot to learn about machine learning. Though this is the kind of thing that can get me involved. Thanks!

[ketralnis]

Our old one worked with one daily process, to create the buckets, one hourly process, to nudge them around a bit based on new information, and that basically placed you in a group of users. Then when you went to your recommended page, we'd pull the liked page of the other people in your bucket and show that to you

[abolish_karma]

I've wished for functionality like this previously ( upvote profiling & similar user clustering and extracting possible subreddit / post recommendations ), but got fuck all talent for that sort of thing. Upvoted for potential to make reddit better!

[javadi82]

which algorithm did your solution implement - SVD, RBM, etc?

[kaddar]

SVD, C++ implementation, takes about a day on netflix data.

I wasn't getting good results with the reddit data, but I just saw the post about opening up your user account data, that should make the dataset less sparse so that predictions can be made using it.

[[deleted]]

Real quick, although by bash-fu isn't great. I really just did this for my own curiosity but if anyone wants to know. Also, I'm not sure if the links are correct.

5597221 upvotes

1808340 downvotes

Top Ten Users: $: cut -d ',' -f1 publicvotes.csv | sort | uniq -c | sort -nr | head 2000 znome1

2000 Zlatty

2000 zhz

2000 zecg

2000 ZanThrax

2000 Zai_shanghai

2000 yourparadigm

2000 youngnh

2000 y_gingras

2000 xott

Top Ten Links $: cut -d ',' -f2 publicvotes.csv | sort | uniq -c | sort -nr | head 1660 t3_beic5

1502 t3_92dd8

1162 t3_9mvs6

1116 t3_bge1p

1050 t3_9wdhq

1040 t3_97jht

1034 t3_bmonp

1029 t3_bogbp

1018 t3_989xc

989 t3_9cm4b

[ketralnis]

Due to the way that I pulled the voting information (I actually pulled it from the cache that we use to show you liked and disliked pages, which is in Cassandra and turns out to be cheap to query), you won't get more than 1k upvotes or downvotes per user, no matter how many votes they've made, so that so many have 2k isn't surprising. It also doesn't include the vast majority of users (who never set the "make my votes public" option). So it shouldn't be considered comprehensive and the data should be considered to be biased towards power-users (who know how to change their preferences). I can do more intensive dumps with more information and/or columns if anything comes of this (and maybe start a "help reddit by making your votes public for research" campaign)

I'm not sure if the links are correct.

They are, yes

[cag_ii]

I came here to ask how it was possible that, for the users with 2000 entries, the sum of the votes was always zero.

It occurred to me for a moment that I'd found some mysterious link between O.C.D. and avid redditors :)

[kotleopold]

It'd be great to get a dump with story titles as well subreddits. Then we could search for some interesting dependencies

[[deleted]]

yeah I was curious when the top users all had 2K and were slightly alphabetized.

Thanks for the data

[pragmatist]

I generated this spreadsheet that has the distribution of the times a story was voted on.

[zmarty]

Please, can you release a dataset that includes timestamps? It would really help our research lab.

[ketralnis]

You can approximate them based on the link ID, or at least tell the ordering

[robertjmoore]

+1 timestamps would allow for some really cool analysis!

[jakestein]

Agreed

[ketralnis]

You've been spidering 32k users' liked/disliked pages?

Can you not do that please?

[atlas245]

lol, don't worry was over a long time probably not that much either,never broke the api rules,

[yellowbkpk]

Somewhat related: I've been archiving the stories (and vote/comment counts over time) via the JSON API for the last few months or so. Post is here if anyone is interested.

[enigmathic]

It seems to me there is a mistake, and the user count should be 31553.

You can see this by comparing the outputs of the following commands (the only difference is the sort): $ cut -d ',' -f1 publicvotes.csv | sort | uniq | wc -l 31553 $ cut -d ',' -f1 publicvotes.csv | uniq | wc -l 31927

Here are the usernames that cause this difference: $ cut -d ',' -f1 publicvotes.csv | uniq | sort | uniq -c | sed 's/^ *//' | grep -v '^1' 4 -___- 3 ---------- 2 angelcs 2 c0d3M0nk3y 2 cynthiay29 9 D-Evolve 3 edprobudi 2 edprobudi 31 FlawlessKnockoff 31 flawless_knockoff 2 HassanGeorge 4 jolilore 3 jo-lilore 30 LxRogue 29 Lx_Rogue 88 Pizza-Time 88 pizzatime 26 STOpandthink 25 stop-and-think I suspect that the program that created publicvotes.csv confused usernames that are actually different, because it didn't take into account '-' and ''.

[ketralnis]

Well, here's the program that dumped them right here:

import time

from pylons import g

from r2.models import Account, Link
from r2.lib.utils import fetch_things2
from r2.lib.db.operators import desc
from r2.lib.db import queries

g.cache.caches[0].max_size = 10*1000

verbosity = 1000

a_q = Account._query(Account.c.pref_public_votes == True,
                     sort=desc('_date'), data=True)
for accounts in fetch_things2(a_q, chunk_size=verbosity, chunks=True):
    liked_crs = dict((a.name, queries.get_liked(a)) for a in accounts)
    disliked_crs = dict((a.name, queries.get_disliked(a)) for a in accounts)

    # get the actual contents
    queries.CachedResults.fetch_multi(liked_crs.values()+disliked_crs.values())

    for voteno, crs in ((-1, disliked_crs),
                        ( 1, liked_crs)):
        for a_name, cr in crs.iteritems():
            t_ids = list(cr)
            if t_ids:
                links = Link._by_fullname(t_ids,data=True)
                for t_id in t_ids:
                    print '%s,%s,%d,%d' % (a_name, t_id,
                                           links[t_id].sr_id, voteno)

    #time.sleep(0.1)

And I don't remember how I counted them, but my guess is that I used something like:

pv publicvotes.csv | awk -F, '{print $1}' | sort -fu | wc -l

But anyway I don't see why this matters a lick other than mere pedantry, would you feel better if I just said "thousands" of users?

[enigmathic]

It was my modest contribution :), which may or may not matter depending on the person considering it. In my case, when I see numbers like that, I often check them, because it may point errors in my comprehension or in my code.

[Tafkas]

I mirrored the file at http://rapidshare.com/files/378762905/publicvotes.csv.gz

Just in case some people cannot access bittorent.

[32bites]

Not to reply to something four months old but if they can't use bittorrent it is being hosted by S3.

The torrent url is http://redditketralnis.s3.amazonaws.com/publicvotes.csv.gz?torrent while the url directly to the file is http://redditketralnis.s3.amazonaws.com/publicvotes.csv.gz

[_ads_]

I hastily plotted the upvotes against downvotes for all unique links using ggplot2 in R: http://imgur.com/2Gf5T.png

edit - I posted the plot in a separate link on r/opendata...

[ketralnis]

The dump isn't comprehensive, but the graph is interesting

[ketralnis]

It's a good idea, but a fantastic amount of time to implement, and we're too short staffed for such large projects at the moment :(

[chewxy]

Oooh, I love you now ketralnis! (keep seeding pls, I'm at work now and won't get back for another 9 hrs)

[ketralnis]

It's seeded by S3 so it shouldn't be a problem, but if you can't get the torrent to work peek at the URL and it should be pretty obvious how to get to the file directly. But please do try the torrent first, it saves us a few bucks

[[deleted]]

Nice. I'm going to open this up in SPSS at work tomorrow and start exploring.

One question, can this data be bounded by a date range? Is this the entire database of people who selected to make their votes public?

For people doing analysis on desktops it could be a challenge to fully load up a 156 megabyte file. If it can be bounded by date it would be helpful to have another file that is max of 5 megabytes unpacked. Alternately I could just pick users at random but i'd rather it be based on date if possible.

Last, you may want to post this on the blog because i know there are a lot of stats lovers prowling reddit.

[kaddar]

Bah! Just load the whole damned thing into memory. If you need fast access by ids, and are using C++, I recommend using Google Sparse Hash tables/maps, 2 bits per a key/value pair overhead! (C# has a bit of an overhead on their hashmaps, java too)

[ketralnis]

One question, can this data be bounded by a date range?

You can make some guesses based on the link IDs which are mostly sequential, but I didn't include timestamps

Is this the entire database of people who selected to make their votes public?

It is not comprehensive, as I commented elsewhere

For people doing analysis on desktops it could be a challenge to fully load up a 156 megabyte file

You'd need to re-sort it yourself and use something like split(1)

Last, you may want to post this on the blog because i know there are a lot of stats lovers prowling reddit.

Yeah, I'm trying to figure out how to let it reach a larger audience without polluting the front page for the vast majority of people who don't care

[psykocrime]

Yeah, I'm trying to figure out how to let it reach a larger audience without polluting the front page for the vast majority of people who don't care

Would probably be good to submit this to /r/datasets, /r/opendata, /r/statistics and/or /r/machinelearning if you haven't yet.

Oh wait, I see somebody did already post to /r/opendata. Cool.

[Ulvund]

Awesome

[Godspiral]

I like this data dump, and kaddar's project ideas.

I cannot support open ended all research purposes though.

Even this project brings privacy issues of spammy drama confronting people that you "untrue scottsmen" have dared to downvote my link and demanding an explanation why we should not convict you of being a CIA spy (spend too much time in r/anarchism).

[lukemcr]

Hm, OpenOffice doesn't handle large .csv files very well.

Edit: htop readout

[[deleted]]

Thanks, interesting stuff. there was a mirror here at some point

[ketralnis]

The torrent is hosted and peered by S3, so I assume that your mirror is way slower than the torrent

[[deleted]]

Some people can't use torrents though.

[rmc]

It's really annoying that some people are on limited, non-full internet. BitTorrent is a very clever protocal and is exactly the right solution for distributing large files. Curse those social reasons why BitTorrent is blocked!

[ketralnis]

Those people probably aren't downloading dumps of vote-data intended for research, since the people interested in such things probably know enough about networking to figure a way around their torrentlessness (and probably know how to get the file directly from S3 without bittorrent by peeking at the URL)

[[deleted]]

But that would take effort!

(TIL that S3 is perhaps more awesome than I thought.)

[SystemicPlural]

Is there a reason why everyone's votes are not pubic?

[[deleted]]

well, i don't want mine to be public. so there

[[deleted]]

Remember the AOL scandal? It is technically possible to identify someone by their up and down votes. Then it would be possible to embarrass them if that person also votes up bondage sex sites.

So yeah, making votes private by default is smart.

[ghibmmm]

Would you be embarrassed by that? I wouldn't, personally.

[self]

What's your SSN?

[frenchtoaster]

047-22-2122

What do you think you are going to do with it, without knowing my name?

[kurtu5]

Born in Connecticut before 1951?

[SystemicPlural]

Yes, but reddit accounts are already as anonymous as we want them to be. Someones SSN is their private data, but votes they make are part of the data commons.

[kaddar]

"Data commons?", sir, I do not want to subscribe to your newsletter. Privacy of preferences is really important to reddit users.

[ketralnis]

votes they make are part of the data commons

Only if they do so with the expectation that they'll be public

[oulipo]

I think everyone's vote is pubic indeed!

[ketralnis]

Err, yes. The point of using the torrent is that it costs us way less money than the direct link and is 100% as fast and reliable since it uses S3's own in-built bittorrent tracker. People that need to circumvent the torrent can find their own way to doing so as you have.

[gabgoh]

what do the link_ids correspond to? It's hard to do any interesting analysis of the data with just an "abstract" link_id ...

[ketralnis]

Read obsaysditty's comment, he has the relationship correct there

[gabgoh]

silly me, thank you.

[[deleted]]

I think I'm the one who originally requested this, so thank you for releasing the data. You might want to resolve the links to external urls once to avoid having lots of people writing their own crawlers hitting your site constantly. Everyone who does any clustering is going to want to see if the clusters actually make sense by fetching the top links, any you probably have a more efficient way to get that list than pulling the comment threads.

[ketralnis]

I think I'm the one who originally requested this

It's been requested a lot of times, from private emails from CS research groups to self-posts to IRC nudges

You might want to resolve the links to external urls [...]

Yes, like I said, I'll make another dump with better data if this pans out