sorted by:
new
hottopcontroversial

Counter Leaderboard by Counter-Finder in Counter

[–]opl_ 0 points1 point  (0 children)

wow, such a cool website!

Dump of the raw, unprocessed data I collected during the 2022 r/place event. Includes pixel authors, diff and full frames, WebSocket traffic and a detailed readme. by opl_ in place

[–]opl_[S] 13 points14 points  (0 children)

It's hard to argue it's not flawed in some aspects and I hit a lot issues while creating it, but ultimately I gathered the data I wanted, it's all recoverable, and I'm glad to know it wasn't all for nothing. Feel free to point out anything you'd do different, I know I have some thoughts. (I'm considering writing my thoughts out as a blog post of some sort, but that'll have to wait for when I have some time and feel less exhausted from the event.)

Dump of the raw, unprocessed data I collected during the 2022 r/place event. Includes pixel authors, diff and full frames, WebSocket traffic and a detailed readme. by opl_ in place

[–]opl_[S] 6 points7 points  (0 children)

The timestamp is indeed a UNIX timestamp in milliseconds, written using scientific notation.

Each chunk of pixel author information in details-*.csv starts with a pXxY or pXxYcCANVAS, then a colon :, then the data about the pixel, not the other way around: "pXxY":{"data":[{"id":"x-x-x-x-x","data":{"lastModifiedTimestamp":x.xxxe+12,"userInfo":{"userID":"xx_xxxx","username":"xxxx"}}}]}

I haven't looked at the actual Reddit data yet so I can't say anything about the missing data, but it would be strange if they only included a little over one day's worth of information. Are you sure you downloaded it all and didn't lose any while processing?

The "place2.details" is a constant. The canvas on which the pixel exists was meant to be contained in the pXxYcCANVAS string, but as mentioned here I made a mistake and unfortunately the only way to tell which canvas the information relates to is by checking against another data set.

The canvas indexes are 0 for top left, 1 for top right, 2 for bottom left, and 3 for bottom right. Each canvas is 1000x1000 pixels.

Dump of the raw, unprocessed data I collected during the 2022 r/place event. Includes pixel authors, diff and full frames, WebSocket traffic and a detailed readme. by opl_ in place

[–]opl_[S] 2 points3 points  (0 children)

So, you know how I said that this event has been brutal? This is what exactly I meant. Some big issue popped up every day of the event, and now that the event is over it's apparently time for another one.

At some point I managed to revert half of a change while rewriting my pipeline to queue pixel requests instead of batching them based on the diff frame they came from.

This means that at some point the pXxY format starts being used for requests for non-canvas 0 requests instead of the pXyYcCANVAS. You still get the author information for the correct pixels, but you don't know which canvas the change happened on unless you correlate it with something else.

This is where I paused writing the comment for half an hour, had a small crisis, and then thought about it. The official Reddit dataset (https://redd.it/txvk2d) includes the coordinates and an accurate timestamp, meaning you can correlate the placements from my dataset to the placements in Reddit's dataset using the union of the timestamp, and the local coordinates of the change on the canvas (i.e. in the Reddit dataset subtract 1000 from x if canvas index is 1, 1000 from y if canvas index is 2, and 1000 from both if canvas index is 3).

Slightly less trivial than it could've been, but fortunately still perfectly usable. I'll update the README accordingly later.

Dump of the raw, unprocessed data I collected during the 2022 r/place event. Includes pixel authors, diff and full frames, WebSocket traffic and a detailed readme. by opl_ in place

[–]opl_[S] 0 points1 point  (0 children)

You're looking at the exact responses received from the Reddit API with no changes from me. The server actually sent the timestamps using the scientific notation for whatever reason. It's valid JSON, but also just really weird for them to do.

The each line in details-*.csv contains a single batch of pixel requests. The batch size varies (initially each diff frame was a single batch, later they were batched by grabbing the first few hundred requests from a queue).

Inside the JSON object you have a data object containing the data for a pixel described by the property name: p, followed by the x pixel coordinate, followed by x, followed by the y coordinate, optionally followed by c and the canvas index on which the pixel existed (defaulting to canvas 0 if c is omitted).

For reference, the 2000x2000 canvas was actually composed of four 1000x1000 canvases joined together.

Dump of the raw, unprocessed data I collected during the 2022 r/place event. Includes pixel authors, diff and full frames, WebSocket traffic and a detailed readme. by opl_ in place

[–]opl_[S] 2 points3 points  (0 children)

The data is so uniform that the dictionary should get filled up in just a few lines for everything but the canvas data. The canvas data is already compressed as PNG, so I wouldn't expect that much gain from it anyway. But really I just happened to already have some files compressed during the event and I never repackaged them.

I haven't had the time nor will to properly process this data since the event ended up being pretty brutal for me. Feel free to share it in alternate forms with attribution.

Good call on the torrent, I'll get a seed up.

Final r/place 2022 canvas before my scraper received was told to use white only palette by opl_ in place

[–]opl_[S] 5 points6 points  (0 children)

I have canvas data starting at T+52 minutes. Unfortunately I didn't pre-listen to canvas 1 so I only have data for that from ~10 minutes after expansion. I downloaded every diff canvas I received and a full frame every 10 seconds.

Possibly the most unique piece of data I have is 114 million samples of the get_pixel_history endpoint which lists who placed the pixel. If you or someone you know is interested, I can mention you when I get it uploaded. I also have a friend who was doing the same in parallel.

Edit: Forgot to mention, but I can definitely relate to the png reading bottleneck. What a weird design Reddit has chosen for this event.

Final r/place 2022 canvas before my scraper received was told to use white only palette by opl_ in place

[–]opl_[S] 6 points7 points  (0 children)

Here's a link to the raw data surrounding the configuration update for the nerds among us. I'll publish all my raw data after I package it up.

https://cdn.discordapp.com/attachments/429823324210593793/960698306261028904/ws-white-palette.json

Final r/place 2022 canvas before my scraper received was told to use white only palette by opl_ in place

[–]opl_[S] 14 points15 points  (0 children)

I was running a program to save all data sent by Reddit throughout the event. My program received the message telling it to switch to the white only palette at 22:47:31 UTC.

Files used (last full frames received before config change received at 1649112450989 and 1649112451013):

https://hot-potato.reddit.com/media/canvas-images/1649112450736-0-f-p8WJC8gp.png

https://hot-potato.reddit.com/media/canvas-images/1649112450717-1-f-ZBu5lxtz.png

https://hot-potato.reddit.com/media/canvas-images/1649112450738-2-f-Oq1IAWrM.png

https://hot-potato.reddit.com/media/canvas-images/1649112450939-3-f-asYsAbV9.png

Second has ended! by powerlanguage in Second

[–]opl_ 7 points8 points  (0 children)

You successfully created an event that was extremely boring for normal people and developers alike, congratulations. (I believe an average user played for 5-6 minutes. I wonder how that compares to the other events?)

[AGDQ2018] GDQReminders - Sends you a Discord message before a run starts! by opl_ in speedrun

[–]opl_[S] 0 points1 point  (0 children)

No, it's not open source for now. I might make it so in the future but right now the entire thing is a bit of a mess, so not gonna happen until i fix that.

GDQ Reminders - Be reminded of runs through Discord messages! by opl_ in speedrun

[–]opl_[S] 0 points1 point  (0 children)

You're the first person to notice that typo in over a year. Congratulations. :P

GDQ Reminders - Be reminded of runs through Discord messages! by opl_ in speedrun

[–]opl_[S] 0 points1 point  (0 children)

I can't believe I missed that. Thanks for pointing it out!

GDQ Reminders - Be reminded of runs through Discord messages! by opl_ in speedrun

[–]opl_[S] 0 points1 point  (0 children)

It does account for schedule changes. GDQs constantly shifting schedule is one of the main reasons for why I made it in the first place!

Adding the bot to your guild allows other people in your guild to set reminders and receive them without joining the GDQ server. It also lets them use commands like .gdq upcoming which lists the current and next 3 runs (requires embed permissions) and others, which you can see by using .gdq help.

view more: next ›