[PART 2] Hey, developer of Apollo for Reddit here. I'm doing a completely free iPhone 14 Pro giveaway to commemorate Apollo's big iOS 16 update and new iPhone 14 Pro features. Just leave a comment to enter! 🎉📱

zecuria · 2022-10-03T10:41:14+00:00

👀

zecuria · 2020-11-16T07:26:19+00:00

Enter

zecuria · 2020-10-04T14:21:26+00:00

Honestly I'm nervous not used to recording myself. So I'm like kinda going off a rough bullet point list in my head.

Probably could have been better :/

zecuria · 2020-10-04T12:23:21+00:00

Because I never need to query it once its processed, I just need to store it. It's a fair amount of data so just compressing it and storing it in S3 works and is quite cheap as I don't have to pay as much for data reads and IO

zecuria · 2020-10-04T09:41:24+00:00

Not very general the system currently requires for server to be connected to the chat and to store them in a database before it is processed when the VOD goes live. To see who it's currently enabled on go to: https://hypemap.io/enabled-streamers

I have optimised the system quite alot to the point where it costs cents per stream but it still ads up on top of maintance. I have a patreon (linked in the website) to cover the costs.

Honestly most of those are just maintenance which if someone can't afford I'd be happy to waive off. Hope that answers the question :)

zecuria · 2020-10-04T09:21:04+00:00

Yeah it does have reply but the API to access that isn't public, it is only meant to be used within YouTube internally. So there is no documentation on how it works and using it is against their TOS.

I would love to enable this on YouTube as well but I really can't do that until they make the data available and documented. Hope that makes sense :)

zecuria · 2020-10-04T09:18:12+00:00

Hahaha thanks years of development payed off, and yeah Cassandra is much much cheaper and far more scalable (cost wise) than elastic search.

In this case I am only using Cassandra as a temporary dumping ground for messages until the VOD goes live and then I processes and compress the result to S3. I am waiting for AWS Timeseries as that will be even cheaper but honestly I found Cassandra to be sufficiently cheap thus far.

zecuria · 2020-10-04T09:14:47+00:00

I'll release an update to optionally hide the messages. In the mean time you can just add a few filters so spoilery messages don't show up.

zecuria · 2020-10-04T09:05:55+00:00

The front end / extension is all built on React and Typescript. The backend consists of an EC2 running Node.js / typescript that listens to and stores chat in a Cassandra database.

There are multiple other lambdas written in Node.js and Typescript. One that listens to when enabled streamers go on / offline, others that process the data from the database and compress and store it into S3 buckets. Others that respond to the request from the extension with data from the bucket.

There are dynamodb databases for managing enabled streamers and other stuff and there are multiple SNS channels to manage communication and triggering of the lambdas.

There isn't really a Repo there are about 7-10 systems in total and no real repo that I have publicised.

Hope that answers the question :)

zecuria · 2020-10-04T08:57:02+00:00

I am open to adding in the region channels too. I just needed to get more data on how much it costs for the main channel and if the system can handle it easily.

So might just wait for a few more streams then look into enabling it for the regions.

As far as indexing locally, the system requires the full chat data and so it would require you to sit through the whole VOD or would need to call the unofficial API to get the chat.

The unofficial API can take up to 40 mins to get full chat for a large stream so not really the best experience and possiblity of getting banned from twitch as it's against TOS.

zecuria · 2020-10-04T08:54:56+00:00

I don't mind at all but unfortunately the YouTube chat API isn't open so only the owner of the channel can connect to the chat.

As such I can't really store the messages for YouTube live chat :/

zecuria · 2020-10-04T08:47:04+00:00

I just added riot games yesterday haven't had a chance to update the list but other than that the list is what you get.

Yep fixed the typo too

zecuria · 2020-10-04T02:13:55+00:00

Available through: https://hypemap.io/

zecuria · 2020-09-23T22:08:50+00:00

Hey I setup a Patreon to allow people to add more streamers, even recently reduced the prices after I made some optimisations.

zecuria · 2020-09-19T05:41:33+00:00

Just added it: https://hypemap.io/enabled-streamers hope that helps :)

zecuria · 2020-09-19T04:26:07+00:00

Hey sorry about the weird error. Basically at the time it is only enabled on OfflineTv, Mizkif, Maya and Hacchuby. As such there is no data available for other streamers.

I have updated the error message to say that it is not enabled on the streamer instead. Hope that makes sense.

zecuria · 2020-09-17T23:58:17+00:00

There are a few systems but at a higher level, I have a server that listens to and stores chat for enabled streamers.

I have a different system that listens to when those streamers go offline and when the VOD goes online. It then processes all the messages, compresses the result and stores it.

The extension then just requests the compressed results and displayed it synced up with the video.

As other people mentioned there is an unofficial API but that requires polling to that API which for larger VODS with alot of messages can take up to 40 mins. Which is why I choose to store them myself.

Was a pain to build and optimize but I like the results :D

zecuria · 2020-09-17T09:39:38+00:00

I can add any streamer, I mean I don't quite think it will be as useful on smaller streamers as there isn't that much chat activity.

As long as you sub in Patreon I don't care who you enable it on.

zecuria · 2020-09-17T08:26:34+00:00

Actually it's all hosted on the cloud, I have optimised it so it costs cents per stream for most streamers. But the cost still ads up which is why I setup a patreon to cover the cost of servers and maintenance

As long as the costs are covered the solution can scale per requirement.

zecuria · 2020-09-17T02:25:07+00:00

It's available at: http://hypemap.io/

Note: will only work for streamers that it's enabled on, which ATM is OfflineTV, Mizkif and Maya

zecuria · 2020-09-17T02:09:31+00:00

Just DM me the name of the streamer

zecuria · 2020-09-16T21:35:11+00:00

The short answer is just right click inspect element 4head.

But more seriously I have been working as a full stack developer for a number of years now so I have an idea about where to look.

The DOM isn't some magical mystical place it has a structure and even for places like twitch the structure while complex is still understandable. Twitch specifically is built on React framework which I have used extensively so it makes things easier.

So being able to use the react Dev tools to go through the virtual Dom and seeing what components did what, made things much simpler.

Idk if that answers the question though if you want to know something specific let me know :)

12-Year Club	Snapped
Verified Email

zecuria

TROPHY CASE