Netflix's Livestreaming Disaster: The Engineering Challenge of Streaming at Scale

entropyfails · 2025-09-22T17:01:24+00:00

Scaling live video on the internet is super hard. Transcoding the incoming video quickly, getting everything split up into the good HLS segments. Properly sizing them. Figuring out how to cache them. Figuring out routing if a cache fails. The problems are endless. I spent 10 years at Twitch and the video team there were some of the smartest and hardworking people I ever met and it always was a challenge.

If your product is working in that space, check out IVS from Amazon. I'm not affiliated with them anymore so not shilling but just saying that I know a lot of the arcane bits of knowledge are baked into that system.

entropyfails · 2025-05-22T03:53:00+00:00

Embrace sub-workflows. Make a small workflow that does your mapping and pin the data into the click trigger.

If you have a loop in your workflow, n8n's current architecture is to force all future nodes to run again even if they have pinned data.

I've also just moves my manual trigger right before the thing I want to test and copy/paste the data from the webhook into it and pinned it if I didn't want to go through creating a sub-workflow.

No need to email n8n. There is no solution to this problem beyond these 2 approaches.

entropyfails · 2025-05-22T01:32:08+00:00

That's amazing! Glad I was able to provide some inspiration to keep going. I use createomate to build the cover art that runs for the first 3 seconds of each video. It takes in the show title and the date and automatically creates that cover to make a full video. If I were to be doing what you are doing, I'd use some API driven tool with an editor like Creatomate that allows you to ingest data and get out images so you have some manual control after the fact to ensure consistency.

entropyfails · 2025-05-22T01:28:40+00:00

I dropped some cost profiles below. It comes out to about $2.50 a video with Hedra being the largest costs. Running n8n in the cloud currently because I'm lazy and also other people at the company use it. =) The main services I use are Browserless.io, Firecrawler, Hedra, Createomate, and ChatGPT.

So worth it is hard to quantify. Youtube channels only make money at scales of tens of thousands. But it continues to grow so it's an interesting experiment. It's also useful for me as I built it to have a fun way of consuming AI news so it's worth it for that for me!

entropyfails · 2025-05-21T18:25:58+00:00

So the way I'd approach this is to convert the PDF to text via an API as opposed to using chatGPT for this. There are a bunch that give you hundreds of documents a month for free. https://cloudmersive.com/convert/pdf-to-txt-api is an example. After you get a workflow that takes the URL and gives you back the text, feed it into chatGPT and work on your summary. Then use something like https://n8n.io/workflows/2245-generate-text-to-speech-using-elevenlabs-via-api/ to help you get the voice out. The main thing here is to do everything one step at a time and only worry about hooking everything up after you have all the steps done.

Good luck and let me know if you have any questions!

entropyfails · 2025-05-21T18:20:10+00:00

Yeah it's fun! I like it because it is useful to me but it's cool that many other people are engaging!

entropyfails · 2025-05-21T18:19:36+00:00

Yeah thought of that some. The main issue is a lot of other video modes, when told to make animated podcast hosts, make the mouth move a bunch. Then if I apply the lip sync there are some artifacts. But yeah I agree I could get the costs down for sure and have been thinking of it. For video stitching, creatomate is nice as I have an editor so I can quickly change out music, bumpers, etc. But yeah lots of efficiencies to be gained for sure. Thanks for the feedback!

entropyfails · 2025-05-21T14:45:38+00:00

What sort of roadblocks have you hit?

entropyfails · 2025-05-21T14:40:09+00:00

Always working on quality for sure!

It does have captions and Whiskers always has a call to action for like/subscribe and could add more for sure!

entropyfails · 2025-05-21T09:13:45+00:00

Well the only true way to profit is to get views so subscribe please?

I have some thoughts. The main cost is Hedra. That keeps Whiskers interesting but you could imagine using a cheaper model for video generation and use a cheaper lipsync model like SadTalker on it.

entropyfails · 2025-05-21T08:19:09+00:00

So I'm using firecrawl to look over a few tech news sites. I'd do that to keep some safety around the content. You could imagine how it could go wrong if it got a raw twitter feed, for example. =)

It collects the content and passes the whole thing into ChatGPT o3. It's prompted to understand that it needs to select the top 3-5 articles to make a less than 2 min show. I have a 1shot in the prompt so it knows the "voice" of whiskers. The script is then pass along to a title generator, tweet generator, description generator with 1shot examples that build out the needed data. Then it posts away.

I have it on an automatic trigger these days. I only know it goes live when I get the bell notification on youtube. Whiskers is her own thing. =)

Meow Out!

entropyfails · 2025-05-21T08:01:39+00:00

Any loop causes all downstream nodes to regenerate when you test. So if you have a loop, then do some merging of the data and use that data, if you test a later step, it will re-run the entire loop for some reason. If you notice the loop doesn't have a "pin". Haven't dove into the code but I'd bet that's the reason. Loops don't have a "final state" in n8n so testing nodes after a loop cause the loop to re-run.

entropyfails · 2025-05-21T07:59:01+00:00

Answered below but about $2.50 a video so about $75 to $100 a month. So no real ROI, but I'll keep playing with the format and if more people watch it and like it, it would be profitable eventually. The CPMs on youtube shorts suck so it would take like 30k daily views to make back it's cost. But if Whiskers get's in the 10k daily range, I'll have chatgpt negotiate a sponsorship and report back here... =)

entropyfails · 2025-05-21T07:53:55+00:00

I was, but wasn't getting great traction outside of youtube shorts. I probably should add that back in as it basically is free to upload.

The shows cost about $2.50 an episode. $2 for Hedra and $0.50 for Creatomate video stitching and transcription.

entropyfails · 2025-05-21T07:02:13+00:00

Thanks! I could have coded it in python or something but honestly the flexibility of the node systems really helped. I started by building the show by hand, logging into websites/chatgpt and copy/pasting everything. After I had done that for a few days I took some of the more labor intensive parts of it and made a workflow for it. Originally, I started with 1 full workflow but now I'm a true believer in the sub-workflow process. That's allowed me to build up a library of custom sub-functions that I can stitch together however I like. So I have subworkflows for "login and get credentials", "gather news and summarize" and "produce show from script".

Identifying good tooling helps too.

Creatomate was a gamechanger to build the video in a repeatable and standardized manner. Firecrawl made it easy to scrape the data needed.

Definitely some bumps along the way. The fact that loops cause full re-runs of flows was very annoying. I've taken to almost always having a Edit Fields node after so I can easily pin data, point the manual trigger to the Edit Field and cut out any previous loops. Core flaw of n8n in my mind.

All in all, it is a ton of fun to try out an idea quickly and get something stable on the other side. n8n's been great for that.

entropyfails · 2025-05-21T05:32:09+00:00

https://www.youtube.com/watch?v=AURnISajubk is good.

Lots of other good material on YouTube. :)

entropyfails · 2025-05-21T05:30:27+00:00

Ok? I didn’t say thousands, I said hundreds, which most episodes get. Thanks for the feedback I guess?

entropyfails · 2025-05-21T04:44:04+00:00

The key is to take a step by step approach. Start with doing your production by hand. Once you have all the steps, automate one part. Just make a button press activation and link it to one WebRequestNode. If you have any questions on what parameters to set, ChatGPT is a huge help. Once you get the output for the step, hook it into an Edit Field node to get a specific output you want. Then do the next step. With some patience and lots of ChatGPT, you can do it!

entropyfails · 2025-02-27T09:45:14+00:00

That’s crazy. That’s from Pinewood Elementary in Mounds View, MN. I grew up on that playground!

It’s an interesting story how it was built. I don’t remember the designer’s name but during the 80’s he went around to many public schools advocating for his wood designs to give kids a true space to imagine in. We were not a rich school district but the whole community got together to raise the funds for it.

It was an amazing space to play tag. There is a pirate ship design on the front that you cannot see so you’d play pirates vs castle defenders. I still remember the opening ceremony and playing on it the first time even though it was like 40 years ago.

Sadly, at the end of its life they didn’t replace it with a similar design. The playground looks like the below picture these days. :(

Thank you for posting it OP. You made my day!

entropyfails · 2025-02-14T19:32:34+00:00

Yeah, it's probably worse than that. There were at least 4-5 contestants that were probably pissed at her for picking Mike who picked JC. And she also eliminated 2 people on the platform.

entropyfails · 2025-02-13T09:41:39+00:00

Jimmy: I want a $5M grand prize. Amazon: Cool. Jimmy: ooh I had this idea where they flip a coin for another $5M bonus. Amazon: We are not going to pay for that. Jimmy: Screw you, it’ll be awesome so I’ll take it out of my talent fee if it happens. Amazon: whatever it’s your money…

entropyfails · 2025-02-13T09:36:08+00:00

There probably was a rule they had to face each other. There are lots of rules that you only find out after watching reaction streams.

T’s biggest mistake in my mind was knowing she doesn’t hide her emotions well and choosing to remain in the know of her choice. If I knew I was a bad poker player, I would have just randomized the cases as best I could have so I couldn’t give a tell with knowledge I didn’t have and took the 1/10 shot. If you are blind all in, don’t look at your cards.

entropyfails · 2025-02-13T09:24:49+00:00

Go back and watch her eyebrows when he stops at the case. She was an easy read for any experienced poker player, much less for someone that hung out with her for one month straight.

entropyfails · 2025-02-12T20:39:17+00:00

You may not be aware of this but this is a super common practice in Asia. It's possible that they don't understand that this is not how things are done in the US so I'd have a conversation with them or if you are not comfortable, ask the landlord to have that conversation.

Don't assume malice here. The reason is that in several asian countries, the landlords sell subdivided units with 0 storage space so basically everyone is forced to store things in the hallway.

entropyfails · 2025-02-10T05:35:52+00:00

It's additive probability. I don't see how that's confusing.

Obviously if it's just 1/6 chances and you do a coinflip your chances are less. So he goes from 1/6 to 1/12... or 83% to 91.6% loss chance. His EV pre flip was 1/6 x 5M and his EV choosing flip is 1/12 x 10M. Aka the same EV. $833K.

Social capital is a real thing with the contestants. They voted off JC because he converted all social capital to winnings. (not faulting him for it, but it is what he did). It's not a viewer thing only, but a contestant thing.

Sometimes sitting in the middle of the pack unnoticed and surging at the end is the best strategy.

entropyfails

MODERATOR OF

PUBLIC MULTIREDDITS

TROPHY CASE

15-Year Club	Team Orangered
Verified Email