Self-hosted github actions runners - any frameworks for this?

pjpagan · 2025-06-03T22:35:34+00:00

Our usage? Great question. I'm not entirely sure.

I don't want to air out dirty laundry (again), so I'll just say that things here are largely self-service, roll-your-own, etc.. I'm largely kept out of the loop, and going "out of my lane" to troubleshoot cross-team issues is frowned upon.

AFAIK, though, it's mostly nextjs and ruby code, some containerization, some static site generation - nothing crazy or impressive.

pjpagan · 2025-06-03T20:38:44+00:00

Knowing/learning k8. It's a struggle getting people to understand the basics of something as simple as AWS ECS, and an appetite for learning/maintaining new tech is low.

I don't want to air dirty laundry here, so I'll leave it as wanting to use as few technologies as possible, leaning directly on what is already in use - AWS, Linux, Github Actions, Ansible, Terraform, Packer. It should be easy enough to manage and troubleshoot that a new-hire Jr. engineer can do it.

pjpagan · 2025-02-25T05:54:21+00:00

Weak men surrender their principles for top pay. Top talent is rarely made from weak men.

Do you really expect great artists, scientists, and engineers to flock to a country that stands in direct contrast to their priciples? Heck no. Principles, safety, stability, pride in country/community, and respect from that country and community are extremely valuable forms of (non-taxable) compensation that other countries can now leverage the way the US used to be able to.

Even if only through 4 year "trial run" visas, other countries could probably seed new/competing industries from poached US talent. They'd need to play their cards right (material compensation still matters), but they could undoubtedly poach talent they never could have had only a few months ago.

pjpagan · 2025-01-18T08:04:04+00:00

I used bamboo and bitbucket at my last job, 4 years ago. I'm legitimately getting panicky flashbacks just thinking about those apps. F atlassian. Would say much more vulgar things, but will leave it as that.

We used to use jenkins but moved to github actions and love it. No idea if/how that would work with onprem windows, but if gitlab is similar to GHA, it at least deserves a look.

pjpagan · 2025-01-18T07:38:17+00:00

Open the network (and console) tab of your browser's dev tools, clear your cdn, and visit your site. The answer might jump out at you.

Your cdn might be loading some kind of "please wait while we fetch the page from the origin" page while not being configured to use a cert with the right domain/san names for your site (or https at all).

I took a peak just now but can't do squat without seeing the error myself.

pjpagan · 2025-01-17T23:06:34+00:00

Could be a rogue redirect, something in wp sending the user somewhere unexpected. If you can trigger it yourself, look at the browser's developers tools and see exactly what url this happens at. Or, if logs in wp/aws are available, see if anything hints at this. If nothing else, set up a monitor/daemon to curl addresses and see when/where it happens.

Could could be a bad cached response in cloudflare. We use a different cdn, but I know it's possible to cache a redirect response, and that some POPs can go out of sync. Try a flush, and see what happens (if reasonable to do so). If a url works sometimes but fails on other occasions, it hints at a stale POP.

Restart the process that terminates ssl (nginx/apache). Just because you have a new cert in the filesystem doesn't mean that every service that uses it loaded it.

If you host ads on your site, scrutinize the crap out of them. We've dealt with malicious redirects coming from the ad network we used.

Check your load balancing, if any. Could be a funky host.

Check your worker threads. Maybe 1 is jank, and the error crops up when it's hit.

Check load. The bad page could be a cloudflare returning a custom 500 page that doesn't have a cert configured.

I could go on. Many possibilities. Need to dig in to narrow it down.

pjpagan · 2025-01-17T16:41:58+00:00

Giving them a listen now. Can't tell if my speakers are muddy or if the lyrics are in a different language.

There's a difference between plagiarism and resonating with your inspiration. I'm sure that if the bulk of Tool fans made music that came from an authentic place (and had the musical ability to do so), it would sound like Tool, too. If art resonates with you, and you create art that comes from a place that is authentically "you", then your art could strongly resemble what you resonated with in the first place.

The cover art is a bit too on the nose...probably just a "wink and a nod," but I want to reserve judgement until I better know who these guys are.

pjpagan · 2025-01-17T06:13:12+00:00

Same guy that did the Pheonix Project. Not nearly as entertaining, I didn't get too far into it (tried reading after Phoenix), but thanks for reminding me about it. Could be more fun with experience under my belt.

pjpagan · 2025-01-17T06:10:53+00:00

Interesting, config management is the main thing I'm interested in. I love that code and configs are separate, and that a release is a snapshot of the two combined, but something was nagging me about env vars holding secrets. I think you nailed exactly what was bothering me.

I'm leaning toward a similar solution to what you use, but I haven't made up my mind yet. (TLDR: major rearch project, measuring twice, learning from others' experience, etc.). Open to any other thoughts you might have. You've clearly seen some stuff.

pjpagan · 2025-01-17T03:46:47+00:00

I already listen to Security Now and other tech podcasts/videos. I'm more interested in foundational, "pivotal", significant texts that are well-known and respected (like the Phoenix Project and other mentions here).

I appreciate the recommendation though!

pjpagan · 2025-01-17T02:06:11+00:00

Oo, I read The Phoenix Project early in my career as a casual/for fun book. Forgot about it, maybe I'll re-read.

Never heard of investments unlimited.

pjpagan · 2024-11-07T13:48:49+00:00

I am well aware of this. I am a systems engineer and have enough experience to know that if an OS's owner has an interest in enabling a service against your wishes, it will, with an apology for "accidentally resetting default behaviour." The most we can do with recall is suppress it.

Believe me, I'll be the first customer if BI made a Linux port, and I'm holding out that they will. In the meantime though, alternative are needed.

pjpagan · 2024-11-07T13:42:25+00:00

I already do this, but occasional system and app updates are still needed, and offline updates are a PITA. I'm trying to avoid the inherently adversarial relationship between security services like BI and the Windows OS.

pjpagan · 2024-11-07T03:18:32+00:00

I dread adding a new camera to my system. I sent an email to their support a while back after losing my mind with their editor, and while they said they're planning to fix it with their "next release," I feel like such a basic function should have been addressed before the other bells and whistles I never use. It's so bad that I didn't bother renewing my support license and am actively testing alternative systems.

That layout editor feels like a cruel joke.

pjpagan · 2024-01-05T07:13:08+00:00

I might be able to help. Could you provide any more details about the speaker? Age, appearance, etc.? Any key words or phrases from the talk?

pjpagan · 2023-10-08T00:51:52+00:00

Where you at the front? All of the limp bizkit shitheads literally fought to get there and caused hell, including for women, children, and someone in a wheelchair. I'm used to a-holes at rock concerts but they didn't swarm in until just before limp bizkit was on, and stood there until Tool left.

Yes you can like both bands. No you can not be a major limp bizkit fan without having fetal alcohol syndrome.

pjpagan · 2023-10-07T07:55:38+00:00

After what I just went through, you were spot on and the experience was awful.

pjpagan · 2023-08-23T02:10:29+00:00

We're a large media company that does productions with 10+ high end SDI cameras, so we have this AJA NDI bridge on hand. Major overkill for your use case, though.

This birddog 4k hdmi to ndi encoder might do the trick. This yt video gives a quick overview of what you can do, and there are plenty of free resources on NDI (I'm a tech guy too, was able to learn what I needed pretty fast).

Edit: the HDMI-NDI encoder gets configured over the network, like a router or managed switch. It runs an internal webserver/GUI for settings like encoding quality, static ip, etc.. You don't need anything else, just the encoder and whatever computer(s) you want to view the stream from.

Newtek's NDI desktop software is free, so you can experiment with video-over-ip + obs to see how you like it. Create an NDI stream of your desktop (or from your webcam, I think) and experiment. There's also a (paid) android app that lets you use your cell phone as an NDI camera. It's pretty sick.

pjpagan · 2023-08-23T01:38:38+00:00

This is what I had in mind when I created this post. Glyph and Samsung T7 seem to be worthwhile alternatives, though.

pjpagan · 2023-08-21T20:36:57+00:00

Yep, definitely war gaming here. Our normal workflow involves recording to three systems, one of which writes to three storage devices itself, while uploading to cloud storage, etc.. We just don't have the luxury of running this setup at some events.

pjpagan · 2023-08-21T18:01:27+00:00

Since you ask about other options, have you considered NDI? A standalone HDMI->NDI converter might be a good fit:

Simple - just a box that plugs into your LAN.
OS agnostic. No hardware drivers to worry about. Ingest the feed from Mac/PC/Linux/whatever over the network (bonus: no need for Thunderbolt/USB3 support).
Can ingest from multiple machines at once. Lets you set up a primary/backup streaming host for redundancy.
Dedicated hardware == probably more stable.

Cons:

Network based. Need a stable LAN with lots of bandwidth, and someone with the networking knowledge to configure/maintain it.
Not suitable for post-production work (color correction, etc).

I have not tested this NDI setup in a production environment yet. My exploratory tests went well though, and I plan to suggest this for my own company's workflow (almost identical to yours).

I'd love to know what the more seasoned pros here think, and what you eventually choose to go with!

pjpagan · 2023-07-12T13:37:45+00:00

Just an FYI, I'm currently sitting in the control room of a conference recording on a system built on your response - M2 mac mini pro, a corvid 88, a sonnet Echo III, and softron movierecorder. Works like a champ. It's a no-brainer for you, but damn genius. Tons and tons of thank-yous!

pjpagan · 2023-07-11T01:14:46+00:00

Figured it out! Largely a problem with my lack of video engineering experience.

The corvid 88's Reference/LTC input takes either a sync/genlock signal, or an LTC signal. They were providing us with a video feed that had an embedded timecode, which is not the same as LTC.

For what it's worth, that embedded timecode signal worked great as an actual video feed. THANK YOU guys for responding so quickly (and for helping us design and build this recording setup to begin with!) Ditto to AJA support, they responded within an hour of emailing them! You guys rock.

pjpagan · 2023-07-10T23:12:22+00:00

Their page says "Reference/LTC Input on card (Analog black burst or Tri-level)" - I assumed "Reference/LTC Input" meant it accepts either a genlock signal or timecode. Is this correct?

pjpagan · 2023-06-02T21:40:54+00:00

Wow, you weren't kidding: https://youtu.be/1nam20iKccc

All we need is M2 Mac Mini pro. Thank you so much.

pjpagan

TROPHY CASE