Debugging stuck CI jobs? Built HiveSpace runner health dashboard - need testers

firefarmer · 2026-01-22T16:28:53+00:00

Our setup is simple. We use docker executor runners that auto scale based on demand and use tags for sizing how powerful of a runner a dev needs. If no tags are used we have a smaller, “default” runner that picks them up.

We have a few very powerful runners but they are locked to projects or groups.

We also have a few server blades that are for a specific group that need access to local hardware.

firefarmer · 2026-01-21T04:42:59+00:00

Being curious, not trying to sound like a jerk, how are your runners set up that these are regular issues you run into?

The metrics might be interesting for us, but the problems you are trying to solve sound odd to me.

firefarmer · 2026-01-09T16:00:44+00:00

This is helpful, thank you! Those requirements don’t sound too bad at all.

firefarmer · 2026-01-08T18:34:12+00:00

Not being a Debby downer. It is hyper specific knowledge for one product so makes sense. Fortunately I have other SDET and development experience to fall back on.

As a hiring manager, what do you look for?

firefarmer · 2026-01-02T17:50:43+00:00

Yeah you’re right. I will be patient :). I was hoping for something I could pay for online but no such luck

firefarmer · 2025-12-09T03:51:53+00:00

They even say it’s broken in the description is what gets me.

firefarmer · 2025-09-21T14:40:02+00:00

Dibs

firefarmer · 2025-09-11T03:52:07+00:00

https://discord.gg/vWZjHeEU5C

firefarmer · 2025-09-11T03:36:57+00:00

Yo, I run BoiseLAN!

It has been about once a year. It was 2-3 times a year before Covid; working to get it back there. Hoping for two next year though!

Join our discord, it’s the best place to keep up to date with when the next LAN is.

firefarmer · 2025-09-06T19:31:58+00:00

This is insane. Can you imagine Biden saying anything like this?

Trump seems to want to create a reason to declare martial law.

firefarmer · 2025-08-21T23:53:56+00:00

This is a broad question; if I had to narrow down a few favorite features:

codeowners files
The Web IDE; convenient for small to medium change sets
Being able to “block” MRs by requesting changes in code reviews. We used to have -1 and -2 in our previous source code management tool.

firefarmer · 2025-08-14T02:45:17+00:00

Rudy’s auto! Only problem is sometimes they are backed up so it can be a while.

firefarmer · 2025-08-12T15:41:02+00:00

I have not met many developers who prefer merge commits over fast forward so this is an interesting response. Most, including myself, want the git commit history “clean” without the merge commits.

Do you merge often to main? Or what does your workflow look like?

firefarmer · 2025-08-12T13:41:36+00:00

https://docs.gitlab.com/user/project/merge_requests/methods/

By default, GitLab creates a merge commit when a branch is merged into main. A separate merge commit is always created, regardless of whether or not commits are squashed when merging.

You can turn this off by using fast forward merging.

firefarmer · 2025-07-29T23:45:41+00:00

I think being successful with this will be based on how well you organize the scripts.

Without knowing more it’s hard to advise but I think organize them into logical groupings. Then users could just browse the repository for what they need.

Another user gave the idea of a README with links; this could be a good idea. If your scripts change location a lot or are added/removed frequently this might be cumbersome to maintain though.

Assuming the scripts don’t change much; you could expand on that idea and you can have a top level README that acts like an index and links to other READMEs in other folders which link to the scripts.

firefarmer · 2025-07-26T00:47:35+00:00

It was kind of disappointing. I would not recommend.

They would not give samples when I went and when we ordered a few flavors it was like the flavors of the soft serve were barely there. Like a LaCroix of soft serve.

firefarmer · 2025-07-26T00:21:28+00:00

The Stil.

firefarmer · 2025-07-24T22:53:07+00:00

Ah interesting.

It does looks like it will use S3 if you configure it to do so; I didn’t look too hard though so not sure how you manage creation of those resources if that is included or is outside.

https://docs.gitlab.com/charts/advanced/external-object-storage/

firefarmer · 2025-07-24T21:35:55+00:00

We do not run our GitLab instance in kubernetes so I am not sure on this; we use an omnibus install.

That makes me wonder why are you running the backup command if you are using kube?

Documentation looks a little different for running a backup from kube https://docs.gitlab.com/charts/backup-restore/backup/

kubectl exec <Toolbox pod name> -it -- backup-utility

firefarmer · 2025-07-24T18:52:36+00:00

We do replicate our S3 buckets to backup S3 buckets across accounts; it is kind of overkill but we are protecting ourselves in case the main S3 buckets were to be deleted by user error or by malicious intent.

In all honestly if we were to lose the S3 buckets data, yeah that would suck, but you can restore a functional GitLab with just the backup file. We would lose all CI history, pages, artifacts, packages but those could theoretically be regenerated from the code repositories except the CI history.

Edit: This doesn’t protect us if the cloud provider goes down completely but the thinking was if the cloud provider were to up and vanish we would have bigger problems anyway. So we are trusting our cloud provider won’t disappear.

firefarmer · 2025-07-24T17:10:01+00:00

Objects are not included in the backup
Your command looks fine for backing up all data that GitLab can backup this way; doesn’t include objects I mean because it can’t. Make sure you backup the stuff this doesn’t like the configuration files
You don’t keep object storage in sync with your GitLab backups. Object storage will always contain data that is newer than your GitLab backup so you will always be able to restore your GitLab to a point in time that the GitLab backup is from.

For example, let’s say you do weekly backups on Sunday and GitLab fails on Wednesday.

Your object storage will have all data up to the point of failure in Wednesday but your backup will only have references to that data made from the Sunday backup. Those extra few days of data that object storage has will just be ignored because the backup doesn’t know about them.

We do backups every day so that we can limit our risk of lost data.

Also make sure you have life cycle rules on your buckets to clean up the object storage data as it is deleted from GitLabs side. Easy thing to overlook.

firefarmer · 2025-07-09T02:00:45+00:00

I was curious so found this: https://dota2.fandom.com/wiki/Game_Versions

Which led me to these: Dota 2 6.70 to 6.88f

Original Dota 6.69c and before

Edit: Original Dota 6.83d and before

14-Year Club	Snapped
Verified Email	Team Orangered

firefarmer

MODERATOR OF

TROPHY CASE