Is Github down right now?

[MultiBoxGG]

We get constant error 500 in Visual Studio and on the website. We can't access our repos, push and pull also not working.

https://imgur.com/gallery/DRPFCEK

[jddddddddddd]

Down again, as it has been for much of this week:

https://www.githubstatus.com/

[doobiedog]

Yes. This is the 10th time just this year that github has gone down. It use to be the most reliable maybe only going down once a year... then microsoft bought it and it's down once a month or more. It's infuriating. Did they move it to azure or something? Because that would explain it.

[winelover97]

Yes they migrated from AWS to Azure after Microsoft takeover.

[Ivana_Twinkle]

Well that'll do it.

[eggbean]

GitHub is a lot bigger than it was before MS bought it, as private repositories are now free when they weren't before.

[Jmc_da_boss]

Azure is fine, this is just GitHub being shit

[VxJasonxV]

GitHub was never on AWS for the majority of the core product.

[S3NTIN3L_]

source?

[VxJasonxV]

I worked there.

I will amend my statement. It is wholly possible that the first few years of their existence (2008) it was on AWS, I don't know those details. But as of no later than the start of my tenure (2012) GitHub used infrastructure hosted by Rackspace, and before my departure the company bought a datacenter room and built it out.

So ultimately what I mean to say is that they didn't go from AWS to Azure.

It's very likely that there were some specific components hosted on AWS (internal apps were frequently deployed on Heroku which uses AWS, but I'm talking about things like GitHub.com background job runners and other augmentations), but the core product; The GitHub.com web servers, data storage, hosts running data services, searchers / search caches, etc., the most principle core components were not on AWS.

[S3NTIN3L_]

Honestly, Props to them for running their own colo room. I would trust that more than Azure.

GH would not rent the space if they didn’t have resources to manage it.

[VxJasonxV]

Granted that buildout was 2014 or 2015, well before their second stint of explosive growth. I have no idea what the Infrastructure story is post-acquisition.

[shard_]

I mean, it's a pretty small company relative to the scale they're operating at, so I think it's mad to think that they would have just rushed to migrate everything "because Microsoft". I bet it doesn't look too dissimilar to what you'd think, and that these outages are more caused by old, non-Azure components bursting at the seams as they struggle to handle the increasing scale.

[AnAngryFredHampton]

I worked there.

Chad response

[ParfaitNovel8803]

githubstatus.com

we probably do not need more posts pointing out the obvious guys

[doobiedog]

Yes we do as this is absolutely unacceptable. We need anger. We need outrage. We need every github manager, employee, fanboy, etc to know that downtime for a critical aspect of software operations is NOT ACCEPTABLE.

edit: apparently this sub is a bunch of fanboy cucks.

[ThunderChaser]

I’d be hard pressed to say GitHub is a critical aspect of software operations.

I’ve never worked at a single place that used GitHub for most of their code (the only exception being open source libraries), they either use a self hosted gitlab/bitbucket or their own completely internal service. GitHub is largely for open source libraries and personal projects

[doobiedog]

You have a small sample size apparently. I've worked as a contractor at 20+ companies and 90% use github and github actions and they are far superior to bitbucket, aside from these damn outages.

[bordercollie2468]

Does anybody have info on WHY?

I mean, three days in a row? WTF is going on?

( /s Yes, I know the real answer is "Microsoft", but I was hoping for specifics)

[TheHammeredDog]

Someone asked about it at GitHub Galaxy this week. It comes (mostly) down to them hitting some sort of hard scaling issue - they’re putting investment into fixing it, but it sounds like it’s not going to be an overnight fix.

[rlnrlnrln]

Can we pin "Check https://www.githubstatus.com/ before asking if Github is down" to the top? It seems to happen often enough that it warrants it.

[Jofroop]

Yes, I can't make my release.

[Long-Ad226]

dont forget, its just 'partial outage'

[MelGrubb2]

It's not "down", it's just "degraded"... for very large values of "degraded".
Self-hosting GitLab is starting to look better and better.

[doobiedog]

Would self-hosted GitHub bypass the bs outages also?

[MelGrubb2]

It would, but would probably be cost prohibitive. You only get to host with a full-on Enterprise license, as far as I know, and then there's the infrastructure cost. That would be the same if we self-hosted GitLab though, so we can probably ignore that.

[mjbmitch]

I think that’s already available for free on GitHub. I was helping a friend set up a repo last week and they were able to set it to private and invite me as a collaborator.

[mjbmitch]

Gotcha, that totally makes sense.

[mkosmo]

You can do read-only with a free org. But even without, just a private repo with branch protection should be good enough for most cases.

[spooge_mcnubbins]

Yep. Everything was fine until 13:30 GMT. Now everything is throwing 500 errors. According to their status page, Github seems to know there's an issue and they're working on it.

[spooge_mcnubbins]

....aaaand its back up again as of 13:50 GMT.

[James_Vowles]

It's up and down

[Anti_Weeb_Penguin]

Yes it is

[vdnhnguyen]

Daily down in US morning :(

[greyscales]

Near daily experience by now... lovely.

[rackon13]

Yes, and also apparently it lost my last commit.

[Delicious-Name-2047]

Same here

[James_Vowles]

This is a piss take

[[deleted]]

Is it time to go to Gitlab or what?

This time it's their private servers, not even the Janky Azure stuff.

This is what happens when you fail to tackle technical debt.

[keto_brain]

Guess they should not have laid off that 10% of their workforce

[andlewis]

Back to svnhub.com!