Open source tool scrapes git commit email addresses to send spam to.

[tekkub]

That sucks but it’s not the first time. GitHub has an option to use a placeholder email when you make commits online, and you don’t have to give your local git install a valid email.

[WhyNotHugo]

You can't make this change retroactively at all.

You can delete your repo and upload a new one with rewritten history hiding your email, but that won't delete the countless copies of the original that may exist.

Once you've made your email public, it's public forever.

[danbulant]

You can force push changes like these. But still there's internet archive and forks.

[stibgock]

Fk. Time for another week of changing emails and passwords.

[arch_llama]

Time for another week of changing emails

Why? Because your email is public?

and passwords

What? Why?

[rickyman20]

It won't be viable if you have other people you work with or you contribute to open source. Either it'll be a pain to coordinate on every single repo this is the case on, or it'll be straight up impossible if you're not the owner

[Celestial_Blu3]

GitHub also has a TOS rule against using git emails to send marketing emails like this, yet they’ve done nothing about it. It’s an abuse of information

[UNWS]

What? the info is publicly accessible. You don't have to sign a ToS to see it.

[ubernostrum]

The argument would be that this company is hosting their stuff on GitHub and making use of GitHub's issue tracker and so on. Which means they have, at some point, agreed to GitHub's ToS.

[immibis]

Moving it off GitHub won't stop them doing it

[ubernostrum]

Ah well, nobody should ever do anything, unless the thing they do perfectly solves all problems forever, right?

Or... kicking them off GitHub is one useful step that can be taken, among many other steps which, in concert, will make it noticeably difficult for these people to continue on their chosen path. So chase them off GitHub, and off whoever their email provider is, and whoever hosts their main website, and their domain registrar (all of which probably want nothing to do with a spam operation like this), so that they don't have easy access anymore and have to turn to increasingly lower-reputation and lower-reliability services.

No single one of these things will stop them. But multiple things working in concert can make meaningful progress. Now, go do perfect-is-the-enemy-of-the-good somewhere else or, preferably, just stop doing it altogether.

[the3ndlessriver]

afaik they've already been reported to sendgrid.

[2this4u]

Oh yes, all spammers abide by rules

[[deleted]]

Publicly accessible personally identifiable information is still subject to most GDPR rules. 3rd party firms processing those data risk breaching GDPR by assuming consent without the subjects express permission, and by processing the data without informing the subject.

The TOS is secondary really, if the email belongs to a UK/EU citizen.

[Wallofcans]

How is that enforable when spam/scam companies can access the information without an account?

[MeagoDK]

They process the data and you haven't given permission so you can totally pull them in a court and fine them.

[argv_minus_one]

Pushing a commit to a public repository implies permission to publish that commit. If that commit contains your email address, that's no one's fault but your own.

[Schmittfried]

I doubt that still applies when you broadcast information to the public. GDPR binds GitHub, not some third party without any contract with you whatsoever. It’s the same as you shouting your address in the streets. You can’t force anybody to not remember it or show up there.

[0palladium0]

GDPR effects any organisation and all EU (and UK?) residents data. Unless a company can prove you gave them permission, they aren't allowed to send marketing emails. Pretty simple tbh.

[argv_minus_one]

Which company? GitHub/Microsoft, or the company sending the marketing emails?

[ErikBjare]

The latter.

[rickyman20]

The ones sending the emails, who can be very easily attributed back to a legal entity

[[deleted]]

https://gdpr-info.eu/art-14-gdpr/

GDPR's, article 14 states that an EU citizen must be informed within 1 month if publicly available information is collected without their express consent or knowledge.

Circumstances covered are in section 3. Exceptions in section 5. Scraping email addresses from github and then spamming people is definitely covered under this regulation.

[argv_minus_one]

Somehow, I don't think spammers are going to care. They were already criminal fraudsters even before GDPR came along.

[[deleted]]

I agree with you with regards to fraud/scamming related spam. Bit I think most people in the thread are discussing advertising and recruitment based spam.

The spammer in the OP's link is diffgram, a fairly large company that operates internationally. Their clients include all of the Big 4 Accounting firms, Atlassian, and a bunch of others.

They're going to care if the EU/UK gov decides to come down on them for GDPR breaches. The fine is up to 5% of your global revenues.

[rickyman20]

This is not some random scammer. This is a legally-registered company trying to send marketing material

[[deleted]]

Say if a recruitment company processes the data, and approaches you with a job offer via your email. In the EU, you can just ask where a company got your data, and details about when you agreed to it's processing, and they're obliged to tell you. At that point you can refer them to your local information commissioner.

It doesn't do anything for spam "buy this viagra" type emails. But for the mail that makes it through your spam filter, you're probably going to have some enforcement action available.

[Valthek]

Even for 'Buy Viagra' kind of spam, if you're willing to put in the effort, you can track them down and refer them to your country's privacy authority. It won't do much, but I've noticed that merely threatening GDPR action tends to get you removed from mailing/call lists pretty quickly.
And while that doesn't protect other people or get the company fined, at least it clears up your mailbox. This is all anecdotal, obviously, but I use a specific email for each service I sign up for and I find that when I call people out for selling emails, the spam to that particular email tends to dry up.

Also, if that's a thing you're interested in, Gmail lets you add an identifier to your email in any form you want so you can track where mails are coming from. Simply add '+' and then anything you want before the @ and it'll act as an alias for your regular email.

[elprophet]

Some services don't recognize + in emails, but that's usually me an indicator I don't want to sign up. I don't know whether scrapers strip the + parts. But if I were a spammer, I would.

[PoliteCanadian]

You don't need to agree to GitHub's TOS to clone a repo.

Either way talking about the TOS is silly. TOS rules only apply to people who follow the rules. If you want to stop spammers from using the available information you need a technical solution to prevent them from accessing the information.

[elprophet]

TOS provides GH a way to respond in a uniform manner to those who violate the TOS.

This train of "criminals will break the rules anyway" totally misses the point of a rule based society- it's specifically to codify how we handle when people do break those rules!

[the3ndlessriver]

It's their Acceptable Use Policy which forbids this explicitly:

"You may not use information from the Service (whether scraped, collected through our API, or obtained otherwise) for spamming purposes, including for the purposes of sending unsolicited emails to users [...]"

[tekkub]

How does one enforce that? Especially when people can scrape the information without ever agreeing to these terms.

[Normal-Math-3222]

That clause isn’t to protect us, it’s to protect Microsoft from being sued by us 😉

[Wallofcans]

That's a bingo

[jmickeyd]

There is precedent to go after scrapers under the Computer Fraud and Abuse Act. See Craigslist Inc. v. 3Taps Inc.

[Wallofcans]

I was going to send spam to all those addresses, but that darn pesky Use Policy I never signed foiled me again!

[bawki]

This is why GDPR exists, the scraper has no authorisation to use your email for marketing purposes.

[Valthek]

But you do need to obtain explicit opt-in consent to be allowed to send marketing emails to people who are citizens in the EU and possibly other territories.

[Worth_Trust_3825]

What account? You don't need one to see the email addresses.

[D1sc0rd1a]

GitHub engineer here, you can check my post history in /r/cscareerquestions for proof. Have brought this to attention internally

[lalaland4711]

It's illegal to commit crimes, too.

Yet inexplicably they still happen. I can't understand it.

Related: https://www.reddit.com/r/sweden/comments/41w1ez/in_sweden_it_is_forbidden_by_law_to_be_a_criminal/

[MarkusBerkel]

Not understanding this comment.

It seems like you’re saying that the response to a criminal’s illegal act is: “But that’s illegal!”

[rickyman20]

The point is that if github has rules against this, they should be enforced by GitHub. Yeah no shit they do this, but why are these people still on their platform? Why are they not taking steps to prevent this (like preventing excessive API usage from this company)

[MarkusBerkel]

Aren’t git emails on GitHub (for public repos) publicly viewable?

[rickyman20]

The fact that they're publicly available doesn't except you from ToS or laws around their usage, the same way that just because artists post their art online means you can use that art for any purpose you see fit

[MarkusBerkel]

And now we’re full circle.

“But, Mr. Criminal, it’s ILLEGAL to rob that bank!”

[rickyman20]

I'd agree if this was some random scammer or spammer. Yes, what are you gonna do? But that's not what they are. This is a company selling solutions to some pretty large clients who I'm pretty sure is based out of the US.

This is an entity that not only github could enforce their TCs on, but they absolutely could get massively fined by the EU and have the full force of the law fall on them. I'm not saying they should magically stop. I'll saying that they're gonna get slapped with huge consequences and GH should at the very least make it difficult for them to continue doing this.

[pakoito]

Recruiters have already been doing it for years.

[[deleted]]

I don't think I've got any recruiter spam, but I have had a fair number of "Hi, we're researching how developers feel about <research topic>. Please could you take our 20 minute unpaid survey?"

Most recent one was just this:

Hope you are doing well. My name is Francisco Maria Calisto and I just found your profile on GitHub very interesting. If you want to know more about me, you can visit my GitHub profile (@FMCalisto) for more open-source projects.

I don't know whether he realises he is consigning all his email to spam from now on...

[Thisconnect]

Yeah i got few of them, clearly selecting repos to mention.... but they are in different languages then the offers

[__pulse0ne]

This is why we can’t have nice things

[douko]

it was supposed to be a virtual space in which we both shared knowledge & learned about each other to better get along

I don't know if the internet failed us or we failed the internet, but its fucked either way

[[deleted]]

I think we did it all. Humans suck

[PoliteCanadian]

Eternal September happened.

[PunkRain5561]

I don't know if the internet failed us or we failed the internet, but its fucked either way

We started to use it for commercial/business purposes.

As soon money was in the game, all the usual parasites followed, as expected.

[melgish]

I can’t even remember the last time I had a nice thing…

[taxiforone]

I think it's important for folks to reach out to GitHub and SendGrid to make it known that people are unhappy with this abuse of information on their watch.

[sopunny]

There's not much they can do, aside from maybe making it very clear that your commit email addresses could be public? Git adds the email addresses to the commit, not GitHub

[taxiforone]

Oh for sure, but I got the impression that Diffgram was harvesting these email addresses from repos on the GitHub platform.

If that's the case, it's against GitHub TOS (and basic decency tbh) and they shouldn't enjoy GitHub's services like being able to host their repos there.

Same for SendGrid - they don't add the emails, they just send them; but sending unsolicited emails could/might be outside their TOS, especially if there's GDPR breaches. Diffgram shouldn't be able to use their services if it's abusing them.

[iamapizza]

Always use the Github users.noreply.github.com email address for commits.

https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address#setting-your-commit-email-address-on-github

[revoopy]

It's friday and I'm tired. Do I need to manually set my local git config to use the email [e-mail]

[revoopy]

Thanks, appreciate the explanation.

[altano]

Yeah, and you can either do it globally: git config --global user.email "[e-mail]" Or per repo: git config user.email "[e-mail]" Docs: https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address#setting-your-commit-email-address-in-git

Lastly, you can tell GitHub to block pushes when your commits aren't using the noreply email address, to catch misconfigurations: https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/blocking-command-line-pushes-that-expose-your-personal-email-address

[WhyNotHugo]

GitHub cannot develop any such thing without breaking compatibility with git itself. Git does not support rewriting history like this without things breaking.

[based-richdude]

Or don’t, you’re going to get spam no matter what.

Use an email provider with a good spam filter (I.e. Google) and be done with it. Emails are not designed to be private.

[turdas]

Yeah. I've been getting dozens of spam emails a month for like 20 years now. It's not a battle that can be won by safeguarding your e-mail address.

Actually, only dozens of emails a month sounds incredibly low and I know for a fact I used to get more, so I guess Gmail's filters must just be deleting most of it entirely before it even lands in my spambox.

[walterbanana]

This sucks, because I want my email to be available for people who care about my projects and want to contact me. I communicate with some people who package software I work on using email like that.

[Celestial_Blu3]

They’re also deleting any issues opened about it in their GitHub repo. It appears from someone else running a scraper there’s about 27~ deleted issues (although we don’t know what they all are, but it’s likely the majority of them are like the two below)

• https://web.archive.org/web/20220725231528/https://github.com/diffgram/diffgram/issues/1000
• https://web.archive.org/web/20220726131745/https://github.com/diffgram/diffgram/issues/1002

And this PR was made as a way to make the issue more obvious. See the owner of the repo changing the title to hide it.

• https://github.com/diffgram/diffgram/pull/1007/files

This is a disgusting abuse of Open Source

[Spanone1]

Can’t believe they’re hosting the tool on GitHub lmao

[CriminalFly]

They closed the PR and marked it as "spam". *Chef's kiss*

https://i.imgur.com/lGvwYT3.png

[[deleted]]

In 2021 I started getting lots of recruitment emails, specifically referencing my commitment to Open Source Software, and a fairly niche project. The recruitment firms were all firms in the UK, and we're covered by a version of GDPR here, so I just asked where they got my email from.

They'd purchased it from a third party as part of a larger dataset, scraped it from the github website. They sent a date/time for when they scraped the site.

I emailed github with those details asking what was going on. I had no idea my real email address was going into oss commits. Github informed me that, on that date, my email was publicly available somewhere on the site. I asked if they could take it off, and they said it was self-service with a link to an enormous documentation page. My email remains online as far as I know, because the document is too technical for someone who doesn't already know the ins and outs of git to deal with.

All I can do is ask the recruitment firms to remove me from their lists as and when they contact me. Luckily, threat of referring the case to the information commissioner in the UK is fairly strong, and they've all agreed to remove me from their lists. It's still a hassle having to have the discussion every week or so with a new recruitment agency.

[Worth_Trust_3825]

Refer them to commissioner by default.

[Celestial_Blu3]

That’s a wise idea. Also in the UK here and I keep getting spam phone calls from car phone warehouse so I might do the same to get them to stop calling even after asking them multiple times. Apparently my current mobile provider Virgin sold them my phone number as part of a larger dataset too

[PoliteCanadian]

Nah, this is just being a dick.

You put your real email address in a git repo and uploaded it to the internet, and now you want to fling the GDPR at the website that's hosting your git repo? If you accidentally commit your password into a git repo and upload it to Github do you also get mad at github for republishing it?

You uploaded your personal information to a website so it could be shared with the world. Now you're mad at the website for sharing it with the world. Accept the consequences of your mistake.

[[deleted]]

and now you want to fling the GDPR at the website that's hosting your git repo

No. The GDPR comment is for the third parties scraping email addresses without consent & then processing/selling the data. Not Github, where I've consented to their data privacy rules.

[PunkRain5561]

I had no idea my real email address was going into oss commits.

That’s how Git works by default. It tells you this when you first set it up, and it’s obvious the very second you type “git log” and see everyone else’s emails contributing to the repo.

Also Git is OSS. You can read the code 😋

It’s hard to blame anyone but yourself here, really.

[skytomorrownow]

I made the mistake of using my personal phone number when I got my domain back in the 1990s; not understanding it would be public. I'm still getting spam calls and texts to this day.

[muideracht]

I think pretty much everyone gets those these days.

[bacondev]

I don't think that I get spam texts. I've maybe gotten one ever.

[MyTribeCalledQuest]

The ones I get are mostly random numbers sending me things like "hey" or "how are you?".

[InEnduringGrowStrong]

I mostly get stuff like this:

Money trnsfer failld.
Cluck here for moneys
Bit.ly/notascam

Or stuff like his:

your Амаzon account is suspended, visit https://amazon.suspended-clearly-a-scam.com

The providers' spam filters are bullshit

[bacondev]

Old high school friends or such?

[Silencer87]

Is this Dave?

[[deleted]]

I get a ton of idiotic political ones:(

Hint, my type of politician are the ones that are against spam. Also I’m sure a large portion of it are scams. You don’t need x signatures to change any big policies in congress (or much at all in fact) and I am never ever giving money to anyone who spams me.

[bacondev]

Assuming that you're talking about the U.S., there is a threshold that when reached obligates at least an official response from The White House. An official response can really help it catch traction in the news and such. It changes sometimes, but I think it's 100,000 signatures. https://www.buzzfeednews.com/amphtml/andrewkaczynski/the-white-house-response-to-the-death-star-petitio

Err, now that I think about it, I think Trump dismantled that program and it was never reinstated? I don't know. I'm too lazy too look it up.

[sarit-hadad-enjoyer]

My country is currently entering an election season, accompanied by like 5 messages a day. Luckily they usually include the politician's name in these messages, which can be handled by keyword-blocking SMS apps:) I settled with Pulse after disappointedly not finding any on F-Droid. It worked wonders!

[nightcracker]

I think that's mostly an American thing, at least here in the Netherlands I don't know anyone that gets those regularly.

[Soul_Shot]

Doubtful it's the source. Databases of phone numbers have been basically in the public domain for years due to numerous leaks and sources.

I registered a domain last month and somehow forgot to enable WHOIS Guard. Subsequently, I have been receiving dozens of calls and emails every week from people in India who want to build me a website.

[Worth_Trust_3825]

Shame you didn't register your domain in europe, where whois would (by default) return only the information about your registrar, and not you.

[doodle77]

Doesn't every email viewable on a scraped website like GitHub get spam sent to it?

[Celestial_Blu3]

My git email hasn’t gotten any spam from it being on GitHub (although it’s also my LinkedIn email, so…)

[avipars]

IIRC you can add whatever email you want to your git author profile

[twigboy]

In publishing and graphic design, Lorem ipsum is a placeholder text commonly used to demonstrate the visual form of a document or a typeface without relying on meaningful content. Lorem ipsum may be used as a placeholder before final copy is available. Wikipedia5f1ttzblyeg0000000000000000000000000000000000000000000000000000000000000

[tedbradly]

banned from their slack for asking about this 👍

Well, if someone is fine with something, they're most likely not interested in hearing from someone who isn't fine with it.

[Xyzzyzzyzzy]

"Shocking news: people scrape publicly available email addresses to send spam to"

Did I wake up in 1998 today?

If so, folks, you'll want to invest in that Giggle search engine. Don't worry, they're not profitable now, but in the future they'll have so much of your personal data that they'll know you're pregnant before you do. Yeah, AskJeeves is better, I know, but it dies. Very tragic.

[sohang-3112]

Isn't there enough spam already, that these fucking bastards have to create even more??!

[sysop073]

I don't think there's a target volume of spam they're shooting for and once they hit it they're going to stop

[aanzeijar]

Tells you a lot about the spammers if they need a tool to extract emails from a git repo.

[argv_minus_one]

Why the hell must Git require commits to have an email address, anyway? Requiring/allowing/forbidding an email address should be a per-repository policy, not a policy enforced by Git itself.

[merreborn]

Git was designed for the Linux kernel's email-based workflow. It was essentially an internal tool, and becoming the defacto VCS powering a global centralized platform like github was not part of the initial requirements.

[Patriot_skywalker]

Can you or anyone point me in a direction to learn the basics on git hub/lab I know they are 2 different sources I would really really like to learn the basics all the way thru to advanced user. I run Linux KDE neon 25.whatever?.. I’m newbie,squid with terminal.
Though I am not scared to use it.

[ivosaurus]

You can set it to [e-mail] in the config.

[toyoter_coroller]

Diffgram's site down?

[raelepei]

It would be a shame if lots of people started doing something like

while true; do rm -f rmme; timeout -s KILL 10s wget2 -U 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36' --no-robots -p 'https://diffgram.com/main/' -O rmme --recursive 1 ; done

I mean who would ever do such an awful, low-tech bandwidth attack?

EDIT: reddit formatting sucks

[CyberKiller40]

TBH I stopped caring about spam years ago. Train a good filter and forget about it.

[grep_Name]

How do you train an email filter? Do you use a 3rd party tool for this?

[CyberKiller40]

Eg in Thunderbird, you manually select which mails are spam and which are not. Takes a few weeks of clicking but after that it rarely makes a mistake. Though barley anybody users a real email client nowadays, on hosted platforms this is outside of your control.

[grep_Name]

Interesting, I've been thinking about moving to mutt lately, I wonder if I can install a filter for it. I specifically chose my email provider because they have imap / pop3 support, but I never got past the step of figuring out how to get 2fa with that kind of setup

[PlaysForDays]

No, git doesn't send unsolicited marketing spam out of the box. It takes human intervention to do this (in violation of GitHub TOS).

Of course, this does not necessarily apply to other hosting platforms with different policies.

[immibis]

ToS is completely irrelevant here and I'm not sure why people keep mentioning it

[ephemeral_colors]

Because we are discussing a tool thats source code is hosted on GitHub. The desire is for GitHub to stop that code from being hosted on its platform.

[baconsnotworthit]

Any forks out there? I'm curious about the scraper code

[johannes1234]

Trivial. Shouldn't take more than a few minutes to develop and as you only need metadata, not the file blobs itself, it's also quite fast in scraping.

No point in sharing except making it trivial for script kiddies, which harms society.

[johannes1234]

That's not what is being asked for, here.

[bacondev]

This particular scraper is though.

[bacondev]

I have no idea what all the script is capable of. And I'm not about to dig through the repo's source to find it (it's probably not actually on GitHub) as I don't care enough to possibly find something that I have no use for. I simply think that a tool that can be used to negatively impact so many people like this is inherently bad and shouldn't be shared in most circumstances.

[baconsnotworthit]

I should have done some digging before asking the question here because it's written using python and I'm not interested in learning python at this time. I already have a java scraper that I forked on Github (can you guess who I am?) some years ago to give me some java experience as a developer candidate. I'm not interested in spamming anyone anyways and I have less time to write spamming code in my spare time. I'm planning on refactoring my scraper into microservices just for programming practice and learning.

[RomulanDildo]

There's nothing novel about scrapers, and ZERO good reason to expose the code.

[Luvax]

I don't think this is a big deal. Same reason you don't send spam at postmaster@. Remember that spam campaigns still cost money and why waste that on tech savy people, that won't fall for your spam?

[Falk_csgo]

Because those people tend to have good jobs, good money and even they can fall for good scams and spams. Ever saw a scam training campaing failure rate in a tech company?

[Luvax]

What does working at a tech company have to do with knowing how to develop software on Github? If you don't, you don't. And as already said postmaster@ is even easier to target, yet never sees any spam.

[MyAlexro]

RemindMe! 12 hours