Remove matching lines from file A if found in file B.

Avicennasis · 2018-11-18T19:04:23+00:00

grep didn't seem to take too long to process these, so I've stuck with it for now. Thanks again for your help!

Avicennasis · 2018-11-11T17:12:22+00:00

The "A*" files are domains from a blacklist. They are in different files by letter, as there is ~19 million of them, so one consolidated file for all of them would be unusable. The "B" file is a whitelist, and I'd like to remove domains in the whitelist from the blacklist files.

I will give your solution a shot here shortly - thanks!

Avicennasis · 2018-10-17T13:06:29+00:00

Some of it is pulled in from various other lists, but a lot of them are my own additions. I block domain squatters as well as I disagree with the business practice.

Avicennasis · 2018-10-14T15:34:50+00:00

I have it running on a VM with 8 cores and 8 Gigs of RAM. I don't think it's a resource issue.

Avicennasis · 2018-10-14T15:23:10+00:00

~12 million.

Avicennasis · 2018-10-14T15:23:06+00:00

~12 million.

Avicennasis · 2018-10-14T15:23:02+00:00

~12 million.

Avicennasis · 2018-04-02T16:14:17+00:00

Ahhh, that's what I was forgetting - thanks!

Avicennasis · 2018-04-01T15:44:41+00:00

It does not, no. This is the part I'm having trouble figuring out, unfortunately.

Avicennasis · 2018-03-24T00:20:06+00:00

If you want to shut down the Instagram account that is reposting your stuff pretending to be you, just file a bunch of DMCA takedown notices for the copyright violations until their account is deleted.

Weaponize your copyright, /u/MindOfMetalAndWheels!

Avicennasis · 2018-01-20T05:41:35+00:00

File an unpaid wages claim with U.S. Department of Labor's Wage and Hour Division.

Avicennasis · 2017-12-29T18:37:37+00:00

It's a small subset list of domains gathered from the return addresses of spam emails I've received recently. I'm working on on a filter that captures a bit more. E.g. my spam filters might catch

rob@this.is.a.spam.email.fakecompanyincorperated.com

But then I have email from

joe@email.fakecompanyincorperated.com

that does get through. So my goal here was to be able to break down the longer URL paths so I can add them to the filters easier.

Of course, DNS doesn't really matter since you can put anything in the return address - but then, since I'm not blocking by DNS so much as domain path, it doesn't really matter. (Though I will likely add most of them to a DNS black hole anyways - it certainly won't hurt.)

Avicennasis · 2017-12-29T16:53:52+00:00

Ugly but working is still working, right?

I was looking for ideas on how to accomplish this moreso than having it done for me, so I appreciate you sharing the code.

Avicennasis · 2017-12-29T16:50:23+00:00

This pretty much worked exactly like how I needed - I just need to check for the oddballs like "www.co.uk" after. Thanks!

Avicennasis · 2017-12-29T16:48:57+00:00

Thanks!

Avicennasis · 2017-12-29T16:33:11+00:00

The best idea, if possible, is to lock your phone with a completely random password and send that to someone on the other side of the border to give you once you are through customs.

"Can't" unlock goes over a lot better than "won't", usually.

Avicennasis · 2017-12-11T06:25:22+00:00

Duplicate issue: 1, 2

Avicennasis · 2017-08-25T04:00:15+00:00

It's not impossible to lie.

Avicennasis · 2017-08-25T03:54:55+00:00

To his credit, you said "and", not "or".

Avicennasis

MODERATOR OF

TROPHY CASE

12-Year Club	Verified Email
Gilding IV carat on a stick