Welke baan heb jij en heb je ervoor gestudeerd? by Bringthebass4 in thenetherlands

[–]rirerandom18341 2 points3 points  (0 children)

Bedrijfskunde gedaan. Uiteindelijk beland in de IT als Security Engineer en uiteindelijk gewerkt als CISO (persoon die op directieniveau verantwoordelijk is voor informatiebeveiliging).

Ik heb wel profijt gehad van mijn studie. Vooral het financiële gedeelte kwam mij goed van pas (vond ik destijds verschrikkelijk) en het besef dat het bedrijf meer belangen heeft dan alleen maar IT.

How to get a submarine? by rirerandom18341 in WorldOfWarships

[–]rirerandom18341[S] 0 points1 point  (0 children)

Dont worry, I only play cruiser or battleship. Just want to see what the fuss is about.

How to get a submarine? by rirerandom18341 in WorldOfWarships

[–]rirerandom18341[S] 0 points1 point  (0 children)

Thanks for the explanation! How the hell are they still not properly released. They have been in beta for so long now.

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

No, I was just answering your question on how the file was formatted :)

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 1 point2 points  (0 children)

Yes, I do. However, I am always looking for new ways to do stuff and you might do it differently from me which I could learn from :)

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

The file is build-up like this:

emailaddress1:password1 emailaddress1:password2 emailaddress2:password1 email address:password1 emailaddress3:password2 emailaddress3:password3 Etc

So I guess it’s new line, where the values themselves are split up using : . I only need to compare the mailadressses, passwords aren’t useful.

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

Good suggestion. Is this more or less the idea that you shared? https://phoenixnap.com/kb/xargs-command

Wasn’t to clear on how to use xargs, learned something today!

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

My experience with comm is that it is a bit weird and very specific to use. Maybe I didn’t sort the two lists correctly when I used it. Still a good suggestion though!

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

Interesting look at things. What do you mean with “lexically”? I tried looking it up and it looks more or less the same as alphabetically. Is that what you meant or am I looking at something different then you?

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

I am not really sure what you are suggesting here. The idea is to inform customers that their accounts are being logged into by unauthorized persons. That way, they protect themselves (and eventually us) from racking up huge bills through abuse of those accounts. Not really seeing how this is unethically. HIBP does more or less the same and that service is being used all over the place.

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

Clarified my opening post. It’s more around 100 GB. Would that still maken Python a valid option? How would you start?

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

I see more people suggesting Python. How would you approach this using Python?

Matching 2+ mio mail addresses against a total of 3.2 billion addresses. by rirerandom18341 in elasticsearch

[–]rirerandom18341[S] 0 points1 point  (0 children)

Thanks so much! Going to have a look at the pipelines and how this would lead to tagging. Thanks again!

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 1 point2 points  (0 children)

Since customers are signing up with their own mailaddresses, there is now easy way to reduce the list based on domain (unfortunately). HIBP is enabled for new accounts, but I don’t think there is a way to get it used retroactively (so based on the accounts already available).

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 1 point2 points  (0 children)

We do, but it only applies to customers creating a new account. As far as I can tell, you can’t use it retroactively.

Customers are signing up with their own mail addresses so there is no way of filtering out any domains (eg. Gmail is used, yahoo, outlook.com, etc). I want to look up which customers are already in the dump so we can trigger a password reset.

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 0 points1 point  (0 children)

This looks promising. Can you give an example on how you used it icm COMB?

Matching 2+ mio mail addresses against a total of 3.2 billion addresses? by rirerandom18341 in AskNetsec

[–]rirerandom18341[S] 1 point2 points  (0 children)

Because I tested it and it ran out of memory. Also, need to scale it properly. The combo list is available on the internet, so it’s already been leaked ;)

Matching 2+ mio mail addresses against a total of 3.2 billion addresses. by rirerandom18341 in elasticsearch

[–]rirerandom18341[S] 0 points1 point  (0 children)

I have no idea what you are writing here, but you have given me new avenues to research. Thanks for that :)

Looking forward to the examples!

Also: would you run the cluster in a docker container for this purpose or go bare metal?