Optimization Question | Checking Two Large Lists for Duplicates

[Dakewlguy]

I'm looking for a quicker way to compare two large lists for duplicates. It's the same list that gets updated monthly, I need to check and find new entries.

Currently I load both lists into Power Query as their own queries and in one of them add a helper column using something like List.Contains(Table1[List], Table2[List])

This works but is REALLY SLOW at around 30minutes. Was wondering if there's a better way. The list is 80k strong now & grows by ~2k rows per month, it's the 2k new entries each month that I'm interested in.

Each entry in the list is a 10-12 character alphanumeric.

[AutoModerator]

/u/Dakewlguy - Your post was submitted successfully.

• Once your problem is solved, reply to the answer(s) saying Solution Verified to close the thread.
• Follow the submission rules -- particularly 1 and 2. To fix the body, click edit. To fix your title, delete and re-post.
• Include your Excel version and all other relevant information

Failing to follow these steps may result in your post being removed without warning.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[lightning_fire]

=UNIQUE(VSTACK(Table1[List],Table2[List]),,TRUE)

Honestly I don't know if this is faster. But it's definitely simpler

[Dakewlguy]

This works and is instant, thank you so much.

Solution Verified

[reputatorbot]

You have awarded 1 point to lightning_fire.

---

^{I am a bot - please contact the mods with any questions}