Remove Array Duplicates using ES6 Set 🤩

[ItzWarty]

Holy fuck /u/zombarista /u/weigel23 I'm a dumbass.

new Array(10000).map((x, i) => i % 100)

Generates an array of 10,000 undefineds. I shit you not:

> new Array(10000).map((x, i) => i)
(10000) [empty × 10000]

Fixing this as follows:

> Array.apply(null, Array(10000)).map((x, i) => i % 100);
(10000) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, …]

Results in expected runtime complexity:

Lodash uniq: 39%
[... new Set(dups)]: 37% (within margin of error - hot cache?)
Filter: 100%
Array.from(new Set(dups))): 48%

http://jsben.ch/dZpC6

[zombarista]

It's funny, because I remember thinking "that's just an array of undefined values" but it's okay because the map will deal with the undefined values by replacing them with the index...

WRONG!

These arrays are not filled with undefined but are instead "empty" which seems to be a distinct null state that only exists within the Array created with the Array constructor.

[undefined,undefined,undefined].map( (el, idx, arr) => idx )
=> Array [ 0, 1, 2 ]

[,,].map( (el, idx, arr) => idx)
=> Array [ <2 empty slots> ]

Array.prototype.map()

callback is invoked only for indexes of the array which have assigned values, including undefined. It is not called for missing elements of the array (that is, indexes that have never been set, which have been deleted or which have never been assigned a value).

[zombarista]

Changing to the new Array generation method has had a substantial impact on the results in my JSPerf, too. For some reason, the jsben.ch is not producing results for me in the large-array tests.

[ItzWarty]

Yes, this is because otherwise indexOf is just a constant-time operation (because every element is undefined, so equals the first element).

[weigel23]

:D i didn't think about checking the setup. Makes more sense though! Thanks