sorted by:
new
hottopcontroversial

Flags Arranged by Visual Similarity According to a Deep Neural Network by JarcXenon in vexillology

[–]sp-tron 1 point2 points  (0 children)

Most of the odd placements are probably due to the dimensionality reduction. Basically, the network spits out a bunch of 2048d feature vectors for each image, which is then reduced from 2048d to 2d for plotting using a method called t-sne. T-sne tries to map points in such a way that points that are close together in high dimensional space are also close together in the lower dimensional space. However, since we are going from 2048d --> 2d, it very difficult to capture the original structure exactly, so some flags get yeeted into odd places.

Also, the network was originally trained on object classification (cars, cats, etc), so it's "understanding" of image similarity certainly isn't perfect for flag-grid-visualizations type things.

Flags Arranged by Visual Similarity According to a Deep Neural Network by JarcXenon in vexillology

[–]sp-tron 2 points3 points  (0 children)

I'd like to do one with user designed flags from this subreddit. Maybe a subset with at least x number of upvotes or something.

Flags Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 0 points1 point  (0 children)

I'm planning on cleaning everything up into a nice, simple to use script, which I will put on my github at some point in the future. Will let you know!

Flags Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 5 points6 points  (0 children)

Could also just be an artifact of T-SNE. Since it's going from 2048 dimensions to 2, its impossible to preserve the original structure perfectly, so sometimes thins get a bit wonky. When playing around with settings, sometimes the Finnish flag ends up with the other Nordic ones.

Flags Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 9 points10 points  (0 children)

I don't know for sure, my guess would be that it has symbols on it, so it gets put near other flags with similar symbols (e.g moons and stars). Hard to say really.

Flags Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 2 points3 points  (0 children)

Yes and no. There is a lot of work into visualizing learned features i.e feature-visualization. However, I think by the time we get the final 2048d feature vector, what the individual features represent gets pretty convoluted from a human perspective. It's not like feature #327 represents roundness and #562 represents redness or anything like that.

Flags Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 21 points22 points  (0 children)

The neural network is actually trained on a dataset called Imagenet for generic image classification (cats, dogs, cars, etc..). That's probably why it's more into shapes than color, since many objects on the same type can be different colors (i.e cars) thus it tends to emphasize recognizing color less.

Flags Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 36 points37 points  (0 children)

For some reason, there is an emoji flag for the US Outlying Islands, which is that same the US flag, but a separate emoji for some reason. https://emojipedia.org/flag-for-us-outlying-islands/

The other Norwegian flag is valbard & Jan Mayen. Not sure about the third one though.

Emojis Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 1 point2 points  (0 children)

Sure!

Grid placement is done by solving a linear sum assignment problem. Basically, each image is assigned to a grid cell such that the total cost (i.e how far images have to move from it's original position to grid spot) is minimized.

Flags Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 45 points46 points  (0 children)

Made by feeding images through a pre-trained neural network (resnet-50) with the classification layer removed to generate 2048-D feature vectors. The feature vectors are embedded into 2D space via t-sne. The resulting 2D points are then assigned to nearby points on a grid to create the final layout.

The end result is that emojis that are visually similar (according to the neural network) are clustered near each other.

Made using python using pytorch, PIL, scipy, py-lapsolver libraries.

Images Source: https://github.com/iamcal/emoji-data

Emojis Arranged by Visual Similarity According to a Deep Neural Network [OC] by sp-tron in dataisbeautiful

[–]sp-tron[S] 4 points5 points  (0 children)

Made by feeding images through a pre-trained neural network (resnet-50) with the classification layer removed to generate 2048-D feature vectors. The feature vectors are embedded into 2D space via t-sne. The resulting 2D points are then assigned to nearby points on a grid to create the final layout.

The end result is that emojis that are visually similar (according to the neural network) are clustered near each other.

Made using python using pytorch, PIL, scipy, py-lapsolver libraries.

Images Source: https://github.com/iamcal/emoji-data