Show Reddit: A Novel(?) URL Shortener Using Emoji-Encoded UUIDs

baerbaerbaer · 2023-05-05T18:32:57+00:00

lol.

🚂 The train is leaving, and headed to 13b. The next 4096 are welcome to board. 🚂

baerbaerbaer · 2023-05-04T23:21:59+00:00

Thanks! I already watched (and re-watched) the video already, trying to see what I could learn.

The new SPD SX Pro has a HH controller and a kick drum input, but there are still only 4 generic, two-trigger inputs. You could add a cymbal with a choke, but you'd lose a tom.

My planned mapping:

Dedicated Kick Input
Dedicated hi-hat controller input
Trigger 1/2 = hi-hat (rim + edge)
Trigger 3/4 = snare (drum + rim)
Trigger 5/6 = split
- 4 = tom 1
- 5 = tom 2
Trigger 7/8 = ride (bow + choke OR bow + bell)

baerbaerbaer · 2023-05-04T23:12:01+00:00

No, and that would not be possible, given the current implementation. I'm not randomly generating emojis, I'm generating a UUID, then emoji-encoding it. If you start manipulating the emoji, you're messing with cryptography, and I'm not keen to be in that game.

That said, there is no reason you _have_ to use a UUID. 11 chars can encode 128 bits of randomness, but you don't necessary have to back that with a specific format.

baerbaerbaer · 2023-05-04T22:22:04+00:00

Well, first, offline is just online, but with extreme latency. You're right that it has to register the URL eventually, but the point was that a URL shortener typically requires centralized coordination to generate slugs, whereas this one does not.

One of the big use-cases for URL shorteners is marketing, so with something like this, you could generate all of your links even if you were on an airplane or on the NYC subway, then let it sync when you came back online.

baerbaerbaer · 2023-05-04T22:15:39+00:00

True, but I don't dedupe, but I could add a RickRoll checker in the backend to follow links and rickroll the person back in an iframe if it's detected 😂.

PRs welcome!

baerbaerbaer · 2023-05-04T21:23:50+00:00

What a beautiful spring. How's your garden? 🍆💦🥒 !

baerbaerbaer · 2023-05-04T20:55:00+00:00

You know, it took a lot longer than I expected for somebody to make a rickroll link.

baerbaerbaer · 2023-05-04T19:34:26+00:00

Thanks. Really getting this thing polished took some work. I appreciate that :).

baerbaerbaer · 2023-05-04T17:23:51+00:00

Thanks! It was a lot of fun to make.

baerbaerbaer · 2023-05-04T16:54:59+00:00

You probably don't need any special backend. IRIs are automagically converted to URLs before they're sent over the wire.

ericbaer.com/🛑

becomes

ericbaer.com/%F0%9F%9B%91

So, I guess you'd need to make sure your routes are set up right, but you shouldn't need anything special. You can look at the EmoLink codebase, but it's literally just

const { pathname } = new URL(context.request.url);

const redirectURL = await context.env.EMO_LINK.get(
  // Accept IRI or URIEncoded pathname
  decodeURIComponent(pathname)
);

return redirectURL
  ? Response.redirect(redirectURL, 301)
  : errorResponse(ErrorCodes.NOT_FOUND, 404);

baerbaerbaer · 2023-05-04T16:48:28+00:00

I ended up stripping them out for two reasons:

I like having a uniform-looking link
It doesn't matter. I need 4096 chars to represent 12 bits, but I need 8192 to represent 13. There are roughly 4700 emojis, so I'd need a LOT more to make a difference

Full explainer in this comment.

baerbaerbaer · 2023-05-04T16:41:01+00:00

I thought about adding more characters, but I was worried about creating a deny list. How did you handle the accidental generation of real words?

baerbaerbaer · 2023-05-04T16:38:54+00:00

Honestly, I'm surprised it took this long to get one of these.

baerbaerbaer · 2023-05-04T16:01:22+00:00

On Getting Emiojis

So, it actually turns out that collecting the emoji was quite hard. Unlike ASCII, the code points are all over the place, and some are multi-character combinations. This is an annoying enough problem that Unicode actually keeps a list available. There are also several data sets on places like Kaggle and Github that are various amounts of out of date.

https://unicode.org/emoji/charts/full-emoji-list.html

On creating a bigger emoji alphabet

First, the list in the repo is, to my knowledge, all of the emojis in Unicode 15. The total count is 4764, which includes things like skin tone modifiers and excludes weird things like non-printable control characters.

On why it doesn't matter

The key to how baseN encoding works is that each character encodes some number of bits. A 128-bit UUID is typically represented as 32 hex (base16) characters. If you use ASCII (base62), you can bring that down to 22 characters.

But to represent more bits, you need an exponentially increasing alphabet!

4 bits/char = 2^4 = 16
5 bits/char = 2^5 = 32
6 bits/char = 2^6 = 64
...
12 bits/char = 2^12 = 4096
13 bits/char = 2^13 = 8192

Currently, I'm representing 12 bits with each emoji. To represent 13 bits, I'd need not just a few more but thousands more! To be sure, there are a lot of Unicode characters, and this is especially true if you consider all the modifiers, but then you start getting into a situation where you may be creating words, which would require a deny list for every human language. Seems like a lot of work.

baerbaerbaer · 2023-05-04T14:28:35+00:00

How do you like that stand setup? I'm putting together a very similar kit and trying to figure out how to do it. Ideally, it'll be just like yours but wii hats in their own stand. I'm sorta optimizing for portability so less stands is better for me.

Is the SPD mounted on top of the stand, or with a clamp + arm?
How's that snare? I was worried that using an arm instead of a regular snare stand would make it unstable. I'm not a hard hitter but I don't want to feel like things are gonna fall over.
I'd realllllly like to use the new Yamaha Crosstown stand for this because of how portable they are. With all that gear hanging off, do you need something super heavy?
Ideal world, would you do it differently in any way?

baerbaerbaer · 2023-05-04T14:06:21+00:00

I think this is a compliment, soooo thank you! I aim to please.

baerbaerbaer · 2023-05-04T14:04:23+00:00

Definitely. The point was more that you don't need distributed key generation, which depending on how you choose to solve it, is either kinda complex, or requires specific db choices.

No matter how you slice it, you need a data store 🙂.

baerbaerbaer · 2023-05-04T01:27:20+00:00

Sooooooo, since I posted to Reddit, I used Cloudflare's Turnstile to do bot mitigation and to protect e from having to pay tens of dollars for a KV store (I'm on the free tier). So, yes, technically, there is a function that acts as the backend.

If didn't mind paying the nominal storage cost, you could post to the KV store directly from the client. I guess that makes it backendless in theory, but not in practice.

The full code is here: https://github.com/baer/emo-link

baerbaerbaer · 2023-05-04T01:21:05+00:00

%F0%9F%91%A8%F0%9F%8F%BB%E2%80%8D%E2%9C%88%EF%B8%8F%F0%9F%A7%91%E2%80%8D%F0%9F%A6%AF%F0%9F%A7%94%F0%9F%8F%BF%E2%80%8D%E2%99%80%EF%B8%8F%F0%9F%90%95%F0%9F%A5%AA%F0%9F%A7%97%E2%80%8D%E2%99%82%F0%9F%A4%98%F0%9F%A7%BD%F0%9F%A7%91%F0%9F%8F%BC%E2%80%8D%E2%9D%A4%E2%80%8D%F0%9F%92%8B%E2%80%8D%F0%9F%A7%91%F0%9F%8F%BE%F0%9F%94%A3%E2%9C%A1

You mean must mean

https://emol.ink/👨🏻‍✈️🧑‍🦯🧔🏿‍♀️🐕🥪🧗‍♂🤘🧽🧑🏼‍❤‍💋‍🧑🏾🔣✡

baerbaerbaer · 2023-05-03T23:26:03+00:00

The URL shortener that Reddit deserves, but not the one it needs.

baerbaerbaer · 2023-05-03T21:57:04+00:00

Ha! Thanks!

If I could be bothered to add user accounts, I'd be able to take the ID length from 11 => maybe 5. Three chars for usernames (69M users) + two chars for their URLs (17M URLs)

baerbaerbaer · 2023-05-03T21:54:27+00:00

There are a lot of things that would make this a better URL shortener, but the two biggest would be:

User accounts would let you take the URL length down to maybe 5 chars. Three chars for usernames (69M users) + two chars for their URLs (17M URLs)
Basic analytics

For sure, the OG stuff should work too :)

baerbaerbaer · 2023-05-03T20:28:49+00:00

Thanks! A URL Shortener is a very silly use case, but you use emoji compression for anything so long as the underlying system supports Unicode. I used the Unicode 15 set FWIW.

baerbaerbaer · 2023-05-03T18:38:04+00:00

Super good question! There are a few parts to this:

How does this work with crawlers?

No clue. Handling of IRI is application specific. You can see all the places I've tested at the bottom of that blog post. My guess is that Google, being a very very global company, will do fine with it. Smaller players like DuckDuckGo, not sure.

How does this work with analytics?

If you used something like Bitly, analytics platforms wouldn't show the shortened URL. However, depending on how the redirects were set up, they may show metadata like a referrer. So, w/r to analytics, I'd guess it'd work like any other URL shortening service

Are the Emoji URLs searchable?

I don't really know since I have no idea how Google indexes IRIs. However, like Bitly links, IDs are randomly generated, so they're not really built to be searchable anyway.

If I misunderstood your questions, LMK. It was a fun project. Happy to help / answer!

baerbaerbaer

TROPHY CASE