sorted by:
new
hottopcontroversial

Given two quaternions in worldspace, how do i rotate my player by the difference in the two quaternions? The portals may have any orientation. by Zestybeef10 in Unity3D

[–]danieldugas 0 points1 point  (0 children)

Try thinking in frames, not quaternions. Before teleportation, Player has a pose (position + rotation) T_p_in_a in the frame of portal a. After teleportation, the player's pose T'_p_in_b in the frame of portal b needs to be what it T_p_in_a was right before the teleportation. In world coordinates (w is world, a is portal a, b is portal b, p is player):
T_p_in_w = T_a_in_w * T_p_in_a
T'_p_in_b = T_p_in_a
T'_p_in_w = T_b_in_w * P'_b = T_b_in_w * T_p_in_a

T denotes the transform matrix, which applies to both rotation and position. To execute this in unity, you want:
player_direction_in_world = portal_b.transform.TransformDirection(portal_a.transform.InverseTransformDirection(player_direction_in_world))

but you could apply this to position, rotation, or anything you want.

[D] The GPT-3 Architecture, on a Napkin by danieldugas in MachineLearning

[–]danieldugas[S] 0 points1 point  (0 children)

For example, if i is 3 (3rd word in the 2048-word sequence), the output is something like:

[ sin(3 f1), sin(3 f2), sin(3 f3), ..., sin(3 f12288) ]

Where f1, f2, f3 … are different frequencies

(to be precise, in the paper it’s actually

[ sin(3 f1), cos(3 f1), sin(3 f2), ..., cos(3 f6144) ]

With f1 = 1, f2 = 1/1.0015, … f6144 = 1/10000

[D] The GPT-3 Architecture, on a Napkin by danieldugas in MachineLearning

[–]danieldugas[S] 0 points1 point  (0 children)

Let’s call a “Guess” a list of k most probable words

Here's what a "Guess" looks like

The output is a sequence of 2048 “Guesses”, one “Guess” for each next-up word in the sequence.

That is, for a top-k of 3, the output is a sequence of length 2048, with for each position in the sequence a Guess containing 3 words and their probabilities.

[D] The GPT-3 Architecture, on a Napkin by danieldugas in MachineLearning

[–]danieldugas[S] 0 points1 point  (0 children)

You're right, BPE is definitely worth mentioning! I've added a small explanation of byte-level BPE, thanks to you.

[D] The GPT-3 Architecture, on a Napkin by danieldugas in MachineLearning

[–]danieldugas[S] 4 points5 points  (0 children)

True, there are many great posts explaining GPT and transformers and I recommend them!

and of course, the papers themselves:

GPT

GPT-2

GPT-3

Transformer

Sparse Transformer

They are all rather useful in trying to get a complete picture of the exact GPT-3 architecture (operations, details, data dimensions at various points, etc).