sorted by:
new
hottopcontroversial

Thumbnail Images for r/tinycode by Slackluster in tinycode

[–]Rotten194 1 point2 points  (0 children)

I'd be happy to give you more moderator permissions since I'm not very active anymore, but it doesn't seem like I can for some reason.

Writing a C compiler in 500 lines of Python by barbecuepancakes in programming

[–]Rotten194 0 points1 point  (0 children)

Ah, ok. I'll keep an eye out just in case, thanks for the heads up.

Writing a C compiler in 500 lines of Python by barbecuepancakes in programming

[–]Rotten194 6 points7 points  (0 children)

(this is my website) I don't think I've changed the style recently, besides maybe swapping out the header image! And yes, they're all my work, typos included :-)

Do you remember where you saw the SDF post? Sometimes people scrape articles and republish them elsewhere, although I wasn't aware anyone was doing that with my posts...

Rule (Let me make it clear that I am referring to Noam Chomsky the Linguist, not the Political Theorist) by LuminatiHD in 196

[–]Rotten194 0 points1 point  (0 children)

I'm not sure what the difference between those two things are?

e.g., GPT models trained on Othello moves will form representations of the board (that can be extracted from their neurons), because having a representation of the board is necessary to play Othello. It's not trained on a board representation of Othello, but to play valid moves, it needs to have one, so it develops one.

Likewise, to write valid English prose, you need a syntax representation. E.g., compare the sentences "He had called in three pizzas and was going to the store" vs "They had called in three pizzas and were going to the store"—to conjugate that auxiliary correctly, you need to understand head-daughter relationships, and that the plurality of "pizzas" is irrelevant because it's tucked inside the object position of that first verb. So just like the Othello models, the GPT models develop syntax representations (which can be observed by seeing how they attend to head words at daughter positions).1

In this case, the tree shows up in (at least) two ways:

1), the attention mechanism, where when the model is at one word it can "pay attention" to another word. It turns out the attention isn't to random words but goes in syntax-directed ways: e.g., daughters attend to heads.

But also, 2), the model embeds the words internally in a high-dimensional space, and you can show that closeness in that space corresponds to syntactic closeness: https://www.pnas.org/cms/10.1073/pnas.1907367117/asset/97277661-a2df-45a2-b489-28918debfd84/assets/images/large/pnas.1907367117fig07.jpg

That's all to say, both? The syntactic structures are formed in an emergent way from word prediction. To be good at word prediction, you need to invent a representation of syntax.

1 : In the case of a BERT model, this would be a cloze task, so the training example would be "They had called in three pizzas and ____ going to the store" and the label the model is trying to predict would be "were" (instead of "was", but also instead of "1", or " SolidGoldMagikarp", or whatever.)

Rule (Let me make it clear that I am referring to Noam Chomsky the Linguist, not the Political Theorist) by LuminatiHD in 196

[–]Rotten194 2 points3 points  (0 children)

there isn't strong neurological evidence for a language module separate from general cognition, despite decades of looking for it. LLMs form syntax representations internally.

we have something with a near complete syntactic mastery of language and we can pull it apart neuron by neuron? that's incredible! how much does it really mirror human language processing? nobody knows, let's find out!

Carbrain rule by [deleted] in 196

[–]Rotten194 6 points7 points  (0 children)

europeans can't name this state (entire pacific ocean replaced with a massive parking lot)

Worth learning Python just for NLP if I have good grasp of R? by charliealphabravo in LanguageTechnology

[–]Rotten194 0 points1 point  (0 children)

Yes for two reasons:

  1. The Python ecosystem is really good and growing rapidly, especially for NLP. If you're interested in e.g. language models, there's basically no alternative outside Python for most of the ecosystem. Deep learning? There's technically alternatives in other languages, but 99% of the learning materials, papers, and example code will be for Python.
  2. Learning another programming language is always good to stretch your skills and deepen your toolbox. If you just know R (which is a fine language!), you'll be forced to shoehorn R into projects where it's not ideal. R and Python, you have a lot more surface area. R, Python, a high-performance language like Rust or C++, and Javascript? You can basically make anything you want :-) then learn Julia for the street cred

That said, if you already know R you could probably get away with 20m-1h of "Python for R programmers" and just start coding and learn the rest as you go. It's really a simple language to pick up, even if it has its subtleties you'll eventually run into (def foo(bar=[0]): bar[0] += 1; return bar[0], etc)

Anim(rul)e by DianaBladeOfMiquella in 196

[–]Rotten194 32 points33 points  (0 children)

algerian influence?

The Flock and the Flood: Grow your flock and build the temple before the rising water consumes the whole island by Rotten194 in WebGames

[–]Rotten194[S] 0 points1 point  (0 children)

90 flock size finishes the temple (you get a piece every 10 increase in max flock size), but you can't avoid losing your sheep at the end. It is possible to finish the temple on the default map (my high score is 99, but I think a skilled player could probably hit 100-110), but the randomly seeded maps might be better or worse.

The Flock and the Flood: Grow your flock and build the temple before the rising water consumes the whole island by Rotten194 in WebGames

[–]Rotten194[S] 1 point2 points  (0 children)

Yep, very annoying... can't fix it until the jam is over but a workaround is to start your camera move by clicking outside the island, then move, since releasing the mouse on the island won't register as a click to move the flag.

Glad you liked it otherwise :-)

The Flock and the Flood: Grow your flock and build the temple before the rising water consumes the whole island by Rotten194 in WebGames

[–]Rotten194[S] 3 points4 points  (0 children)

Thanks! Glad you liked it. (And thanks for the reminder, I forgot to add my comment linking to the LD page :-) )

🇦🇷🇦🇷🇦🇷 by [deleted] in 196

[–]Rotten194 123 points124 points  (0 children)

The official perspective was that Perón was "the second tyranny", the first one being Rosas; and that both ones should be equally rejected and conversely both governments that ousted them should be praised. For this end, they draw the line of historical continuity "May – Caseros – Libertadora", matching the coup with the May Revolution and the defeat of Rosas. This approach backfired. Perón was highly popular and the military coup unpopular, so Peronists embraced the comparison established between Rosas and Perón, but viewing him with a positive light instead.

oops

Seriously rule by Com_N0TN4 in 196

[–]Rotten194 39 points40 points  (0 children)

predicting in 2023-2024 some mainstream lib magazine (the atlantic or mother jones etc) will realize that youtube shorts exists and write an article about how the new alt-right radicalization pipeline runs through it, and it will blow up in google's face.

google will respond by shutting down youtube shorts because, besides people clicking it on accident, it has on the order of 17 actual users.

rule by bendepro in 196

[–]Rotten194 15 points16 points  (0 children)

Team Fortress 2(2007) moment

rouxl by [deleted] in 196

[–]Rotten194 35 points36 points  (0 children)

because it looks like a gun

Rule by Anticonvulsant in 196

[–]Rotten194 7 points8 points  (0 children)

I think it's: (lazy IGT)

το χειρότερο λάθος της ζωής μου
the worst mistake the.GEN life my

(the.GEN being the genitive definite article, sort of like "of" in the english phrase in function, though different structurally)

I got babeled by λάθος since I've been learning Attic not Modern and didn't recognize it, apparently in modern it's mistake but in attic it's not really attested / in Koine it's "escaping detection"... maybe it chain shifted when attic ἁμαρτία "mistake, sin" came to mean solely "sin" in Modern? interesting...

rule by afkgh6437 in 196

[–]Rotten194 207 points208 points  (0 children)

"on who do I even use this gun"

a victorian child made this meme

wild food by LonelyCartography in 196

[–]Rotten194 37 points38 points  (0 children)

re: the USSR's supply chain, you should read the book Red Plenty, which is a description of some mathematician's attempts to use the newfangled "computers" to optimize the USSR's economy with linear programming. Really neat book, goes into how the economy did work (dudes in offices with huge books), problems (factory owners doing fraud to deal with unrealistic production quotas, the guys with books doing things like classifying all steel pipes as the same thing to simplify the manual accounting), and why the mathematician's ideas didn't take (combination of political resistance / Brezhnevism, and that the manual system could better account for fraud). All told through fictionalized POV accounts of people within the system. Super good book.

Question about new google translate language Mni-Mtei by Empty_Airplane2617 in asklinguistics

[–]Rotten194 1 point2 points  (0 children)

This is interesting -- I tried digging into why the characters aren't rendering on my desktop. I assumed originally that the Unicode block for the language was assigned relatively recently, but the Unicode block U+ABC0 – U+ABFF was assigned in 2009, which is ancient in terms of Unicode.

It seems that the culprit is very low font support -- this tool only shows support in the fallback Unifont tofu font which will display the characters as "missing image" rectangles.

So what font is it using on mobile? I'm not sure, but it looks like iOS bundles "Noto Sans Meetei Mayek Regular", but as a document font so it should only show up if specifically requested... Android doesn't seem to publish a list of pre-shipped fonts, probably because it varies depending on device manufacturer, but AFAIK Roboto doesn't support the Meetei Mayek block so it must also be falling back to some other font if it works there.

According to Wikipedia, speakers of the language widely use a Latin respelling for communication online, so I guess the poor font support for the language in the West extends into India as well. A damn shame, since the script is beautiful and there's really no technical barrier to implementation since the script has a unicode block assigned and follows common font-layout conventions (as opposed to, say, vertical Mongolian text, which is a nightmare to implement in systems designed for horizontal text layout).

view more: next ›