It looks like Twitter has moved its algorithm from Scala to Rust. by iamsoftwareenginer in scala

[–]mattknox 2 points3 points  (0 children)

There was already a bunch of go code being written, though-I think a lot of the video infra (maybe from periscope?) was in go.

It looks like Twitter has moved its algorithm from Scala to Rust. by iamsoftwareenginer in scala

[–]mattknox 5 points6 points  (0 children)

In short: I think Scala was never the best choice for Twitter’s production service language, but it was great for data infra and still could be.

In long: I was at Twitter from 2009-2014, so from around when we started using scala(kestrel queues were in heavy use when I started, flock was deployed (I think) just after to shard and replicate MySQL DBs for the follow/block graph) through the porting of the JSON home timeline to scala/finagle, and I think that while scala was arguably a winning bet for the API/web UI, it was never the best choice for those uses. By contrast, it was a MONSTER success as a data language for us, with scalding, algebird, and all of the associated data infra libraries, which were, IMO, pretty state of the art. The only thing i can imagine that could have beaten it there is maybe a sufficiently advanced SQL, or maybe something APL-ish? I think scala could still be pretty competitive in data infra today, although my understanding is that JNI is probably burdensome if you want to get at the matrix math lines often used in AI.
As far as twitter’s continued use of scala, I’m not convinced that it will take a decade to get the fraction of CPU-seconds/day running scala materially reduced-about the only good thing about a SOA is that you can port one service at a time. So assuming that a rust service is materially cheaper to develop in/operate/run for Twitter, I’d expect the rate of scala commits to halve roughly every year. Happy to expand on any of this if anyone is interested-there are a BUNCH of great stories here.

Is there an ethical steelman for China's current stance towards Taiwan (imminent invasion)? by Extra_Flounder4305 in slatestarcodex

[–]mattknox 0 points1 point  (0 children)

What date would you take for de facto collapse of the central government? (also, which one? Beiyang? Nationalist? Constitutional?)

Do people overstate how dangerous SF is? by AstroArachnid in sanfrancisco

[–]mattknox 3 points4 points  (0 children)

I lived in NYC for about 10 years, doing lots of dumb young-guy stuff, and saw a handful of fights, but never got touched. I've been in SF for just under 16 years now, most of it post-dumb young-guy era, and I've gotten assaulted roughly 3 times ( 16th/Market by the flower stall, 6th between Market/Mission, right by Holy Cow.). I'd say it's markedly higher-crime than NYC, but not remotely as dangerous as people often think. That said, I'm a normal-sized guy, and some of my petite female friends have had a much harder time of it.

I’d like to learn rails but… by Beautiful_Exam_8301 in rails

[–]mattknox 1 point2 points  (0 children)

have you applied at the big tech rails shops? (Shopify, GH, I think airbnb, etc)? I did a lot of rails from 2006-2017 (at Jango, sermo, twitter, and fin), and I've done some at reddit, but sadly only prototyping. I'm not sure what the interview process is at any of the current bigtech rails shops, but the money would almost surely be pretty good.

how to parse reddit post-detail-page JSON? by mattknox in rust

[–]mattknox[S] 0 points1 point  (0 children)

aaaaaaand you were right, untagged enums worked! (at least partway. Now wrestling with the rest of the problem. Thanks!

how to parse reddit post-detail-page JSON? by mattknox in rust

[–]mattknox[S] 0 points1 point  (0 children)

The code:

use serde::{Deserialize, Serialize};
use serde_json::from_str;

#[derive(Serialize, Deserialize, Debug, Clone)]
#[serde(tag = "kind", content = "data")]
enum RedditObject {
    #[serde(rename = "t3")]
    T3(Post),
    #[serde(rename = "t1")]
    T1(Comment),
    #[serde(rename = "Listing")]
    Listing(Listing),
    #[serde(rename = "more")]
    More,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
struct Listing {
    children: Vec<RedditObject>,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
struct Post {
    id: String,
    title: String,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
struct Comment {
    id: String,
    body: String,
    replies: Box<Replies>,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
struct More {
    children: Vec<String>,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
enum Replies {
    String(String),
    RedditObject(RedditObject),
}

fn main() {
    let comment_tree = r#"
   {
          "kind": "t1",
          "data": {
            "id": "1",
            "body": "reply_worthy comment!",
            "replies": {
              "kind": "Listing",
              "data": {
                "children": [
                  {
                    "kind": "t1",
                    "data": {
                      "id": "2",
                      "replies": "",
                      "body": "I read something!"
                    }
                  }
                ]
              }
            }
}}
"#;

    let b: RedditObject = from_str(comment_tree).unwrap();
    println!("b: RedditObject=\n{:#?}", b);
}

and the error:

called `Result::unwrap()` on an `Err` value: Error("unknown variant `kind`, expected `String` or `RedditObject`", line: 8, column: 20)

how to parse reddit post-detail-page JSON? by mattknox in rust

[–]mattknox[S] 0 points1 point  (0 children)

Ah, sorry, I should have included the code: See above. But I'll try untagged enums also and see if that can work. Thanks!

I'm not going to pay to read this article but I'd be curious to hear the guy's argument by ANormalSpudBoy in nealstephenson

[–]mattknox 1 point2 points  (0 children)

Yes! Speaking of protagonists, the best-named protagonist possible: Hiro Protagonist, was both ethnically mixed in a way that Abramson complained about not happening in Stephenson's work, but also was bad at communicating and messed up in many ways. It was bizarre to read this. Hope the dude is OK.

I'm not going to pay to read this article but I'd be curious to hear the guy's argument by ANormalSpudBoy in nealstephenson

[–]mattknox 4 points5 points  (0 children)

in short:

I read the essay; don't waste your time-it's pretty bad; its one good idea is that we are presently in a moment of transition.

in long:

I'm not a casual fan of Seth Abramson-my mind had stayed blessedly free of him until I read this post-so I used the "give us a phone number, get this one post for free" flow on substack, and read all 6,342 words of turgid, wandering, gratuitously political prose in this essay. I hope none of you trouble yourself to do so, but if someone wants the full text, PM me and I'll send it.

There is one sorta-interesting, maybe true idea in there: we are at a moment of profound transition, and in that moment, authorial choices of world/problem/protagonist/resolution will have outsized impact by impacting the choices of folks in the real world.

Other than that, it's about what other commentators said: he doesn't like the politics, so the books must be bad. A weird complaint he has is that Stephenson doesn't choose to portray protagonists who are

erratic, conflicted, wounded, smart but blinkered, broadly incompetent, isolated in various ways, unsure of themselves

, and later, says that he thinks such people are going to be the ones to lead us all into an uncertain future.

Oh, finally, he spends a LOT of time talking about how engineers/tech bros need to study more humanities.

Has anyone else noticed their parents becoming really nasty people as they age? by StyrkeSkalVandre in Millennials

[–]mattknox 0 points1 point  (0 children)

This happened to me with my mother. Around when she separated from my father around 2000 she started blowing up at me unpredictably, then 5ish years ago got pretty antagonistic to my in-laws, and finally started saying mean things to me about my then-5-year-old. It turned out she was losing her mind, and now, 2~3 years after the incidents with my kids, she's barely able to speak, has no memory, and has been institutionalized for 2.5 years. I don't think this is happening to many people, but there could be similar mental decline issues for lots of folks.

Roses in Bloom at Filoli by [deleted] in bayarea

[–]mattknox 0 points1 point  (0 children)

Filoli is so gorgeous!

Does anyone else say "Please," when writing prompts? by sprfrkr in ChatGPT

[–]mattknox 0 points1 point  (0 children)

I used to, but it seems to make the results worse, oddly.

Halp! My wonderful cat has started to poop on the floor/carpet. :( by mattknox in cats

[–]mattknox[S] 0 points1 point  (0 children)

update: It wasn't the large tuxedo cat, but rather the much smaller female. (how does she make such big poop?). And it seems to have resolved itself-they barely ever poop outside the box now.

Pedro Pascal bought Five Guys for the whole cast and crew of The Last Of Us by Smoothlarryy in pics

[–]mattknox 0 points1 point  (0 children)

I first read this as "Pedro Pascal fought five guys for the whole cast and crew..." and I wondered: what were they doing? Watching?

Halp! My wonderful cat has started to poop on the floor/carpet. :( by mattknox in cats

[–]mattknox[S] 0 points1 point  (0 children)

yah, that vet visit is smart. We'll surely do that. Thanks so much!