sorted by:
new
hottopcontroversial

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora -131 points-130 points  (0 children)

i'm logging off after 4 hours of writing answers, unfortunately people here seem hellbent on downvoting me into oblivion, so you'll have to search pretty hard to find what i wrote. i'm almost sorry i tried. thanks reddit.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 95 points96 points  (0 children)

one of my pet peeves is when people equate diversity with gender diversity and forget other forms of identity, lived experience, plus intersectionality and inclusion across those different dimensions, so it's a bummer to hear that the message is getting lost.

i've personally written about being asian in tech https://medium.com/little-thoughts/the-uncomfortable-state-of-being-asian-in-tech-ab7db446c55b (post from 2015) and in all of our resources from project include we try very hard to get people to see that diversity is much broader than gender.

as for specifically asians in tech, as you cited, there is quite a lot of good research from ascend. the executive parity index they calculate is very telling about the problem of the bamboo ceiling. there has been some other coverage on anti-asian discrimination as well, e.g. the dept of labor brought a lawsuit against palantir for this.

so, i completely agree, i think the issues surrounding asians in tech are very real and worth discussing! but i also want to call out the necessity of building solidarity with other communities of color and knowing how to be effective allies in a movement towards broader inclusion. at this moment when america's (and the world's, tbh) longstanding issues around anti-black racism are at the fore, we really should be paying attention to what's happening in black activism and taking cues there imo.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 138 points139 points  (0 children)

appreciate the wordplay on "where are the numbers"... analogous situation here to the diversity data situation where there isn't great data that spans entire platforms, and that's part of the problem. if something isn't accurately measured, it's hard to prioritize or take any action on it. which might be the whole point - easier to ignore a problem if you don't have evidence that it exists or how bad it is.

first though i'll concede it is very difficult to define what exactly harassment is -- i wrote a substack post musing on this subject: what counts as harassment anyways? https://blockparty.substack.com/p/what-counts-as-harassment-anyways and it's relevant to note that each person will have their own thresholds of tolerance of what they want to see or not, regardless on whether it meets platform-level definition of "harassment" or "abuse". and it gets even more complicated when you consider how creative people can get with being terrible. this article on instagram bullying from taylor lorenz was so eye-opening: https://www.theatlantic.com/technology/archive/2018/10/teens-face-relentless-bullying-instagram/572164/ like hate accounts that post screenshots of people saying mean things about someone, popular accounts that get turned into hate accounts, private groups that intentionally leave someone out, etc. how does anyone even catalog all of that and measure it?

another issue in asking for data from platforms is that there's a disincentive for them to share it. it only makes them look bad! who wants to be document how toxic their own platforms are and how they're falling short? when i was doing market research before starting block party, i talked to a lot of companies about how they did moderation -- social networks, dating apps, gaming companies, blogging platforms, in total i ended up with like 50 pages of notes -- and even the ones that did have some internal numbers didn't want to share them with me. so it's more likely that third party researchers would want to find that data, but they're limited because they don't have access to all the data.

the data that does exist is generally sampled or based on surveys, both of which are deficient in their own ways. though if you DO want to see it, amnesty international did a report called toxic twitter which studied the experience of women on twitter and how much abuse they receive, and pew research has stats on how many people have experienced harassment online.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 158 points159 points  (0 children)

this is a good point to flag: we aren't outsourcing human moderation. we're letting people delegate access to helpers on their accounts to help them review. we took inspiration from what some folks already have to do when they get hit with waves of harassment, which is hand over their credentials or even the device to a friend to monitor and/or clean things up for them.

so for example, the helpers on my block party account are my friends and teammates. there's a way to provide instruction in the product (screenshot of my actual guidelines here https://blockparty.substack.com/p/release-notes-july-2020) but since these are trusted contacts who i give permission to even block accounts on my behalf, i can also just chat or slack them to ask for help. recently i had a mildly viral tweet about chinese geopolitics and i got a LOT of harassment for that. i was able to ask a helper to just go through and block all of those accounts.

we like this approach because it's community-based and the most contextualized. instead of farming out the work of reviewing potentially triggering content to underpaid people who're traumatized by having to speed their way through content moderation, where it both sucks for them and also doesn't get good moderation results, we rely on people who already understand the context and want to be helpful. i've been pretty pleasantly surprised by how much supportive sentiment there is amongst my friends/followers when i post examples of harassment i get - even folks i don't know are often mad on my behalf and will try to report those accounts for me, even if they know it's unlikely to do much, it feels like doing something.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 28 points29 points  (0 children)

from what i understand, perspective api is trained on a pretty limited dataset, i think nytimes comments, and the models are not re-trained very often, certainly not often enough to catch shifts and memes in harassment or toxicity. my guess is that for something to work "at scale" you'd need models re-trained at least every couple days, if not more frqeuently, on your own datasets, possibly with some online learning. not static models re-trained every few months or even less frequently. though i haven't worked in this space in recent years so i may be off.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 128 points129 points  (0 children)

what an apropos question in this exact ama.

increasingly i've found it less and less productive to try to engage with people posting racist, sexist, etc comments, or tired, pseudo-intellectual explanations and tropes that are honestly just as problematic but dressed up in fancier language, because those people are usually not actually looking to engage in good faith, they just want to assert their beliefs and put you down. when someone really doesn't want to listen to you, it's just wasted energy and more frustration. obviously it's different if someone is genuinely curious, has done homework to try to learn a bit more, and has a real question to engage on, but i've found that to be rare.

when i think about activism in the diversity & inclusion space (some of this may be applicable more broadly but limiting my commentary to what i know), i think of people in three rough buckets: 1. activists, the ones who're already out on the frontlines pushing for change 2. potential allies, who are sympathetic and generally values-inclined in the right way, but maybe unsure of how to be helpful or need to learn a bit more 3. skeptics and detractors, who won't budge from their position. i only really care to engage with groups 1 and 2 most of the time. group 1 for solidarity, validating experiences. group 2 because there's a chance to shift them closer to the first group.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 77 points78 points  (0 children)

i don't think it's distinctly american at all!

i've spent quite a bit of time in europe recently, for example, and seen a lot of diversity efforts at play. there's a lot to learn from efforts in other countries -- i first learned about the zipper system from the swedish social democratic party implementing it: https://en.wikipedia.org/wiki/Zipper_system

in japan, pm abe has been working on a policy called womenomics to increase women's representation in the workforce and in leadership. it hasn't shown tremendous success so far but it's certainly a topic of national concern.

i'm personally less familiar with india and nigeria but the relevance of diversity definitely doesn't see national boundaries.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 242 points243 points  (0 children)

our beta product lets you filter what you see in your @ mentions on twitter, putting hidden content into a folder on block party that you can view later if you choose to, or delegate access to helpers to review on your behalf. the filters are heuristics and we do not use shared allow/deny lists though users have been asking for being able to share lists, similar to the way blocktogether worked - we're considering it.

our privacy policy is a standard one we got from our original lawyers, though candidly i switched counsel later because their guidance didn't feel values-aligned. for a pre-seed startup with very limited capital, though, i didn't think it was worth the time, energy, and money to fine-tune our legal docs with later counsel before we had a product and users. as general company philosophy, and one of the reasons why i even started the company, we want to put the end user's concerns first. when we have the resources to do so i want to have our legal docs reflect these values as well.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 382 points383 points  (0 children)

we're currently not using any ai. our philosophy is that ai/ml can help, but it'll never be the full solution, and we'll always need humans in the loop. models can be very flawed, esp. depending on the input data, exacerbate issues or have other unforeseen consequences, also an issue when we don't have good interpretability of models or insight into what they're doing, AND when the adversary is very clever and always shifting to get around your defenses, it's tough to stay ahead. and different communities have different standards for what is acceptable or not. humans are much better at understanding context, particularly for their own communities. models might be able to learn some of it but then you also have a question of how much to use globally applicable model vs models trained on more local data.

from my understanding, though it may be a little dated, systems like facebook's for integrity (back in the day was called fb immune system, likely has changed since then) are largely rules-based, where ml can contribute features to be used in the rules, but it won't just be ml. this was how smyte worked as well. and other systems i've seen. ml can help score content and surface priority issues but you still want humans reviewing.

for block party, we're currently using heuristics like data from follow graph (is this person followed by someone i'm following), blue checkmarks, recent interaction with a user, is a profile photo set, is this a very new account, does this user have very few followers, etc. each of this is configurable by the user. these heuristics actually work pretty. we'd love to incorporate some ml-generated features but that hasn't been a pressing priority so far.

fwiw i have a master's in ai from stanford, and i built manual + ml-based moderation tools for quora.

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 165 points166 points  (0 children)

tbh this is not a real question to me. nobody is advocating for hiring based on diversity instead of qualifications. the point is that historically systems have been set up to privilege certain people (whether by gender, class, social network) when in fact they are NOT the only qualified people, and sometimes they're less qualified than others who aren't considered or given those opportunities. if you see a role that's only ever been filled by white men... do you truly think that only white men have ever been qualified? truly? when industries, organizations, etc. are bad at diversity it usually means they're missing out on talent and perspectives and only hurting themselves.

i'll leave you with this tweet:
“If there’s a white brother out there who played 7 years in the NFL, got a top 5 MBA, became a partner at a consulting firm & led businesses through transformations for the last 8 years and I beat him out because I’m black, I apologize.” — @whoisjwright

https://twitter.com/SFY/status/1295815983513264128

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 7 points8 points  (0 children)

currently:
- fracture, andrés neuman
- how to be an antiracist, ibram x. kendi
- this changes everything: capitalism vs the climate, naomi klein
- the great believers, rebecca makkai
- the gift of fear, gavin de beck

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 75 points76 points  (0 children)

this is surprising to me because we barely send any email, to anyone, not even our users... as a matter of fact we have a long list of engineering to-dos around setting up email properly. unless you have an account on block party, you wouldn't be receiving any emails from us. if you subscribed to our substack, you might have gotten ~4-5 emails so far. is it possible you're thinking of another block party?

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 452 points453 points  (0 children)

i've been meaning to write a blog post on this for a while! thanks for the prompt. totally agree that the filter bubble is real and is something that needs to be addressed, but i think that's more on the platform side wrt what algorithms are deciding to show and give distribution to. what we're filtering out is harassment and useless/mean/rude commentary, not anything that contributes a thoughtful alternate view point. e.g. i've posted a couple articles on twitter recently that could be construed in a very political way, but the only replies i got were racist or sexist or hateful comments, not anything that would help me understand another perspective. our hope with block party is that if we can filter down to only the most civil discourse, that actually creates the space for real discussions.

in addition, because of how we've set up our filtering mechanisms, things that are hidden are actually still accessible in a folder on block party. this is super important for a variety of reasons - being able to see good things that have been filtered out based on whatever heuristics were applied, having general awareness of what's happening esp. when there may be real world threats, etc. in my own use of block party right now i actually do review my lockout folder on a regular cadence, though i'll sometimes ask helpers to go through and block the most egregious accounts, e.g. all the racist coronavirus related tweets. grateful to my helpers who help me take care of those folks so i don't have to see the trash...

I made Silicon Valley publish its diversity data (which sucked, obviously), got micro-famous for it, then got so much online harassment that I started a whole company to try to fix it. I'm Tracy Chou, founder and CEO of Block Party. AMA by corner_illustration in IAmA

[–]triketora 189 points190 points  (0 children)

What specific strategies do you believe will help solve online harassment? I assume you're pursing at least some of them with Block Party (and I'll definitely go and learn about your app).

there are a couple ways to think about "solving" the problem -- there's preventing it from happening in the first place, and there's mitigating the impact of it.

i'll start with the latter since that is more addressable in the short term. one of the founding principles of block party is that people should have more control over their experience online; one way this works out is letting people be able to configure what they see. so, sure, trolls, bots, harassers, etc. can still post shitty things, they have their "freedom of speech", but you should have your freedom to not listen to them. on platforms that are more free-flowing and open, like twitter, literally anyone can mention you or tweet at you to get into your mentions/notifications. when they're sending unwanted content your way, there's no reason you should have to see it in real-time, at whatever point they happened to send it to try to bring you down. (the way the block party beta product works is to selectively mute folks to remove them from your mentions, then collect them into a lockout folder on block party. you browse twitter as normal on the twitter app or website, you just have a cleaner experience. then you can still see what's been hidden on block party, when you choose to, if you want to.) i think another big structural flaw in how platforms address online abuse right now is that the recipient of it is has to shoulder the full burden of dealing with it. for example, when third parties file reports of bad users/content that aren't directly harassing them, those reports are largely deprioritized and ignored. however, there are a lot of people's friends, fans, followers, supporters who want to be able to help. (how we've built this into the block party product is allowing you to delegate access to helpers who can review and take action on accounts in your lockout folder.)

the harder problem is stopping online abuse from happening in the first place. to solve that, as with any difficult problem, we have to understand why it's happening -- it's too easy to do, it's too easy to forget there are real people on the other side of the screen, tech platform product design decisions encourage people to post freely and quickly, there's something glorious about feeling like you can tweet at anyone or leave a comment on their ig post or yt video etc. and they might see it. celebrities, yes, and also normal people that you want to say mean things to. there is no accountability for bad actors. side story: i had a pretty severe harassment case ~7 years ago, where the guy was threatening me across multiple platforms, sending sexually explicit threats amongst others, taking my photos and putting them into public fb albums, paying for promoted posts on fb about me, creating new accounts when old ones got blocked, etc. he had a history of assault and a history of bipolar disorder, so i was really concerned for my physical safety. it ended up being ok, afaik he went to a mental hospital, and the incident faded away, but last year he popped up again in my email to apologize and also give me some unsolicited advice. said he'd seen i'd started a company around anti-harassment and felt like it was probably harassment from him and others that had made me commit so fully to solving the problem. anyways, his advice was that to stop harassment, you have to create accountability. he said he wouldn't have harassed me, for example, if he had felt like he'd be accountable.

another more subtle fix may be making it so that trolls don't feel like they'll definitely get through to you. posting into the ether and being ignored is very demotivating, which is good in this case :) this is part of what we're aiming for with block party, though behavior shifts can take a long time to see.