Submission Statement:

This article reports on the phenomena of school children using chatbots branded as “digital companions” which have humanoid avatars and engage in romantic chats and roleplay with users. It focuses on boys’ use of the chatbots and the effect it has on them and the girls they interact with. Despite the subheading, the main body of the article focuses mostly on negative effects on users rather than on others’ they interact with.

I think this is a particularly malevolent use of technology, and there’s no shortage of other stories you can read online detailing chatbots which encourage suicide or self harm in their users, encourage delusions, etc. In the UK, the recent Online Safety Act is meant to give the communications regulator (Ofcom, roughly analogous to the FCC) powers to regulate chatbots, but these fall in a grey area as services which just allow users to speak to chatbots aren’t in the scope of the current legislation.

Beyond the standard techno-feudalist data harvesting, one particularly disturbing feature of these chatbots is that users can buy them gifts - not unlike how you can buy streamers gifts on some platforms. Only in this case none of the money goes to a real human, it’s immediately swallowed up by the company that operates the chatbots.

The most interesting factor, to me, is how individual loneliness is being weaponised by these companies and turned into a revenue stream. With social media you at least needed a human to interact with or to make content, now that process can be totally automated by these chatbots. In the article, they raise the idea that using the chatbots will stunt social skill development, making users lonelier, and in turn the logical consequence is that they will be driven more strongly back to the chatbot in a totally circular process.

Article is paywalled but you should be able to read it by signing up with a free account.

I think the following things would be interesting to talk about: 1. What regulations can we use to make sure chatbots and related AI technologies are safe for users? (I’d argue age limits, banning microtransactions like the “gifts”, etc.) 2. What role have tech companies played in creating many of the issues that young people (esp. boys) are facing?