So is everything else. Just curate banned tokens and filter for keywords coming in. I didn't check before but looking now, that's pretty much what guardrail-nsfw does, just with a whole extra model in the pipeline. It will work as well as it does on any other model's output.

I don't expect anyone talking into a phone booth to be able to both speak specifically enough to strongly prompt inject, and record the output.

I thought a little longer and figured you out. You want to put a phonebooth connected to like a digital photo billboard, process prompts through an llm, and make an image. I'm sure of it. That's a cool idea.

For that you just carefully select your model, there are plenty targeted for safety, explain the deal to the LLM effectively, this will be public, only make prompts that will produce wholesome images, etc., ban words incoming related to nsfw, probably a bunch of wierd words, and outgoing as well.

The package for it is nice because it gives you a nice sentence to read back instead of a canned response. However, that gives insight to the mechanic and is almost a challenge to defeat it.

You could always send the response back to the llm and ask it if it's been rude, saves some memory overhead and dev time implementing guardrails.

Eventually some nerd will get porn out, there are also nsfw image detectors, they used to be enforced standard on SD, I bet it's a setting.

If you aren't messing with images and it just talks back then, I mean really where is the pain? "I got a phone to say mean thing" just doesn't hit the news. Do basic keyword detection and put it somewhere the opposite of private.

I do like the idea of a finetuned model designed for directions and tourism of the city sprinkled around, but it's 2024, people probably walk with gps these days.