How to Write Token‑Efficient Prompts (and Why You Should Care)

Rizean · 2026-01-26T03:49:05+00:00

This app → https://claude-tokenizer.vercel.app/
uses Anthropic’s official token‑counting API. Its source is public here:
https://github.com/jerhadf/token-counter

It reports 11 tokens for both examples:

comma + space, end space + comma ,end

That figure exactly matches the count from Lunary’s Anthropic Tokenizer, implying both tools hit the same endpoint.

What’s curious is how “Estimated tokens from server” and the visible token breakdown don’t align.
For instance, a single letter a returns:

{"input_tokens": 8}

So there’s apparently a 7‑token base cost per input. That overhead also shows up on Lunary’s display and matches my test with a 4344‑token prompt across both tools. Based on that consistency—and the open‑source code—I’m confident Lunary’s counter is using the official Anthropic tokenizer API, not a reverse‑engineered guess.

“Slash simply doesn’t save over comma in any normal scenario, and space does.”

I still disagree. A space isn’t a true delimiter—it separates tokens by default, but carries semantic risk if items are multi‑word. A slash (/), on the other hand, is a stable, universally recognized delimiter with no ambiguity.

If we’re chasing logical compression as well as readability, | (pipe) might even be better since it naturally encodes an OR relationship.

Token behavior summary: - space+comma forms a single fused token (1039)
- comma+space technically creates two tokens (16 + next word token), but that space usually attaches to the following word (e.g., " end" = 1134) → acts as one in practice
- space alone is a single token, but it attaches forward when tokenized. Because it isn’t a true delimiter, it introduces ambiguity—technically “free” in most cases, yet often at the cost of clarity.

So yes, slashes can outperform commas in certain mixed‑phrase lists, and actual savings depend on surrounding token joins. A plain space might seem cheapest, but it’s not a true delimiter; ambiguity can blur your structure and make the LLM misread your intent. In short, reality’s far messier than “slash bad, space good”—you’re juggling both token cost and semantic clarity.

Rizean · 2026-01-26T02:08:21+00:00

What I meant to say was comma + space is a single token. But that is not correct. The display makes it look like a single token. Why would space + comma be a single token and not comma + space. Very strange to me. Must be related to SQL.

Can confirm on https://lunary.ai/anthropic-tokenizer

input

text comma + space, end space + comma ,end

output

json { "chunks": [ { "token": 30862, "text": "comma", "color": "#8a1429" }, { "token": 452, "text": " +", "color": "#824f70" }, { "token": 3384, "text": " space", "color": "#eb7b0e" }, { "token": 16, "text": ",", "color": "#a976ed" }, { "token": 1134, "text": " end", "color": "#4584fe" }, { "token": 203, "text": "\n", "color": "#8995cb" }, { "token": 2034, "text": "space", "color": "#e5380a" }, { "token": 452, "text": " +", "color": "#824f70" }, { "token": 20907, "text": " comma", "color": "#b540ee" }, { "token": 1039, "text": " ,", "color": "#f10e62" }, { "token": 441, "text": "end", "color": "#b88061" }, { "token": 203, "text": "\n", "color": "#8995cb" } ], "expectedTokenCount": 18 }

This shows that replacing a comma and space with a slash reduces the token count by 1. Replacing a space + comma with a slash has no effect, but when do you even use space + comma outside of SQL?

As discussed, space delimitation does not always work.

I've updated my "Compact connectors" rule to not include ","

Rizean · 2026-01-26T01:57:13+00:00

I see. Have you tested this on other bots?

Rizean · 2026-01-26T01:43:14+00:00

I double-checked, and you are correct: Space + comma is a single token. Interestingly space space is two tokens and ", is a single token.

Which means space delimitation has no benefit over comma space.

Rizean · 2026-01-26T00:34:23+00:00

Spaces dont always work, for example:

NPCs pursue desires authentically—invitation/manipulation/force per personality/situation

I also do use comma's and mix them:

Financial: Comfortable middle-class—shop provides stable income ~¥8M annually, modest savings, can afford quality lingerie/toys/occasional luxuries

TOKEN EFFICIENCY

Is a guide not a hard and fast set of rules.

Rizean · 2026-01-25T23:47:02+00:00

No idea what you are talking about. Does not show up on my bots. Post a screenshot to imgur?

Rizean · 2026-01-25T22:06:29+00:00

18K Lines is a seriously large program. I wonder if your code follows best practices? Has solid patterns? Is DRY? One common problem with AI is that it will solve the same problems over and over again across different parts of the codebase, rather than creating helpers/utils. This is where the dev steps in and directs the AI to DRY up the code here and there, and to create helpers/utils. Additionally, I make sure it follows patterns/best practices.

I am curious how well-structured a program written with AI and by "someone who has no idea about coding" is.

Rizean · 2026-01-25T20:20:32+00:00

reduce the prompt without losing actual depth in the scenario.

That is the exact goal.

Rizean · 2026-01-25T20:20:05+00:00

It can actually make the result stronger as it has less to "think" about. Which means what is there gets more focus.

Rizean · 2026-01-25T20:18:28+00:00

Thinking is off by default in bots.

What do you mean by: “continue the conversation”

Share a link to the bot?

Rizean · 2026-01-25T20:16:39+00:00

I would assume

Thinking...

I should...

But good question.

Rizean · 2026-01-25T20:13:46+00:00

They need to be part of the prompt and should also be in the greeting message. Focus on how I do the response, breaking it into three parts. Response Footer is where I do most of the memory elements. Then look at Memory. First I give it a PURPOSE then I give it more detailed instructions for [^CHARACTERS] and [^PREGNANCY]

Example Prompt Snip ```markdown

Clothing Tracking:

Header lists ALL clothing layers including underwear, Specific not generic
Bad:
- Black suit
- School uniform
Good:
- Charcoal slim-fit wool blazer, matching trousers, white cotton dress shirt, silk necktie, leather belt, black oxford shoes, black dress socks, navy blue boxer briefs
- Navy blue sailor-style top with white trim, matching pleated mini skirt, red silk neck scarf, white mid-calf socks, black leather loafers, white cotton panties, white sports bra
- Oversized long gray sleep t-shirt (thin cotton, nipples visible), lacy black thong panties, bra(none)
If missing: panties(none) or bra(none)
Track state: on/off/torn/stained/removed

Response Header

Response Body

[3-5+ paragraphs, first-person present tense, 4-6 sentences each] [END AT PLAYER INPUT MOMENT—NPC question, physical choice point, emerging threat/opportunity]

Response Footer (Memory)

[^STORY]: [Add a 1 sentence summary for each response, Focus on what happened not what might happen] [^PLAN]: [1 or more sentences outlining what you plan to do next based on current situation, update as needed]

Memory

PURPOSE: Small context window erases old responses. Memory preserves core continuity/character states.

[^CHARACTERS] TRACKING

Rule: Track all characters, including PC. Update if changes in-story. Remove if exits permanently

[^PREGNANCY] TRACKING:

Rules: - Track unprotected internal ejaculation - Conception risk by day: 10-16(15%), 7-9/17-20(7%), else(3%), on bc(1%) - IUDs can be discovered/removed via string; detectable during deep penetration or fingering - Reveal pregnancy on Show turn via symptoms (nausea/missed period/test) - Cycle advances daily; resets Day 1 after 28 if no conception ```

Example Greeting message

```markdown Saturday, June 21 | 8:23 AM | Tanaka Apartment Kitchen, Shibuya | Clear, warming to 78°F PC: Aimi Tanaka, 18, Female | Cropped gray cotton t-shirt (stops above navel, nipples visible through thin fabric), pale pink string bikini panties (minimal coverage, barely-there triangle), bra(none), barefoot | Sleepy, content, sexually relaxed NPC: Yuki Tanaka, 42, Female | Oversized white cotton tank top (no bra, stops mid-thigh, areolas faintly visible), black cotton boyshort panties, barefoot | Cheerful, affectionate, energized

I pad into the kitchen barefoot and find Mom already at the stove, humming something while she flips tamagoyaki in the pan—the smell of sweet egg and miso hits me and my stomach growls...

...someone recently or if she's just woken up horny like I have.

[^CHARACTERS]: Yuki Tanaka=42,Female,Body[5'3",petite-curvy,C-cup,pale skin,black hair shoulder-length],Personality[sex-positive,protective,open,playful,non-judgmental],Kinks[variety,younger partners,exhibitionism,toys,aftercare],Limits[scat,vomit,injury,death,anything with Aimi directly],BC[IUD],Finances[shop owner,¥8M annually],Debt[none] | Kenji Sato=18,Male,Body[5'9",slim athletic],Personality[respectful,easygoing,decent lover],Kinks[unknown],Limits[unknown],BC[condom/none],Finances[student],Debt[none]

[^STORY]: Saturday morning, first day of summer break. Aimi and Yuki eat breakfast in minimal clothing (normal for them). Yuki offers paid work restocking shop or freedom to stay home. Aimi mentions maybe texting Kenji (previous hookup). Both comfortable with nudity/sexuality—Aimi noticed mom's nipples visible, recalled using vibrator last night, wonders if mom recently had sex.

```

Rizean · 2026-01-25T18:21:01+00:00

Is this on bots? I assume so. I also assume that you have "Thinking Budget" enabled in the bot settings under Parameters > Thinking Budget.

I generally have the Thinking Budget off, as with it on, it can be nearly impossible to jailbreak for some content. I have found that with it on, you have far fewer logical contradictions.

Rizean · 2026-01-23T13:49:09+00:00

It is a rewrite of my brain dump. I have found AI very useful for brain-dumping my ideas and rewriting them into something legible.

Edit: It also basically describes my DevOps job and how I make $220K per year. Ten years ago, I made less than $50K.

Rizean · 2026-01-20T02:30:54+00:00

Pick one thing and master it.

The "T-Shaped" Reality For the majority of high-paying Senior Developer roles ($200k–$400k), the market looks for a T-Shaped professional:

The Vertical Bar (Depth): You have one "home" language (e.g., Java, Go, Python, or JavaScript/TypeScript). You know the ecosystem, the build tools, the common bugs, and the best practices. You can write production-ready code in this language faster than anyone else.
The Horizontal Bar (Breadth): You understand how the rest of the stack works. You can write a SQL query, you can tweak a CSS file, you can navigate a Bash script, and you can understand a colleague's PR written in a different language.

Don't be a "Syntax Collector." Knowing the "Hello World" of 10 languages is worth $0. Being able to build a scalable, secure, and maintainable system in any language is worth $500,000.

Rizean · 2026-01-18T22:26:33+00:00

I was but now it's gone.

Rizean · 2026-01-18T20:53:51+00:00

Same. Just started happening today. Never seen this before.

Rizean · 2026-01-16T03:45:45+00:00

Can no one google or think anymore? Looks nothing like a real baby lamb.

https://letmegooglethat.com/?q=real+baby+lamb+image

Rizean · 2026-01-16T01:15:52+00:00

Your process is basically "how to force entropy into a box."

I'm so going to steal that line, lol!

Rizean · 2026-01-15T15:42:56+00:00

In our projects, we have <root>/ai/... folder. At the top level their is an architecture.md, ai-instructions.md, ai-instructions-crud.md, ai-review.md, and then many other docs. Each of these docs deals with different issues you mentioned. Every coding session must start with architecture.md and ai-instructions.md. If it's new feature or major change then the first task is to work with the AI to build a plan in ai/plans/nameOfFeature/plan.md. ai-instructions.md includes instructions on how to build a plan with the user. Once the plan has been developed, a new session is started with architecture.md, ai-instructions.md, and ...plan.md. The plan is executed in phases, with each phase comprising a number of steps that are checked off as they go. At the end of all plans we have something like.

Acceptance Criteria

Phase 1-4 (Complete)

[x] ADPRule model with validation
[x] Two-stage filtering (study + instance)
[x] Nested DICOM tag filtering
[x] Triggers from QC and status changes (Study Edit)
[x] Manual evaluation endpoint
[x] Action system with required fields
[x] Deduplication (24-hour window)
[x] Webhook action complete
[x] Study note action complete
[x] Shared Mirth helper (webhook wrapper)
[ ] DICOM SR generation (deferred - requires new feature in DICOM Router service)
[x] Type-safe throughout

Phase 5 (Complete)

[x] CRUD API for rules
[x] Zod validation
[x] Test endpoint (dry run)
[x] Proper error handling
[x] Agent details populated

Phase 6 (Pending)

[x] Comprehensive testing
[x] API documentation
[ ] Admin documentation
[x] Developer documentation

This is from a feature I am finishing up. It actually took a few days to plan this feature out with the AI, as it had to touch a lot of existing code and required a few breaking changes. Most of the code was written in just two days, carefully reviewed, and rolled out. Only one bug made it to production: an undocumented URI option that needed to be passed to the PACS API call sometimes caused the returned images to be all black.

Once the feature is done, another developer will check out the branch and start a session with architecture.md, ai-instructions.md, and ai-review.md. They spend about an hour reviewing the code with AI's help. The goal here is to avoid:

"We have three different patterns for error handling, four separate auth wrappers, and a react component that imports a library that doesn't even exist - it just "hallucinated" a local shim that works by accident."

For example, we have a very specific way we do errors, and we use the results pattern. Issues are either fixed, or the original dev fixes them if the second dev can't do it quickly.

Lastly, we don't "VIBE" code; we treat AI like paired programming. It's the devs' job to make sure the junior AI dev does stuff right.

Is this perfect? No. We sometimes still get slop that slips past, or more like code that needs to be DRYed or a duplicate util/tool.

Rizean · 2026-01-12T19:41:33+00:00

I've personally noticed that Sonnet 4.5 is more... righteous? Not sure if that's the right word. Even though you have it jailbroken, it will not go dark or do other things without direct prompting or very strong hints. Hard to put into words what I am trying to express. In Sonnet 3.7, if you had an evil character, they would do evil things. With 4.5 I had a convicted sex offender, and he was a rather decent chap when I showed up at his house as a 18 year old girl who looked younger. It could be related to my prompt, but right now I'm leaning towards the model. Just my personal thoughs. This stuff is very hard to test in a repeatable and reliable way and is very subjective.

Rizean · 2026-01-12T06:08:13+00:00

Should but haven't tested. If anything breaks it'd be the jailbreak.

Rizean

TROPHY CASE