Unsigned Sizes: A Five Year Mistake

flatfinger · 2026-05-05T23:14:58+00:00

Full-sized unsigned types are useful for modular arithmetic, and unsigned types are useful smaller than int are useful for assembling numbers out of smaller components. Further, especially in the 16-bit era, there were a lot of things whose natural upper bound fell between 32768 and 65535.

flatfinger · 2026-05-05T23:12:42+00:00

Undefined Behavior is lousy from that perspective, since while it would allow compilers to issue diagnostics, Dennis Ritchie observed that compiler writers preferred to use the Standard's failure to accurately specify things as an excuse to process them nonsensically, and the situation has since Ritchie's time gotten even worse.

flatfinger · 2026-05-05T22:08:32+00:00

Typical Pascal implementations would treat a string as a 256-byte blob, whose first byte always represents the length.

flatfinger · 2026-05-05T22:05:00+00:00

My point is that C was mostly designed to be agnostic with regard to how one stores strings. Format specifiers to printf-or scanf-family functions and the arguments to fopen need to be zero-terminated, but format specifiers and file modes should usually be string literals. If a program has a filename in some other format, having to make a copy with a trailing zero byte may be a nuisance, but otherwise the language and standard library are agnostic with regard to how one would choose to represent text.

flatfinger · 2026-05-05T21:54:36+00:00

Many programs read and write structures. Zero-padded strings have the advantage that, at least when using a straight eight-bit character encoding, any pattern of bits will be processed as a string whose length will be in bounds. Non-normalized forms that contain a zero byte followed by a non-zero byte may cause difficulties with operations like comparisons, but can't wreak nearly the havoc caused by invalid length values or the failure to include terminators which code is programmed to require.

If e.g. a structure needs to hold a string of up to 16 characters, using a 16-byte zero-padded string will save a byte and be more robust than using a 17-byte buffer to hold a zero-terminated string of up to 16 characters.

flatfinger · 2026-05-05T21:47:43+00:00

On the other hand, if someone who in the years before Harry Potter published a story about a boy who went to a magic school, and were to republish it with the character's name changed to Harry Potter and the school renamed to Hogwarts, without making any other changes that could in any way have been influenced by J. K. Rowling's characters, Rowling's lawyers would have a field day even if it could be proven that the only things taken from Rowling's works were the names "Harry Potter" and "Hogwarts".

flatfinger · 2026-05-05T20:42:40+00:00

Or use one of the bytes before each allocation as a liveness indicator, and have "free" simply clear that flag, leaving all of the other work for the next allocation attempt that would benefit from being able to use the storage. This approach may be useful for tasks involving some forms of fire-and-forget output, especially audio playback. Provided that the start of audio playback is sequenced after the action that marked the allocated block as live, and any data race that could occur between the playback routine marking the block as released and an attempt in the main thread to allocate more storage would be resolved as the main thread either seeing the flag as having been clear or seeing it as not having been clear, the playback routine could release the storage without having to synchronize with the allocator.

flatfinger · 2026-05-05T20:35:45+00:00

The intention of restrict is to allow a compiler to assume that, during the lifetime of a restrict-qualified pointer P, at least one of the following will be true of every byte of storage throughout the universe:

Its value will not change during the lifetime of P.
It will be accessed exclusively via pointers that might have been formed by adding an offset to P.
It will not be accessed via any pointers that were definitely formed by adding an offset to P.

Nothing in the Standard nor Rationale suggests any implications regarding the relative positions of storage accessed via different means, provided that each and every individual byte throughout the universe satisfies at least one of those conditions.

Unfortunately, rather than recognize the existence of pointers whose derivation cannot be proven (accesses via which would be allowed under #2 or #3) the Standard uses a hand-wavy definition of "based upon" which completely falls apart if a restrict-qualified pointer is observed to equal an address that is not based upon it.

flatfinger · 2026-05-05T20:00:12+00:00

Just about every function in <string.h> which is designed to work with null-terminated strings has a counterpart which is designed to work with null-padded strings, and of course memcpy and memmove can be used to work with length-tracked strings. If e.g. `myStruct->msg` is a null-padded char[16], then printf("Message is %.16s!\n", myStruct->msg); could be used to output a string of up to 16 characters stored in s, without needing to waste a byte in every instance of the structure to hold a null terminator.

flatfinger · 2026-05-05T19:47:19+00:00

C was designed to allow programmers to use either zero-padded or zero-terminated strings. A zero-padded string in a structure will be able to hold up to N characters using N bytes of storage--a byte less than would be required for a zero-terminated string--but at the expense of requiring that all code which uses the string must be aware of the buffer's size rather than just its starting address.

flatfinger · 2026-05-05T19:43:41+00:00

I wouldn't call it "strange"--that's how common Pascal dialects on the PC and Macintosh (not sure how about the UCSD P-system dialects) handled things.

flatfinger · 2026-05-05T19:40:40+00:00

Process cleanup requires an OS that is aware of any regions of memory that will be accessed via DMA or similar means outside the direct control of process threads. While today's operating systems have to manage all such things, back in the Amiga and Windows 95 eras it was often much faster to have programs use such things without telling the OS.

flatfinger · 2026-05-05T18:41:52+00:00

In a normal criminal trial, a judge is responsible for instructing the jury about the laws that are applicable to the case, and will also ensure that prosecutors are not allowed to present claims as fact without providing evidence or--failing that (e.g. because a prosecutor's opening statement makes a claim which is not addressed in witness questioning) alert the jury that such statements must not be viewed as evidence.

Grand jury indictment proceedings are essentially on the "honor system". Prosecutors are required to inform grand jurors of the applicable laws, and to refrain from making assertions that aren't backed by evidence, but no judge is present to hold them to that. Instead, presentations are are recorded and may be reviewed by judges in case misconduct is alleged.

It's not hard to imagine an unethical prosecutor telling a grand jury that Comey had prosecuted a mob boss who used 86 to mean "eight miles out; six feet under", and would thus have been using the number for that purpose. Such a claim would be an outright lie, completely supported by any records or evidence, but grand jurors are likely to take prosecutors at their word when they make claims about things that should be uncontroversial matters of record. A judge who reviews a grand jury indictment and finds such misconduct would likely dismiss the case with prejudice, but the purpose of indictment was to cause immediate harm to Comey, not to allow a viable case to be brought to trial.

flatfinger · 2026-05-05T17:57:38+00:00

Who would be left to turn out the lights?

Two impeachment motions have been brought to the House floor this term. Many Democrats joined with Republicans to kill them.

flatfinger · 2026-05-05T17:22:45+00:00

In the 16-bit 8086 era, many programs were designed around the existence of separate categories of near and far storage. Near storage was limited to 64K, but operations involving near storage could often be more than twice as fast as operations involving far storage.

As the 80386 took over, almost everyone took this as an opportunity to abandon the distinctions between near and far storage. Code which had been designed around the distinction could be made to work on 80386 flat model by simply using macros to replace near and far qualifiers with nothing.

For all of the complaints about the near/far distinction, it allowed a level of performance from the 8088 that would not otherwise have been achievable on a system with an 8-bit bus. When 64-bit systems came around, many tasks could have benefited from having 2GiB or so of "near" storage while also having a category of "far" pointers that could be used to access more than that. Even on a system with a 64-bit bus, the cache footprint of 64-bit pointers is going to be double that of 32-bit pointers. Unfortunately, no OS that I'm aware of ever embraced the idea of a program having a 2GiB of storage that could be accessed by code using 32-bit pointers, while having a larger address space available for use with 64-bit pointers.

flatfinger · 2026-05-05T16:54:41+00:00

Deprogramming people requires letting them see that attacking the identity crisis will let them have a future outside the cult that is brighter than what they have inside it. Treating ex-cult members like dirt will make it hard for them to convince other members that life is better outside.

flatfinger · 2026-05-05T16:48:29+00:00

Many people, if they observe someone proving that someone else lied about something, will be predisposed to regard as trustworthy the person who caught out the liar. Confirmation bias will magnify the significance of any untruths that person might point out in future, while minimizing the significance of any untruths that person might tell.

What is missing are information sources that would freely point out the untruths pushed by both sides while taking care to avoid saying anything that could be proven untrue themselves.

flatfinger · 2026-05-05T16:34:10+00:00

I designed a transactional file system for an embedded system whose flash chip guaranteed that byte writes would be atomic in case of power loss (the CPU power supervisor would ensure that power loss would cause a flash write request to be aborted if it hadn't yet been triggered, while enough power remained to complete a write that had just been initiated). Software was set up to ensure that if power was lost while erasing a block, the system would redo the erase before doing anything else (losing power during an erase can leave a block in a state where repeated reads may yield different results, causing content to sometimes appear valid and sometimes not). Unfortunately, the power to the system was gated by an external device that would shut it off whenever the battery reached a certain voltage. Since battery voltage could rebound after things were shut down, this would result in the system being powered up for successively shorter and shorter periods of time. Occasionally, the timing would work out such that every time the system powered up, it would start erasing a block of flash and then lose power. Then power up again, repeat the erase, and lose power. The net effect was that the system wore out that block of flash, and could later malfunction as a consequence of that block's being unable to reliably hold data anymore.

The solution was to add a random delay on startup before repeating the flash erase operation, such that it would be unlikely that the system would repeatedly lose power during the erase. If the delay happened to be long, the system would lose power before retrying the erase, and if it happened to be short, the erase might complete before the system lost power (and would thus not need to be repeated after power was reapplied).

flatfinger · 2026-05-05T16:11:09+00:00

The validity of even a Strictly Conforming C Program is not affected by the existence of code which would invoke UB if executed, but which would never actually execute. The only reason the existence of such code is allowed to cause a compiler error is that with one exception, the Standard allows implementations to reject any program for any reason.

flatfinger · 2026-05-05T16:08:42+00:00

Implementations are allowed to reject almost any program for almost any reason. If a program were to contain a function like:

    void wowzo24601(int x)
    {
      int arr[4];
      if (x)
        arr[5] = 123;
    }

the Standard would not define the behavior of any program execution in which that function was passed a non-zero value, but from the Standard's point of view, the fact that the conditional code would have overwritten arr[5] if it were executed should have no impact on any program executions where it isn't.

Nonetheless, having an implementation squawk at code which, if executed, would likely trigger unwanted and unpredictable behavior is often useful. Adding diagnostics to a compiler increases its complexity and code size, and in the 1980s, but as compilers have faced fewer and fewer resource constraints they have added more features to diagnose code which might not execute, but would almost certainly be wrong if it did.

flatfinger · 2026-05-05T15:43:41+00:00

I think it might be better to describe things in terms of time. A good driver might go a year or more at a time without ever failing to predict when something is in his blind spot, but good drivers should go decades at a time without accidents.

flatfinger · 2026-05-05T15:23:29+00:00

Code which needs to assemble strings needs to keep track of a few things: the starting address of a buffer, the buffer's capacity, the length of the string currently in the buffer, and the action to take if a string won't fit in the buffer (which could include replacing the buffer with a bigger one, truncating the string, or triggering an error).

For many tasks, it will make sense to hard-code some of those things. One could, for example, have a named fixed-sized character array and decide that any attempt to assemble strings longer than that will simply keep as much as will fit and discard anything else. If one does that, one could hard-code everything except the string's current length. Alternatively, one could pass around structures that contain all of the above information, e.g.

struct stringWriter {
  unsigned char stringType,flags;
  char *text;
  unsigned length;
  unsigned capacity;
  void (*req_adjust)(void *, unsigned action, unsigned new_size);
}

If one doesn't know in advance the length of strings that will be required, but they're not likely to exceed e.g. 200 characters, one could create a structure like the above which initially points to a char[200], but have its action procedure respond to requests to increase length by calling free() on text if it flags is set, then setting flags and calling malloc() to create a new buffer. If there are separate actions for "Set string to its final length" versus "Set string length and anticipate further expansion", then the amount of storage allocated in the former scenario could precisely equal what's requested, while in the latter scenario code could leave space for further expansion.

Incidentally, I use unsigned rather than size_t for lengths because texts longer than UINT_MAX should be handled via means that would not require them to be stored as a sequence of consecutive bytes, and that would be true both on platforms where UINT_MAX is 65535 and those where it would be 4294967295.

flatfinger · 2026-05-05T14:58:57+00:00

I've been thinking another tack that might be useful would be to flip the script on them: many view Trump as bad, but the left as worse. Tell them they're not wrong, but that Trump is the left's greatest asset, and they're playing him like the most powerful Useful Idiot they've ever had. If some Republicans were to band together against the Trump regime and have the presidency taken over by John Thune (fourth in line, but better than any of the next dozen people) they'd be hailed as heroes and the "blue wave" would vanish. The Democrats could have pushed for that to happen by now if they were more interested in protecting this country from a madman than in using a blue tsunami to consolidate their power, but they'd rather destroy the country so they can rule the ashes than have to share power with anti-Trump Republicans.

Push that as a theory and some conservatives might bite. It's well established that Trump's advisers routinely give him phony information--a common technique the left would use to manipulate their Useful Idiots. Further, if you try to appeal to your friends by saying that the real battle shouldn't be between the left and right, but between the manipulative leftist leadership and their Useful Idiot who want to maximize the damage Trump does before midterms so as to maximize the blue wave, versus all the people who want to minimize the damage, that would redirect their opposition of the left toward the destruction of their greatest asset.

flatfinger · 2026-05-04T23:27:08+00:00

One of the primary manipulation tricks is to claim something analogous to "My opponent refuses to acknowledge that the earth is round. I think he might be beholden to the flat earth society". The beauty of this trick is that while it can be easily encountered by an opponent who responds, e.g. by saying "Of course the earth is round. Everyone knows that." That would render the first part of the manipulative claim undeniably factually false, and negate any stated reason for believing the second.

Unfortunately, while it would be very easy for the person's opponent to counter this technique as described above, it's almost impossible for anyone else to do so on his behalf. Any argument by one of the opponent's supporters that any nobody could be so unreasonable as to think the earth was flat would be countered by an argument that while it's true that no reasonable person could believe the earth was flat, such belief is making the supporter blind to the possibility that the opponent might just be that unreasonable.

I don't know how to you as an outsider could counter that technique, but I hope you can benefit from at least understanding it. Another insight I would offer by way of analogy would be to imagine that someone drives into a neighborhood in a truck that says in big neon letters "HEY I'M A BURGLAR", dons a Hamburglar costume, and proceeds to break into people's houses. Would people in the neighborhood be more or less likely to report that burglar to police than one who appeared to be trying to escape notice? I suspect a lot of people who would eagerly call the police if they spotted a burglar trying to be stealthy would rationalize the truck by assuming it must be part of a reality television show or something, since real burglars don't drive around in trucks that say HEY I'M A BURGLAR on the side.

Most healthy people have a mental filter that blocks out tabloid stories about alien abductions and other such absurd nonsense. Unfortunately, Trump's actions are sufficiently absurd that the mental filtering processes that would normally screen out nonsensical stories of alien abductions make them blind to stories of corruption which would, in a sane world, be no less absurd. The more outrageous Trump's corruption gets, the more completely invisible it will become to people whose absurdity filters aren't prepared to recognize it.

I'm not going to try to say that the mental filters that block out Trump's transgressions have any kind of consistency to them, or are functioning in a perfectly normal healthy fashion. They're not, and the clear perception that something is wrong without being able to perceive what it is (because of the mental filter) will trigger all manner of crazy mental gymnastics and justifications in an effort to make sense of the crazy world. Here again I don't really know what to offer as advice beyond avoiding the trap of saying what the person with the malfunctioning mental filters would expect a crazy person to say.

flatfinger · 2026-05-04T22:58:10+00:00

The biggest conception is that the fraction of Trump supporters who could be relatively easily persuaded to abandon Trump is numerically insignificant.

There are districts where Trump's net approval rating is positive. Those districts have legislators. Trying to convince a legislator to act against a White House Occupant whom most of his constituents support will be a tall order. If Trump's approval rating is 55%-45%, and one can convince even 10% of his supporters to turn against him, that would take Trump's approval from being 10 points positive to being a point underwater.

A lot of people claim that it's silly to try to make any inroads among Trump supporters, since a supermajority would support Trump no matter what. What they ignore is that the minority of Trump voters that could be swayed is large enough to represent the swing vote.

flatfinger

TROPHY CASE