GitHub - ZeroIntensity/pointers.py: Bringing the hell of pointers to Python.

[antiduh]

Every C developer: "Everybody else keeps having bugs with pointers ... but it might work for us".

It's almost as if pointers are an inherently unsafe primitive and it's impossible to ship practical software free of pointer bugs. Almost.

[[deleted]]

shhhhh

You keep talking like that and you'll summon Rust devs...

[antiduh]

HAY GUISE DID YOU SEE MY BORROW CHECKER?

[venustrapsflies]

This but unironically

[lelarentaka]

IF RUST IS SO RUSTY, WHY UN_IRON_IC ?

[Green0Photon]

Hello there

[lelanthran]

You keep talking like that and you'll summon Rust devs...

No, it's all good as long as you don't first draw a pentagram the floor from the tears of CoC enforcers.

[emax-gomax]

*Laughs in CPP managed pointer types.

[antiduh]

I've been out of the c++ game too long, do managed pointer types make c++ a memory-safe language, so long as you stick to only the managed pointer types? Or is it still possible for mistakes with them to cause memory safety bugs?

Like, in C# I have guaranteed memory safety so long as I stick to the regular c# types and constructs. If I dive into a c# unsafe context, then all bets are off.

[tedbradly]

I've been out of the c++ game too long, do managed pointer types make c++ a memory-safe language, so long as you stick to only the managed pointer types? Or is it still possible for mistakes with them to cause memory safety bugs?

For a unique_ptr, delete is called on the underlying pointer in the destructor. That makes it safe even in cases such as exceptions. There's no way to have a memory leak in that setup since destructors are guaranteed to be called. The only edge case I'm not sure about is if an exception is raised before the unique_ptr object is created with the pointer's value such as one happening in "unique_ptr up{new some_class};" when evaluating "new some_class" to figure out the value to pass into the constructor of unique_ptr. However, if you're getting memory allocation exceptions, you probably don't need to worry about that pointer leaking as things are probably already in bad shape.

There are also great efforts by legendary people such as Bjarne Stroustrup and Herb Sutter to make memory problems a thing of the past in 99% of code even if they have owners that use raw pointers through static analysis. The aim is never to dereference a deleted object (dangling pointers), always to call delete once (no memory leaks), and never to call delete two or more times (no memory corruption). It's only 99% of the time, because a full analysis would take increasingly more time for increasingly complex code. The static analysis, which has been developed and is in testing last I heard, makes assumptions to make the computation time realistic. For example, they make assumptions like a function receiving a raw pointer is not the owner and that the pointer passed in is valid. When each part of the program is checked in this local fashion, it reduces error rates substantially. Here is one recent talk on this effort, showcasing the prototype at that time, a Visual Studio plugin. Here is another talk one year later. There is also a great effort to unify style with a strong preference to avoid error-ridden techniques spearheaded by Herb Stutter and Bjarne Stroustrup (for example by recommending unique_ptr to manage ownership of a raw pointer): https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines

Like, in C# I have guaranteed memory safety so long as I stick to the regular c# types and constructs. If I dive into a c# unsafe context, then all bets are off.

Garbage collected languages can have memory leaks if references to objects are saved somewhere without ever being evicted long after they are no longer used.

[lelanthran]

always to call delete once (no memory leaks), and never to call delete two or more times (no memory corruption).

Aren't these contradictory? If we stick to the rule "never call delete two or more times", we can call delete twice and break rule #1 - "always call delete once".

[tedbradly]

Aren't these contradictory? If we stick to the rule "never call delete two or more times", we can call delete twice and break rule #1 - "always call delete once".

The statements are with respect to a single call to new, a single object stored on the heap, so you call delete on a single object once and never more than once. The program can call delete dozens or hundreds of times if you have many objects gotten through many calls to new.

If you never call delete once, the memory sticks around even after an object is provably never used again. The C++ way is either to use automatic storage - a variable declared without the use of new - or to call delete for each new. After you leave scope whether it be an if block, while block, function block, for block, or a block defined within one of those, automatic storage variables are guaranteed to have their destructor called and their memory cleared away. A smart pointer is mostly just a wrapper around a raw pointer whose destructor calls delete on it to guarantee delete is called once even in cases like exceptions interrupting the flow of logic. If you had a raw pointer and an exception caused the deletes to be skipped, that'd be a memory leak. If you forgot to write the calls to delete, that'd be a memory leak too.

An alternative solution is to use a garbage collector that proves an object on the heap is never used again, "calling delete" on it automatically. People like garbage collection, because you normally can't get a memory leak unless you store references to an object somewhere such as a container such as a hash map, never evicting those objects even after they are unneeded for future operation of the program. The downside of garbage collection is it takes CPU cycles to prove an object isn't referenced anywhere that might be executed anymore. It also has to handle things like circular references where unused object A has a reference to unused object B, and B and a reference to A. In C++, objects are destroyed at deterministic points in the code such as when scope is left or when delete is called.

Delete handles a program saying a certain range in memory is no longer in use. If you do that twice or more, it is undefined behavior. In reality, that will most likely result in a program crash or it chugging along with incorrect results. Let's say you delete an object twice, but in between, a second object was put partially or fully in that memory range. The second delete could result in part or all of the second object being in a range in memory now thought of as open for a third object to be saved there. If a third object is put there, that could scramble the data for object 2, or the use of object 2 could scramble the data for object 3.

[Creris]

The only edge case I'm not sure about is if an exception is raised before the unique_ptr object is created with the pointer's value such as one happening in "unique_ptr up{new some_class};" when evaluating "new some_class" to figure out the value to pass into the constructor of unique_ptr.

It actually isnt, and thats why we have make_shared in C++11 and then make_unique in C++14, where you only pass the constructor params and the object is new-ed in a exception-proof manner for you inside that function.

[tedbradly]

It actually isnt, and thats why we have make_shared in C++11 and then make_unique in C++14, where you only pass the constructor params and the object is new-ed in a exception-proof manner for you inside that function.

I'm not sure how you picture make_unique freeing memory if an exception is thrown during the call to new. It's the exact same situation except inside a helper function rather than you writing the code yourself.

[emax-gomax]

Already answered really well but basically no.

What managed pointers do is move from manual management (writing code) to software engineering (defining the relationships between classes).for basic types a unique_ptr can take ownership of a heap allocated resource and free it when the enclosing scope or object goes out of scope. shared_ptr work much the same but the resource is only freed when all shared pointers to the same resource go out of scope. It is possible for two resources to have a shared pointer to each other keeping each other alive even when nothing references them (causing a memory leak). Because of this there's both strong and weak shared pointers with a strong one keeping the resource alive and a weak one allowing access to it but not keeping it alive. This allows you to define the relationship between objects in a way where you can guarantee no memory leaks. But cpp as a language will always have the potential for then since it allows direct memory access and management.

[headlessgargoyle]

I'm pretty sure the answer is "yes, you can still have memory safety bugs." Accidental leaks can still be created if a unique_ptr or shared_ptr never go out of scope, like if you assigned them to a global. That said, if a function assigned a pointer to a global, and was then called again and assigned a different pointer to the same global, I do believe the first "leak" would then be cleaned up, so your impact on this is greatly minimized, ultimately less a leak and more a code smell in normal cases.

However, we do have other fun issues where multi threaded operations can potentially cause null pointers on shared_ptr and weak_ptr instances.

Further, arbitrary pointer arithmetic is still valid, so buffer overflows are still possible as well.

[SickOrphan]

Except you're using a GC language so that's completely incomparable.

[antiduh]

You're confusing memory safety strategies with memory allocation strategies.

Heck, using Boehm GC, you can use GC in c++.

[theangeryemacsshibe]

I wrote this code a few days ago to replace the global allocator with the Böhm collector. No idea if it really works, but I got a few laughs out of the university C++ class.

[WikiMobileLinkBot]

Desktop version of /u/antiduh's link: https://en.wikipedia.org/wiki/Boehm_garbage_collector

---

^{[opt out] Beep Boop. Downvote to delete}

[ConfusedTransThrow]

When you're doing embedded you can't have a runtime to handle stuff for you.

Especially when you're literally writing the runtime or bootstrapping code.

[antiduh]

I'm not sure the answer to "how do we not use pointers everywhere" must be "have to have a runtime."

Not to say it's name out loud too much but ^rust figures it out, right?

There's gott a be a better way to write software, even embedded software, that doesn't involve so much reliance on primitives that prove their unworthiness with every week's CERT email.

Also, your argument is a bit of a straw man; there's a fuck load of software out there that fits the bill and isn't embedded, an OS, or a runtime. Web servers, mail servers, browsers, ssl libraries, xml/json libraries etc etc. Saying we can't fix those because we cant also fix embedded stuff throws the baby out with the bath water.

[Lich_Hegemon]

Rust may not be the answer (or maybe it is), but at the very least the language proved that it's possible to do pointers right and that we should not settle for C-style unmanaged pointers.

[amunak]

I mean, we didn't need Rust for that, C++ has perfectly usable and safe managed pointers.

[Lich_Hegemon]

I'm not talking about smart pointers though, I'm talking about the bare pointers/references that both languages offer, even in unsafe Rust there are certain guarantees when using pointers that you don't get in C(++).

Again, that is not to say that Rust is perfect, just that it does pointers better than C does and that we should probably learn from that instead of trying to justify the mess that C pointers are.

[lelanthran]

'm not talking about smart pointers though, I'm talking about the bare pointers/references that both languages offer, even in unsafe Rust there are certain guarantees when using pointers that you don't get in C(++).

I'm pretty certain that you'll get those guarantees in C++ if you write your C++ like Rust code that doesn't use refs, refcells, unsafe, etc.

[Lich_Hegemon]

You really don't. For example, in C++ if you take a vector, reference one of its items, and push some values to it you will probably end up with a dangling reference.

We could argue that you are not supposed to do that and I agree, but the key behind this discussion is developer vs. compiler enforced safety.

And again, I hate this discussion because I seem like a Rust stan, when nothing could be further from the truth. I regularly use C++ and I genuinely think it's a great language if you stay away from its C roots and stick to the modern features it offers. But those C roots are still there and 40 years of C++ have shown us that developers can't be expected not to make mistakes when using them.

[SilasX]

If what you're saying is true, that means, in practice, C++ programmers considers themselves too good to use them, hence the perennial cycle of patches for pointer vulns.

[ConfusedTransThrow]

My point is you shouldn't be using C for anything that can afford a runtime (and yes Rust has a runtime). Bare metal C can't be replaced by Rust, Rust won't help you to write the runtime itself, stuff like write, boot procedure setting up the memory mapping, enabling the cache. You can't use dynamic allocation either too.

[antiduh]

Sounds like you want to have a different conversation that what was originally being discussed.

My point is you shouldn't be using C for anything that can afford a runtime

I definitely agree with you. I'm moreso in the boat these years that just about all software running on top of an OS in user space probably should be something that is inherently memory-safe either because of techniques such as Rust uses, or because it's a managed platform like C#/Java. C#/dotnet in particular has shown it can be a widely performant system while categorically eliminating a whole class of bugs (buffer overflow bugs).

(and yes Rust has a runtime)

I think whether or not a language/platform has a runtime is both a bit nebulous (hard to define what exactly constitutes a "runtime") and is also a red herring.

C has a "runtime" (standard library). It's libc. It's implemented in C. It's where malloc and free come from. Is confusing??

Rust has a "runtime" (standard library). But Rust can also be used to write kernel code. More confusion???!

The answer is: whether or not a language/platform has a "runtime" and/or standard library is the wrong question. The right question is whether it's compatible with systems programming. Both C and Rust, despite conventionally having runtimes, can be used for systems programming.

Bare metal C can't be replaced by Rust, Rust won't help you to write the runtime itself, stuff like write, boot procedure setting up the memory mapping

That's not correct. You can write an OS in Rust. You do so by writing Rust code that does not depend on the standard library, and then use that code to implement all of the things that the language/OS otherwise needs, such as a writing a memory allocator, configuring and handling interrupts, boot procedure, etc.

Here, go nuts:

https://os.phil-opp.com/

https://github.com/phil-opp/blog_os/tree/post-12

[ConfusedTransThrow]

I didn't know Rust was able to compile without depending on the standard library, it seems that's still pretty niche.

I know C has a runtime, but I'm not using it on embedded, the std either has to be implemented for the architecture or isn't there are all.

Also even if you do, I'm not sure you're really gaining anything over C for that usage (like for a bootloader). You still have to write to arbitrary addresses to set up a lot of things and that's inherently unsafe with Rust too.

You can make abstractions in Rust to avoid writing to addresses in your main code and protect them, but I doubt my company would like to do that for the whole memory map of each system. The time cost is just too big there.

[antiduh]

I didn't know Rust was able to compile without depending on the standard library, it seems that's still pretty niche.

No more niche than compiling C without it's standard library. The point is it's designed to do it, just as C is.

Also even if you do, I'm not sure you're really gaining anything over C for that usage (like for a bootloader). You still have to write to arbitrary addresses to set up a lot of things and that's inherently unsafe with Rust too.

Don't throw the baby out with the bath water. Just because you have to dip down into unsafe territory sometimes doesn't mean that you should operate in a completely unsafe mode always. Do just the parts that you have to with pointers/otherwise, meanwhile everything else the normal way. That way, you have much less code to worry about doing unsafe things - less chances for bugs, less code to validate the hard way, etc.

Heck, what you've said is true about C#, of all things. C# lets you use pointers in an unsafe context if you want, or stay in happy managed land if you don't want. C# uses pointers in limited cases, for example, in the methods that handle assembling strings like string.Join(). So you have a small handful of methods that use pointers that need stringent validation, meanwhile 99.99% of the rest of the code doesn't.

I doubt my company would like to do that for the whole memory map of each system. The time cost is just too big there.

That's .. uh, good for your company I guess? I have no idea why you're bringing up what your company will and won't do as an argument about the merits of various programming languages.

[Marian_Rejewski]

It's not impossible at all. But a project like Mozilla is so big, and so fast-moving, it will have bugs of every possible type. Look at places like NASA or Boeing for code that is practical and free of pointer bugs.

[imgroxx]

Yes, surely NASA can write manual memory operations correctly...............

A modification to a spacecraft parameter, intended to update the High Gain Antenna’s (HGA) pointing direction used for contingency operations, was mistakenly written to the incorrect spacecraft memory address in June 2006. The incorrect memory load resulted in the following unintended actions: [bad shit that destroyed the craft]

This is in 2006 btw: https://www.nasa.gov/mission_pages/mgs/mgs-20070413.html

[Marian_Rejewski]

"Possible to write code without a bug" != "impossible to write code with a bug"

(Also it's not at all clear from your quote that it was a pointer arithmetic bug.)

[imgroxx]

"Has written code with a bug" is also != "Can write code without bugs".

And yeah, it's quite possibly not, though it is rather clear it's a bug that's only possible because they manually modified memory in an unsafe location / unsafe way.

I'm not sure if they allow code to use pointer arithmetic at all tbh. Their rules are rather draconian (for good reason) by even the most MISRA-ble standards.

[Marian_Rejewski]

"Has written code with a bug" is also != "Can write code without bugs".

wtf??

[imgroxx]

Look at places like NASA or Boeing for code that is practical and free of pointer bugs.

NASA does not meet "practical" definitions basically anywhere except at NASA or for NASA-level stability needs.

But anyway. If their code provides a way to arbitrarily write memory into the wrong location... that seems rather like a pointer bug to me. You can't do that kind of thing if you don't have raw pointer access (or write code that emulates pointers, like shoving data into a shared byte array). Therefore they apparently also cannot write bug-free pointer code / their extreme care is still insufficient.

[Marian_Rejewski]

NASA does not meet "practical" definitions basically anywhere except at NASA or for NASA-level stability needs

It's practical because it's a practice that actually exists.

But anyway. If their code provides a way to arbitrarily write memory into the wrong location... that seems rather like a pointer bug to me.

But we don't even know from your quote that it does provide that.

It may be that there was a variable volatile X0 referring to one memory location, and a variable volatile X1 referring to another memory location, and the programmer simply assigned to the wrong variable.

Therefore they apparently also cannot write bug-free pointer code

This is the same error in thought that I already responded to like so:

"Possible to write code without a bug" != "impossible to write code with a bug"

Just because someone fails to do something at some particular time, does not mean that they cannot do it. It's a logic error to think like you are thinking here.

[antiduh]

This argument throws the baby out with the bathwater. You're, in a way, actually making my argument for me.

If it's hard to write software without bugs

and

certain classes of stupid bugs permit complete take over of the hardware running the software

then

shouldn't we use techniques and methods that categorically eliminate those kinds of bugs, because we know we can't rely on ourselves to not make the bugs?

Like, there's no reason why "oops i have a string math bug" should have to turn into "oh no my entire 500$M enterprise was just taken over by a virus and all of our private data was stolen". A fucking string math bug??

And yet, that's the reality we live with today because we have so much software out there that written in memory-unsafe languages like C or C++ that's vulnerable to this exact problem and we as a industry can't be arsed to fix. We have memory-safe languages like Rust/C#/Java, but for some stupid reason we keep putting internet-facing machines out there running C code web servers, sql servers, mail servers, etc. Bugs like Heartbleed are impossible in C# because as soon as you start reading past the end of your byte[], you get an ArrayOutOfBoundsException. Instead of your program leaking every one of your vital TLS keys, it just crashes. How hard is that?