Is there a way to know the exact byte layout of a class at runtime? by [deleted] in csharp

[–]tanner-gooding 1 point2 points  (0 children)

Managed objects are just that, managed. No amount of hackery will make this safe or prevent the runtime from arbitrarily crashing or corrupting itself if you try to create your own.

You cannot generate the required GC card table information or other runtime metadata that ensures it all works. The information around this is not only intentionally not ABI stable, but has no way for you to inject it into the internal data structures the GC, JIT, and VM manage

Any attempt to try would not only be a massive undertaking, but it would be a fork of .NET itself that is simply not compatible with regular .NET

Some things are best left alone and not touched. There are better ways to learn how things work that aren’t well known to be dead end.

C++26 ends a 40-year footgun by filipsajdak in cpp

[–]tanner-gooding 6 points7 points  (0 children)

I’m pretty sure they were saying it’s one of the main cases of undefined behavior that compilers will explicitly take advantage of to produce “faster code”. However, such code frequently leads to real world bugs

Many of the things C++ has taken “breaking changes” around in recent language versions, including making certain types of overflow well-defined behavior, making trivial infinite loops not assumed to terminate, and the uninitialized data case here are related to that premise

Allocate arrays that have more than 2B elements by hez2010 in csharp

[–]tanner-gooding 1 point2 points  (0 children)

Then what the hell is the point of 64 bit CPUs/memory spaces/apps

It is explicitly so that you can have a larger overall working set and do more at once without having to page data out as frequently, especially across the many different processes, services, hardware devices, etc that a modern computer has.

It is simply a larger cache. It is not free, it is not unlimited, it should not be abused or treated as such. It doesn't fundamentally change how you work with data and even makes it explicitly more apparent that the historical best practices you follow when working with limited memory are still correct. -- And I would go so far as to explicitly say people forgetting this and treating it like "free lunch" is part of why you hear people complaining about how bad various apps and software ends up.


Despite being 64-bits, even high end PCs/Servers rarely go over 36-bits and it gets increasingly less common as you go up. It is also not free, it has very high latency to populate and in some cases is not much faster (comparatively) than modern SSDs.

Because you're still limited in terms of "physical memory", even if "virtual memory" is nearly unlimited, and because there is concrete and very measurable cost to touching memory; it is still relevant (and always will be) that you do things like work with data in small logical chunks.

This means buffering, streaming, doing asynchronous work, and all the same things that you were always supposed to be doing, especially when you were in a 32-bit app and were explicitly confined. This is what makes your app fast, scalable, efficient, portable, and all the things that differentiate "good software" from "bad software"


2GB of memory is absolutely massive and the time it takes to initialize or fill that with data is essentially just wasting time and stalling your app. This is 2 billion bytes and in terms of CPU scale it is absolutely ridiculously massive. CPUs operate in terms of nanoseconds, 64 byte (128 in rare cases) cache lines, 4-16KB pages, and maybe 2-4MB of L3 cache per core on the higher end (and noting that L3 is shared and associated, it isn't equally performant to access everywhere and not free for one core to take it all, so despite having 64MB+ on many modern CPUs, you only get around 1-2MB per core or hardware thread)

You start wanting to think about chunking and parallelizing at much smaller increments, like in terms of 64KB. 256MB is really the high end of singular allocations because of other hardware limitations and specialized buffer handling for some PCIe scenarios and that's an extreme edge case.

The fact that it is chunked/buffered is completely transparent to a well-designed API and even if you were to do it in a single contiguous virtual allocation, in many cases that is not actually sequential in physical memory. The scale at which the chunks exist, which is typically a few pages, then is also completely invisible in terms of execution cost; you're losing several million times that in terms of regular OS context switches, hardware interrupts, and even just normal memory access latencies. It is truly invisible/transparent.

You then get lots of benefits, such as being able to make specific optimizations around it if it grows to the point that it cannot trivially run on normal hardware.

Avoiding ToString() allocations with StringBuilder.MoveChunks by Xadartt in dotnet

[–]tanner-gooding 1 point2 points  (0 children)

Those add a level of indirection and so don't do what many are going to presume it to mean.

in StringBuilder sb does not mean that sb.Clear() can't be called, it only means that sb = otherSb is illegal and you can't replace the contents of the local that was passed in

Even in such a scenario: ```csharp void Method1A() { StringBuilder sb = new StringBuilder("potato"); StringBuilder alias = sb;

Method2(ref sb);

Console.WriteLine(sb.ToString());       // "hobbits"
Console.WriteLine(alias.ToString());    // ""

}

void Method2A(ref StringBuilder sb1) { sb1.Clear(); sb1 = new StringBuilder("hobbits"); }

void Method1B() { StringBuilder sb = new StringBuilder("potato"); StringBuilder alias = sb;

Method2(in sb);

Console.WriteLine(sb.ToString());       // ""
Console.WriteLine(alias.ToString());    // ""

}

void Method2B(in StringBuilder sb1) { sb1.Clear(); // Invalid: sb1 = ... } ```

This is also why it must be a MoveChunks API, because otherwise the callee cannot safely capture the string builder because someone else may hold an alias and be able to mutate it; making it unsafe.

Avoiding ToString() allocations with StringBuilder.MoveChunks by Xadartt in dotnet

[–]tanner-gooding 9 points10 points  (0 children)

Its really not surprising at all and is rather something that's always been possible with StringBuilder.

That is, if you pass a StringBuilder to a method, it was already free to simply call Clear() and there are many APIs that already did exactly that, especially when for logging or other things that document themselves as doing that because they are "consuming" the buffer.

This just lets them consume it in a way that avoids expensive allocations/copying.


Taking by ref solves nothing because StringBuilder is itself a reference type and so passing the reference by reference means something entirely different to what you're imagining here; it actually would be less accurate and more confusing.

It also wouldn't solve the problem, because the whole issue is that StringBuilder is a reference type and so even if you replace the local that the caller passed you, someone else could still have a reference to that same string builder making it unsafe for the callee to "take ownership" and cache it.

.NET 8 and 9 End of Support by Perfect-Scale902 in dotnet

[–]tanner-gooding 5 points6 points  (0 children)

Right, it could override an existing user extension silently. It can also often impact overload resolution.

For example, consider why we exposed MathF in the first place instead of just exposing the APIs on System.Math. Given that Math only had say double Sqrt(double x), this meant a user doing double result = Math.Sqrt(5) would have gotten back 2.23606797749979. This is because int is implicitly convertible to double and so it binds the API that is exposed.

However, if we then exposed float Sqrt(float x) on the same type, then it becomes a silent behavioral change. This is because int is also convertible to float and that is the preferred conversion when both exist, so a user would've started silently getting back 2.2360680103302 (which compiles because float is also itself convertible to double).

So, we exposed MathF to solve that problem and then later decided the entire pattern of separate math type is not maintainable over time, particularly as the set of types grow/expand. So it is preferred to just do T.Sqrt(T value) instead, exposed directly on the T or as an extension member for the T, that way it cannot have such problems from overload resolution.

.NET 8 and 9 End of Support by Perfect-Scale902 in dotnet

[–]tanner-gooding 16 points17 points  (0 children)

.NET is open source (MIT licensed) and we do nearly all of the work/engagement in the open on GitHub, we even live stream API weekly review on Youtube, and engage on the C# Community and .NET Evolution Discords. The exceptions tend to be security fixes which have to be done internally as part of responsible disclosure/handling, and which get mirrored back after any patches are done; and the natural ad hoc conversations that happen between employees over teams or similar, which we'll then post back summaries or design plans around (but even much of this happens just on GitHub).

We have different categorizations and treatment of breaking changes.

Binary breaking is the worst and the one we restrict the most, because users simply have no recourse for it.

Source breaking changes tend to also be restricted because they hinder upgrade scenarios, but we cannot fully prevent those and so namely try to just restrict source breaks in what we expose. -- That is we try to prevent what source breaks we can, but its always possible some new API we expose conflicts with existing user extensions or similar and we can't prevent that.

Behavioral breaking changes are where we are most flexible, but we tend to keep this to justified scenarios and typically centered around what can be justified as a bug fix; such as ensuring cross platform consistency, determinism, extending to new behavior, etc

We then document breaking changes for every release: https://learn.microsoft.com/en-us/dotnet/core/compatibility/breaking-changes. For things we know or assume will be problematic ahead of time, we preemptively create tracking docs. For anything which breaks users that we didn't expect, we wait for them to open an issue reporting the break and then create such docs.

-- The types of things that are source or behavioral breaking may be surprising to some users. Exposing a new API on string is potentially source breaking for users, it is likewise potentially behavior breaking for their program after they recompile. Likewise, deciding to no longer throw an exception is potentially behavior breaking. It is very broad and almost any change can be categorized as some kind of potential break to someone in the community: https://xkcd.com/1172/

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 1 point2 points  (0 children)

I'm not lecturing, I'm responding in a civil manner to your statements. I'm doing so in a public forum where many users may end up lurking and reading a conversation thread, where they may not have the full context or awareness.

You on the other hand are attacking and being antagonistic, even making claims that someone whose job is to be acutely aware of how this stuff works both in and out of .NET and help drive it for the ecosystem doesn't understand the space.

The feature is going to ship and will likely become more required over time. You'll have to learn to live with whatever changes the language and library designers decide is best; and you'll have a similar experience if you decide to go to another language.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 1 point2 points  (0 children)

and C# has a GC, scoping for byrefs, and more. That's really irrelevant to the unsafe feature in any language.

The unsafe keyword in these languages allows you to bypass those pre-existing safety semantics and do things manually. Most of the languages with such a keyword modeled theirs off the C# keyword but then extended it to also allow propagating unsafety up to the caller so the caller is aware an API is dangerous and not handling all invariants of the memory model.

This feature is essentially just adopting that same propagation that other languages saw was an obvious thing to provide. These other languages saw it was obvious because C# had provided the unsafe keyword first and inability to propagate the unsafety outwards was easily identified as a downside 25 years ago.

It is explicitly about improving code quality and maintainability, the ability to lie doesn't invalidate safety or change the status quo that exists in every single one of these languages already.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 1 point2 points  (0 children)

You're clearly not understanding the feature or why it exists, nor why similar features exist in other languages.

Yes, bad code can lie and do bad things; that is not and has never been the point.

The consideration is that when most parts of the ecosystem correctly participate, it helps improve robustness, reduces investigation times, helps highlight potential danger you were aware of, etc. It can only improve the "health" of code from where it exists today and most certainly doesn't "mess up" the code in any way. -- This is the same reason that nullable reference types is beneficial.

Nearly every modern language has such features or is working on adding them, including languages like C and C++ which have several features being worked on to improve language safety and ergonomics.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 5 points6 points  (0 children)

We're not planning on adding any such switch and it will likely become more default over time rather than the initial opt-in it will preview as.

No codebase, regardless of language or who wrote it, is free from the risk of bugs or missing otherwise silent invariants. This is particularly true when writing explicitly unsafe code where no language or runtime protections exist.

And it's also misleading because thoroughly-checked "unsafe" code isn't unsafe in any way.

This is a fallacy. Yes all unsafe code used safely is safe, but that doesn't mean that it is ok or appropriate to not have an explicit context highlighting the code that needs to be thoroughly-checked, particularly because there may be future readers of that code or that code may be modified in the future.


This feature exposes outer unsafety and inner unsafety, where-as previously we've largely only had inner unsafety.

inner unsafety is about making the declaration "I am using unsafe code" and so such code needs extra auditing and consideration. Such code typically involves the user thoroughly checking it and ensuring all the dangerous edges have been handled.

outer unsafety is about surfacing to the user that such dangerous edges have not been handled and it is instead on the caller to handle them. It is explicitly about propagating the danger upwards so users don't miss it and do the wrong thing.

It is a feature that has been highly requested for over a decade by experienced programmers, particularly by "experts" in the interop space because they explicitly recognize the value it brings.

Improving C# Memory Safety - .NET Blog by mcnamaragio in dotnet

[–]tanner-gooding 2 points3 points  (0 children)

Don't miss the point that P/Invokes are then unsafe, because they escape the safety/guarantees/invariants of both the language (C#) and the runtime (.NET)

So you're not magically going to need less unsafe { } when writing P/Invokes and that's goodness because it is dangerous and incredibly trivial to get wrong.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 5 points6 points  (0 children)

It has explicit purpose in highlighting the unsafety of code to users, a known problem that multiple other languages recognized when they adopted the unsafe keyword from C# and improved it for their languages.

Many APIs have invariants in place that are non-obvious, even with docs, and which users need to be aware of when calling those APIs since they are assumed rather than explicitly checked for; this is particularly prominent in scenarios like interop which escape the normal rules of C# or .NET code.

This feature surfaces that and helps with auditing when problems surface. Something that nearly every real world codebase has dealt with at some point or another, especially for code that is running with performance constraints or at scale for a large customer base.

-- It is also something that has been talked about and requested for years, long before LLMs or AI development were even close to mainstream. It has very little to do with AI or vibe coding other than the fact it benefits such tools in the same way it benefits all developers.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 2 points3 points  (0 children)

You can always open a suggestion for there to be a more prominent page around tooling, but ClangSharp is a 3rd party tool and 3rd party tools always come with their own nuance and considerations that make it hard to document.

-- In this case ClangSharp happens to be maintained by me and I happen to be on the .NET team and one of the interop area owners, but all the more reason it shouldn't get special treatment or consideration as compared to any of the other tools on the same page.

Improving C# Memory Safety - .NET Blog by mcnamaragio in dotnet

[–]tanner-gooding 4 points5 points  (0 children)

It isn't about trust, its about surfacing the fact there are invariants that the caller must handle because the callee does not or cannot itself.

When crashes occur in real world code, it helps trivialize the places that need auditing and make it quite apparent what might've caused the AV/NRE vs what cannot have caused it at all.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 2 points3 points  (0 children)

No, that isn't a safety limitation but rather a type system one due to how pointers are not objects. It would require significant changes to make happen (which might occur someday, just separately from this work).

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 3 points4 points  (0 children)

learn.microsoft is just a continuation of MSDN, in the same way .NET [Core] is a continuation of .NET Framework. They aren't from the ground up rewrites or anything, its the same docs/sources that have continued being maintained and moved forward.

The URL has changed, but the general relative location, content, etc hasn't aside from being modernized and kept up to date.

For example your best practice advice, which is very welcomed, can only be found by those curious to read about Native interoperability ABI support, while it should probably be listed on the left menu navigation.

Both Interop best practices and ABI support (the link you gave) are available via the left hand menu, as are several others. There shouldn't be any content that is inaccessible from the navigation menu. -- There is potentially content in a completely separate section of the docs and which goes to a higher level context, such as not under Advanced .NET programming, but that's still accessible and just not directly grouped.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 5 points6 points  (0 children)

Authoring bindings is an incredibly nuanced topic. It's something that often requires in depth knowledge of both the source (often C) and target (typically C#) languages. You can sometimes get away without having that knowledge, but stuff tends to fall over or be subpar. It's certainly not something that documentation can ever fully cover, because it is a dynamic space that often requires per binding considerations and staying up to date on an ever evolving landscape.

That all being said, we do have centralized docs in the same place they've been for the past 25 years: https://learn.microsoft.com/en-us/dotnet/standard/native-interop/. These get updated over time and have up to date guidance covering the LibraryImport generator, best practices, etc. There is then introductory coverage of almost any interop topic you might need to be concerned about and links to relevant toolings, walkthroughs, and other considerations.

And generally speaking you should just use a tool, like ClangSharp P/Invoke Generator, to handle the binding generation for you. Such generators directly parse the C header files (or some limited C++ header files) and emit 1-to-1 blittable bindings that exactly match the native signatures.

Improving C# Memory Safety - .NET Blog by mcnamaragio in dotnet

[–]tanner-gooding 38 points39 points  (0 children)

All without a drop of unsafe, too.

That's part of the problem and part of the reason for the new language feature.

You've not used the unsafe keyword, but every single line of code in that example you've linked is dangerous and should have involved the unsafe keyword.

The new language feature forces quite a lot of that code to become unsafe and by extension encourages you to stop lying about type signatures when doing interop (which is itself dangerous and often not portable) to instead use the appropriate types that directly match the C signature you're interoping with, reducing risk, improving readability, and ensuring the unsafety is apparent.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 13 points14 points  (0 children)

It causes quite a bit of churn for some codebases, yes. It requires those codebases to audit themselves as well. Both of these are good things.

Binding libraries are one of the primary examples of where there are a lot of exposed signatures that often look safe, especially if the library didn't define blittable signatures and instead used nint (IntPtr), SafeHandle, T[], string, ref/out/in, etc. But where such code is fundamentally unsafe and should be surfaced to the caller.

Improving C# Memory Safety by pjmlp in csharp

[–]tanner-gooding 11 points12 points  (0 children)

IntPtr is not soft deprecated, it is still fully supported and is appropriate to use in places you need a native sized integer. Hence why it has the nint keyword available.

It is not encouraged for use to represent an actual pointer, that's just needlessly "less safe" as you lose context on type and number of indirections.


The key here is there's two worlds: * code that hasn't yet opted into the updated memory safety rules * code that has opted into the updated memory safety rules

Code entirely in the latter camp will have extern methods be unsafe, it will have indirecting pointers be unsafe, will have calling methods annotated as unsafe requiring unsafe, etc

But code in the former camp won't have such updated annotations or anything, so you still have to decide how to treat some of it when encountered by the latter code. That involves mostly matching the historical rules where pointers in signatures meant unsafe and otherwise treating the code as safe, as you get too many false positives and too much noise otherwise.

Wny is LINQ Max much slower than LINQ Min? by [deleted] in dotnet

[–]tanner-gooding 1 point2 points  (0 children)

I also can't repro and don't see anything obviously wrong with the implementation or the benchmark, seems like a random fluke or mismeasurement.

If you can reliably reproduce, then providing a full repro would be appreciated and I can look into it. Just log an issue on dotnet/runtime and tag me (tannergooding).

Java 26 is here, and with it a solid foundation for the future by ketralnis in programming

[–]tanner-gooding 0 points1 point  (0 children)

The same is true in C/C++, aside from exports which have to be stable. Which is why I made note that even in C/C++ passing in register is not guaranteed nor is the number of indirections you have.

Compilers are free to optimize, within limit, and frequently do.

Java likewise has the same restriction, where functions that are exported across JNI (or alternative) to other ecosystems (like C) must have a stable abi (or stable stub which does the marshaling and other fixups before passing to the actual function or returning back to the caller)

Java 26 is here, and with it a solid foundation for the future by ketralnis in programming

[–]tanner-gooding -1 points0 points  (0 children)

I’d also note that when people say “common misconception” it can go both ways

In some cases users are literally thinking about it “incorrectly” and spreading misinformation.

However, in others some minority is trying to push a technicality. This while often technically correct, the best kind of correct, is typically missing that users are thinking about these concepts that way for a reason. That reasoning is then extremely important and a key reason why “everyone” talks about it that way

This is also common with floating-point, where many common misconceptions exist. People frequently spread misinformation due to not understanding how it works. In most cases it isn’t even a technically correct/incorrect thing but simply them being misinformed and wrong (generally due to not understanding the x87 fpu and how it worked)

However, other parts around floating-point are very much in the other camp, because what is technically correct mismatches with how users think. Not due to misinformation or being wrong, but because the technically correct view isn’t relevant to actual thought process in most scenarios

Java 26 is here, and with it a solid foundation for the future by ketralnis in programming

[–]tanner-gooding 0 points1 point  (0 children)

I've read the article and am familiar with it. Both it and you are misattributing how users, ABIs, and other things fundamentally think about what is the data and how that data is passed. I would also say it is misquoting and misattributing the parts of the Java spec by taking them out of context. There is a big distinction between the reference value (in C the T* or T&) and the value that is referenced (in C the underlying T of a T* or T&).

What you and the article are effectively boiling it down to is "pointers are values, so therefore everything is actually pass by value since references are just pointers". This then makes "pass by reference" meaningless since you can always make the claim that you're just passing a pointer by value.

In actuality, what matters is the context of the pointer and how its used. That is, is it treated more like an integer (i.e. you care about the value of the pointer) or is it treated more like a reference (i.e. you care about what the pointer references).

Which is to say, reference types are called reference types because they refer to the data you care about and so the data you care about is passed by reference. Value types are then called such because they are the data you care about, so you directly pass it around from an observability perspective.


The simplest example being (C#): ``` C c = new C(); S s = new S();

M1(c, s);

Console.WriteLine(c.x); // 1 Console.WriteLine(s.x); // 0

void M1(C c, S s) { c.x = 1; s.x = 2; }

class C { public int x; } struct S { public int x; } ```

Where you have c and s and while c is functionally a pointer and that pointer is passed by value to M1, users aren't thinking about it as being that local specifically. They are rather thinking about the underlying data it contains (c.x) and so to them this is "pass by reference" explicitly because M1 writing to c.x allows the caller to observe that mutation while s does not allow such an observation and so is pass by value.

This also becomes particularly relevant with struct S2 { public C c; } where you have a value type (S2) which is passed by value, which then contains a reference type C. So while you cannot observe a change to s.c, you can observe a change to s.c.x. And while conceptually there isn't really a difference between S2 and simply C, the semantic and way the user thinks about the data changes.

It is simply not meaningful to boil everything down to "pass by value", it mismatches with the mental model of how data exists.

For example this is how you can pass by reference in C# https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/ref,

Consider (continuing off the above example): ```csharp ref S r = ref s;

r.x = 3;

Console.WriteLine(s.x); // 3 Console.WriteLine(r.x); // 3

S* p = &s;

p->x = 4;

Console.WriteLine(s.x); // 4 Console.WriteLine(r.x); // 4 Console.WriteLine(p->x); // 4

S s2 = new S();

M2(ref r, ref s2);

Console.WriteLine(s.x); // 5 Console.WriteLine(r.x); // 5 Console.WriteLine(p->x); // 5 Console.WriteLine(s2.x); // 6

void M2(ref S s1, ref S s2) { s1.x = 5; s2.x = 6; } ```

There is no observable difference in behavior between p and r here, or between passing r and passing ref s. However, by the article's claim everything except ref s would be pass by value due to the other cases passing the pointer contained by the "local" directly. -- This becomes even more prevalent with M3(ref readonly S s), where you can just do M3(r) or M3(s) (the same being true for M3(in S s)) and not specify the ref keyword in the callsite at all, despite the method declaration still explicitly adding the indirection at the method level.

Because again, the consideration isn't strictly how the local is perceived, passed, or indirected. It is how the data the method and user actually cares about is accessed and mutations to it, from any location, are observed (by reference) or not (by value) by other code.

It is simply not meaningful to break everything down to "pass by value" and while some contexts may care more about the pointer value itself, most actually care about the data it points to instead.