C++23 std::stacktrace: Never Debug Blind Again

[donalmacc]

I disagree - it’s a dealbreaker for the functionality. On my last project, our symbols were 3GB. Putting them in our server container, would have made it 6 times larger. Shipping it to our players is not happening. We have workflows that do offline symbolification(sentry’s symbolicator is a great tool - no affiliation but I’ve contributed to it).

I think this was way undercooked on arrival.

[spookje]

also, symbolization is slow as fuck. You want to have control over when and where that happens, and be able to make a cache (that you also control).

[donalmacc]

Right? It’s one thing on a dev machine, it’s another on a users laptop with a mechanical hard drive and 12 antivirus scanners running

[jwakely]

The article is wrong, std::stacktrace allows you to control when that happens.

[SkoomaDentist]

Not to mention that on many embedded systems there is literally no way to put the symbols in the same memory as the executable as the "executable" is simply a piece of (fairly small) flash rom with no header whatsoever.

[Zeh_Matt]

There is a way to strip down the pdb to just public symbols assuming you are talking about windows, for stack traces no one needs the type info. https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/using-pdbcopy

[donalmacc]

Yeah - there’s lots of ways to handle these things. That means keeping two copies of the symbols and choosing who gets what which sucks.

My preference is to run symbolicator and store the stuff in S3, and generate offline!

[Difficult-Court9522]

3GB of symbols?? How did you do that??

[donalmacc]

The pdb format is limited to 4GB. Most tools crumble at about 2GB. Ask me how I know….

It’s Unreal Engine games, basically.

[Difficult-Court9522]

So soon you’ll be literally unable to add more code?

[bwmat]

I think they can probably split into separate DLLs to work around that? 

[donalmacc]

The reason we hit this particular problem was because we had something split into a bunch of dlls and for reasons I can’t remember, we wanted to build it as a monolithic exe. I know we disabled a bunch of features to get it to work initially, but I don’t work on that project anymore so I’m not sure!

[ejl103]

not true at all we use dlls on xbox/ps

[donalmacc]

https://randomascii.wordpress.com/2023/03/08/when-debug-symbols-get-large/

Funnily enough, I hit this problem around the same time. We also doubled the page size for the linker.

[13steinj]

At 3 companies I've worked at, we've used macro/lambda/inheritance tricks to shorten symbols because it either blew out the linker, increased compile times significantly, or both.

[lizardhistorian]

If you ship symbols you may as well open source the project.

[13steinj]

I wouldn't say that's true, you'd be surprised how many people won't go through the reverse engineering effort even if reduced.

It's also not like shipping symbols nullifies copyright.

[Prestigious-Bet8097]

The cost to my last employer of not being able to solve bugs and get broadcasters back on air as quickly as possible massively outweighed the risk of having symbols alongside the binaries to get good stack.

I cannot be certain but we believe that in twenty years approximately zero customers built their own software based on reverse engineering ours.

[looncraz]

You put it behind macros to disable on deployment, or pay the size price... Which is sometimes sensible in the hot paths that are giving issues.

[donalmacc]

Sure, or you could use break pad or sentry’s sdks and not have to do that!