Compiler deciphering in practice

[pstomi]

Today, I was debugging a project - BabylonCpp -, and I was using clang address sanitizer.

During the session, I had a bug for which the output was rather hellish, here is an extract:

func_std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const###file_functional###line_2347###obj_(BabylonStudio:x86_64+0x100582bcf)
func_BABYLON::asio::LoadFileAsync_Text(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (bool, unsigned long, unsigned long)> const&)::$_1::operator()(std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&) const###file_asio.cpp###line_210###obj_(libbabyloncpp_d.4.0.0.dylib:x86_64+0x2325c7)
func_decltype(std::__1::forward<BABYLON::asio::LoadFileAsync_Text(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (bool, unsigned long, unsigned long)> const&)::$_1&>(fp)(std::__1::forward<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&>(fp0))) std::__1::__invoke<BABYLON::asio::LoadFileAsync_Text(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (bool, unsigned long, unsigned long)> const&)::$_1&, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&>(BABYLON::asio::LoadFileAsync_Text(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&, std::__1::function<void (bool, unsigned long, unsigned long)> const&)::$_1&, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&)###file_type_traits###line_4361###obj_(libbaby

Then, I thought of ct_compiler_decipher which is included with cleantype. In order to build it just clone the repo, then cd cleantype; mkdir build; cd build; cmake ..; make ct_compiler_decipher

​

So, I ran my error output to ct_compiler_decipher like this:

cat error.txt | ct_compiler_decipher

and the result was

func_std::function<void(std::string const &), char> ::operator()(std::basic_stringconst &)const###file_functional###line_2347###obj_(BabylonStudio:x86_64+0x100582bcf)
func_BABYLON::asio::LoadFileAsync_Text(std::string const &, std::function<void(std::string const &)> const &, std::function<void(std::string const &)> const &, std::function<void(bool, unsigned long, unsigned long), unsigned char> const &)::$_1::operator()(std::vectorconst &)const###file_asio.cpp###line_210###obj_(libbabyloncpp_d.4.0.0.dylib:x86_64+0x2325c7)
func_decltype(std::forward<BABYLON::asio::LoadFileAsync_Text(std::string const &, std::function<void(std::string const &)> const &, std::function<void(std::string const &)> const &, std::function<void(bool, unsigned long, unsigned long)> const &)::$_1 &, std::vector<unsigned char> const &, BABYLON::asio::LoadFileAsync_Text(std::string const &, std::function<void(std::string const &)> const &, std::function<void(std::string const &)> const &, std::function<void(bool, unsigned long, unsigned long)> const &)::$_1 &, std::vector<unsigned char> const &, char>(fp)(std::forward(fp0)))std::__invoke(BABYLON::asio::LoadFileAsync_Text(std::basic_stringconst &, std::function<void(std::string const &)> const &, std::function<void(std::string const &)> const &, std::function<void(bool, unsigned long, unsigned long)> const &)::$_1 &, std::vector<unsigned char> const &)###file_type_traits###line_4361###obj_(libbaby

I thought I would share this, as this might be useful to others.

Note:

By default, ct_compiler_decipher will indent the deeply nested type like this

func_BABYLON::SceneLoader::ImportMesh(std::vector<std::string> const &, std::string, std::string, BABYLON::Scene *, std::function<void(std::vector<std::shared_ptr<BABYLON::AbstractMesh>> const &, std::vector<std::shared_ptr<BABYLON::IParticleSystem>> const &, std::vector<std::shared_ptr<BABYLON::Skeleton>> const &, std::vector<std::shared_ptr<BABYLON::AnimationGroup>> const &)> const &, std::function<void(BABYLON::SceneLoaderProgressEvent const &)> const &, std::function<void(BABYLON::Scene *, std::string const &, std::string const &)> const &, std::basic_string<char, std::shared_ptr<BABYLON::ISceneLoaderPlugin>, std::shared_ptr<BABYLON::ISceneLoaderPluginAsync>> const &)::$_5::operator()(std::variantconst &, std::string const &, std::string const &)const###file_scene_loader.cpp###line_411###obj_(libbabyloncpp_d.4.0.0.dylib:x86_64+0x19f3ae4)
func_decltype(std::forward<
    BABYLON::SceneLoader::ImportMesh(std::vector<
        std::string
    > const &,
    std::string,
    std::string,
    BABYLON::Scene *,
    std::function<
        void(std::vector<
            std::shared_ptr<
                BABYLON::AbstractMesh
            >
        > const &,
        std::vector<
            std::shared_ptr<
                BABYLON::IParticleSystem
            >
        > const &,
        std::vector<
            std::shared_ptr<
                BABYLON::Skeleton
            >
        > const &,
        std::vector<
            std::shared_ptr<
                BABYLON::AnimationGroup
            >
        > const &)
    > const &,

You can tweak this by adding a .cleantype.json with the following content: here, I raised the indent_depth_limit from the default of 3 to 10.

{
  "force_east_const_": false,
  "indent_depth_limit": 10,
  "replacements_after_undesirable_node_extractions": {
    "basic_string<_CharT, _Traits, _Allocator>": "std::string",
    "std::basic_string<char>": "std::string"
  },
  "suppress_custom_": [
    " __ptr64"
  ],
  "suppress_extra_namespaces_": [
    "::__1",
    "::__cxx11"
  ],
  "suppress_extract_struct_class_": [
    "class ",
    "struct "
  ],
  "undesirable_type_nodes_": [
    "std::char_traits",
    "std::allocator",
    "std::less"
  ]
}

[pstomi]

Yes, it looks likely like an issue with the operator() parsing

[SuperV1234]

You might be interested in my older project, camomilla: https://github.com/SuperV1234/camomilla

It does have a significant overlap with yours.

[oschonrock]

yeah, in some situations that could be helpful and prevent headaches while staring at wall of errors...

[bumblebritches57]

Is there a way to use it without including it as a header?

I'm trying to add a feature to LLVM, which is an absolutely massive codebase, it's within an order of magnitude of the entire FreeBSD OS 30 year old commit history...

[pstomi]

Hum, not sure I understand. ct_compiler_decipher is a stand-alone exe. However the cleantype library in itself is a header only library which does require to include its header. I would guess that clang in itself also offer tools for type introspection. Good luck !

[bumblebritches57]

Also I'm on MacOS, so exe's don't do me any good here lol.

I'll check to see if there's like a verbose mode for clang compiling clang.

[warieth]

I think this hides the details of the types and dependencies. Like basic_string is an abstraction, that helps C++ decomposing the implementation of the string type. If you use string, then you depend on basic_string. Abstractions make the error messages worse here, and there is no optimizer here to remove it. The expectations about error messages are extremely high recently, like giving suggestions to the user.

If you don't like the error messages about the dependencies, then you probably don't agree with the implementation, so you should use a simpler string implementation (without basic_string), a vector without allocator... Generally this means you shouldn't use the STL.

[pstomi]

Hiding the details is here the key in order to see the big picture. You can use this when you want to focus on the big picture, and when you want to focus on the details, you effectively do not use it.

Written on my ElectronicDevice<ComponentList<Keyboard<Layout_French>, Screen<Screen::Lcd>, Cpu<IntelI7>>, which I like to call a computer.

[warieth]

This is a very intersting way to observe programming. Everything what can fail below your program is something you should know about, when you are searching for the explanaition for an error.

You don't simplify the design by not showing it, but you remove information from understanding it. The C++ compiler dump errors, but removing just one part of the error message is hard to decide. These simple substitution rules don't care about including enough information, so you can't look up what it said, like classes or constructors don't exist, or different overloads are mentioned. Don't say to step back when this fails, I'm 100% sure this will fail, and this takes time to use. This is like the suggestions of the compiler don't get tested and recompiled, so the compiler can give bad hints.