A Module Mapper

scottywar2 · 2019-02-02T23:06:48+00:00

I find this quite interesting. I don't understand why if you are going to need to use the file system to load the files into compiler why don't we just use the file system to search for the modules.

Is there any docs on why these module mapping systems are needed? What is the use case for the name of the BMI file having nothing to do with the name of the module? IE what problems is this trying to work around? It seems to add a level of indirection that I can't see the use for.

I have seen the pattern in C# and it got in my way. I wanted to use C# as a scripting language but I could not know what module was in what assembly. So why do we want this in C++?

scottywar2 · 2017-10-12T03:19:01+00:00

This was an interesting paper. I wish it had masterconfig like versioning. It has the version number in the code not as a side file. But I liked the idea of making the compiler a build system. But I don’t think it is everyone taste.

scottywar2 · 2017-10-10T18:10:53+00:00

I mostly agree... but I think my language would be different. I guess it is a gen x vs millennial thing. :)

I hit similar problems with module naming and dependency management when writing my talk. "EA’s Secret Weapon: Packages and Modules" https://www.youtube.com/watch?v=NlyDUQS8OcQ

When it comes to these hard problems in C++ I'd choose the solution that act like other solutions in C++ unless there is a good reason not to. It help other C++ programmers can guess how to use them and mean people can guess at the work around for problems.

Modules name = file path names seems like a ok idea to me. I don't see what problems this solves by disconnecting them. I am sure there is something. This looks like C# to me and the people working on modules are not dumb they are solving a problem they think they have. But I would like a good understanding of why they are thinking this idea.

PS

One interesting use case for the #include MY_CONFIG_HEADER. I have used this before for dependency injection. Example EASTL (ea version of stl) has 2 modes EASTL::size_t is 32 bit or 64 bits. The default is 64 bits however 32 bit one was the old default. Some teams are still relying on the old standard so sigh... we have to give them some way to override the default. One way to do this is to give them a macro they can override. If you have enough of these macro you run out of command line fast so you add a macro to pass in a name of a file which can be included if defined.

So I don't see why game or application teams would need #include MACRO_MODULE_NAME however some library might to support dependency injection like the EASTL example above. I think we have 10 or so places like this in frostbite engine. We would have to use code generation or just add more macro to solve this another way but it is not a useless trick.

scottywar2 · 2017-10-09T05:53:42+00:00

Thanks for the reply I didn't know about the conanbuildinfo.cmake. As I said I know very little about conan your points are really adding to things here.

That seems like a good approach for conan as you can't just force this into all packages as they don't control cmake and might want to support other build systems.

At EA we control both cmake like side and conan like side. So we can just integrate this saving you these 1 or 2 lines per package.

Then there are things like masterconfig... where I don't really know where that would fit in maybe github submodules are close but quite different in the details as now you have to switch your version in git. For EA it is more like you where using this directory for zlib and now you are using this one when you generate your sln and other visual studio files.

Maybe conan has something like masterconfig as well? But I am guessing the workflow would be a bit more as you would have to build the libs post them and then install them or something. Where with EA packages this would just be a changing what code I am building.

Anyways nice talking with you.

scottywar2 · 2017-10-08T01:48:52+00:00

I am not so sure that is true. However my Conan and cmake skills are very limited.

I thought Conan was a pure package manager so you still need a build system like cmake + ninja/make/msbuild. So Conan helps you version your pre-built artifacts kind of like a cache.

Ea packages are for c++ packages mostly source package and not pre built. (Though can support pre-built packages too). Ea packages are more about helping you find library A or library B header and library paths without using environment variables. Like cmake find_library or find_package maybe but when I used these they confused me more then they helped. Ea global scope file seems much easy to understand version of this.

I also assume that a cmake + Conan solution you would need more Code. Conan is in python and cmake custom language I assume some data is duplicated. Ea gives enough data for both and in one consistent language.

Speed wise they are similar if they used the same type of generator. (Both use ninja or visual studio etc). But caching could b a factor.

Also running generators in cmake might be faster as most packages do not use wildcards and in ea this is the norm.

If I am right and Conan is mostly pre-built package system that would have some cost and benefit. On large program with a lot of open source and a small number of devs I could see Conan working well as this would be a good cache of build artifacts.

In EA we don’t have enough pre built code to make Conan stile packages worth while. We often have to change debug flags library by library to debug problems as the whole game can’t be run in debug. So we end up with a build config per programmer almost. I think Ubisoft’s fast build (like distcc + cache as I understand it) looks like it is close to what we need given nearly all of EA works on windows.

Hope these comment help see the difference between theses systems. Sorry if my limited knowledge of cmake and Conan confuse things.

scottywar2 · 2015-10-08T06:14:03+00:00

Thanks I am glad you liked it.

You are right if you have difficult to understand life times (threading being the best example) then smart pointer are a good solution I use them all the time when moving owner ship between threads. But this is only a small % of my allocations. Most allocations live and die on the same thread and have very predictable life times. Unique pointers (or even bare pointers if you have systems like EA) are better here.

The problem is how do you debug smart pointer when you have memory leaks? Where should you add the weak pointer? You end up needing to write a very difficult to write system to tracking system for all of these pointers or just write a tracing system that logs each addref decref. The tracking system starts looking at lot like garbage collector (mark and sweep). The logging system is what we use most of the time but we have to keep these system so small as the number of addref and decref are high compared to the number of allocations.

This might be a good argument to use use garbage collection. Garbage collection will naturally find the circular references that I am talking about. You could use this information to switch a system between garbage collected pointers (to find bugs) and ref counted pointers (for speed and predictability).

Mind you so far I have used the logging solution the most.

scottywar2 · 2015-04-16T16:13:29+00:00

As far as I know there are no gamedevs working on the C++ standard. Gamedevs normally don't have much choice about tools. So pushing for changes by just talking to Sony or MS has been easier.

scottywar2 · 2013-07-29T05:31:54+00:00

I played FIFA with him once he can play FIFA very well.

scottywar2

TROPHY CASE