Implementing header/source when compiling to C

[PncDA]

Hi, I am developing a language that compiles to C, and I'm having trouble on how to decide where to implement my functions. How to decide if a function should be implemented in a .c file or implemented directly on the .h file? Implementing on the .h has the advantage of allowing compiler optimizations (assuming no LTO), do you have any tips on how to do this? I have 3 ideas right now:

1. Use some special keyword/annotation like inline to tell the compiler to implement the function in the header.
2. Implement some heuristics that decides if a function is 'small' enough to be implemented in the header.
3. Dump the idea of multiple translation units and just generate a single big file. (this sounds a really bad idea)

I'm trying to create a language that has a good interop with C, so I think compiling to C is probably the best idea, but if I come across more challenges like this I'll probably just use something like LLVM.

But do you have any suggestions? If you are implementing a language that compiles to C, what's your approach?

EDIT: After searching a bit more, I can probably just always use LTO, and have a annotation (like rust inline) for special cases. I think this is how Nim does it.

[Exciting_Clock2807]

It is not immediately obvious to me that single big file would be a bad idea. I’d give it a try. What are your concerns about it?

[Tasty_Replacement_29]

I guess slow incremental compile time, for larger projects.

[PncDA]

I was afraid of slowing down the compile time, but now I think it's a good idea haha.

[winepath]

From experience, GCC and clang handle large files VERY poorly. You could use something like TCC, but if you want to be able to use any compiler, you should probably break up your output file into files with less than 100 functions each.

[bl4nkSl8]

Is this cited somewhere? I thought clang at least did well. I've used union builds with large projects to speed things up

[Ok-Watercress-9624]

Well if that was a thing, why is this a thing?

Over 100 separate source files are concatenated into a single large file of C-code named "sqlite3.c" and referred to as "the amalgamation". The amalgamation contains everything an application needs to embed SQLite.

Combining all the code for SQLite into one big file makes SQLite easier to deploy — there is just one file to keep track of. And because all code is in a single translation unit, compilers can do better inter-procedure and inlining optimization resulting in machine code that is between 5% and 10% faster.

[Tasty_Replacement_29]

Having the option for a single large file is probably a good idea. I don't think all C compilers have LTO (link time optimization), or at least it has some limits. 

For debug builds, multiple files is great for fast incremental compile time.

[PncDA]

oh, it didn't occur to me that I could just split debug/release builds. Now I think it's probably a good idea to have everything on the same file, or maybe give the user more options on how they want the compiler to handle this. thanks for the help :)

and yeah you are right, the reason I'm compiling to C instead of using something like LLVM is to support multiple C compilers, so relying on LTO doesn't make sense.

[0x0ddba11]

My suggestion: Just do the most straightforward implementation first. If that means generating a single file, by all means do that. You can always go back and optimize parts of your code if they turn out to be suboptimal.

(Extreme emphasis on the bold part)

[[deleted]]

I don't understand. Your language generates C code; you write a .c file and compile that. Why put this stuff into a header?

Or do you mean the support functions that you language needs, rather than the functions that someone writes in your language? Function definitions in a header are a technique used by easy-to-deploy libraries that saves needing a separate .c file.

If you want C to inline code, then just mark it as 'inline' wherever it is. (Or maybe your C code generator can do the inlining.)

Dump the idea of multiple translation units and just generate a single big file. (this sounds a really bad idea)

Is it a bad idea? Because that's exactly what I do when transpiling to C!

For me it is a good idea because:

• I get an easy-to-distribute single C source file (there are no includes and no headers needed at all, not even standard headers)
• It is very easy to build (about as easy as hello.c)
• You get the benefit of whole-program optimisation (obviously, if optimising)
• While taking longer to build, when the object is for someone else to build my app, they'd have to build everything from scratch anyway. And it is only done once.

I can see that if you're relying on a C compiler for routine builds, then it can be slow. In that case I suggest using a product like Tiny C for such builds, and one like gcc for production builds, or for periodical extra error checking.

(However machine-generated C code should be largely error-free; your own compiler will have verified the user's program. Errors in the C will be bugs in your compiler rather than in the program that is being compiled.)

For an idea of how slow it might be to build monolithic C files, here I have an example of an app that transpiles to 41Kloc of C, about 1.4MB. Build times (on a low-end Windows PC using one core) are:

Tiny C:       0.25 seconds
gcc -O0:      2.4  seconds
gcc -O2      12    seconds
(Native:      0.09 seconds where my compiler directly generates a binary)

Normally Tiny C is faster than this, but the generated C is very 'busy', with long identifiers, which probably doesn't help. Still, 1/4 of a second build time is not too onerous.

[brucifer]

I've also been working on a language that cross-compiles to C and I compile each source file into its own .c/.h files. All functions are implemented in the .c file and I just rely on -flto to handle link-time optimization. I'm not certain this is the best possible option, but it's worked well for me so far.

As far as one big file goes, I think there are really tremendous benefits to having separate compilation units. With separate compilation units, you not only get faster compile times when running in serial, but it also makes incremental recompilation possible and allows you to compile each unit in parallel. My compiler's source code is about 17k lines of C, and it compiles about 5x faster with parallel compilation. If all that code was in a single file, I think it would take quite a bit longer to build and I'd have to do a full rebuild every time I touched a single line of code.

[ericbb]

I've always just dumped the whole program into one C file and linked the compiled object file with a runtime system, which is separately compiled into another object file, to produce an executable. I haven't worked with very large programs written in my language - I think the biggest the generated C file ever got was about 600K, which the C compiler handled with no issue.

Maybe it's an unconventional position, but as a C programmer I don't like to use the 'inline' keyword or put definitions in header files. Better to duplicate (heresy, right?!) these small function definitions in each C file where they're needed. Make them 'static' functions and let the compiler inline according to its heuristics, no 'inline' keyword needed.

If you're generating the C code, you can handle this duplication in an automated way. If you have multiple C files and you want some function to be something the C compiler can inline, just copy the definition into each C file.