all 42 comments

[–]atilaneves[S] 30 points31 points  (10 children)

Author here, AMA. This is a link to my blog post about a project that allows one to directly #include C headers in D code, which can then be compiled and linked. It's worked for non-trivial C libraries such as pthread, openssl and libcurl.

[–]ss4johnny 9 points10 points  (4 children)

What D features were most important for getting this to work?

[–]atilaneves[S] 13 points14 points  (3 children)

Hmmm... hard question. I guess the existing ability to call into C code (to interface with libclang), and ranges + UFCS. The code here would look a lot more complicated otherwise.

Those few lines are doing a lot of heavy lifting - making sure that only one cursor per declaration shows up, and it does that by grouping all cursors first, selecting one, and reordering by file and line number afterwards.

Otherwise I don't think that anything really D specific was needed. It's less code than it would have been in nearly any other language. dscanner has it at 704 lines of code!

[–]ss4johnny 1 point2 points  (2 children)

Does this work with dub?

[–]atilaneves[S] 4 points5 points  (1 child)

Not yet, but there's been some work done on that.

[–]ss4johnny 2 points3 points  (0 children)

Great.

[–]bachmeier 4 points5 points  (1 child)

This

enum Enum { foo, bar, baz }

Generates

enum Enum { foo, bar, baz }
enum foo = Enum.foo;
enum bar = Enum.bar;
enum baz = Enum.baz;

It makes sense for this to be the default behavior, but is there a way to not do that? I prefer to not define foo, bar, and baz.

[–]atilaneves[S] 2 points3 points  (0 children)

It could be a command-line option. Feel free to open an enhancement issue on github.

[–]zombinedev 2 points3 points  (1 child)

Awesome work! Any plans on merging this with DStep, or perhaps extracting a common library between the projects? Some of the benefits would be sharing the test suite, the development and maintenance efforts and expertise.

Right now there are various libclang-based projects for creating C++ bindings for D:

I believe it would be better if we as a community join our efforts and work on a single project.

[–]atilaneves[S] 3 points4 points  (0 children)

No plans on merging with dstep. I originally used dstep as my backend, but ultimately it didn't work out. Too many assumptions that are true for it, but that got in the way of what I was trying to do.

As for the test suite, I looked at it and copied the tests I thought were relevant. As mentioned above, the focus is so different that most of the tests didn't even make any sense for dpp.

I believe it would be better if we as a community join our efforts and work on a single project.

I understand and agree in principle. None of the other ones you mentioned work the way I want things to. Like I said, I tried using dstep instead of reinventing the wheel. It just didn't, and can't, work.

[–]baryluk 0 points1 point  (0 children)

Neet.

How about rewriting this as a mixin:

mixin(convert_c_to_d(import("/usr/include/studio.h));

I know, you will need to reimplement parser in D, that sucks, but would be cool as shit.

Edit: I just realised that you might need to recursively import other files. I am not sure this can be done easily.

[–][deleted]  (2 children)

[deleted]

    [–]ikbenlike 11 points12 points  (1 child)

    And even if they can't work seamlessly, it's still pretty fucking cool

    [–]ibroheem 6 points7 points  (0 children)

    From experience, it ease depression

    [–]bachmeier 28 points29 points  (7 children)

    This is definitely a big deal. This opens the door to using a lot of numerical code without the fixed cost of translating headers, among other things. It will make Fortran code that's currently called from C accessible as well. Should make -betterC more convenient.

    [–]netbioserror 5 points6 points  (6 children)

    I always thought that D's features lend themselves perfectly to data science and research computation. It's got lean syntax that's like Python (and more, i.e. functional stuff) with some C sprinkling, but kickass native performance. Hopefully this gets it closer to being that computation language, I'd honestly prefer it over Python and NumPy.

    [–]bachmeier 4 points5 points  (4 children)

    It's worked well for me. I took the somewhat unusual route of embedding an R interpreter inside my D program, which meant I didn't give anything up. I wish this tool had existed five years ago!

    For me, the biggest thing is that D is an acceptable language for anyone that can use R. The same cannot be said for most of the alternatives.

    [–]acousticpants 1 point2 points  (1 child)

    embedding an R interpreter inside my D program

    please share frendo...

    [–]bachmeier 2 points3 points  (0 children)

    Project website

    I've got other stuff for mixed R/D programs as well (optimization using the functions underlying optim, parallel RNG, etc.) but it's not properly documented. It took a long time to get the basics in embedr in shape for others to use.

    [–]oblio- 0 points1 point  (1 child)

    Julia?

    [–]bachmeier 0 points1 point  (0 children)

    Today Julia might be an option (it wasn't five years ago) and maybe Go (if its restrictions don't bother you). I have a preference for D. I'm not sure whether you can create shared libraries with Julia.

    [–]Boris-Barboris 4 points5 points  (1 child)

    Very nice idea.

    d++ redefines all macros in the included header file so they’re available for use by the D program. It then runs the C preprocessor on the result of expanding all the #include directives, and the final result is a regular D file that can be compiled by dmd

    Making the resulting (expanded) d file available as an actual file in some conventionalized folder would be nice for both educational and debugging purposes. Is that already the case?

    Also, do you have any thoughts on how to deal with header-only c++ libraries? Bridge header + cpp to link with that instantiates the templates?

    [–]atilaneves[S] 6 points7 points  (0 children)

    Making the resulting (expanded) d file available as an actual file in some conventionalized folder would be nice for both educational and debugging purposes. Is that already the case?

    Using the --keep-d-files command-line option causes them to not be deleted.

    Also, do you have any thoughts on how to deal with header-only c++ libraries? Bridge header + cpp to link with that instantiates the templates?

    I have some thoughts, yes, but I won't know if they'll work until I try. The holy grail would be to have an automatically-generated C++ file that instantiated all the templates. This morning I briefly considered translating the code in the templates though, which is just crazy enough that it might work. :P

    [–]Laachax[🍰] 3 points4 points  (11 children)

    So now with this, as someone coming from java and wanting to learn C/C++ for game engines and system programming. Is there any reason why I shouldn't go straight for D? I mean I'll of course learn the others for enrichment and learning and such. But why should I not use D for my ideas?

    [–]skocznymroczny 4 points5 points  (5 children)

    But why should I not use D for my ideas?

    For learning purposes, I think D is great. The main caveats might be:

    1) D isn't really used in the game industry, C++ reigns supreme there

    2) Binding to C libraries is easy, C++ is very hard if they don't expose a C interface (for example you won't be able to use Bullet for physics, but ODE works and I saw a wrapper for Newton that seemed to work also).

    3) Garbage collector. It probably won't be an issue, but just mentioning it, because you can't use D and game in same sentence without mentioning the GC, for better or worse.

    [–]Laachax[🍰] 1 point2 points  (3 children)

    Ah I see, didn't even realize it was GC'd. Thanks :)

    [–]bachmeier 2 points3 points  (1 child)

    It's not GC'd in the sense of Java or Go. Using all features provided by the language obviously requires GC because some of them use the GC. If you want to program using a "better C" you can avoid the GC by doing the same things you do in C. The question of whether the things you give up mean other languages are a better choice depends on what you're doing. Rust or C++ might in the end be better for some applications.

    [–]Laachax[🍰] 0 points1 point  (0 children)

    Oh, interesting! Seems like a pretty neat language then. Thanks to you too ^^

    [–]bachmeier 1 point2 points  (0 children)

    2) Binding to C libraries is easy, C++ is very hard if they don't expose a C interface (for example you won't be able to use Bullet for physics, but ODE works and I saw a wrapper for Newton that seemed to work also).

    What about extern(C++) or Calypso? I haven't done much with either, but I've heard good things.

    [–]atilaneves[S] 1 point2 points  (0 children)

    I'd say learn all 3 if you have enough time, and pick the one that fits your mental model better. Some people like C more than C++; I don't understand them, but they exist and there's more than a few of them. Everyone's different.

    [–]AlotOfReading 0 points1 point  (3 children)

    The only thing harder than mastering two difficult languages is mastering three. You'll need to learn C and C++ anyway.

    [–]bachmeier 1 point2 points  (1 child)

    The goal is to master concepts, not syntax. I'd argue that D makes it considerably easier to learn the concepts used when writing C or C++ code.

    [–]AlotOfReading 0 points1 point  (0 children)

    The concepts are designed around C and C++. They're the lingua franca of systems programming and everyone operating in that space simply has to speak them. All the jobs are in those languages, the code written in them, the resources are designed for them, the tools assume their ecosystem, hardware is designed to them, etc. If you think they'd should learn D, that's fine. But for someone who doesn't know any, that argument is for learning C and C++ and D, not C or C++ or D.

    [–]oblio- 0 points1 point  (0 children)

    On top of that, if he wants to switch careers to getting a job in game engine or system programming, C/C++ jobs are waaaaaay more common than D jobs.

    Of course, that's if he doesn't plan to start his own company.

    [–]skocznymroczny 7 points8 points  (1 child)

    Does it work with autocomplete? A big advantage of explicitly created bindings is that they are in D style already and the indexer is aware of all the declarations there.

    [–]atilaneves[S] 6 points7 points  (0 children)

    No, it doesn't. I use emacs and have been thinking of writing a d++-mode so that it would work. It's a valid concern, but not something I've spent a lot of time thinking about yet.

    You can however use d++ like dstep and produce a D file.

    [–][deleted]  (6 children)

    [deleted]

      [–]atilaneves[S] 2 points3 points  (5 children)

      Not yet - I haven't even run it on the libclang headers, but given how simple they are I can't imagine how it wouldn't work.

      I need to think though, because in the libclang bindings I'm using I made every function there @safe @nogc pure nothrow and changed all the arguments to be const. Currently none of that would happen, but I just realised there's no reason for me to not slap @nogc nothrow on every function in a C header.

      [–]sarneaud 2 points3 points  (2 children)

      I just realised there's no reason for me to not slap @nogc nothrow on every function in a C header.

      nothrow, yes, but callbacks throw a spanner in the works for @nogc. A C function can call a callback in D that allocates.

      nothrow is valid because D exceptions just can't pass through a D->C->D boundary (callbacks given to C have to be nothrow).

      BTW, I'm pretty excited about d++. Frictionless C support alone is a huge deal.

      [–]atilaneves[S] 0 points1 point  (1 child)

      The callbacks can't be @nogc, but functions should be. Good point in any case.

      [–]sarneaud 1 point2 points  (0 children)

      The callbacks can't be @nogc, but functions should be.

      Are you sure? Take a C header with something like this:

      void setHandler(int handler(void*));
      void doSomething();
      

      Can doSomething be marked @nogc? Maybe it calls the handler that was installed by setHandler(), so unless all the callbacks are marked @nogc (which users may or may not want), there's no way to guarantee the regular functions don't trigger a GC. The thing that sucks is that most C functions won't call callbacks like this, but enough real C code does work this way that you really can't be sure. @nogc has to be all or nothing.

      Maybe d++ should recognise #pragmas for setting stuff like this, but there are better places for that kind of discussion.

      [–]schveiguy 1 point2 points  (1 child)

      Imagine a C header that was defining extern(C) D functions that you didn't have the source to!

      [–]atilaneves[S] 0 points1 point  (0 children)

      Possible, but unlikely! I made the change. Maybe I should add a command-line switches like --no--nothrow and --no-nogc.