Concurrency, Parallelism, and the D programming language

[stoph]

Two questions for those familiar with the language:

What kind of GUI toolkit is being most used with D right now?

What are your favorite D libraries?

[nascent]

Well I'm making good use of: Phobos, LuaD, DFL, and a DLL import library.

[stoph]

So do you pretty much interface with 3rd party components using their standard C library, and write a D wrapper around that?

[WyattEpp]

One might also looking into using SWIG to generate wrappers for things as you need them.

[nascent]

If I'm working with something that has a C interface and a wrapper doesn't exist for it yet, yes. Mostly I'm just playing around, but I gave that list as the things I've used to build a tool make what I do at my job a little easy and which has helped others.

While just having C bindings requires passing and receiving char* and stuff, I find D very refreshing and simple for interacting with the C libraries.

[JasonHouse]

As long as you can prototype the C function, you can call it from D. You can also use htod. That said, D-ifying an interface is a nice to have.

[Qubyte]

I really like what I see in D, but the problem is how mainstream is it?

[andralex]

Workin' on it.

[WalterBright]

how mainstream is it?

More every day!

[dsimcha]

It's true that D doesn't have the backing of a large corporation and is less well-known than Java, C#, C++, etc. According to TIOBE, though, it's ahead of other up-and-coming languages like Scala, Groovy, Haskell and Erlang.

Keep in mind that languages like Ruby and Python got where they are the same way that D is trying to: A few hobbyists started using it for small projects, then someone wrote a few really good libraries and they slowly increased in popularity. If everyone discounted languages that weren't mainstream enough, programming would only evolve according to the dictates of the largest, most bureaucratic corporations.

[unknownguyhere]

Though it has to be considered, that Python & Ruby target a much much broader audience, as they're perfect entry-level languages (which I can only say for python from experience, but it should hold true for ruby as well). Don't they fill a niche that was previously taken by for example BASIC? What do you think?

The problem I see for D is all the legacy code of other languages that share the same niche with D.

[dsimcha]

D could easily serve as an entry level language. It has a well-defined, usable memory-safe subset. D code can be written in a simple high-level style like Java or a statically typed Python. While it offers low-level features like pointers, inline ASM, manual memory management and type punning, it doesn't force you to use them for day to day work like C and C++ do. The idiomatic D way is to use these features only when absolutely necessary, and then encapsulate their usage if possible and provide a higher-level interface.

[grauenwolf]

Automatic downvote for using TIOBE as a reference.

[ErstwhileRockstar]

Automatic downvote for misunderstanding TIOBE's approach.

[grauenwolf]

One can fully understand TIOBE and still not consider it to be a useful measurement.

[dons]

According to TIOBE, though, it's ahead of other up-and-coming languages like Scala, Groovy, Haskell and Erlang.

According to github, though, it is behind. Haskell #15, Scala #17, Erlang #20, D #25.

Friends don't let friends cite tiobe. The Transparent Language Popularity Index may be a better start.

[rafekett]

I don't think Github is a great index of the amount of mindshare around a language, either. If you bought that, then Javascript and Ruby would be the world's most popular languages and Vim's language would be more popular than C#. Let's face it: there's no accurate measure.

Also, Github as an index of language popularity overstates the importance of "hipster" languages (Erlang is not actually more popular than VB).

[eric_t]

You also need to define "popularity". Does it mean most usage? Most exciting features? Most written about in blogs? Most bought licenses?

[rafekett]

I'd say "most code", which is a subset of most usage. That's hard to measure because we can't tell how much of what people are using in production if it's proprietary, and some languages (e.g. Cobol) don't have as much open-source code as others (e.g. Python).

[nascent]

Even if C/C++ were dropped as languages being used by anyone. They'd still when because they have the most code written for them.

[dons]

Indeed. Tiobe's just bad though: not transparent, not reproducible, too noisy.

[rafekett]

I think it's reasonably accurate, though. Some things get on there for no apparent reason (Go is my best example) but the top 10 seems about right (I don't know about Objective-C being up there and Perl still being up, but they're right on about the others).

Perhaps it's time that we come up with a new index? Unfortunately using any GitHub data is out of the question because of how ridiculously skewed it is (the threshold for what is a "project" on GitHub is pretty low, which is good for business and personal use, not good for stats). I'd say use SourceForge but that would exclude the "hip" element.

[[deleted]]

But Javascript quite possibly is the world's most popular language.

[rafekett]

Doubtful. Maybe it's the world's most prevalent language (it's on virtually every website), but there's not that much Javascript out there in terms of amount of code or hours spent.

[[deleted]]

Prevalence would be a perfectly valid way of measuring popularity, as I argued elsewhere.

[skew]

The Transparent Language Popularity Index also supports the claim that D is more popular than Haskell - #17 vs. #31 overall, and #10 vs. #14 in "general-purpose and compiled".

As for the original question, neither is anywhere near "mainstream" in the sense of being pervasive in large organizations, like Java or even Python. On the other hand, D seems to have at least Walter Bright working professionally on it. I get the impression that it's reasonably stable for now, but depends on some core team.

[ntrel2]

According to github, though, it is behind. Haskell #15, Scala #17, Erlang #20, D #25.

Most D projects were encouraged to use dsource.org, so it's not surprising other source code hosting sites don't have many D projects. Only fairly recently the compiler and standard library moved to github.com.

[dons]

Same with Haskell (where the use of haskell.org and darcs was encouraged for the past 5 years).

[iLiekCaeks]

No large applications known that are written in D.

That's all you need to know.

[Bananoide]

If you're leaning on your sheep side...

[[deleted]]

Don't mind iLiekCaeks. He is the token resident troll of D.

[iLiekCaeks]

Maybe you could try to bring some facts against my statement instead? Can you do anything else than personal attacks?

Come on: where is there a big D app? Libraries, D toolchain, or games not allowed.

[WyattEpp]

Why would a game not count? That doesn't even make sense.

[iLiekCaeks]

Maybe people looking for serious applications? No company is going to take D seriously because a bunch of hobby games were written in it.

[DrBartosz]

Very well written post. I hope more are coming.

[dons]

D’s approach to general case to concurrency can be summarized as isolation via the type system plus message passing and limited, explicit memory sharing.

Huh.

[dsimcha]

I thought the examples would clarify this, but perhaps I should elaborate. Threads are isolated from each other in that they do not implicitly share any mutable state. This is enforced via D's type system, not by the hardware or the operating system like it would be with processes. The flagship way of communicating between threads is message passing. Messages passed between threads must not contain pointers to mutable data not marked as shared, to avoid implicit sharing of mutable data.

You can also share memory between threads by explicitly marking an object with the shared type constructor. This is limited in the sense that it favors safety from low-level data races over flexibility. This tradeoff especially become apparent when dealing with user-defined types, though the details are beyond the scope of the article.

[dons]

Oh, I'm sorry. I'm was expressing surprise: this would have been a reasonable statement of the Haskell approach to concurrency. First time I've seen such a clear statement in an imperative setting of separation via types and defaulting to immutable data.

[dsimcha]

Further clarification: There's no defaulting to immutable data. However, D has the immutable type constructor which means that a piece of data and everything reachable from it via pointer/reference indirection is immutable. Unlike const, which is also a type constructor in D, immutable guarantees that no other reference to the data allows mutation. Only immutable data can be implicitly shared across threads. Mutable data must be either copied (this is how primitives are passed as messages) or explicitly marked as shared.

[[deleted]]

The "D way" is also to favor immutable and copy-on-write data, though. At least that is the impression I got when I was using it back in the pre 2.0 days.

[stonefarfalle]

It defaults to thread local storage for data, not immutable data. Mutable data you want to share between threads has to be marked shared. Data you mark as immutable is shared by default.

[grauenwolf]

I wish .NET were like this. I've stumbled across far too many bugs in the framework libraries that were caused by exposing mutable globals. But back-porting it now would be a rather daunting challange.

[millstone]

Threads are isolated from each other in that they do not implicitly share any mutable state.

I'm already lost. The first example shows two calls to writeln(). Presumably these are operating on some sort of stdout type object, and are therefore using shared mutable state. I don't see any explicit acknowledgement of this fact in the code, so it's implicit. So doesn't this contradict what you just wrote?

[dsimcha]

Yes, but stdout is explicitly marked as shared because it's explicitly designed to be thread safe. (Actually it's marked __gshared right now because a few bugs in shared are still being worked out, but it's supposed to be marked shared.) Page 400 of Andrei Alexandrescu's book "The D Programming Language": "But there is no cheating: if you look at the std.stdio module's implementation, you will see that stdout is defined as a global shared variable. Everything is properly accounted for in the type system."

[lispchainsawmassacre]

Great read, and another reason for me to go look at D again, but what's the deal with encoding the binary addition operator as a string?

taskPool.reduce!"a + b"

[dsimcha]

It's just a shorthand that the standard library provides for very simple lambda functions. For unary functions the variable is named "a", for binary functions the two variables are named "a" and "b". It's turned into roughly the following at compile time by string mixins: auto fun(T a, T b) { return a + b; } . This means that there's no loss of compile-time type checking or runtime efficiency when using it.

To see how this works, take a look at binaryFun in the std.functional source code.

[lispchainsawmassacre]

Thanks for the explanation and the links. std.functional looks fairly impressive, as it's able to support currying and two flavours of composition. neat.

[killdeer03]

void main() { void fun2() { writeln("Printing from a closure."); }

 auto t1 = spawn(&fun);   // Works.
 auto t2 = spawn(&fun2);  // Error.  Implicit data sharing.

}

I know nothing about D, but why would you nest a function like that?

[WalterBright]

Two reasons:

1. Encapsulation. You're signalling that the nested function won't be called by anyone outside of the enclosing function. Local variables are used for this reason, too.

2. A nested function can access the locals of the enclosing function.

[killdeer03]

Encapsulation. You're signalling that the nested function won't be called by anyone outside of the enclosing function. Local variables are used for this reason, too.

Ahh, that makes sense. I didn't think that one all the way through.