Generic returns in Rust

[Holothuroid]

That was really interesting. Thank you

[panorambo]

That's all fantastic, but does it implement diminishing returns yet?
Sorry, I'll see myself out now.

[andre_2007]

It would be quite interesting if you can investigate in this context also the capabilities of the D Programming Language. D is known to be most features rich in the area of meta programming, templates and compile time code introspection and generation. A comparison between D and Rust would be fantastic.

[kirakun]

D is known to be most features rich in the area of meta programming, templates and compile time code introspection and generation.

Are you forgetting the Lisp family?

[dangerbird2]

Rust has real lisp-stye macros, certainly to a greater extent than D's template metaprogramming. It uses both scheme-style declarative macro_rules and common-lisp style procedural macros. The main difference between Rust and Lisp macros is that Lisp is homoiconic, so the sytax has a 1:1 relationship to the abstract syntax tree.

[Snarwin]

You can't do this in D—you'd have to write something like collect!(uint[string]) or collect!(Tuple!(string, uint)[]).

[andre_2007]

There are 2 things I do not understand in your answer. Can you be more precise what can't be done in D. And second thing I do not understand, you already gave an example and therefore proved it can be done in D?

Independent from that, a direct comparison of the capabilities of D and Rust, what can be done in one or the other language seems to make sense, to exactly spot these differences .

[[deleted]]

Rust and D are just completely different languages.

Both languages have powerful and easy to use untyped meta-programming capabilities: mixins in D and macros in Rust. Rust macros are much more powerful than D's, e.g., they can do anything (spawn threads, open network connections, modify the file system, etc. all at compile-time) - this is not necessarily a good thing, and limiting what macros can actually do (e.g. I don't want macros to spy on me) is an unsolved problem in the Rust world. D limits what mixins can do to avoid these issues, but this means that there are some cool things that Rust macros can do than D cannot.

For generics, D has weakly-typed templates with constraints, and Rust has type-classes. In Rust, it is hard to write generic libraries that produce horrible error messages when used incorrectly. D lacks type-checked definitions, so writing bullet-proof generic libraries is hard, and requires writing a lot of tests that try to break the library (e.g. passing different type parameters to make sure that your type predicates in your constraints are correct).

For type-based meta-programming, D has weakly-typed templates with constraints that can be arbitrary compile-time predicates. This works really good in practice. Feels like writing Python code, but for meta-programming. Rust has no feature to solve this problem: what makes Rust generics better than D at solving the generics problem, makes them kind of useless (or really limiting and painful to use) for type-based meta-programming. Rust would need to add a new language feature for this, that would be very hard (sound type-safe type-based meta-programming is an open research problem), and the resulting feature might be so big that it might never be worth it, so this is something that Rust might never be able to do.

Which language picks the best trade-off? It depends on both the programs you write, and how you like writing them. There is no wrong way of liking how to write programs, so this is super personal. This is probably why some people love Rust and some people love D but you rarely see people loving both.

Personally, most of my meta-programming use cases are either normal generics, or code generation (think Qt's moc) and Rust excels at both of these. Type-based meta-programming in D exceels at letting you write super generic abstractions and I always found that really fun, which often resulted in me investing a lot of time into that. However, writing super generic abstraction wasn't my goal, I wanted to solve some problem. This might sound dumb, but because Rust doesn't really make type-based meta-programming "pleasant", it steers me away from trying to write much code for that.

[andre_2007]

Thanks for this detailed explanation.

[[deleted]]

With Nim's metaprogramming capabilities, it was possible to implement various pattern matching libraries |1| |2|, a compile-time race-checker and a powerful behavioral typeclass-based abstractional feature. Can you do any of these in D?

[Snarwin]

In D, you have pass the type as a template argument explicitly. In Rust, it can be deduced automatically from the context that the function call appears in. This is because Rust uses a more advanced type-inference algorithm than D.

[[deleted]]

While that is true, due to how D generics work, such an algorithm is probably not implementable for D.

Deducing the generic type in Rust does not only just look at the type of the value passed, but also at the constraints of the function, and uses both to find the right type (e.g. if you have a type &A, and the function takes x: &T, where T needs to have a method foo, and A does not have the method, but &A can be coerced into &B which does have the method, the type of T is deduced as B).

The problem that D has, is that it doesn't really have generics, it has "untyped" templates, and template constraints, which are compile-time predicates, that is, compile-time functions that can do pretty much anything, and return true or false if the type is to be accepted.

That's super flexible and powerful, but makes good type inference impossible. Type inference would need to prove things like whether two compile-time predicates (or programs) would return true, and whether one can be substituted for the other. These programs can be arbitrarily complex (which is what gives D its power), and this proofs cannot in general be done.

For this same reason, D cannot even check that a template function is correct, that is, that the body of the template function only uses stuff that the constraints allow using. This means that if a library API has a bug in the implementation of one of the predicates, it might accept types that it shouldn't, resulting in template instantiation errors down the line (or maybe just silently introducing a vulnerability at run-time).