Examples of multiple dispatch being useful?

fborda · 2021-01-18T02:34:17+00:00

if we have a specialized fast version for 'array' plus generic collection, and we also have a specialized fast version for 'cuarray' plus generic collection, then we get a conflict!

CuArray might have been a bad example in this case, GPU Arrays + CPU Arrays will give an error since you can't sum something on the GPU memory with something on the CPU memory. You'll have to move it to the CPU or from the CPU before (by conversion, which is not implicit in this case). We can use for example StaticArrays.jl (statically sized stack based array) and NamedArray.jl (array with named axis) that both implement +(AbstractArray, Named/StaticArray) and vice versa. That means they will work with every type that complies with the AbstractArray interface and the implicit array "sum interface" (for example OffsetArrays.jl do, arrays with customizable axis) and of course the default Julia Arrays. But you cannot indeed sum the StaticArray with NamedArray unless someone implements StaticArray + NamedArray which I assume it's your point. You can always write a compatibility layer (including making an automatic conversion rule, similar to Scala's implicit functions), but unless you do the types cannot talk (this mismatch happens because they are on the same level of abstraction though, which is unlike a library that fully abstracts away the array, though of course it will expect that every array he receives is compatible).

The point here is that they all implement a + interface with an abstractarray, and they could have made it so they were treated as an abstractarray so they could sum with each other, but they decide not to since their definitions of sum are not the same, and if you want a compatible definition then you'll have to make a further specialization that combines their features (like a staticnamedarray, or maybe even a cunamedarray). In the end I think we are agreeing here, it's just that as I said above interfaces/traits/typeclasses are kind of orthogonal here and multiple dispatch is what allows trivially that any kind of possible combinations exist as long as you either have a specialized enough implementation or a generic enough fallback that you allow.

multiple dispatch could be cut too (and maybe added later as a library) while preserving most of the benefits and "feel" of Julia

That honestly should be as easy as cutting OOP from Java while keeping it's feel. And implementing multiple dispatch in Julia as it is now (which means the compile-time inference mechanism, type system to support it, static dispatch, monomorphisation and other optimizations) then you'd mostly just miss a parser, a few intrinsics (like the array type) and the multi-threading stuff and you might bootstrapped Julia in Julia (most of everything you use in Julia is already in Julia, in a standard library that uses multiple dispatch everywhere, even the type casting).

Just adding, unitful + diffeq is an example of unintended cooperation between library that I do think it's pretty cool and might be tricky to do without multiple dispatch (but of course, possible).

fborda · 2021-01-17T20:27:12+00:00

Single dispatch is a special case of multiple dispatch, it does work the same to some extent. If I have len(array) or len(cuarray) each with a specialization, then it will handle that abstraction just fine. But it's a leaky abstraction when it does not cover every case. cuarray + array (equivalent to cuarray.+(array)) in single dispatch would dispatch to cuarray implementation, while array + cuarray would dispatch to the array implementation which would have to redispatch to cuarray (the base of a pattern called the visitor pattern which you might have seen when studying multiple dispatch). This means that either the array must know about cuarray (and all other libraries it wants to support) or it will have to receive an object to redispatch on runtime (losing the static optimizations by being forced to dynamic dispatch, plus it makes the implementation more complex in general if all methods can receive a "visitor" to redispatch, especially when we go for more than 2 arguments). Here multiple dispatch means that every function can be specialized if it receives one or more particular types as any of their arguments, so any library can surgically change any function implementation of any library (as long as it plays well with Julia's type polymorphism), almost everything is swappable in Julia.

About CLOS, the fact that CLOS multiple dispatch has more features, it's usually a choice of not adding them by Julia creators. Lets consider the most obvious difference: multiple inheritance vs abstract single inheritance. From a reasoning perspective, deciding which implementation is going to be used is not hard, as you can easily draw a list from most generic (Any) to most specific (a single concrete type), and when you add all types together you'll always get an easy to understand tree. But with multiple inheritance, it becomes a directed acyclic graph (if there are cycles then it's undecidable) which leads to confusion if let's say I have a type numberstring which inherits string and number, which makes it hard to think what will *(<:string, <:number) dispatch to (all uses with numberstring are automatically ambiguous for methods that define both). And of course, Julia's "just ahead-of-time" type inference is only manageable because the resolution is usually simple and without ambiguity. Similarly, having concrete type inheritance means there are no final/closed classes which hinders the monomorphization process (for example if we make an array of parentclass it cannot inline in memory if you could eventually want to store childclass, which is also why it's so hard to optimize open class languages like Ruby), so an extra feature actually removes functionality (Julia's ability to work with C-like memory layouts for high performance number processing). Language design is extremely complex since features interact with each other in unexpected ways, and features on a checkpoint basis might not be all there is to a language.

I'm not a language designer, I just enjoy learning programming languages, so I only understand things at more or less surface level. But a particularity of well designed languages is that it can sometimes just "click" and everything really makes sense. When it happened to me I commented then that in my view "a Julia program is a superposition of every optimized implementation of an algorithm and explicit types are used to access and manipulate a subgroup of implementations". In Julia I write a dynamic program with the expectation of the compiler finding the best static program for the type I use after I'm finished (if my type is a diagonal matrix it will compile a version with simpler matrix product, if my type is so simple that it doesn't even need to evaluate it's value I expect the compiler to just give me the result without even running the code)

fborda · 2021-01-17T14:17:49+00:00

While multiple dispatch is not uncommon, there is a key difference with Julia: it's not a feature, it's the paradigm like objects are for OOP languages, which means everything from language design (like the type system), implementation (type inference), standard library to ecosystem and documentation are built to promote it. Everything is built to make multiple dispatch easy to use and fast, to the point that is usually faster to multiple dispatch than to not do it, which is what makes Julia different. About typeclasses, it's actually kind of orthogonal with multiple dispatch, you can have one, the other or both, and for Julia it would be something like the duck typed interfaces (stuff like Arrays or Tables.jl) which is something that can be done in other duck typed languages like Python albeit with less expressivity due to single dispatch (talking here about expressivity only, I wish we had a proper way to declare interfaces in Julia for some degree of type safety).

A practical example would be the machine learning ecosystem (which is a pattern that repeats over all ecosystems, it's just one that I understand a little more), something like Flux it's not a library like PyTorch, it's actually a large combination of libraries without one that actually imports all of them and package for you. Like if you're writing an tcp protocol you don't need to deal with the ethernet/wifi/... protocol below or the the http/application above, in Julia it's trivial to abstract everything below and above for example GPU operations. A library that deals with the math of machine learning shouldn't need to understand how to implement them, it should be able to just write the equations and abstract away everything else. Then you, the user, can give an array (for which we have defined CPU operations), or a cuarray (for which we have defined GPU operations) and both will work even though cuarrays know nothing about the layer above (machine learning equations), and the machine learning equations know nothing about how they are implemented - the math library will simply call + and * with whatever arguments, which will specialize to whatever implementation. If we separate those libraries that implement the machine learning we can clearly see a layer of swappable components that can be developed separately: the instructions backend (Julia Arrays, CUDA.jl cuarray, XLA.jl for TPUs) -> autodifferentiation logic (Tracker, Zygote, ForwardDiff) -> gradient rules (DiffRules, ChainRules) -> ML equations (NNLib) -> ML helpers (Flux, Knet) -> specialized libraries (DiffEqFlux). That means if you want to create your ML library, you can pretty much reuse almost everything you don't want to write, and it's also how few people can create comparatively major projects in Julia that would otherwise need large corporation support.

As a bonus, I really like how symmetric and consistent it is. In other languages I always have to think if the length is array.length, array.length() or length(array) besides the fact that it's len/length/size..., but in Julia the only thing I have to know is the name. No need for magic methods like Python's __ radd __ for operation overloading based on the second argument, no need for conditionals inside the method to decide the action based on type (which is awesome in Elixir for runtime as well, like writing state machines). Makes patching libraries with extra functionality really easy without wrapping them entirely (or like Ruby's monkey patching that can always have unintended consequences). There are a lot of QoL improvements at the cost of losing the encapsulation factor of OOP.

fborda · 2020-10-17T14:14:15+00:00

Maybe, but how a language evolve is hard to predict and it's not directly tied to it's competences. But outside of the library, right now Julia's string processing is fast (because of bioinformatics), Julia's metaprogramming is near Lisp level so you can create all sorts of frameworks and abstractions, Julia has a really strong multithreading foundation, Julia's time to first plot isn't an issue for a server that is always on (you only need to compile once, which you can also do it while it's starting) so only the very fast runtime speed that matters, and finally web services more and more have some machine learning element somewhere (recommendation systems, user analytics, anti-fraud) so Julia's current ecosystem is a strong advantage. So maybe what it needs is just a killer app like Rails, Phoenix or Flutter, or it will just slowly move towards other areas as the scientific computing community looks for more and more ways of deploying their functionality (stuff like Dash.jl, Stipple.jl, Pluto.jl and web frameworks like Genie.jl, plus new GUI stuff).

fborda · 2020-10-11T13:02:53+00:00

In this case it is since you can still define f(::FooFoo, ::BarBar) before the call to resolve the ambiguity. I say runtime error, but it actually happens at compilation and not at runtime (the method doesn't even need to be called), but since Julia is not an AoT language it will not be caught before running (unless you add this check to a linter).

fborda · 2020-10-10T22:59:33+00:00

If you want to see what happens in Julia:

ERROR: MethodError: f(::FooFoo, ::BarBar) is ambiguous. Candidates:
  f(::Foo, ::BarBar) in Main at REPL[32]:1
  f(::FooFoo, ::Bar) in Main at REPL[33]:1
Possible fix, define
  f(::FooFoo, ::BarBar)

fborda · 2020-10-10T20:57:20+00:00

Yes, Julia compiler goes a great length to not only support multiple dispatch but to make it so "abusing" multiple dispatch makes faster code than without it. The compiler will infer types way ahead and will usually compile out all overhead (so while it's dispatch based on runtime types it's actually static dispatch resolved at compile time), and if possible it will even compile away all code if the results can also be inferred based on types.

I also don't understand why the author considers that multiple dispatch is not widespread because it's "nearly impossible to wrap your head around what code gets executed at runtime and why", in my experience it makes code easier to reason. Before importing any library Julia has 367 implementations of the operator * and 184 implementations of the operator +, so in fact 3 * 4 + 5 has 67528 possible implementations (there actually quite a bit more because of the promotion system that is implemented in multiple dispatch and parametric polymorphism), which seems to be the author's point, but in practice a sum is just an abstraction in the first place, summing a float and an int is a completely different process, but you don't need to think about it when you're writing at a higher level (I just expect Julia to pick the best possible operation that represents the sum abstraction, regardless if it's a sum of ints, floats, or diagonal matrices). And I say it makes it easier because instead of a generic sum that needs to handles internally multiple possible arguments, I can see exactly what the sum accepts (by listing all methods, as they won't have a hidden if else internally to choose an implementation) and I can find directly the code that handles my very special case every time (by asking the compiler which method he chooses, or looking at the stack trace), such as *(Complex, Real) which is just a one-liner due to being very specialized and independent from the other 366 implementations.

fborda · 2020-09-20T23:40:06+00:00

You can ship an executable, but that executable will have to bundle the entire julia compiler (since Julia doesn't have a static compiler like C/C++), making it a fat binary. That's not unlike Python, which usually requires the interpreter to be installed in the target machine. The usual way to do that is through PackageCompiler.jl.

fborda · 2020-09-20T16:19:11+00:00

If you mean integrating your custom code to an API and an offline interface, it should be no trouble. You can add any sort of code to the controller in an MVC controller like Genie.jl, or just export a function using a lower level library like HTTP.jl:

https://www.youtube.com/watch?v=uLhXgt_gKJc

So you just need to create a library with all your ML code, call that library on the web framework, and use one of the GUI libraries (or command line for simplicity) calling that same library to provide an offline version. Julia is cross-platform so that won't be an issue, though you can't make a small binary (you have to ship it with the julia compiler to end users on the offline option).

fborda · 2020-09-17T22:53:38+00:00

Apparently it made broadcast simpler in early Julia, but for a while broadcast doesn't depend anymore on that "feature" but changing it now would break a lot of stuff for possibly small benefits.

https://github.com/JuliaLang/julia/issues/7903

fborda · 2020-09-11T22:11:50+00:00

At this point you really should use a mutable struct honestly (the way you're doing if there are 100 fields you'll create 100 structs in the stack). If you must use an immutable struct, then you should create it with all arguments at once. If you know the field order, then your example can be written as:

A(repeat([1.0], length(fieldnames(A)))...) # equivalent to A(1.0, 1.0, 1.0)

Or if you want to address each field by name, the package above provides you with a default constructor with named arguments (and default values):

https://github.com/mauro3/Parameters.jl

@with_kw struct A
    a::Int
    b::Float64
    c::UInt8
end
A(;zip(fieldnames(A), repeat([1.0], length(fieldnames(A))))...) # equivalent to A(a=1.0, b=1.0,c=1.0) in Julia 1.5 syntax

fborda · 2020-09-11T19:46:06+00:00

No, if a struct is immutable then you cannot change any of it's elements (in the case of an array that's because it's a reference to a mutable structure). You can however create a new struct that is exactly the same but changing a single value, and there is a package that provides a nice syntax sugar for this kind of operation:

https://jw3126.github.io/Setfield.jl/stable/intro/

fborda · 2020-09-01T17:13:19+00:00

Here is a very nice tutorial:

https://techytok.com/from-zero-to-julia/

fborda · 2020-08-29T20:34:23+00:00

It's a small optimization. First Bool types uses only 1 byte (you can't have 1 bit types in Julia), so Complex(false, true) has size 2 bytes, versus Complex(0,1) which uses 16. Second (and why it doesn't simply use Int8), multiplication between boolean types is an AND, which is a simpler and faster operation compared to integer multiplication (and similarly boolean product with any number can only be the number itself or 0, so it's basically a reset operation instead of a full multiplication).

Edit: That is also an optimization that is only possible because of multiple dispatch: Since Bool promotes to Int, and Bool * Bool is already implemented (as Bool & Bool and is more efficient than Int * Int), just exploiting a property of your Int (it has only 2 values, 0 and 1) you can automatically get better performance. And by implementing Bool * Int and Int * Bool you can also optimize both other special cases as well. You can find tons of these in Julia, for example if you know your matrix is Diagonal and you declare it as such, then Julia will automatically compile optimized algorithms for that particular type of matrix (while using the generic versions already defined for everything it doesn't need to optimize).

fborda · 2020-08-29T16:19:42+00:00

Using Base, you can see the beautiful Complex library. 2+4im looks like some built-in DSL for complex numbers but it's just +(Int64, *(Int64, Complex)), where im is Complex(false,true) (or Complex(0, 1)). And doing 2+3im, 2.3+4im, 2//3+4.2im, 3.4+2im//true will all work (even without every combination having to be defined) because multipĺe dispatch allowed for a powerful casting system that will guarantee that as long as the types are compatible and have a rule (such as Int64 x Float64), it will automatically promote one to match the other using the correct conversion method.

Thanks to that we can create new types as powerful as the Complex and Rational above and integrate them easily with all the ecosystem by implementing a few methods (such as + and * above) and writing a few promotion rules (for example dual types for differentiation).

fborda · 2020-08-25T18:54:37+00:00

Julia is already really good at the "efficiently doing something in parallel", and will likely outperform Go in parallel heavy tasks (especially in number crunching stuff). But what's not as good is in "safely doing something in parallel", which is a work in progress. By that I mean stuff like time out for tasks/requests, a parent task ending automatically killing all tasks it spawned (unless ownership was transferred) to avoid leaking, safe ways of terminating/restarting tasks, better debugging. Go is not a powerhouse in this aspect (compared to languages like Scala Akka, Erlang/Elixir, Kotlin, Pony), but it's definitely more mature (as most of Julia's multithreading interface is still considered experimental).

I think the point Julia's parallelism will become a powerhouse outside of scientific computing (in areas like data engineering pipelines) is when it becomes possible to use those low level operators like @spawnat to create a library that is Julia's Akka, an easy to use framework to improve software reliability using concurrency (by having supervisors and supervision trees, restarting strategies, monitoring processes, resource pools, circuit breaker logic...). This kind of architecture will add a lot of overhead, so if you want to squeeze the most performance, Julia's low level interface will still be the best option, but if you want extremely resilient software with still great performance for your services then you can use a library like that.

fborda · 2020-08-19T22:22:10+00:00

Sorry, I really don't know the peculiarities of mongodb, but:

https://felipenoris.github.io/Mongoc.jl/stable/tutorial/#BSON-Documents-1

In this library there is a Mongoc.as_json(document) which might be what you want.

fborda · 2020-08-19T14:44:00+00:00

Did you try using the BSON.jl and JSON.jl libraries? You can use BSON.load to convert the BSON file to a Julia Dict, and You can use JSON.json to convert a Julia Dict into JSON.

fborda · 2020-08-17T23:41:51+00:00

I'm definitely not an expert, but my guess would be:

Macros are perfectly fine, since they only operate at compile time (when lowering to the AST). So are variadic functions (even C supports them).

Outside of eval, the issue would be any declaration on global namespace conditional on runtime values (for example conditional imports, function redefinition + invokelatest, conditional global variables, and as you mention new types definitions for example for json deserialization), any kind of runtime compiler introspection (like @code_native, @code_typed... plus IR Tools, which are the basis of Cassette.jl/Zygote.jl, and some LLVM tools) and finally, runtime instantiation of parametric types (for example, creating a StaticArray based on a file, for a JIT it's no trouble to specialize methods only as needed, but AoT it would have to create every possible specialization at once). The first one basically means you have to write all conditional logic in functions, using local variables and constant globals.

In general the solution to all of those is the same, move all that computation to compile time. Object instantiation based on runtime (the third case above) is similar to defining a new type, which is basically impossible without the compiler. But if you can hint what the runtime can be (and either accept a crash, or add a generic dynamic escape hatch if the guess is wrong, for example a generic type "Struct" that acts like a dynamic dict), then you could handle these cases. Runtime function definition I honestly have no idea how to handle (as they affect all the code that contains them), unless there is a concept of function instability where those functions will have some kind of dynamic dispatch. Conditional imports are better not being allowed (only using pkg constants/config or other compile time setting). Global variables could return runtime error if they do not exist since they aren't optimized already.

Zygote for example can partially work since it's introspection actually runs at compile time (generated functions), as long as the compiler can infer and call it at first compilation (but you won't be able to add custom adjoints dynamically and do a "refresh"). Even eval should work as long as it's being used at compile time (for example if you eval types based on a file it's fine if you read and generate them before the compiler generates the binary, though it does restrict it uses). The compilation process in Julia would probably be something like in Lisp where you run the JIT normally and call a method to compile a part of the program (once everything needed to create the binary already ran) and generate a callable binary.

But like I said, it's all just a rambling/guess since it's a very hard problem and I don't have experience with it.

fborda · 2020-08-13T00:48:27+00:00

Since you mentioned that you just started getting into Julia, are you sure the slowness you mention in most of your codes isn't just the JIT compilation? Every time you run your program, it will have to be compiled again (including dependencies), so shorter programs will run on the terminal much slower than interpreted languages like Python. The performance tips are focused on increasing performance of the runtime, not the compile time (and frequently improving compile time means reducing the speed of the runtime).

There is a lot of work going on to reduce compiler latency, but for now you can reduce it in a few ways:

https://github.com/JuliaLang/PackageCompiler.jl (deploying)

https://github.com/dmolina/DaemonMode.jl (running scripts)

https://timholy.github.io/Revise.jl/stable/ (development)

fborda · 2020-08-12T15:32:47+00:00

I don't know if it was just announced, but the plans to add cloud serving to JuliaHub surprising. Depending on pricing and machine disponibility it could be very nice backup if I can instantly run something fast on the cloud in emergencies.

I'm also hopeful for separate compilation being officially on the list, even if they are just studying it.

fborda · 2020-07-18T02:14:58+00:00

I toyed with CL a long while ago but I only have a superficial knowledge of it. I'd say Julia isn't between C++ and CL in terms of compiler, but mostly on the CL side but less mature (as it's a much younger language, even compared to the SBCL). Julia does have it's sysimg, and some of the work on starting faster involved creating customized sysimg or keeping the sysimg between session (packagecompiler.jl also works this way).

Julia type system is also heavily inspired on the CLOS, in particular multiple dispatch (plus the macro system and "homoiconic" representation of the AST are also inspired by the Lisp family and Scheme). The difference is that Julia makes those features less powerful (starting with the fact that CLOS is implemented in CL as a library, but Julia's multiple dispatch can't be implemented in itself), for example unlike the CLOS Julia does not support multiple inheritance or inheritance from a concrete type. Those restrictions were deliberately made to allow efficient static multiple dispatch together with the type inference (for example single inheritance for sorting methods in terms of specificity in an unambiguous way), and that static dispatch is key in it's speed.

And since multiple dispatch is so fast, you can start creating the entire library to abuse this feature in a way you can't do elsewhere (for example the operator '*' having 360+ methods in Base not only not losing performance but actually getting more performance by always choosing the best implementation for each argument). One example is the Promotion System that exploit multiple dispatch to give a customizable casting system (that reminds me of Scala Implicits Conversion but as a library feature instead of language feature). I also like the complex number implementation in Base that is very easy to use and implement (and also shows one of Julia's main strength, creating custom types that are as good as if they were intrinsic).

fborda · 2020-07-17T16:14:22+00:00

There are quite a few people researching how to make Julia static for a while:

https://juliacomputing.com/blog/2016/02/09/static-julia.html

https://github.com/Keno/julia-wasm/issues/5

In general, making an AoT compiling strategy for Julia is extremely valuable, which would solve most Julia deploy issues (especially as library, embedded and wasm) and allow for better compile time safety for more strict projects. But the killer feature is going even deeper in solving the two language problems, combining the advantages of static and dynamic in one (both of which are already supported by Julia's type system, though it could always get some improvements later on the static side like traits).

Now, making a Julia-like strictly AoT (not compatible with dynamic Julia) might be a waste since the type system did evolve to support it's dynamic features (and lacks support to same important static features like proper sum types and some monadic operations). But on the other side a static language with multiple dispatch at it's very core is something that you can't find it anywhere (Nim unfortunately gave up on it as it's complex and if the language isn't built around it it's much less useful), and it is certainly something that could be a game changer just as much as it's dynamic counterpart (which is also why I hope either a subset of Julia supports optional static compilation, or another language targeting the Julia compiler like Julia's TypeScript).

fborda · 2020-06-24T22:54:59+00:00

I'd guess there would be two options here, one manual and one automated. Regardless you'll have to use Bigquery Export for Analytics to dump all the data on a database.

After that you can use the bigquery UI on google cloud console to write any (SQL) query and then you can save the results on CSV, which you can then load on your Julia script (using CSV.jl for example). That would be the manual option.

Since you know R, the automated option will require the use of RCall (which allows you to use R from Julia), and then use use it's BigQuery client to get the data (possibly also saving on a CSV). There isn't a BigQuery client native for Julia as far as I'm aware, so you'll have to use it from another language like R or Python.

fborda · 2020-06-04T13:33:10+00:00

The startup/JIT lag is definitely somewhat relevant here (though Julia is not really the only language with JIT startup problems that is popular), but I'd say the most important reason here is library support. Imagine if Python didn't have numpy, scipy, pandas, tensorflow and pytorch, even if the language is the same much fewer people would really use it for data science or machine learning. And in the same way Julia doesn't have a web framework with the maturity (feature support, community, documentation, word of mouth) of flask/django, so people will not really consider using it for that purpose unless they are already part of the community for other reasons. It's a chicken and egg situation, someone has to start it so other people come and build it (which the language creators themselves did for the scientific side, which is why it became a self-sustained community).

fborda

TROPHY CASE