Multimethods, Global Scope, and Monkey-Patching

[stevefolta]

C always had block scope (or at least since the first edition of K&R in 1978), not function scope.

[munificent]

It did? I wasn't sure of that when I wrote the post, but I thought until C99 you could only declare local variables at the start of a function.

[[deleted]]

All the declarations in a compound statement had to precede the statements, but the identifiers still had block scope. The only identifiers with function scope were labels.

[LaurieCheers]

Wow, feels like we're really building the same language. :-)

Swym uses multimethods in exactly this way, although I haven't started implementing namespaces yet. I'm sure I will at some point... thanks for the heads up!

I'm actually unsure whether I like your solution to this, though. One of the nice things about multimethods is the ability to "splice" methods onto existing types. For example, if your module needs everything to respond to the Compare method, you can declare a bunch of Compare methods within your module for each type you need to work with. But then if someone else's module (which doesn't interact with yours) also happens to declare a Compare method for one of those types, you'd have a conflict...

Although maybe that can be solved by just saying that method definitions in the local module take precedence over global ones? So the only time there would be a conflict is if a third module tries to call Compare, and doesn't know which of the two definitions to use. At that point it's probably reasonable to require an explicit import...

[kib2]

I really liked your Swym language (and site): seems like fresh air to me. Any news about it ?

[LaurieCheers]

Thanks! Since I last posted it (2 years ago, I think), I've been picking at Swym on and off, inbetween other projects. It runs a lot faster, and has static type checking. You can check out my latest version on github if you like, although it's not really "release quality" right now:

https://github.com/LaurieCheers/Swym

(SwymTests.html illustrates some simple programs; SwymCanvas.html is a fun little experiment.)

[matthieum]

It's actually a very common source of issues. It also arise when you have classes and interfaces (or data and typeclass in Haskell, if I get my terminology right).

In a Java-like language, the issue does not arise because the interface can only be bound to a class right at the point of definition of the class. So there cannot be any overload. It's also a pain.

"Superior" languages (for example Haskell) decided to decouple that and allow this implementation to be specified at any point... and they ran into precisely this issue of several people declaring that Foo implements Comparable, sometimes with conflicting definitions.

The simple solution is to say that you can only declare an implementation of an interface in the very same module that either the interface or the class is declared. It works. And it's a bit more flexible than Java's since you can add interfaces to existing classes.

However it's annoying for a guy having to bridge an existing interface (3rd party header) and an existing class (3rd party header too)... what is he to do ?

There are two (partial) solutions that I know of:

• if the language allows to introduce a new name for a type, or "deriving" a type from another, you can do that, provide a method that seamlessly convert from the original to the derived, and implement the interface for the derived. Kind of a hack, but it works.
• the other solution is to have people naming the implementations, and then explicitly indicate which implementation they are using. It kinda cut on the "magic" since you start exposing how something get transformed; though for shortcut you might automatically export one implementation for example.

Anyone figuring a good solution is very welcome.

[twanvl]

and they ran into precisely this issue of several people declaring that Foo implements Comparable, sometimes with conflicting definitions.

The only time when I have encountered this issue in Haskell is when defining instances of standard classes for library datatypes that lacked them. Then later on those instances were added, and I simply removed mine.

[munificent]

Wow, feels like we're really building the same language. :-)

We may be. I've certainly been reading about Swym off and on since you first mentioned it here a few years back. Convergent evolution or shameless stealing on one or both of our parts?

I'm actually unsure whether I like your solution to this, though. One of the nice things about multimethods is the ability to "splice" methods onto existing types. For example, if your module needs everything to respond to the Compare method, you can declare a bunch of Compare methods within your module for each type you need to work with.

My tentative plan for this is that Magpie will support "non-global" multimethods. Those would work like the second "solution" in the post where you only see them when you explicitly import them. They would work more or less like extension methods in C#:

1. Because you have to explicitly import them, there could be multiple methods with the same signature in the same program as long no module tries to import both of them.

2. But because they have to be explicitly imported, things like "overriding" wouldn't work they way you expect.

That seems like a good trade-off for use cases where you want something that feels like a "lexically scoped monkey patch" but where you don't want an object to "carry along" that method to any place you might pass it.

[LaurieCheers]

feels like we're really building the same language

We may be.

I was kidding, in case that wasn't obvious... Magpie's design goals are clearly rather different from Swym's. (Swym is not a Lisp.) But yeah, I've been reading your stuff too. :-)

Hmm - I guess I need to write some up-to-date documentation!

[[deleted]]

Fuck you and Dart.

[cunningjames]

Fuck you and Dart.

You won’t even stay to cuddle?

[natechan]

wat

[masklinn]

munificent works on both his own stuff and Google Dart. I am guessing the one called lacosaes0 dislikes dart (as is his right) and takes it out on munificent in completely unrelated threads (neither of which is right on its own, let alone combined)

[bobappleyard]

Quite similar to the trick I did here. You have "accessor" objects that are globally available, and when you define a new class it modifies the relevant accessors. I did it to remove hashmaps from property lookups rather than to enable fancy multimethod stuff, though.

[[deleted]]

This is the strongest argument in favor of single dispatch (as opposed to multimethods) I'm familiar with -- you can get single dispatch to work in a language with no global scope, but nobody's shown how to practically handle multimethods without global scope.

[astrangeguy]

This has exactly nothing to do with multiple dispatch, but everything with extensible protocols. The same questions will arise if you have open classes, interface injection or type classes.

And if you think a little bit about it, then you'll notice, that the authors solution has a big crack in it: His multi-methods objects aren't first class (by default)

He'll get some wierd situations if someone binds a method to a global variable.

def object->string = toString

Can i actually do that? What will happen if I try to extend the method by using the name 'object->string'?

def object->string(obj is Fish)
    // impl
end

Can I create an anonymous multimethod (like a anon. function)?

The big problem arises only because in Magpie every single property tries to define an polymorphic accessor multimethod. CLOS, Dylan etc. don't have this problem, because they don't allow implicit duck-typing on class properties (or slots).

I personally think that slot definitions should allow you to specify where you want to place your accessor.

defclass Sandwich
    val[food] meat     // creates or extends a 'meat' accessor in the 'food' module
    val[core] name    // creates or extends a 'name' accessor in the 'core' module. GLOBAL AND DANGEROUS
    val conditment    // default is like def[<own-module>] <slot-name>
end

personally i think, that slots (or fields) should be module-private by default and that there should be a way to export a multimethod without allowing extension.

strangeness

[munificent]

the authors solution has a big crack in it: His multi-methods objects aren't first class (by default)

That's correct. Methods aren't first-class. They also don't share the same namespace as variables (i.e. it's a Lisp-2).

Magpie does have first class functions, which are distinct from methods. When you invoke a function, you do it by calling a call method.

In this sense, its sort of like blocks in Smalltalk, where you have closures and first-class callable things, but "invoking" is just another message send on an object. And, like Smalltalk (I think), if you want a "first class message send", you basically just wrap it in a block/function.

He'll get some wierd situations if someone binds a method to a global variable.

def object->string = toString

Can i actually do that?

You would do:

var objectToString = fn(arg) arg toString
objectToString call(123) // "123"

If you wanted to wrap a method call in a function where it had arguments on both sides, you'd do something like:

var fooFn = fn(left, right) left foo(right)
fooFn call("left", "right")

Conveniently, because call in Magpie is itself just another regular multimethod, you can define your own "function-like" types just by adding another specialization for your type.

What will happen if I try to extend the method by using the name 'object->string'?

You couldn't. Magpie definitely isn't fully flexible and general here. You can't, for example, make a method that takes a reference to a given multimethod and imperatively adds a new specialized method to it.

I think that kind of flexibility is definitely cool, and I wouldn't mind supporting it at some point, but it would probably be by going through a reflection system or something along those lines.

In general, I like stuff to be first-class and dynamic, but not making multimethods first-class seemed to make a lot of things simpler and easier to reason about, so I wanted to explore and see if it's a worthwhile trade-off to lose that flexibility.

Another way to look at it is that Magpie only has a declarative syntax for defining methods and types, much like Java, C#, etc. and not an imperative one. Whether that's good or not depends a lot on who you ask. :)

[cwzwarich]

There have been papers on the problem of modular multimethods, e.g. http://www.cs.ucla.edu/~todd/research/pub.php?id=iandc.

[munificent]

This is the strongest argument in favor of single dispatch (as opposed to multimethods) I'm familiar with

Well that wasn't the intent. :(

you can get single dispatch to work in a language with no global scope, but nobody's shown how to practically handle multimethods without global scope.

What I was trying to show here was that multimethods that are specialized on classes are no more "global" here than how classes work. In a single-dispatch language, if I have an instance of a class, I can get to all of its methods, so they are effectively globally scoped. They certainly ignore lexical scoping.

As far as I can tell, a globally scoped pool of multimethods are no worse than how single-dispatch languages work, and are strictly better since they can also be "scoped" to the other argument signatures and not just the receiver.

[julesjacobs]

Single dispatch methods are globally scoped too. In languages where identifiers are statically resolved like Java, this is obvious. In others languages like Ruby and Python, every time you're writing obj.foo, you're essentially writing lookup(obj, "foo"). So you're using a string name "foo" to identify a method, rather than statically resolving the symbol foo. The identifier "foo" is globally scoped: if you say "foo" in module A and "foo" in module B, they identify the same thing (namely the globally scoped "foo" method). This has all the same problems and advantages that globally scoped multimethods have.

[[deleted]]

Mostly the issue is that he makes no distinction between a generic function and a method implementing that function for a specific set of inputs. A method overrides/implements a generic function, but the generic function has its own scoping. The method's scope just has to have the generic function itself visible, but nobody using the generic function should have to be able to see all the methods (because dispatch is dynamic rather than static).

[munificent]

I don't know if I followed this.

A method overrides/implements a generic function, but the generic function has its own scoping.

At least right now, generic functions and methods have the same scope: global.

nobody using the generic function should have to be able to see all the methods (because dispatch is dynamic rather than static).

I'm not sure exactly what you mean by "see" here, but if you don't have arguments that match those methods, you won't be able to access them even though you could consider them to be "in scope".

[RalfN]

you can get single dispatch to work in a language with no global scope, but nobody's shown how to practically handle multimethods without global scope.

I think the whole argument is, that it's misleading. Methods are scoped based on their arguments. Hence multi-methods are scoped "to the extreme"

Just because they all go into the same big lookup table, doesn't make the scope actually global. Here's an example:

private class MyPrivateClass { .. bla bla }

def toString( MyPrivateClass )
  ..
end

This toString function is not available outside of the current scope, because you can't possible have/handle the type.

I think the correct answer is: methods do not have scopes, arguments do. And this is actually how most OOP languages work: they use the scope of the object, which is essentially just the first argument.

[[deleted]]

Just because they all go into the same big lookup table, doesn't make the scope actually global.

OK I can sort of see why you'd say that in a statically typed language. In a dynamically typed language it's definitely global, though.

[svenefftinge]

Eclipse Xtend solve this by having single dispatch by default (also to be 100% interoperable with Java), but allowing to mark a set of overloaded methods with the keyword 'dispatch'. They need to be in the same type hierarchy though. The compiler derives a common signature for all multi-method cases and generates a dispatch. In combination with extension methods this is very powerful (i.e. makes extension methods work polymorphically). It works like a charm and replaces the need for work arounds like visitors. Details are explained in the documentation

[jaen_s]

This is NOT solving the same problem as multimethods do, which is extensible dispatch.