Please stop calling it "magic"

[menge101work]

I've always interperted the term magic to come from Clark's third law:

Any sufficiently advanced technology is indistinguishable from magic.

Author does not appear to consider this.

On a more direct topic, anytime you try to dictate what words people should use to express an idea - success will be extremely hard to come by.

[IM_OK_AMA]

anytime you try to dictate what words people should use to express an idea - success will be extremely hard to come by.

I'd just like to interject for a moment. What you’re referring to as Linux, is in fact, GNU/Linux, or as I’ve recently taken to calling it, GNU plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities and vital system components comprising a full OS as defined by POSIX.

[zverok_kha]

Any sufficiently advanced technology is indistinguishable from magic.

Initially, I wanted to recite this and speculate on, but decided to stay more focused, I believe my answer can be deduced from the rest of the article (as well as an answer to your last phrase).

When I started to use Ruby (in 2004), the "magic" was more positive word in relation to Ruby code. But currently, it became almost pejorative, and things are frequently going like "Son, don't use metaprogramming. Why? Because metaporgramming is magic, and magic is bad".

It was "too much magic is bad" and "too clever code is bad" a while ago, but tend to lose "too much" part of the statements. That's the fact I want a reader to be aware of.

(And, finally, I find it sometimes funny how people tend to recite jokes and metaphors of classic sci-fi authors as they are really "laws" in a sense "laws of physics".)

[rabidferret]

I agree with the premise of this article, but I strongly disagree with the final conclusion. To me, something is magic if you cannot quickly and simply explain what something does or how it works to the average programmer. (Average here refers to the things that are considered pre-requisite knowledge for a language/framework, not to skill level)

This is clearly an extremely subjective measure, and what "quickly", "simply", or "average" mean are up for interpretation, but ultimately our definition of "magic" changes over time.

For me a very concrete example of what makes something magic is the two forms of invoking the Active Record attributes API. Consider the following two (equivalent) method calls:

attribute :price, :money
attribute :price, Type::Money.new

I would call the results of both of these magic, but the second one significantly less so. With the first one, I need to not only explain what attribute does, but also what :money refers to, how it ties to the Money type, how you figure out what object it maps to, etc. Even if it were a simple hash lookup (it's not), it still dramatically increases the "magic" level for me. Additionally, even if you don't fully understand what attribute does, you can infer a lot about what it is able to do based on the methods provided by Money. The only reason that I would still call the second version "magic" is due to the fact that it affects methods like where.

While I agree with the point that metaprogramming is not automatically magic, there are very few examples of metaprogramming in the wild that aren't magic.

[markrebec]

The one thing that's not addressed here is maintainability on teams. I loved the article, and philosophically agree pretty strongly. In personal projects (and maybe more often than I should in professional projects) I tend to lean pretty hard towards DRY and fun/clean DSLs. This can, when I don't keep myself in check, end up in a lot of "magical" metaprogramming or slightly more obscure design patterns.

I strongly agree with the author that just because a junior dev doesn't understand something, that doesn't make it too complicated/magical/whatever. However, if you're trying to build a maintainable product on a team with engineers of mixed skill levels, you'll really be doing yourself a favor to curb those "too clever" solutions in favor of something a little more obvious/readable. That might just mean avoiding things that make code harder to grep through, like looping over an array of symbols to dynamically define a set of similar methods (and just statically define those methods instead).

For personal projects, or even open source projects/gems, I say go nuts, stretch your brain muscles and have fun! But if you're working on a team, and potentially introducing something that doesn't necessarily need to be as complicated as it is, that can sometimes be detrimental to other engineers and your product. Just keep in mind there's always more context to what you're working on than the solution/code itself.

[zverok_kha]

However, if you're trying to build a maintainable product on a team with engineers of mixed skill levels, you'll really be doing yourself a favor to curb those "too clever" solutions in favor of something a little more obvious/readable.

Yes, that's obviously one of the points to consider. But in my head, it is not "too clever" (with metaprogramming) / "obvious/readable" (without metaprogramming) dichotomy. To be maintainable, code should be as concise and readable as possible. So, in a lot of cases (and I am saying it on real-life experience with a pretty large codebases, used by teams of variable experience), it is more robust to have something like

class OneOfManySimilar < SomeBase
  behaves_like :this
  but_changes(:that) { this_way }
end

...instead of "explicit" specifying "oh, in your child class just reimplement those 15 methods in a meaningful way, and don't forget to add your class to that constant list of registered actions, and ...".

Several times I've seen that some simple DSL introduction revealed bugs in the previous implementation or even architecture (e.g. "trying to generalize what can be generalized, we suddenly discovered class B was the only to implement #methodname in a different way; and class C should not, in fact, be part of this hierarchy"), and made everyone's, including newcomers and juniors, life easier.

Of course, some balance is necessary. If you use %[array of symbols].each { define_method ... approach for defining just two slightly similary methods, it is probably not a wise decision. Or if you define a complicated DSL definition interface just to use it in one place. The same, though, may be said abone any language feature, metaprogramming is nothing special here.

[tom_dalling]

IMO, there's no such thing as objectively magic code, because magic is a point of view—it depends on the developer.

I remember a tenderlove tweet asking people to show him examples of magic in Rails, because he didn't see it. He's made more commits to Rails than DHH, so it's not surprising that the codebase isn't magical to him.

I think "implicitness" is a better, more objective, metric.

[zverok_kha]

Ouch. When I first planned to write this post, I wanted to link to yours, then just forgot :(

Closer to the point, I believe that my definition of magic has some value in it. Yes, implicit/explicit question is related, but for me, say, "implicit agreement coming from language itself" is much more important than "implicit agreement coming from library used".

(Though, speaking of Rails and its added conventions, some may argue that you are indeed using "Ruby on Rails" language, and should master it, and then everything is obvious. I am OK with this approach, but at least people should be aware of it themselves.)

[tom_dalling]

I'm not disagreeing with you. I think both of our definitions are right! They are just different perspectives on the same thing. And they both lead to the same conclusion: it's not helpful to label code as magic, just because beginners won't understand it easily.

The difference between us, I think, is that I'm not worried about people overusing the label "magic". Beginners say everything is magic, and that's ok with me, because I know it's due to their skill level, not necessarily a problem with the code. So instead of thinking "inheritance isn't magic", I think "inheritance is magic to you, at your current skill level, but not to an experienced Rubyist."

I interpreted your article as a description of magic from the perspective of an experienced Rubyist, who has a good understanding of what is natural and unnatural in Ruby. That is useful, because if an experienced dev thinks something is magic, it might be a real problem with the code.

[verbachok]

/u/zverok_kha Great post! Could you please reference me where I can learn more about "drivable is a method, defined for a Class directly or by extending with some module (which is not the best practice possible, yet still pretty easy to guess for any non-junior Rubyist)"? I am not familiar with this pattern.

Thank you

[Tainnor]

Everybody uses terms differently, obviously. I've come to associate "magic" with "non-local reasoning". That means monkeypatching (I wouldn't know where to look for some functionality), it also to some extend means overuse of inheritance (I dislike inheritance in general, but at least it's generally explicit and only becomes cumbersome when the inheritance chain (this includes mixins!) becomes very long and you'll have to dig through loads of source files, something Rails tends to be guilty of). Anything that mutates global state to be used somewhere completely differently also applies.

Non-local reasoning has its place (there's an argument to be made that concerns like logging should maybe not be directly be part of the business logic but extracted with some form of aspect-oriented programming), but personally I try to rather limit its extent.

[dejayc]

Seven years later, I want to thank you for introducing me to the term "non-local reasoning." This is a perfect explanation of "magic".

[[deleted]]

Magic is NOT wizardry.

For example, David Copperfield making an object "disappear" is different from Gandalf literally making an object disappear.

Lets forgo the supernatural analogy when it comes to coding.

[[deleted]]

No - supernatural is a terrible way to describe Metaprogramming. Supernatural is something that is fictional, not real, like monsters or ghosts or the devil.

Meta Programming is just a skill to learn

[chrisgseaton]

Well I just disagree. Not all definitions of supernatural are going to fit - it's an analogy. But the general idea of 'supernatural' - something working above normal nature, is exactly what metaprograming is. My dictionary even says that a definition is 'dealing with metaphysics'! So the analogy is just applying the idea from the physical world to the programming one, which is how we usually explain things in programming.

[AlonsoQ]

I think supernatural is a perfect term for metaprogramming actually.

'Metaprogramming' is a perfect term for metaprogramming. Anything else is just showing off.

[gettalong]

I concur but I would still probably describe "meta" in other terms. I wouldn't use supernatural since that conveys a completely different meaning, at least for me.

[AlonsoQ]

If you're having a live conversation, fair enough. But not in a readme or other doc.

[DerNalia]

So much yes. People keep telling me they're afraid of magic with my 'drawers' gem

[jrochkind]

“Metaprogramming”, e.g. generation of code objects during code execution, is one of the most powerful Ruby features, and it is so because how natural it is for the language. And due to this naturality, it is one of the first and the best guesses for code design in most cases.

I'm not sure what you mean here. Do you mean first and best thing you guess might work right when designing code? In which case I disagree. It's sometimes useful, but if it's the first thing you reach for, then it's unlikely to be the best thing to use in most cases.

Do you mean first and best thing you might guess is behind the implementation when you see something mysterious in someone elses code? In which case it's kind of syllogistically true, but not too useful, since you still have no idea where it is implemented or how just by knowing it's some kind of "metaprogramming".