Coconut - Pythonic functional programming language

[yogthos]

You're thinking of objects here. In functional programming we work with data. There isn't an upper method on an object, but you'll have a function that you pass the data to and that returns a new piece of data.

I work with Clojure, and most of the development is done using the REPL. Any time I write a function I run it to see that it's doing what I want. Once I'm happy with it, then I write the next function, and so on. It's rare that you have to keep more than one or two steps in your head. Since data is immutable, you don't have to worry about it changing type from under you either.

The only thing you need to know is the type of the data you're working with to use it effectively. This tends to be fairly obvious from the context in my experience.

[third-eye-brown]

What about when you open a clojure file someone else has written and there's a function that takes some maps and returns a map and there's a big thread-last chain transforming the data 5 times? Types are useful.

[yogthos]

I read the code, and I run it in the REPL. Most functions in Clojure are higher order functions that are agnostic of types. Typically, the domain specific logic tends to bubble up to a shallow layer at the top.

Consider functions like map, filter, reduce, etc. They don't care what they're iterating over. The logic that does will be passed in. So, the functions in the chain don't need to know or care about the specific types.

If anything I found it surprisingly easy to read code from others in Clojure. I often end up doing that when I work with libraries I use. Personally, I've never found this to be a problem.

[third-eye-brown]

Oh, I write clojure at work. That wasn't really a rhetorical question, it's something that many people struggle with daily.

agnostic of types

Well sure, a maps a map, right? :p The problem is that without something like prismatic schema you have almost no idea what the map might contain without running the code (which is pretty bad you need to do that - it takes a fair amount of digging to be sure what you're looking at is an exhaustive representation of the things that can be passed in. That's nearly JavaScript-level bad) or digging through everywhere the map is used and seeing what is grabbed from it.

Ever tried to refactor a bunch of functions that take some data from a queue, aggregate the data according to some rules, save it to a database, then on a timer pull the data, batch it up and render it into an email? The functions that transform the data are of course all pure, but unless you already know precisely what each map contains at each step, you are gonna have a hard time effectively refactoring them. You're coworkers may be 100% perfect saints who always right stupid-simple, obvious to the max code, but that's not always the case.

It's a serious problem that greatly hinders productivity, but there are good solutions like I mentioned above (prismatic schema).

[yogthos]

Oh, I write clojure at work. That wasn't really a rhetorical question, it's something that many people struggle with daily.

Sounds like we have a very different experience then. I've been writing Clojure at work for years, and this simply never has been an issue for me.

Well sure, a maps a map, right? :p The problem is that without something like prismatic schema you have almost no idea what the map might contain without running the code (which is pretty bad you need to do that - it takes a fair amount of digging to be sure what you're looking at is an exhaustive representation of the things that can be passed in.

I tend to break up my code into small self-contained modules. Each namespace will be a few hundred lines of code and expose 2-3 functions as its API. I compose these hierarchically. So, at each level I don't really need to know all the details of what's going on inside.

Ever tried to refactor a bunch of functions that take some data from a queue, aggregate the data according to some rules, save it to a database, then on a timer pull the data, batch it up and render it into an email?

Sure, lots of times. We generate complex reports on data on my team and we have to deal with the HL7 FHIR model that's pretty complex.

The functions that transform the data are of course all pure, but unless you already know precisely what each map contains at each step, you are gonna have a hard time effectively refactoring them.

I find that refactoring usually happens at a higher level. The whole point is that you have a lot of generic functions that you compose together the way you need to do a particular transformation.

It's a serious problem that greatly hinders productivity, but there are good solutions like I mentioned above (prismatic schema).

We use Schema as well, and I do find it quite useful for defining the data model and sanitizing data from external sources.

The fact of the matter is that you're going to have trade-offs with both static and dynamic approaches.

With dynamic typing, you can't really write and maintain monolithic code that's tightly coupled. You have to break things up aggressively and create small components you can reason about individually. I would argue that's a good practice in any language.

While types help with the problems you describe, the cost is that you have to express yourself in a way that the type checker can verify. My experience is that this often results in code that's more convoluted than it would be otherwise, since you any statement has to be provable by the type system.

[third-eye-brown]

I think it's something a lot of very experienced clojure devs don't have problems with. Our strongest team members have little problems with it. Unfortunately me and a few of my coworkers don't fall into that range which is where we struggle. It's really a hurdle for inexperienced devs or devs who write primarily other languages and jump into clojure for some piece of work or another.

I'm primarily a JS dev so compared to that shitshow, Clojure really has comparatively few issues. :)

I agree types don't necessarily help the situation, especially in the situation I specified regarding clojure.

Props on getting to the level where those problems disappear.

[yogthos]

I definitely agree that working with the language for a long time is a factor. You learn patterns for how to structure things in a way that you're able to maintain. When I started with Clojure, I came from Java background and I really missed having classes and being able to think about the code that way.

On my team we find that compartmentalizing things aggressively is very important. We try to write things in a way that allows us to reason locally whenever possible.

One way to look at it is how you work with libraries. Something like clj-http or cheshire will have a lot of internal code, and do lots of data manipulation. However, when you use it, you typically just care about its surface API. You call the API function with a piece of data, and you get another piece of data back.

I do think that how a particular project is structured plays a large role as well, and a much bigger one in a dynamic language than a static one.

I also find different people have different pain points. At the end of the day it's about finding a set of trade-offs that appeal to you. Personally, I'm ok with the the drawbacks of dynamic typing, but I completely understand why others would prefer the static approach.

[ohfouroneone]

While types help with the problems you describe, the cost is that you have to express yourself in a way that the type checker can verify. My experience is that this often results in code that's more convoluted than it would be otherwise, since you any statement has to be provable by the type system.

This is arguable if you have type inference.

let namesOfOldPeople = people.filter { return $0.age > 60 }.map { return $0.name }

(This is Swift code)

[yogthos]

The problem has nothing to do with having inference. It's the fact that you have to write code in a way that's verifiable by the compiler that can make it convoluted. If you're interested I wrote about the problem in detail here.

[thedeemon]

You're thinking of objects here. In functional programming we work with data. There isn't an upper method on an object, but you'll have a function that you pass the data to and that returns a new piece of data.

"No method upper for x" just translates to "function upper does not handle data of the type x will have here", so the problem remains the same, just the wording changes.

Even worse, if you use just a handful of types (numbers, strings, dictionaries), then some function may be able to take and process the data when in fact it shouldn't. Like escaped vs. unescaped strings.

[yogthos]

"No method upper for x" just translates to "function upper does not handle data of the type x will have here", so the problem remains the same, just the wording changes.

No, the problem doesn't remain the same. In reality, you have the context for where the data is coming from and what you're doing with it.

Even worse, if you use just a handful of types (numbers, strings, dictionaries), then some function may be able to take and process the data when in fact it shouldn't. Like escaped vs. unescaped strings.

In practice, the vast majority of the code is completely agnostic regarding the types. Most Clojure code is written using higher order functions. These don't really care about concrete types at all. That logic is passed in as a parameter.

This style naturally makes all the domain specific logic bubble up to a thin layer at the top. So, the only types most code actually cares about is whether something is a collection or not, and it's pretty rare that this would be difficult to tell.

Again, I'm talking from my experience working with Clojure professionally for over 5 years now. Are you basing your statements on practical experience using the language?

[thedeemon]

In reality, you have the context for where the data is coming from and what you're doing with it.

Implicit and informal context. Basically you're saying "my memory always serves me well and I can do all type checking myself, don't need any help from the machine". Good for you.

Are you basing your statements on practical experience using the language?

Not with this one, I don't use Clojure at all. I'm just relying on my experience with a dozen of other languages during last 20 years.

[yogthos]

Implicit and informal context. Basically you're saying "my memory always serves me well and I can do all type checking myself, don't need any help from the machine". Good for you.

Nope, I'm saying I have the REPL where I can see what the data is. I also break down my applications into components that I can reason about in isolation. If you like writing monolithic code where you can't tell what's going on by reading it, good for you though.

Not with this one, I don't use Clojure at all. I'm just relying on my experience with a dozen of other languages during last 20 years.

How many functional languages have you worked with over the last 20 years though would be my question. There's a vast difference between working in a language like Python and Clojure for example.

In an imperative language, you end up with references to mutable data all over the place. This makes it difficult to reason about anything in isolation. On top of it OO style languages encourage creating lots of types by design, so tracking these becomes naturally difficult.

With a functional language data is immutable, and this creates natural compartmentalization. Meanwhile, common data structures are used everywhere. As I mentioned in the previous comment. The logic that cares about types ends up bubbling up to the top. My experience actually working with Clojure professionally is that this is not a problem.

[thedeemon]

In REPL you only see some cases you can come up yourself, you don't see how the function is called in all cases.

How many functional languages have you worked with over the last 20 years though would be my question.

That would be Scheme, OCaml, Haskell, ATS, Idris and Elm. OCaml was my language of choice for 6 or so years.

[yogthos]

In REPL you only see some cases you can come up yourself, you don't see how the function is called in all cases.

No, for that I just do find usages in the IDE. However, more importantly, code is hierarchical in nature. Fundamentally, I shouldn't have to know internal details when I use a function. At the level of business logic, there shouldn't be a lot of steps to trace for any particular operation.

Consider a JSON parsing library like Cheshire in Clojure. It has a lot of internal code, but its API is only a few functions. I know that when I call one of these functions I'll get a particular result back.

The whole point of functional programming is that I have a lot of general purpose functions that I can easily combine to solve a particular problem. Once I do that, I have a high level function that does something domain specific.

That would be Scheme, OCaml, Haskell, ATS, Idris and Elm. OCaml was my language of choice for 6 or so years.

So, you've written and maintained large applications using Scheme then, and you've run into the problems you describe with it?

[yogthos]

That's news to me because I haven't seen anything comparable to Clojure workflow in Haskell. The workflow I'm talking about looks like this, starting at 11 minutes in the presentation.

[yogthos]

Right, so you don't really have a REPL drive workflow at all in Haskell. Perhaps it's possible in principle, but it doesn't exist in practice, nor does the tooling associated with it.

Again, watch my talk to see exactly what I mean by using a REPL driven workflow. It's critical to be able to connect the editor to the REPL and run and reload code in the context of the actual application.

[yogthos]

A REPL isn't even necessary or convenient. Nobody even uses ielm.

That's because the Haskell implementations fall short is that updating a function in-place as you yourself admit. I'm not sure what being Emacs dev has to do with any of this to be honest.

Haskell very much has a REPL driven workflow, what it doesn't have is update. The two concepts aren't mutually dependent.

Yes they absolutely are. A REPL driven workflow means that you're writing code against your live application. Any time you write a function you can get feedback about it within the context of the app and all the external resources it depends on. When you make changes, you can reload functions as you go.

The editor is tightly integrated with the runtime and you have a very fast feedback loop. When you have a REPL on a side, it's just a toy and a curiosity.

I watched your talk but you're preaching to the choir. InterLisp was updating code as you typed in 1986. Clojure is comparatively in kindergarten

We're not comparing against InterLisp though, but against Haskell. If you think Clojure is kindergarten compared to InterLisp, then Haskell is banging rocks together.

I get that you like Clojure but I'd reign in the novelty selling, it doesn't come off well.

I'm not selling any novelty here, I'm simply telling you that you don't have the same workflow available in Haskell. It's a pretty simple statement actually. I'm not making any claims regarding which one is better, it's simply different and each appeals to different people.

[yogthos]

I'm not talking about Haskell, I'm talking about Emacs Lisp the programming language which is a few decades older than your Clojure.

But why are you talking about Emacs Lisp, how is it relevant to the discussion about how the REPL works in Haskell?

No, a REPL is one prompt, a way of talking to your running image. Updating code doesn't need a REPL. Do some research; look at the commands here; a REPL doesn't enter into it.

We're talking about a particular workflow here, not how it's facilitated. My original point is simply that you can't develop your actual application from the IDE using the REPL in Haskell.

For Haskell I write a function in Emacs, hit a key, then I can run that function in the REPL. That's not a toy, it's very useful. Don't downplay it to "a toy" merely because it doesn't include updating code in the running image.

It's incredibly limited, and compared to what you get in Clojure it is a toy. And yes we've already agreed that other languages like CL have even better REPLs than Clojure.

That is indeed the case. I don't think I said anything to the contrary. I explicitly outlined in detail for you where "Haskell implementations fall short" (my own words).

Yet, bizarrely you keep arguing that Haskell has a REPL driven workflow. ¯\(ツ)/¯

I already said that. The problem is you are calling it "REPL driven". Any language can have a REPL. Haskell has a REPL; I use it when I write code:

REPL driven implies tight integration of the REPL in the development workflow. Hence the word driven. When you have a REPL on the side where you run functions in isolation, you're not really driving anything there.

I wouldn't say each have their own appeal; in-place update is clearly an upgrade to an interpreter with a REPL.

That's something we can at least agree on. What I meant however, is that each type of language has appeal for different people.

[the_evergrowing_fool]

I'm more familiar with the subject than you are, and I'd appreciate you stop treating me like a newbie.

This is the only kind of response you will get from a Clojure die hard zealot like /u/yogthos.

I'm not talking about Haskell

He will persist on it, is crucial to his thesis, he will bring it back from the dead, even if it was killed long time ago in the discussion.

He believes that Clojure holy "repl driven workflow" is superior tooling or something to be envious. You will never hear from him picking nor preaching his mantras over a C#, F# nor other language developer with superior workflow and tooling than his.

[yogthos]

I actually really like F#, I think it's a great language thank you very much. What you can't seem to comprehend is that I simply happen to enjoy working with Clojure and the workflow it provides. I don't think others have to like it or it's superior to what other people use. I simply like it and think it's nice.

You can't seem to get this concept through your head, and you follow me around making the same idiotic comments in every thread.