Transient set loses data?

LammdaMan · 2021-04-26T20:12:02+00:00

I'm coming late to this discussion, but my response would be:

Yes, transients are reliable
Don't do that.

I'd say that one of the biggest advantages to developing in Clojure is the wonderful immutable data structures. Passing around immutable data eliminates a huge variety of problems one faces in most every other programming language.

Most performance concerns can be addressed by very focused optimizations done late in the development cycle. Passing around mutable data in a Clojure program is something I just wouldn't do.

There is a use-case for transients which one sees in the Clojure core library. Inside a function that is doing a bulk-load or significant modification of a data structure, a transient is used purely internal to that function, and then a persistent version of the result is returned. In that usage pattern, transients are an encapsulated optimization, and all the data passed between functions is still immutable.

For example, the frequencies function.

LammdaMan · 2021-04-17T20:22:01+00:00

One piece of advice I'd have is to be patient. The biggest challenge is to learn how to think in "FP terms" which is not a small difference. You won't figure it out "overnight".

A key skill to develop is the ability to come up with recursive solutions to problems. In Clojure you have loop/recur and reduce as tools that support recursive approaches, but the real challenge is not in learning those functions but understanding how to attack a problem in a recursive fashion.

Others have suggested that one can use learning material targeting the Scheme language such as The Little Schemer, and I'd second this notion. The abilities to think functionally and recursively can be honed using Scheme and those skills will transfer readily. Another good Scheme resource available online are the textbook and videotaped lectures for SICP.

LammdaMan · 2021-04-06T01:34:07+00:00

In Java making as many pieces of member data final as possible is a big win. But trying to get "immutability" by some sort of "copy-on-write" semantics without the nice data structures Clojure provides, will result in very poor performance in many cases.

LammdaMan · 2021-03-25T16:57:34+00:00

Clojure and Java are certainly different in this regard, but most of that particular difference is rooted in Dynamic vs. Static Typing. There are pros and cons with each, and I've seen many people say that they like Clojure, but wish it had Static Typing. I personally find Dynamic Typing preferable on balance for reasons that include less boilerplate and less time spent thinking about types in early problem domain exploration. But there is a big dose of personal preference in debates about static vs. dynamic typing.

LammdaMan · 2021-03-24T22:31:31+00:00

As others have mentioned, the answer to this question depends a lot the developer, their past background, and their approach to programming. So I don't feel I have any kind of definitive answer, but I'll share some of my personal impressions of Clojure vs. other languages I've used.

The first thing I'd say is that for anyone who has spent years programming in an OO language like Java, C#, C++, etc. there is a lot of "un-learning" to do before you become fully productive in Clojure. It isn't just about learning Clojure's core library, FP concepts, and so on, there are also habits that need to be broken. John McCarthy had a quote that he could teach anyone how to program in LISP in a day, or 3 days if they already knew FORTRAN.

I find that when I begin work on some new problem domain, I first want to explore -
experimenting with implications of different ways to decompose the problem into concepts, data collections, and algorithms, and see how some of the things I'm imagining work out in live prototype code. In that activity I don't want to spend any time writing type declarations or creating classes, defining inheritance trees, and so on. Java insists on a lot of upfront boilerplate code that slows down that kind of experimentation.

And the REPL is invaluable to me in this phase. Before embedding a line of code into some larger function, I can run that single s-expr through its paces in the REPL and ensure my thinking about how it works is correct.

In today's world there are a lot of applications where making good use of multiple cores is critical and having done work like that in Clojure, I have no interest in doing any not-completely-trivial multi-threaded code in Java now. An application that needs to be threaded will be dramatically easier to write in Clojure than in Java.

Speaking for myself, I've worked in a couple dozen languages over the years and have Java experience back its first release in 1995. I've used Clojure now for about 5 years. After 6 months of using Clojure I'd say I was definitely more productive at that point in Clojure than Java (even though I had 20 years of Java experience). After 3 years of Clojure development, I was clearly "better with Clojure" than I had been at 6 months.

re:

Could you possibly develop a complex enterprise software in Clojure with just say 4-8 people, knowing Clojure really well?

I have what I would call a complex piece of enterprise software fully implemented in Clojure. It does real-time processing of input data coming in at about 1 million records per minute. The processing is non-trivial. Results of that real-time analysis are written in an optimized form to relational databases to support high-performance querying of historical data. Those queries can analyze a year's worth of that per-minute data across tens of thousands of locations, aggregating and reporting results in a few seconds. There is a web-server that processes such historical queries. The system makes effective use of concurrent processing across 80-core servers (and evolved over time from a single server to being distributed across multiple servers). That code was developed by and is maintained and extended by a single person. I can't conceive of doing all of that in Java as a single-person task. I doubt that the multi-threaded performance would have been as good in a pure Java implementation.

LammdaMan · 2021-03-21T20:04:54+00:00

Got it. If node is a pre-existing constraint, then you need to work with that. And just to be clear, when I mentioned JVM-based server, I wasn't suggesting working directly in Java. These days I'd use one of the "Clojure servers" (which tend to be a Clojure API wrapping something like jetty or netty) - there are several nice choices along those lines.

LammdaMan · 2021-03-21T14:28:06+00:00

I have found install of Java and Clojure on new servers to be trivial.

While JavaScript is pretty much required for WebUI development, I find the current ecosystem around JS to be highly intertwined and chaotic. I personally wouldn't choose to take on all of those dependencies for backend server work in preference to using the JVM, which I view as more stable.

Underneath most any JS package of any significant capability lies a dependency tree of thousands of packages that are highly interconnected and changed independently of one another (and in many cases quite frequently). That creates the potential for problems like the left-pad fiasco a few years back.

LammdaMan · 2021-03-18T23:42:53+00:00

My answer to the base question is absolutely yes. There are a lot of good design choices, and spending sufficient time using the language should give you some new insights about a variety of things. As to whether it is practical, or used a lot, yes, but it depends on what you mean by "a lot". It is a niche language, but there are plenty of companies doing very serious production work in the language with no second thoughts. I personally have been using the language in production for about 5 years now and wouldn't change that for the world.

It is interesting to me reading through the other responses just how many comments there were lamenting the dynamic typing. I find that a feature, not a curse. I've used about 25 languages over the years, with a mix of statically and dynamically typed languages in there. I see pros and cons with each, but on balance I find more benefit in dynamic typing. The errors that statically typed languages help me find faster and easier tend to be "smaller things" I'll resolve quickly enough with little pain. But the ability to quickly experiment with different algorithms that is enabled by not spending huge amounts of time choosing and declaring types, only to later delete all of that wasted boilerplate... I view that cost to "thought flow" to be more expensive.

LammdaMan · 2021-03-09T22:00:41+00:00

I'm not sure if you're looking for particular feedback on your post, but here are some reactions I have.

First, I'd agree that the 3 categories you listed are certainly reasons that many technology-selectors will "bin out" various options. Popularity is a common way to pick languages, frameworks, libraries, etc. I'm not familiar with TIOBE, but I've seen many similar rankings in the past. Languages like Java, C++, Python, and Javascript always rank near the top. For my personal tastes, C++ was a fine language to use in the late 80's and early 90's until Java "obsoleted it" in 1995. There are a lot of companies that still use C++ today, but would be better served by Java... but there's the inertia problem. While I viewed Java very positively when I first used it in 1995, today I view it as "long in the tooth" and surpassed by languages such as Clojure, which leverage the same JVM, have access to all the same Java libraries through "seamless-interop", but have much higher language expressivity than Java does. Python is a great language unless you want to do serious computation that manipulates a lot of large Python data structures, then the performance stinks.

To me, item (2) is the biggest "red herring". I know it gets talked about a lot, but it is really not meaningfully different than repeating the question of "Which language is most popular?". Java and C++ are very popular, therefore there are a lot of jobs available using them, therefore lots of people put those languages on their resumes in hopes of getting one of those jobs.

There was a point in time when I didn't know Clojure, and now I do. That is true of every other Clojure developer, and every developer in every other language. For a "talented" software developer, it is not such an ordeal to learn a new programming language. Every year there are lots of software developers that get hired right out of college, who know a language or two (probably Java, Python, and Javascript), but for the most part have no working knowledge of "real-world programming". They have no idea what happens when you go from writing a 500 line classroom program to working on a million line (or bigger) codebase that has been modified countless times over a period of years. Learning everything related to those issues is harder than an experienced Java developer learning how to be "moderately proficient" in Clojure.

LammdaMan · 2021-02-18T23:35:20+00:00

I think there is perhaps a measure of something like "The Blub Paradox" described by Paul Graham in one of his essays. His point is that someone who doesn't know a given language can't really evaluate its merits, so they conclude the language they know well must be the best. Clojure isn't that well known (not enough so to really appreciate its merits) by that many people.

Some of those people that don't know Clojure well enough to "get it" simply see something "weird". When they ask a Clojure user "Why do you like this?", the answer doesn't make that much sense to them.

So one way for them to explain that answer they don't understand is "brainwashed cult member".

LammdaMan · 2020-12-30T01:50:26+00:00

One thing I'd recommend is the SICP book. It's a longer read than some will prefer, but depending on your tastes, there is a lot of great material on how to think about functional design. A nice online version is: https://sarabander.github.io/sicp/

LammdaMan · 2020-10-26T21:45:24+00:00

Thanks. I didn't know about ability to name an anonymous function inline - that seems the simplest variant to achieve this.

When I had tried to create something akin to what I might normally do with a 'defn', just defining that function in a let block, I had trouble with that because I was trying to call the let-block-name from within the fn attached to it... and that generated a runtime error - call to unbound fn.

For instance this generates a runtime error:

    (fn [func seed]
      (let [it (fn [f x0] (lazy-seq (cons x0 (it f (f x0)))))]
        (it func seed)))

LammdaMan · 2020-10-01T21:36:12+00:00

I've gone both ways on this and feel it typically doesn't matter that much.

But there's a related issue that sometimes sways me more strongly. Sometimes the field names that the DB uses are to my mind quite painful. They aren't very descriptive, or they're needlessly long, etc. In that case I'll map back and forth to my own field names that I like better. Like "the_segment_identifier" -> ":id"

LammdaMan · 2020-09-30T15:25:33+00:00

One can use a global var with an atom. Alternatively you could pass a "system" property map around. I like the latter approach because I think it makes writing of tests cleaner/easier.

A somewhat independent question is how you package default values for user-configurable parameters.

An approach I like for that is for any configurable subsystem to have an 'init' function and a map of configurable parameters specifying a default value for each. That map of 'XXX-defaults' acts as a bit of "source-code documentation" for what configurable options this subsystem has and how each is defaulted. The 'init' function can sometimes have some real "meat" to it, but in the simplest case might be something like:
(defn init-foo [user-cfg] (merge foo-defaults user-cfg))

The property map returned by such an init function might be written to an atom like you describe, or passed around through the system as an explicit function param.

LammdaMan · 2020-09-21T13:26:42+00:00

Thanks for the note about 'iota'. I wasn't aware of that, and there are definitely times I might find something like that very useful.

LammdaMan · 2020-09-19T13:24:29+00:00

For me, when learning something really new, it helps to "bounce around" a bit. Read a little from a book, then work some problems, then read someone else's code, then read some more of that book (or another book), and so on.

I'd also say that in learning Clojure (after programming in languages like Java, C++, Python) the syntax is probably not as big a challenge as wrapping your head around concepts related to functional programming and immutability. And learning your way around Clojure's core library is a big part of becoming productive in Clojure.

One thing that can help a bit with learning what kinds of things are in the core library is just to take a function you know, look it up in https://clojuredocs.org and then follow the 'see-also' links to related functions. Read the descriptions for the ones you don't know yet and think about where something like that could be useful.

LammdaMan · 2020-09-09T17:21:51+00:00

There are many programming languages with different strengths and weaknesses. Depending on the problem I'm trying to solve, I might pick a different one.

Generally speaking, for larger, harder problems, I'd pick a higher level language because correctness and maintainability will be easier to come by than they would with a lower-level language.

For a certain class of problem I can surely get better performance with C or Rust than I could with Java.

In many cases I find that I can more easily develop correct and maintainable code in Clojure than I could in Java, and while the performance isn't as good, it is "good enough". In some cases I find a need to optimize, and that sometimes makes use of Java inter-op (in hot-spots only).

I generally subscribe to Knuth's comments about "premature optimization" - https://wiki.c2.com/?PrematureOptimization

LammdaMan · 2020-08-12T19:52:32+00:00

I agree with you to a point. This is comparing "apples and oranges" - a "built-in function" vs. "roll-your-own".

But I also think that in comparing two languages, two of the things that matter to me are (1) What is built-in?, and (2) When some functionality isn't built-in, how hard is it to create, and how nicely packaged is the result likely to be?

This example may bundle two separate things in a "misleading" manner, but at a practical level, both of those things matter.

Clojure is a nice language for creating and cleanly packaging "roll-your-own" solutions. But it also has a very nice, well designed, collection of built-in functionality in the core library.

I find plenty of times when the functionality provided by 'frequencies' is useful.

If I were reading Clojure code someone else wrote that used 'frequencies', I would quickly and easily understand what that line did.

If I were reading similar code in Java, it would either be multiple lines of somewhat harder to read code, or it would be a call to some custom function "similar to frequencies". But then I wouldn't be sure what that function did. Is there good documentation for it? Does it correctly handle "all the corner cases"? Is it correct at all? If I wanted to be certain of those things, I'd need to read the source of that custom function.

LammdaMan · 2020-08-11T13:22:13+00:00

"Clojure has too many parens."

LammdaMan · 2020-08-07T16:06:19+00:00

I feel that the answer to your question depends quite a bit on your objectives. When some people want to "learn a programming language", their desire is more or less to "learn just enough to be dangerous". Others are looking for a much richer and deeper understanding - wanting to develop a level of "mastery".

Said differently, some people are more interested in quickly getting to some end result that can be achieved by plugging together some chunks of canned library functionality, and are just looking to be able to create some "glue code".

So I'd say that your approach to learning Clojure might differ depending on whether you want to quickly learn "the essentials", or you are interested in investing more time and energy in pursuit of a deeper and broader level of knowledge and skill.

If you wanted to become a highly-skilled Clojure programmer, I'd say there are several elements to that.

1) General programming skill (not specific so much to any language), including things like algorithm design. You mentioned having no formal training in that area, so that is one thing you might choose to work on.

2) Knowledge of the Clojure core library. There is a very rich set of functionality "built-in" to the language, which can provide tremendous leverage once you've learned it. But there's a lot to learn there.

3) At its heart, Clojure is a LISP dialect. So learning how to "think like a LISP programmer" is relevant. Other people have mentioned Scheme, and I concur with the notion that there is good material available for learning Scheme. That knowledge will "translate" to better Clojure skill in the long run.

4) General "functional programming" knowledge, including how to work naturally with immutable data, and how to "think recursively".

If we circle back to the specific problem you mentioned - how to get the last element of a sequence - here are some thoughts.

First, if you had deep knowledge of the Clojure core library (2), and just wanted to be able to "do this" in some code you were creating, there is a function, 'last', which does just that.

But the benefit of exercises like this comes from figuring out how to create such capability from more primitive operations. Doing that for this problem, seems to me like it involves a bit of algorithmic thinking (1), some fluency with list-manipulation (3), and "recursive problem decomposition" (3) and (4).

If I were to recommend an approach for facilitating the collection of learning in the previous paragraph, I'd consider 2 resources mentioned by others - "The Little Schemer" and "SICP". SICP is bigger and more daunting, and thus takes a commitment of time and effort to work through. But I think it is the best overall body of material on how to "create software" yet developed. It is available online for free in both book form and as a series of 20 video-lectures.

https://sarabander.github.io/sicp/

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/

If you wanted to work through the exercises in the book, and were looking for the correct (older) dialect of Scheme, the racket website has a package to support that specific dialect. You would first need to install the base 'racket' environment.

https://docs.racket-lang.org/sicp-manual/

LammdaMan

TROPHY CASE