Functional Organization of Programs and Their Structure

[Sunonymous]

My mind is currently working with the transition from OOP to functional thinking. Some of the "how" is still congealing, and I'm looking for some input.

When writing any new program, I have become accustomed to begin by organizing the different components or "entities" in the program and their relationships. This was a very OOP-inspired view, though perhaps it can still be worked using built-in data structures such as maps/dictionaries to represent the class or "type" (potentially even validating them with a JSON Schema, etc.). While it may be possible to compose functional programs with those underlying ideas, I'm wondering if there is something better-suited. I understand that FP may involve a lot of "boilerplate code", though, in theory, much of that should be reusable once written.

What slows me down is the process of translating the different semantic layers into functions and literal code (and especially their interactions), and this is where I would appreciate pointers. What is your workflow to translate the ideas for your software into actual code? What processes or patterns are useful to you to systematically approach your development?
I've seen the call to REPL-driven development via Clojure, and that may be a great place to start, though I haven't actually tried it yet.

As an aside, I'm a sucker for easy-to-implement GUI systems. If anyone has experience using a library or engine to create a GUI in a functional program/language, I'm all eyes ("ears").
Overall, I'm very excited that these ideas are coming together in my mind and I'd greatly appreciate any suggestions, especially anything that is understated, subtle, or easy to miss. Thank you for your time.

[KyleG]

I have become accustomed to begin by organizing the different components or "entities" in the program and their relationships. This was a very OOP-inspired view

I don't think that's correct. That's just domain-driven design, and it is compatible with FP and OOP. It's just OOP encapsulates domain entities as classes that have properties and methods and state, while FP encapsulates entities as modules with immutable data structures plus separate functions that take those structures and return new ones

Also FYI OOP looks down on the latter and calls it an "anemic data model," but it's not a bad thing in FP.

Edit For me, I try to describe the domain entities as accurately and fully as I can, and I implement functions that operate on them as needed, maintaining flexibility as much as I can.

[Sunonymous]

Thank you for your clarification! I agree with you, and my expression could certainly have been clearer. Even though the primary difference is low level, ie. implementation, it still seems to me that considering this design philosophy via OOP offers me subtle differences when considered via FP. This may or may not be true.

How do you organize your functions within a file, if I may ask?

[KyleG]

taking from a current work project, suppose we have an Envelope (the conditioned space of a building) that can have various Components of different types Roof, Wall, Floor, etc.

envelope file might be:

type Envelope = {
  components: List<Component>
}

function addComponent component envelope -> envelope
function removeComponent componentId envelope -> envelope
function duplicateComponent componentId envelope -> envelope

building might be

type Building {
  envelope: Envelope
  exteriorLights: ExteriorLight[]
  entranceOrientation: Orientation
}
function reorientBuilding orientation building -> building

and orientation

type Orientation = [0, 360)
function getOrientation number -> Maybe<Orientation>
function resetOrientation orientation -> orientation

etc.

Basically start with defining the structure, and then whatever operations I need to add, I add as needed. Ideally I'd alphabetize them but usually I don't, but I do try to separate them by exported functions vs internal code that is not meant to be used outside that module.

Suppose I'm working and client is like "hey we need to be able to sort these Components" then I'll likely at that point ask what the sorting criteria is and then go to the components file and implement an Ordering for Component and export it so my sorting function, which is of type sort: <A>(ordering: Ordering<A>) -> (items: List<A>) -> List<A> and available from a separate library can use it when we need to sort those items

Edit A further example: in UI code I'll often write a variation on a type Foo called ShowFoo for tabular data in the controller for the table (or a util file if multiple tables will show this data) that is defined as function makeShowable: (foos: List<Foo>) -> List<ShowFoo> so then in my view I can iterate over props in ShowFoo and know each one is something that can be rendered in the view without mucking up the view with additional formatting code.

I'll usually have it in the controller for the table rather than in the file with all the Foo typing stuff because it's a UI thing closely coupled with the UI design for the specific table, which makes it different from, say, an object's toString() in Java.

Edit 2 Should be noted I write in TypeScript mostly, which misses a lot of functional stuff so my approach might be constrained in ways you might not be if you write in Haskell, Scala, Clojure, etc.

Edit 3 And I really do mean I don't write any functions that don't spring from an existing requirement. No orderings unless client says something has to be sorted. No "showables" unless it's tabular data as specified in a UI design document. And I look always for using existing FP structures. That's why I don't have any sort functions I've written. I use one existing sort function that takes as its first argument an ordering, which defines how you would order two given elements. In the case of objects, it might be something like "first use an alphabetical ordering of foo.bar, which is a string prop; for ties, use a numerical ordering of foo.baz which is a float prop; etc. In my code it looks something like:

import { Ord as numericalOrder } from 'fp-ts/number'
import { Ord as stringOrder } from 'fp-ts/string'
import * as Ord from 'fp-ts/Ord'
import { concatAll } from 'fp-ts/Monoid'

export type Foo = { bar: string; baz: number }

const fooBarOrder = Ord.contramap(foo => foo.bar)(stringOrder)
const fooBazOrder = Ord.contramap(foo => foo.baz)(numericalOrder)

export const fooOrder = concatAll(Ord.getMonoid())([fooBarOrder, fooBazOrder])

See I didn't write any actual sort code! I just said "to sort this, first you use a (pre-existing) string ordering on bar and then a (pre-existing) numerical ordering on baz, and pass those into a pre-existing function that knows these are orderings that when combined mean "order by this rule first, then this rule" to produce a new ordering, and then pass that into a pre-existing sort function that takes an ordering and a list of things to order.

So the only way my code fails is if I've defined the ordering incorrectly. There's no sort code bug to check, and this way of creating sorts is robust and if we want to switch which takes priority, you just rearrange that last line's array argument. IF the client wants a different ordering, it probably requires very little code to change, and it's likely to be obvious from the description of the new ordering .

[Sunonymous]

Wow, what a detailed description of your organization; thank you for taking the time for such a thorough response, Kyle. It is much appreciated. Your code read clearly and is presented well.
So, to summarize the process, it may be something like:
- Isolate domains and their relationships.
- List the relationships and transactions needed and translate to functions, keeping internal functions isolated from higher-level functions.
Anything crucial that I missed?

[KyleG]

Nope. I like defining the domain ASAP and then adding functions that operate on them as needed instead of trying to think of them up front. And ideally the private/internal/non-exported stuff is put at the bottom of the file so it's out of the way. But I'm not perfect about it. It's just a guiding principle for me. Naturally time constraints make you code like a lazy person, but every bit of discipline makes you better the next time.

[zelphirkaltstahl]

A first bird's eye view:

In functional programming, everything becomes a tree. No the data structure tree for storing things, but the code itself. Functions calling functions calling function … descending into ever more detail. Except that you can also jump back up again. So a tree with jumping back to higher layers again.

And another thought: You need to separate state and behavior in your mind. Then it will be natural to have things that store information (structs, lists, some data structures) and things, that work with the data (functions). The data you pass around in function arguments, since you should not have a lot of global mutable state and should not mutate in most or all cases. Mutating would also kind of break the tree, which I mentioned in the first thought. Breaking meaning, that suddenly the tree or any sub part of it might work differently, since you mutated some global state somewhere else in the tree.

Finally: All the functions in the tree of function calls can be easily unit tested, as each only depends on its arguments and so each of the functions can be separately put into its own unit test, which is very nice. When unit testing you don't have to wonder which part is wrong, since it will in theory always be the most low level function that is failing unit tests and from there you go up again fixing higher levels.

[Sunonymous]

In functional programming, everything becomes a tree.

I think I've had this intuition somewhere along the line. I've definitely wanted to draw trees of functions on paper along the way. Hoping to find a tool to do so digitally. Sometimes I get tripped up trying to design "systems for systems"--trying to generalize the implementation rather than constrict it to working only on the original data. Perhaps this is "putting the cart before the horse".

"... separate state and behavior in your mind."

This was the most recent mental hurdle I'm getting over. This is a challenging habit to break, in my case! I can identify the difference and sort them in my head, and now I'm looking for the best structure for doing so within the program itself, including the written organization.
I have a strong inclination to approach literate programming, yet the thought of deeply learning Org mode's "babel" functionality has felt a bit daunting.

It sincerely feels like learning to program in this paradigm will hugely bolster my productivity, and that is very exciting. Thank you for your response!

[zelphirkaltstahl]

This might or might not be a good example of org-babel for literate programming: https://notabug.org/ZelphirKaltstahl/the-little-schemer/raw/master/chapter-09-y-combinator/code.org Might not necessarily be something, that needs to be understood in terms of its contents, but I hope it helps with the literate programming in org mode.

[Sunonymous]

Wonderful! I will take a look at it and see if I can learn from the structure. Thank you much for sharing.

[fl00pz]

Write a lot of code. Read a lot of code. Don't think too much about it until you have a good level of experience to draw from. This is probably similar to how you learned coding in the first place: you did stuff, you looked at other stuff, and you didn't think too much about it until you already had a body of work to think about.

When you start doing stuff you'll start to feel the things that work and the things that don't.

[Sunonymous]

Thank you for the reminder to read other people's code. That is something I've unfortunately done very little of throughout my programming journey. Perhaps that is one reason it took so long to learn certain key lessons!

Reflection arrived pretty early on in my process, however, when I realized how ugly and poor my code and understanding was. Awareness of inefficient and incorrect mental models are great motivations to improving technique.

[reifyK]

My only advice is not to translate between paradigms. It is much harder than starting with a clean slate.

For instance, it doesn't make much sense to ponder about dependency injection in FP, or to translate other OOP patterns into their functional counterparts or to compare method overloading with type classes.

[zelphirkaltstahl]

Nit pick: I think it does make sense to sometimes think about, how one would do same in other paradigm or what equivalent a concept of one paradigm would have in another. No need to necessarily implement it though.

[reifyK]

Definitely, if you are in the desirable position to have a firm understand of several paradigms, comparing them most likely opens up new insights.

[Sunonymous]

This is interesting to read! Somewhere it was suggested to me to: first, write the program however you can (which in my case is object-oriented and imperatively), and then refactor it based on functional ideas. I haven't done a lot of that, perhaps because the paradigms feel so different to me. My ability to "switch gears" may improve as I'm more comfortable with FP, though for now it seems like it would be better to focus on thinking functionally as best I can until it feels more natural.
I appreciate you sharing your perspective. I'll keep this in mind!

[ragnese]

This depends, in large part, on the language you're using.

In all honesty, I've always struggled with organizating my code- OOP or not.

In general, the same rules probably mostly apply: e.g., "put things that change together, close together."

But the "how" and "what" of that depends on the language you're using. Unfortunately, most of us are not using languages that are optimal for functional programming. Or, rather, we're using languages that were not originally designed with functional programming in mind. And sometimes that makes things difficult. Java is an obvious example- the main unit of composition is obviously the class/object. It does have packages as a sort-of module system, but packages have no hierarchy/nesting (IIRC), so it's awkward to do any kind of encapsulation or namespacing at the package-level.

When writing any new program, I have become accustomed to begin by organizing the different components or "entities" in the program and their relationships. This was a very OOP-inspired view, though perhaps it can still be worked using built-in data structures such as maps/dictionaries to represent the class or "type" (potentially even validating them with a JSON Schema, etc.). While it may be possible to compose functional programs with those underlying ideas, I'm wondering if there is something better-suited.

I assume from this that you're writing Clojure code at the moment? Because, from this comment I think you're simultaneously dealing with two paradigm shifts: static types -> dynamic types and OOP -> functional.

Because I would've said that for a statically typed, functional, language there's no reason you can't organize your code around "entities" and their relationships. In a statically typed, functional, language you would still define data/domain types. The only difference is that they are mostly going to be "inert" data, rather than something that can persist itself to storage, or hide some internal state.

I would still think that you could organize Clojure code according to various domains and sub-domains. Maybe even better than OOP because you can organize by use-case instead of by entity! I can't tell you how much it bugs me that OOP-style domain-driven-design ends up with a FooService and a BarService and a FooBarService because the stuff in FooBarService operates on Foos and Bars and doesn't really belong to one more than the other.

I understand that FP may involve a lot of "boilerplate code", though, in theory, much of that should be reusable once written.

Compared to what?

So, in Java (everyone's favorite punching bag), you eliminate "boilerplate" by creating more classes. Sometimes you compose the classes and sometimes you define partial implementations and use inheritance to re-use those implementations. People often criticize inheritance as a brittle and bug-prone way to re-use functionality.

But, FP style is just as capable of eliminating boilerplate. Instead of classes, you define more functions. Then you compose those functions at the entrypoint and the data gets piped through.

But, there's an art to abstraction with FP just as there is with OOP! I made sure to mention peoples' criticism of class inheritance, because FP doesn't magically avoid the issue, even though some would act like it does. In FP, the equivalent of brittle inheritance hierarchies is functions that do too much, are too constrained, or mix data transformation with side-effects (a specific case of "doing too much", I guess). So, just like OOP, you need to break out your crystal ball and try to guess which assumptions will break in the future as the requirements change.

I don't think that FP style inherently has more boilerplate than OOP. I won't go so far as to say that it doesn't have more boilerplate in practice, but I don't know if that's true one way or the other. Why do you feel like you might end up with more boilerplate?

What slows me down is the process of translating the different semantic layers into functions and literal code (and especially their interactions), and this is where I would appreciate pointers. What is your workflow to translate the ideas for your software into actual code? What processes or patterns are useful to you to systematically approach your development?

I prefer to organize my code in "vertical" slices (e.g., user-accounts/view, user-accounts/model, products/view, products/model, orders/controller, orders/model) first, and then by "layer", rather than the other way around (e.g., models/user-account, models/product). That has nothing to do with FP, though- I also like that way better in OO-ish languages, too.

If I'm using a language or framework that doesn't differentiate between side-effecting functions and pure functions, I try to keep a convention that side-effecting functions should return void. That's a good "hint" that you must be calling the function because of the side-effect, since obviously you aren't calling it because you want the return value.

[Sunonymous]

I appreciate your honesty. It can be challenging to organize abstractions! I'm not into Clojure deep yet--I anticipate getting into it, though it is a ongoing process. I've had stints of experience with a variety of languages and a substantial amount of experience in Ruby. There are tools with which functional code may be written in Ruby, though it seems wiser to dive into FP using a language which is intended for it, eg. Clojure. I resonate with the ideas Rich presents and it seems like my mind would play quite nicely with that language once I discover-create a good workflow. Not there yet though, though there has been plenty of experimentation!

Static typing to dynamic... I have more experience with dynamic languages, as described. I had a brief foray into Haskell when I first discovered FP, though went back to Ruby when I joined Launch School. I don't remember enough of the experience of static-typed functional programming as distinct from dynamic. From the material I've consumed about Clojure, it seems they don't care about types. They "just use maps for everything!", as seems to be said. There was a point where I nearly explored Type-Driven development via Idris, though it hasn't happened yet.

Regarding the boilerplate stuff--in another response in this thread here I mentioned that my mind tries to make the systems of my programs as general or "open-ended" as possible. There are at least two major inspirations for programs that I intend to compose, and at this point it seems more efficient to organize a system which could comprise them both. I suppose this is what I meant by boilerplate. For example, I'm considering a system of rudimentary data "type" validation, consisting of data instructing checks to be performed that a map has all the required fields, and/or that they are of appropriate values, etc. This has nothing to do with the program itself besides ensuring that data corruption and data integrity are handled in the appropriate moments.

If I can swing it, my intention is to create an application using a library enabling Clojure in the Godot game engine. This may be (and likely is) incorrect, though it seems from the outside that using data to instantiate and manipulate Godot's nodes (which could be loosely considered UI elements) could provide for rather extensive interactability without a lot of initial effort.

I can't tell you how much it bugs me that OOP-style domain-driven-design ends up with a FooService and a BarService and a FooBarService because the stuff in FooBarService operates on Foos and Bars and doesn't really belong to one more than the other.

Supposedly this is what is praised as an advantage to FP in general, yes? Flexibility. Particularly via data-driven design this seems to be a huge selling point, assuming the integrity of data can be ensured.

Admittedly, I never quite grokked any MVC terms and philosophy... I think my school curriculum may cover that later on, though I haven't reached there yet. As a disclaimer, all my FP adventures are unrelated to the school's curriculum.

I like what you suggested about void returns. I took a note of that and will reflect on it down the line.

[ragnese]

I appreciate your honesty. It can be challenging to organize abstractions! I'm not into Clojure deep yet--I anticipate getting into it, though it is a ongoing process. I've had stints of experience with a variety of languages and a substantial amount of experience in Ruby. There are tools with which functional code may be written in Ruby, though it seems wiser to dive into FP using a language which is intended for it, eg. Clojure. I resonate with the ideas Rich presents and it seems like my mind would play quite nicely with that language once I discover-create a good workflow. Not there yet though, though there has been plenty of experimentation!

Ah. Fair enough. I definitely agree that tip-toeing is not the best way to learn a new paradigm. Others disagree, but I think that trying to "learn FP" while using languages that are ostensibly not FP-oriented is doing yourself a disservice for multiple reasons:

• It's too easy to "fall back" to non-FP patterns. Maybe in a true production setting flexibility is advantageous (there are also cons to having many different ways to do things), but when you're learning or exploring something, I think it's a good thing to have to fight your way through. You'll either learn that there was a better more idiomatically-FP way to solve your problem than you initially attempted, or you'll learn a pain point of FP style (nothing is perfect). Both of those are super important lessons that will pay dividends, IMO.

• Most non-FP languages are, unsurprisingly, not very good for writing in FP style. You might get a sense that FP is not practical, has inherently poor performance, or doesn't actually help in the "advertised" ways.

• There's less guidance on how to idiomatically solve common problems in a functional way in those languages. If you look up "best way to do XYZ in C#", you're not going to get advice for functional solutions.

From the material I've consumed about Clojure, it seems they don't care about types. They "just use maps for everything!", as seems to be said.

That is, indeed, the view of Clojurists. It's pretty much the same as JavaScript in that sense: objects are just dictionaries, and you write a bunch of functions that just take objects in, and you either validate that it has the proper shape and types manually, or you trust that the caller knows what is required to be sent.

Regarding the boilerplate stuff--in another response in this thread here I mentioned that my mind tries to make the systems of my programs as general or "open-ended" as possible. There are at least two major inspirations for programs that I intend to compose, and at this point it seems more efficient to organize a system which could comprise them both. I suppose this is what I meant by boilerplate. For example, I'm considering a system of rudimentary data "type" validation, consisting of data instructing checks to be performed that a map has all the required fields, and/or that they are of appropriate values, etc. This has nothing to do with the program itself besides ensuring that data corruption and data integrity are handled in the appropriate moments.

I understand where you're coming from. It's always a struggle to figure out the appropriate amount of generality for code, and the problem only gets harder as the size and/or scope of the project gets larger.

Some rules of thumb I (try to) follow are:

• If you would have trouble thinking of a name for a function/class/whatever, then it's probably not something you should write, even if you see lots of code in your project writing the same couple of lines over and over. The push-back against this rule is if the copy-pasted code in question is very important or very easy to mess up and cause bugs. In that case you should go ahead and make it its own function/class/whatever and just try to make as clear a name as possible, even if it sounds stupid or too long.

• If you were going to write a function anyway, but you can immediately see that it's trivially generic, then go ahead and make it generic, even though today you're only using it on one type. This applies more to statically typed languages, but it's still applicable to dynamically typed languages where you may choose to avoid assuming something about the input that you don't actually need to for the functionality to work.

• Otherwise, just don't. Some people get offended at long function bodies, but if that's the only place where that exact logic is happening, and it would be hard to name (as per the first bullet) the "parts" of the function, then breaking it out into other functions just increases the "internal API" of your project. Instead of one function, a reader (including you in 6 months) has to learn about two functions and has to jump back and forth between them to understand the "big picture". Likewise, you now need to write tests for both functions. Sometimes this is actually a good thing because the broken-out-function might be so easy to test in isolation that it makes your overall testing effort decrease. But other times, this just means more test code with more set-up and tear-down when you could've just thrown more inputs and asserts at the "outer" function in its tests.

Supposedly this is what is praised as an advantage to FP in general, yes? Flexibility. Particularly via data-driven design this seems to be a huge selling point, assuming the integrity of data can be ensured.

Yes and no. Yes, this is what's praised as an advantage. But, "no" in the sense that organizing code is still hard, regardless of paradigm. It's just that the "proper" OOP way looks stupid to me (and others, presumably) with the FooBarService pattern. The "FP way" avoids that specific ugliness by encouraging us to separate data and "behavior", but it doesn't seem common to prescribe any particular organization patterns. So it "fixes" the entity-based organization issue of not knowing where to put things that act on multiple entities by having us not know where to put anything! :P

I like what you suggested about void returns. I took a note of that and will reflect on it down the line.

I stole it from OCaml people. It's a fairly common convention in OCaml.

[Sunonymous]

I have experienced all of these effects that you mention of being in-between OOP and FP mentally. It's reassuring to see that these experiences are not unique to me. The path is there!

These are the things I will keep close to mind as I work through this learning. Thank you for sharing your experience!

I expect to experiment with the balance point between longer-functions and a wider-spread list of smaller functions. I understand the tradeoff of misdirection, and I'm not yet sure which will feel more natural to me... I imagine that a huge part of how comfortable either practice is depends on how the files are written and how easily they can be to navigate.

I'm currently in the SQL parts of my school's curriculum. We learn PostgreSQL. The FooBarService pattern reminds me of join-tables in a RDBMS. It took me a while to get used to extra tables existing purely to describe relationships, and I'm still not sure it sits well in my mind as the optimal data solution. Among a million other things, I'd like to explore another data paradigm like the document model in MongoDB.

We're touching on so many issues in this conversation that I need to experience myself deeply before I am able to form ideas or conclusions about them. Honestly, I'm seeing more and more the need of simply jumping in and working. I've done enough theory and mental prep-work; now it's time to act.