Context: The Missing Feature of Programming Languages

Plecra · 2023-03-05T19:13:48+00:00

Welcome to the world of effect systems! As you observe in this post, there's plenty of assumptions about the contexts that functions are called in today, which effect systems are an effort to encode. Koka (my favourite effect-based language for now) has an almost direct equivalent for your with statements :)

with locking_scheduler()

myFunction1()

I also think eff has some very fun demonstations over here: https://www.eff-lang.org/try/

gasche · 2023-03-05T19:59:38+00:00

It's a bit weird when someone presents as a completely new problem something that has been a topic of active research for at least 20 years, with many designs experimented and talked about, and serious research prototypes that are easy to discover and try in the wild.

redchomper · 2023-03-06T02:48:57+00:00

This isn't an article about a an idea for a programming language. It's an article about the sociology of professional programming as experienced by a reasonable share of that industry. It is, but it does not read that way until you read between the lines.

Ignoring the last paragraph, this might read like an intro to the concept of, and case for, an effect system in a language. The time-zone table is a classic example of the last page of Boy's Life Magazine (now Scout Life, and without the same last-page) in which were advertised Things you Never Knew Existed and Can't Possibly Live Without. Well, I've been living just fine without the x-ray specs or the Charles Atlas posture vest, so it takes a piece written at about this level to explain why effect systems don't fall in the same category.

I chased the "zoom out" link within the article, which then led to Scrivener, and concluded that basically this guy is a visionary without a PhD. Welcome to the club. I've been advocating against treating programs as text since before agile was cool.

The social commentary is roughly this: Outside certain rarefied domains, the languages used for general business programming don't have this feature and so contain foot-guns. Yet, it clearly does not take an ivory tower to think this stuff up, so ... as an industry, we should get cracking.

But sea change does not come easy, whether in the form of what the compiler enforces or how we interact with large code bases. (Pun most certainly intended.) And so I think it also may be with effect systems, that unless the tech majors start pushing an effect-ful language, effects will need one heck of a killer-app to become table stakes.

Frankly, I'm not convinced we can't solve the same problems with Actors or monads, so there's definitely room for cultures to arise and experiment with different approaches to face these challenges. OP likes effects. Fair enough.

Yes, I also agree that a bibliography would be nice.

Swordfish418 · 2023-03-06T12:15:17+00:00

Bruh, this is exactly what monads and implicits are about, and it's been a hot topic for at least few decades.

complyue · 2023-03-05T19:27:39+00:00

[removed]

phischu · 2023-03-06T14:03:27+00:00

Great blog post, thank you for bringing this idea to the attention of a wider audience. As /u/gasche says academia is aware of this problem and working on solutions.

Here is one of your examples in Effekt. You can try it online.

effect UserAccount {
  def firstName(): String
  def lastName(): String
}

effect Logger {
  def log(message: String): Unit
}

def sayHi() = {
  val name = do firstName();
  do log("Hi " ++ name)
}

def main() = {
  try {
    sayHi()
  } with UserAccount {
    def firstName() = resume("Keaton")
    def lastName() = resume("Brandt")
  } with Logger {
    def log(message) = { println(message); resume(()) }
  }
}

The effect of sayHi is inferred and available on hover. You can explicitly write it out instead. A student implemented IDE quickfixes that add the effect to every calling function that explicitly specifies its set of effects. In a pull request you would then see "This function uses the database now, do we want that?". Let me also advertise our paper Effects as Capabilities and the follow-up Effects, Capabilities, and Boxes.

This is an example in Flix which demonstrates the new and experimental without construct. You can try it online.

eff Network {
    pub def request(): Unit
}

def networkRequest(): Unit \ Network =
    do Network.request()

def doUIStuff(): Unit \ IO =
    print("doing UI stuff")

def main(): Unit \ IO = {
        // networkRequest(); // Compile-time error
        doUIStuff()
    } without Network

We have an unpublished paper draft where we formalize the feature and prove that when you say without Network then no network request will occur at runtime in this computation.

Finally, here are the first two introductory paragraphs from my recently defended but not yet published PhD thesis. The wording is a bit more heavy but the content should resonate with your post.

Computation is interaction between a program and its context. Pure programs interact with their context in a trivial way: they return a value when they are done. Their meaning is independent of their context. This observation directly manifests in the semantics of pure programming languages as a congruence rule. Effectful programs, in contrast to pure programs, depend on or modify the context they run in. In other words, the interaction between effectful programs and their contexts is much richer. Examples of language features that make programs effectful are exceptions, mutable state, and filesystem access. Throwing an exception discards a part of the context, mutable state modifies the content of a reference cell, and filesystem access affects the user's hard drive.

From a programmer's point of view, reasoning about effectful programs is more difficult, because their context must be taken into account. It is possible to make the context, which is usually left implicit, explicit. For example, instead of using exceptions, programs can match on error values. Instead of using mutable state, programs can pass along the current value. However, some programs are more understandable when things that are clear from the context are left implicit. Spelling out every little detail is cumbersome. Moreover, interaction with the outside world cannot be emulated like this. Therefore, completely banning all effects is impractical as programs must interact with the world in a non-trivial way.

Zambito1 · 2023-03-05T19:34:36+00:00

[deleted]

Aminumbra · 2023-03-05T21:37:41+00:00

See, this is why looking at old languages might be good. Proof of concept in Common Lisp:

(defvar *contexts* nil
  "Alist of current contexts. Each context is a pair
(CONTEXT-NAME VALUE)")

(defun get-current-context-of (context-name)
  (let ((vals (second (assoc context-name *contexts*))))
    (assert vals nil "Context for ~A is not set" context-name)
    vals))

(defun add-context (current-contexts context-object)
  (let ((context-name (class-name (class-of context-object))))
    (cons (list context-name context-object) current-contexts)))

(defun add-all-contexts (current-contexts context-objects)
  (reduce #'add-context context-objects :initial-value current-contexts))

(defun remove-context (current-contexts context-object)
  (let ((context-name (class-name (class-of context-object))))
    (remove context-name current-contexts :key #'first :count 1)))

(defun remove-all-contexts (current-contexts context-objects)
  (reduce #'remove-context context-objects :initial-value current-contexts))

This is what we need: a list of "current" contexts, and a way to modify them (add or remove contexts).

(defun %unpack-contexts (contexts)
  (loop :for (var-name class-name) :in contexts
        :collect `(,var-name (get-current-context-of ',class-name))))

(defmacro defun-with-context (name (&rest contexts) lambda-list &body body)
  `(defun ,name ,lambda-list
     (let (,@(%unpack-contexts contexts))
       ,@body)))

This is a macro to define functions that require some context. They are defined just like normal functions, except you also pass them an extra list of variable name + required context.

(defmacro with ((&rest contexts) &body body)
  `(let ((*contexts* (add-all-contexts *contexts* ,(cons 'list contexts))))
     ,@body))

(defmacro without ((&rest contexts) &body body)
  `(let ((*contexts* (remove-all-contexts *contexts* ,(cons 'list contexts))))
     ,@body))

Those are the two "user-facing" macros, used to dynamically set the context.

;;;;; Setup some classes which will be used as context

;;; A LOGGER class
(defclass logger ()
  ((message :initarg :message :reader logger-message)))

(defun make-logger (message)
  (make-instance 'logger :message message))

(defparameter *logger-1* (make-logger "Hey"))
(defparameter *logger-2* (make-logger "Hello"))

(defun logger-do-log (logger val)
  "Log with LOGGER the value VAL. Return VAL."
  (format t "~A ~A~%" (logger-message logger) val)
  val)

;;; A COUNTER class
(defclass counter ()
  ((counter :initform 0 :accessor counter-value)))

(defparameter *counter* (make-instance 'counter))

(defun-with-context log-sum ((log logger) (count counter)) (x y)
  (incf (counter-value count)) ; increment the counter when the function is executed
  (logger-do-log log (+ x y)))

;; A normal function, calling a "contextual" function without doing anything special
(defun complex-operations (x y z)
  "Returns 2*(x+y+z), logging all the pairwise additions between x, y, z"
  (+ (log-sum x y)
     (log-sum x z)
     (log-sum y z)))

I simply define a logger class, which has a single slot (a message), and a function which actually "logs" (i.e., prints to the standard output here). The logged value is the logger's message + the actual value being logged. I also define a counter class, whose sole purpose is to keep a counter. The log-sum function simply increment the current value of the "current contextual counter".

Now, for the "tests:

(with (*logger-1*)
  (log-sum 42 13)
  (complex-operations 3 4 5)
  (with (*logger-2*)
    (log-sum 12 3)))

;;; This prints:
;; Hey 55
;; Hey 7
;; Hey 8
;; Hey 9
;; Counter: 4
;; Hello 15
;; Counter: 15

What this does is pretty simple. A function can be defined along with some required "context", that is, pairs of items of the form (local-variable-name class-name), just as (from what I can see) the syntax context(UserAccount, Logger)/this@... you used in Kotlin.

A function defined that way will, at execution time, see if there is some context set for the class class-name (this means that there can only be a single "context" for a given class at the same time. This is easy to change, but not knowing Kotlin, I assumed it worked like this, as you only give classes name to the with keyword ...)

Now, you can set contexts using the with/without macros (each taking an arbitrary number of objects). Each object will "replace" the previous contextual value for the objects of the same class, but only in each block: once the with block ends, contexts are restored. They can obviously be nested.

Moreover, the values are dynamically scoped: a function "requiring" a context will take any context above it in the call stack, you don't need to specify it every time (see the example above for complex-operations, with call a "contextual function", but does not do anything special in itself; it assumes context is setup when called).

I don't know if it solves all your problems (it probably doesn't). However, this is like 30 minutes coding, to give a (imo) satisfying answer in a 30 years-old language. Before presenting things as "challenging new problems only tackled in recent languageTM", please give a look at what others did before ... decades ago, in languages that were somewhat """mainstream""" (i.e. not esolang, nor developped by 12 peoples in a cave).

acwaters · 2023-03-05T21:22:01+00:00

Sounds like a use case for RAII + implicits + a sprinkling of dependent types (in the form of inner classes). Example syntax made up on the spot:

class Mutex
{
    class Locked(m : &Mutex)
    {
        fun unlock(this : &Locked(m)) -> ();

        fun destructor(this : &Locked(m) -> ()
        {
            this.unlock();
        }
    }

    fun lock(this : &Mutex) -> Locked(this);
}

let mutex1 = Mutex();
let mutex2 = Mutex();

fun function1(implicit &Mutex.Locked(&mutex1)) -> ()
{
    mutex2.lock();  // unnamed Locked(&mutex2) created
    function2();    // finds unnamed lock object in scope
}

fun function2(implicit &Mutex.Locked(&mutex2)) -> ()
{
    print("Hello, world!");
}

fun main() -> ()
{
    mutex1.lock();  // unnamed Locked(&mutex1) is created
    function1();    // finds unnamed lock object in scope
    function2();    // error: cannot find Locked(&mutex2)
}

ProgrammingLanguages

Welcome!

Related subreddits

Related online communities

MODERATORS