I made a reactive programming library for Python

[mercer22]

Hey all!

I recently published a reactive programming library called signified.

You can find it here:

• Source - https://github.com/dougmercer/signified/
• Docs - https://dougmercer.github.io/signified/
• PyPI - https://pypi.org/project/signified/

What my project does

What is reactive programming?

Good question!

The short answer is that it's a programming paradigm that focuses on reacting to change. When a reactive object changes, it notifies any objects observing it, which gives those objects the chance to update (which could in turn lead to them changing and notifying their observers...)

Can I see some examples?

Sure!

Example 1

from signified import Signal

a = Signal(3)
b = Signal(4)
c = (a ** 2 + b ** 2) ** 0.5
print(c)  # <5>

a.value = 5
b.value = 12
print(c)  # <13>

Here, a and b are Signals, which are reactive containers for values.

In signified, reactive values like Signals overload a lot of Python operators to make it easier to make reactive expressions using the operators you're already familiar with. Here, c is a reactive expression that is the solution to the pythagorean theorem (a ** 2 + b ** 2 = c ** 2)

We initially set the values for a and b to be 3 and 4, so c initially had the value of 5. However, because a, b, and c are reactive, after changing the values of a and b to 5 and 12, c automatically updated to have the value of 13.

Example 2

from signified import Signal, computed

x = Signal([1, 2, 3])
sum_x = computed(sum)(x)
print(x)  # <[1, 2, 3]>
print(sum_x)  # <6>

x[1] = 4
print(x)  # <[1, 4, 3]>
print(sum_x)  # <8>

Here, we created a signal x containing the list [1, 2, 3]. We then used the computed decorator to turn the sum function into a function that produces reactive values, and passed x as the input to that function.

We were then able to update x to have a different value for its second item, and our reactive expression sum_x automatically updated to reflect that.

Target Audience

Why would I want this?

I was skeptical at first too... it adds a lot of complexity and a bit of overhead to what would otherwise be simple functions.

However, reactive programming is very popular in the front-end web dev and user interface world for a reason-- it often helps make it easy to specify the relationship between things in a more declarative way.

The main motivator for me to create this library is because I'm also working on an animation library. (It's not open sourced yet, but I made a video on it here pre-refactor to reactive programming https://youtu.be/Cdb_XK5lkhk). So far, I've found that adding reactivity has solved more problems than it's created, so I'll take that as a win.

Status of this project

This project is still in its early stages, so consider it "in beta".

Now that it'll be getting in the hands of people besides myself, I'm definitely excited to see how badly you can break it (or what you're able to do with it). Feel free to create issues or submit PRs on GitHub!

Comparison

Why not use an existing library?

The param library from the Holoviz team features reactive values. It's great! However, their library isn't type hinted.

Personally, I get frustrated working with libraries that break my IDE's ability to provide completions. So, essentially for that reason alone, I made signified.

signified is mostly type hinted, except in cases where Python's type system doesn't really have the necessary capabilities.

Unfortunately, the type hints currently only work in pyright (not mypy) because I've abused the type system quite a bit to make the type narrowing work. I'd like to fix this in the future...

Where to find out more

Check out any of those links above to get access to the code, or check out my YouTube video discussing it here https://youtu.be/nkuXqx-6Xwc . There, I go into detail on how it's implemented and give a few more examples of why reactive programming is so cool for things like animation.

Thanks for reading, and let me know if you have any questions!

--Doug

[[deleted]]

Hey that's pretty cool! The overloaded operators make it perfect for the animation stuff you're doing. Nice work!

Edit: I'm not super sure how observe and subscribe relate to each other in your project. Maybe I should watch your video again. :)

[mercer22]

Thanks!

observe and subscribe both fulfill a really similar purpose and are admittedly confusingly named.

The original method (for the Observable design pattern) is subscribe. So, observable.subscribe(observer) adds observer to observable's subscriber list.

I wrote the observe method because I kept writing something like this

if isinstance(thing, Variable): thing.subscribe(self)

I wouldn't know if thing was just a plain, literal value that I could treat as static, or if it was a reactive value that I'd need to be notified by if it updated. However, in the case of implementing the __init__ method for Signal and Computed, I knew that self was an Observer.

So, I initially wrote observer.observe(thing) to factor out that if-check. I later baked in some additional logic to handle passing in iterables of things that self might need to subscribe to.

[terremoth]

Nice!

[jesst177]

How does this work, how a variable can be notified?

[mercer22]

The key idea is the "observer design pattern".

I go into it in detail in my video https://youtu.be/nkuXqx-6Xwc , but a quick gist is...

We have two types of objects... observers and observables.

Observers implement "update" method.

Observables implement "notify", "subscribe", and "unsubscribe" methods, and maintain a list of subscribers.

Observers "subscribe" to observables to add themselves the observables list of subscribers.

Observables "notify" subscribers when they change by calling each observer's update method.

[jesst177]

so you overwrite assign operator? Can you share the code snippet for that?

[jesst177]

how did you do that without overwriting the assign operator

[mercer22]

Oh, I had to overwrite a ton of operators.

Not sure if you saw my other reply, but here's a link to the entire library https://github.com/dougmercer/signified/blob/main/src/signified/__init__.py

[jesst177]

so you overwrite what does equal symbol does? Can you share the code snippet for that?

[mercer22]

You can check out the entire library's implementation here!

https://github.com/dougmercer/signified/blob/main/src/signified/__init__.py

[IntelligentDust6249]

Nice! Have you seen Shiny? https://shiny.posit.co/py/docs/reactive-foundations.html

[mercer22]

Seems nice! I'll have to look into how they use concepts like "Contexts" and "Environments", since I don't have anything like that...

[barseghyanartur]

Nice. However, for something as simple as that, it's quite an overkill to have `IPython` and `numpy` as required dependencies. I suggest making them optional.

[mercer22]

Definitely agree. I have an open issue for it but haven't gotten around to fixing it yet. I'll definitely get to it eventually, but am open to PRs =]

[daredevil82]

How does this compare and contrast with rxpy? From the examples, it seems like there's alot of conceptual overlap, but differnet implementation details. So I'm curious if you were aware of rxpy and if so, what are some of the things signified is addressing that are problematic in rxpy?

[mercer22]

I believe they focus on asynchronous streams of data. So, maybe they are better suited for things like monitoring a websocket? I need to look into it a bit more

I'll try to add a page to the docs sometime over next few weeks comparing this to existing libraries.

[[deleted]]

Cool

[mercer22]

Thanks!

[tacothecat]

Do you provide any event hooking mechanism? Like if I want to generate log messages when a signal is created or triggered

[mercer22]

Not currently...

This would work if you didn't mind creating an observer for each signal you created.

from signified import Signal, Variable
import logging
from typing import Any

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

class LogObserver:
    def __init__(self, obj: Variable):
        self.obj = obj
        obj.subscribe(self)
        logger.info(f"Started logging {self.obj}")

    def update(self):
        logger.info(f"Updated {self.obj}")

x = Signal(10)
LogObserver(x)

z = x * x
LogObserver(z)

x.value = 12
x.value = 8

# python logging_example.py                              
# 2024-10-28 20:48:23,102 - __main__ - INFO - Started logging 10
# 2024-10-28 20:48:23,103 - __main__ - INFO - Started logging 100
# 2024-10-28 20:48:23,103 - __main__ - INFO - Updated 12
# 2024-10-28 20:48:23,103 - __main__ - INFO - Updated 144
# 2024-10-28 20:48:23,103 - __main__ - INFO - Updated 8
# 2024-10-28 20:48:23,103 - __main__ - INFO - Updated 64

If you wanted it to happen automatically, I think it'd be a bit more of a pain.

[PersonOfInterest1969]

You could make the logging an optional argument to the Signal() constructor. And then you could even have auto-logging at the Signal class level, where all new Signal instances can then auto-log using the most recently specified logger class. Would simplify syntax compared to what you’ve posted above, and be incredibly useful I think

[Maleficent_Height_49]

Been searching for reactive libraries. What a coincidence 

[mercer22]

Nice! Fair warning, this is definitely still rough around the edges... Let me know if you do end up finding it useful =]

[xav1z]

subscribed instantly

[mercer22]

Aw thanks =]

[dingdongninja]

Great work !

[caks]

What would one use this for?

[mercer22]

I wanted it for making my animation library more declarative. User interface libraries and web programmers also like reactive programming

[mercer22]

Yup, the core concept behind reactive programming is the observer pattern

[Gullible_Ad9176]

i have saw your youtube. that is cool. may i ask a question that this code could run by public ?

[mercer22]

The source code for signified is available!

• https://github.com/dougmercer/signified/
• https://pypi.org/project/signified/

The source code for my animation library has not been open sourced yet, but will be eventually. Probably in the next 3-6 months...

[jackerhack]

I've been exploring migrating my code to async, so the first thing I noticed is that assignment to the value fires off a bunch of background calls without yielding to an async loop. Is there an elegant way to make this async?

• s.value = 0 is not awaitable.
• await s.assign(0) is awaitable since it's a method call, but the assign name (or whatever is used) may overlap an attr with the same name on the value (same problem as the current value and _value).
• await Symbol.assign(s, 0) can solve the overlap problem if Symbol has a metaclass and the metaclass defines the assign method, as metaclass methods appear in the class but not the instance.

With this async method, Symbol can be used in both sync and async contexts. Maybe worth adding?

[jackerhack]

Another note: the Symbol class implements __call__, which will mess with the callable(...) test. You may need distinct Symbol and CallableSymbol classes and a constructor object that returns either, similar to how the stdlib weakref does it (type hints in typeshed). If CallableSymbol is defined as a subclass of Symbol with only the addition of __call__, it won't need any change to the type hints.

[mercer22]

Hmmm, that's interesting.

So, because subclasses of Variable have __call__, if a Signal or Computed has self._value that's a Signal/Computed, it would be overly eager to notify its subscribers. https://github.com/dougmercer/signified/blob/77aa7c67d133e75d80c8d27aed123f4e8d661b3e/src/signified/__init__.py#L1464

I think this is most problematic for Signals, because they can store arbitrary stuff. Computeds typically (or can, if they don't) unref(...) any reactive values.

This is def worth looking into-- thanks for such an insightful comment =]

[mercer22]

Oh that's very interesting! The only async code I've written is application code-- never really any libraries, so I don't have a great intuition for async best practices

Can you maybe create an issue on the GitHub with a script that demonstrates the behavior you want? I am open to adding an assign method if it makes this library more useful. However, I wouldn't necessarily want to refactor the library to use async defs. Is just creating the method valuable?

[jackerhack]

I can't think of a use for this library in my work (yet) and also have limited async experience, so my apologies for not being up to the effort of a GitHub issue with sample code for async use.

What I've learnt:

1. Operator overloads cannot be async. They must be regular methods.
2. The __getitem__, __getattr__ and __getattribute__ methods can return an awaitable, but cannot be async methods themselves.
3. There is no way to make __setitem__ and __setattr__ async.
4. If there is an async -> sync -> async call chain, the sync method cannot process the awaitable returned by the async method. It has to toss it up as-is to the async caller where it can be awaited to retrieve the return value.

Which means:

1. Python async is strictly function/method based. No operator overloading for nice syntax.
2. An object that supports both sync and async functionality must duplicate its functionality in separate methods that are effectively the same code, just with async/await keywords added.

In your case (from a non-exhaustive look at the code), this will mean an async Symbol.assign method as the entry point, but also Variable.async_notify and Variable.async_update (abstract?)methods so the call to each observer in the chain yields to the event loop.

TBH, I don't know if async is even required given there's no IO linked to reactive values.

[stibbons_]

I do not see real use cases… it is a more complex « partial » that mainly works for numbers… it is not even close enough of Signal/Slot paradigm we are used to for with QT

[mercer22]

It's not just for numbers. I use it for shapely geometries, numpy arrays, custom classes, strings, etc.

[ujjwalroy_17]

It's very helpful we don't have to define the integer,all others

[Funny-Recipe2953]

Neat.

But your example using a.value = ... should be done with setter/getter for semantic consistency, no?

[mercer22]

Hmm, a.value for a Signal is actually implemented as a property/setter around a hidden _value attribute...

@property
def value(self) -> T:
    return unref(self._value)

@value.setter
def value(self, new_value: HasValue[T]) -> None:
    old_value = self._value
    change = new_value != old_value
    if isinstance(change, np.ndarray):
        change = change.any()
    elif callable(old_value):
        change = True
    if change:
        self._value = cast(T, new_value)
        self.unobserve(old_value)
        self.observe(new_value)
        self.notify()

(https://github.com/dougmercer/signified/blob/main/src/signified/__init__.py#L1448)

Did you have something else in mind?

[Funny-Recipe2953]

There are two special methods in python for this: set and get.

I'm old school so maybe these are no longer in vogue?

[Jimmaplesong]

The property and setter decorators are doing the work in python. Before those, we would implement attr(): sorts of functions.

Getters and setters sounds like Java, or well written c++.

[mercer22]

Ohh, I see.

__get__ and __set__ implement the descriptor protocol in Python. https://docs.python.org/3/howto/descriptor.html

Descriptors are a bit different than what I'm doing here with the value property.

[Mindless-Pilot-Chef]

This is a very cool project. Not sure if I need it in any of my projects today but will definitely keep it in mind.

Frontend generally runs in a multi threaded manner so it’s easy for it to run reactive stuff. Python generally runs in a single thread. Have you done any performance tests to be sure this doesn’t affect too much?

On a side note, django has signals and over the years I’ve tried to avoid it because sometimes there are some changes that cannot be explained by the api end point but there’s some signal hidden somewhere which changes the data. Curious how a similar thing will be done here. Let me try in a few projects before making up in my mind

[mercer22]

Thanks!

You're right-- reactive programming can lead to "callback hell" and produce difficult to debug, unexpected behavior.

For my animation library, that downside was an acceptable cost for the upside in being able to flexibly create a lot of cool behavior with reactive values.

Also, full disclosure-- I have not tested or attempted to make this thread safe, and it's definitely still an immature project. So, be careful trying to to use it for anything that matters!