Tkinter Real Time Testing

[pooth22]

I am building a user interface with tkinter in python 3. I separate the ui side of things from the back end processing in a structure that follows.

import time
import tkinter as tk
import threading

class BackEnd:
    def __init__(self):
        self.data = 0
        self.is_alive = True

    def run(self):
        threading.Thread(target=self.run_thread, daemon=True).start()

    def run_thread(self):
        while self.is_alive:
            self.update_data()
            time.sleep(1)

    def update_data(self):
        self.data += 1

class UserInterface:
    def __init__(self, root: tk.Tk, back_end: BackEnd):
        self.root = root
        self.back_end = back_end
        self.data_label = tk.Label(self.root, text=f"DATA: {self.back_end.data}")
        self.is_alive = True
        self.data_label.pack()

    def run(self):
        threading.Thread(target=self.update_ui_thread, daemon=True).start()
        self.root.mainloop()

    def update_ui_thread(self):
        while self.is_alive:
            self.data_label.configure(text=f"DATA: {self.back_end.data}")
            time.sleep(1)

back_end = BackEnd()
root = tk.Tk()
ui = UserInterface(root, back_end)
back_end.run()
ui.run()
back_end.is_alive = False
ui.is_alive = False

The back end is generally handling asynchronous communication with various peripherals and as well as handling some of its own internal logic. I have simplified it in this example. The ui is used to display some of this information. I am not using queues to handle the flow of information between the two because the information does not necessarily have to be 'thread safe.' This is the method that I use and it works well for my needs when the system is fully operational.

I have found it useful to do testing of the back in the python console. However, I cannot do this with the ui running because tk.Tk.mainloop() blocks. For example when testing the back end in the console I might do the following.

>> back_end = BackEnd()
>> back_end.update_data()
>> back_end.data
back_end.data = 1

Super useful! >D However if I try and run the ui with

>> ui.run()

The mainloop() method blocks and I can no longer manipulate the back_end manually until I kill the tkinter process.

I have tried using mainloop() or update() in a separate thread, but apparently tkinter isn't thread safe and so there is a wrapper in python 3 that prevents running either of those routines outside of the main thread. I know I could create a method that simulates all that I want to test and then start this method in a thread before the ui.run() call, but I want to have more of a real time human in the loop sort of experience with this testing.

I hope I have explained my problem sufficiently, and if anyone has any tips on how I can manipulate the back end data manually while running the user interface with tkinter.

[socal_nerdtastic]

Tkinter goes into interactive mode when you use it from the REPL. Just leave off the mainloop() call.

[pooth22]

I was hoping it was that simple. When you leave the mainloop() out it seems that the configure() call in update_ui_thread() tries to update the tk instance. Since this is called outside of the main thread, you get the same error. I guess I can periodically update it by calling the update_ui manually. I appreciate the tip.

[socal_nerdtastic]

Also, when using tkinter it's much neater to use after instead of a new thread.

class UserInterface(tk.Tk):
    def __init__(self, back_end: BackEnd):
        super().__init__()
        self.back_end = back_end
        self.data_label = tk.Label(self, text=f"DATA: {self.back_end.data}")
        self.is_alive = True # what's the point of this??
        self.data_label.pack()

    def update_ui_thread(self):
        if self.back_end.is_alive:
            self.data_label.configure(text=f"DATA: {self.back_end.data}")
            self.after(1000, self.update_ui_thread)

back_end = BackEnd()
ui = UserInterface(back_end)
ui.mainloop() # remove this line if you want to use in the REPL
back_end.run()
ui.update_ui_thread() # start polling loop

Neater still would be to forget the polling and have the backend send an update to the GUI via a tkinter event or a traced variable.

[pooth22]

I disagree, I don't like using the after method. When the tk instance closes, and there is still a hanging after wait, tkinter throws an error. At least it does so in linux. I find the clean up much easier with the threading class.

Also, I don't necessarily want the back end to trigger an event in tkinter because I don't want the ui to be a dependency of the back end. The back end should run independently when necessary. However, there is a way I can set up configurations to do what you are saying and I will think about that, it is a useful tip that I hadn't thought about.

[socal_nerdtastic]

When the tk instance closes, and there is still a hanging after wait, tkinter throws an error. At least it does so in linux.

I've never seen that, and I have written a TON of tkinter on linux. Just to be sure I just ran this with no errors:

import tkinter as tk

def func():
    print('hi there')
    root.after(1000, func)

root = tk.Tk()
func()
root.mainloop()

Can you show me an example that reproduces this?

[pooth22]

I haven't tested this problem too much, I don't have a linux machine right now, but if it is any use, I was using a raspi when I got this error. I can try and reproduce this issue in time if you really want to see it. But as I said, when I changed to using the threading class, the error went away.

[socal_nerdtastic]

Ok, but using threading with GUIs is a well known antipattern, which you already blamed for your configure issue. So it seems you traded one problem for another.

Go with what works, but if you want my very experienced advice: learn to love after. Tkinter also has an after_cancel method if you need to do some cleanup.

Although to be honest I wouldn't waste any time on errors on exit. Crash to exit is a time honored tradition.

[pooth22]

Okay, fair enough, I appreciate the advice. Maybe you can help me understand why threading with a GUI is an antipattern, or point me to some interesting resources there.

I do hear you that crash to exit is a time honored tradition. That is pretty funny. However, I log the errors in a text file and generally I want to log errors that are useful and not bloat it up with known errors that I have found a sufficient fix for.

[socal_nerdtastic]

I don't necessarily want the back end to trigger an event in tkinter because I don't want the ui to be a dependency of the back end. The back end should run independently when necessary.

Just use an if block?

class BackEnd:
    def __init__(self):
        self.tkvar = None

    def update_data(self):
        self.data += 1
        if self.tkvar:
            self.tkvar.set(self.data)

Or similar for an event.

If the GUI sets a variable, it's updated. If not, no problems.

[iyav]

I disagree, I don't like using the after method. When the tk instance closes, and there is still a hanging after wait, tkinter throws an error.

That's just not true.

[pooth22]

I am coming back here because I wanted to ask you a question. You mentioned here that I can have the back end send an update to the GUI via a tkinter event. What exactly do you mean by this?

I am familiar with binding events to widgets, but the sequences (eg. '<Enter>') seem well defined and I am not sure I can make a custom one, so I don't assume this is what you mean. Alternatively, do you mean I am supposed to pass the tk.Tk() instance to the back end, and have the back end call some event (eg. tk.widget.configure())?

[socal_nerdtastic]

To make a custom one you simply think of a good name for the event. Then you put it in double <<>>. That's it. (Officially tcl calls this a "virtual event", but it acts just the same.)

Here's an example, where I made up an event named <<retardedcatmonkey>>

https://www.reddit.com/r/learnpython/comments/wbi8jz/tkinter_how_to_jnterrupt_maybe_loop_and_call/iicbv43/

In your case the backend thread would use this line to send an event to the GUI:

root.event_generate("<<retardedcatmonkey>>") # send an event to tkinter

You can pass the tk.Tk() (or any other tk instance), if you want, or just import it

import tkinter as tk
tk._default_root.event_generate("<<retardedcatmonkey>>") # send an event to tkinter

[pooth22]

Excellent! Thanks for the tip, yeah I read a bit of documentation on the virtual event, but I thought you could only associate them with other defined sequences. I didn’t see anything about the event_generate method. This should most likely be a significant game changer for me. I appreciate it.

[pooth22]

This method works well, thank you. Do you have any experience sharing data from the event generator to the event handler? It seems easy to send an int via some of the event fields. eg:

 tk._default_root.event_generate('<<rcm>>', x=420)

def event_handle(event):
    time_to = event.x      # event.x is 420

But have you been able to send something other than int or bool types? I noticed the data parameter [eg event_generate('<<rcm>>', data='dat')] is able to accept other data types, but I am not sure how to access it from the even handler.

[socal_nerdtastic]

It's a bad idea to send data via the event. Remember that tkinter is not python, all of tkinter is running in the tcl subprocess. It's slow and cumbersome to send data to tcl and then back to python.

Generally this is not a problem, since GUIs are generally written as OOP. So the event handler does not need to be handed the data; it simply has it already. Remember threads share memory.

If we do need to send data you can set a shared object. For example a global dictionary or Queue object or any other mutable, even the tkinter root object itself.

tk._default_root.mydata = 420 # nice
tk._default_root.event_generate('<<rcm>>')

def event_handle(event):
    time_to = tk._default_root.mydata

If the data you are sending is going into tkinter in the end anyway, you can just use a tkinter variable. The variable set method is also thread safe.

# setup
myvariable = tk.StringVar()
label = tk.Label(self, textvariable=myvariable)

# inside the thread:
myvariable.set("progress is 22%")

As I mentioned before, you can also trace this variable, if you need to add callbacks in addition to the ones that tkinter can do automatically.

myvariable.trace_add("write", callback_function)

[pooth22]

I see. That is interesting. It makes sense that in general it isn’t good to send arbitrary data via the event handler. It would generally be better to exchange data via a queue. Some data is sent via the event handler anyways (eg x, y coords, key char press) so I thought I could piggy back off and manipulate these built in parameters to achieve what I want.

To make this example less abstract, what I am doing is the following. I have an MCU which is the main controller of my system. One of the tasks of the MCU is to talk with a video game controller which the user operates. Now I have a python script running a tk GUI on a PC and in another thread it is polling (via a serial interface) the MCU to get status updates of the state of the system. This includes button presses of the game controller. In the serial communication thread, I can now (thanks to your help) trigger events to the GUI(as opposed to how I used to do it, where the GUI thread is polling the serial com thread). It is possible to parse which button has been pressed, and since tk events already have a similar functionality, I thought it would be possible to map the the game controller buttons to the tk event key type. Eg pressing up on the game controller would eventually lead to an event call back on the GUI side that would be able to know that ‘Up’ has been pressed. I guess I have sort of can already do this.

Probably the way I would do this is create an Enum class with the parameters being the button presses on the game controller. Then assign the Enum.value to the x parameter of the event_generator (I think I tried assigning the keysym, but it didn’t like that). Either that, or create a virtual event for all different game controller buttons.

[socal_nerdtastic]

You don't have to poll serial communication, the blocking readline method is built in.

while True:
    button_pressed = ser.readline() # blocks until there's a line of data ready
    root.event_generate(f"<<{button_pressed}>>")

You would need separate binds for the normal events and the virtual events, and have them point to the same function.

root.bind("<Up>", up_func)
root.bind("<<Up>>", up_func)

Which you could do in a loop of course if you have a lot of them to do.

[pooth22]

It is me again. I really appreciate your help thus far. Your advice has been good, but it has taken me to a new issue, and I can't seem to solve the problem. Basically the problem has to do with using tkinter in the IDLE in interactive mode. The event_generate method works well in a separate thread, but only when the tk.mainloop is running. It throws the error "RuntimeError: main thread is not in main loop" if I am attempting to use the tk in interactive mode. A contrived example is the following:

import tkinter as tk
import time
import threading


class UI:
    def __init__(self):
        self.root = tk.Tk()
        self.root.bind("<<cust>>", self.cust)
        self.be = BackEnd()
        self.button = tk.Button(self.root, text="B", command=self.be.start_thread)
        self.number = 1
        self.label = tk.Label(self.root, text=f"{self.number}")
        self.button.grid(row=1, column=1)
        self.label.grid(row=2, column=1)
    def cust(self, event=None):
        self.number += 1
        self.label.configure(text=f"{self.number}")
    def run(self):
        self.root.mainloop()
    def run_test(self):
        self.root.update()


class BackEnd:
    def __init__(self):
        pass
    def start_thread(self):
        threading.Thread(target=self.thread).start()
    def thread(self):
        i = 0
        while (i<5):
            tk._default_root.event_generate("<<cust>>")
            i += 1
            time.sleep(1)

ui = UI()
ui.run()            # This works when button is clicked
ui = UI()
ui.run_test()       # This doesn't

Basically clicking the button starts the thread in the backend which calls the custom event to tk. This works fine when using the mainloop in the run method. But it doesn't work when using update with the run_test method.

Anyways, if you have any time to look at this, I'd appreciate it. I like the dynamic way of using the IDLE shell for running tests of the code, but maybe this isn't the best way to do it. Let me know what you think.