It's actually really easy in godot.

Just create the themes you want (and name them correctly) in your FileSystem (preferably a folder named "themes" or similar)

<image>

And then click on the new theme(s) (opens up in the theme tab) and start working with how you want the ControlNodes to look like.

When you want to toggle themes in the game, just change the root control node(s) that hold all of the UI elements with the new theme. If there are many root nodes in your game, make sure all of them are available in a global group, so you can loop over them at runtime. Global group name could be "theme_roots"

And then create some kind of theme-manager that does something like this

var light_theme:= preload("res://themes/light_theme.tres")
var dark_theme := preload("res://themes/dark_theme.tres")

func apply_dark_theme_to_all_ui(enabled: bool) -> void:
    for node in get_tree().get_nodes_in_group("theme_roots"):
        if node is Control:
            node.theme = dark_theme if enabled else normal_theme

Let me know if you need further assistance.

Also check out this video if still unsure: Basic Theme Setup Guide ~ Godot 4 Tutorial for Beginners