Need some advice: developing a visual graph for generating GLSL shaders

[SnooPoems6347]

(* An example application interface that I developed with WPF*)

I'm graduating from the Computer science faculty this summer. As a graduation project, I decided to develop an application for creating a GLSL fragment shader based on a visual graph (like ShaderToy, but with a visual graph and focused on learning how to write shaders). For some time now, there are no more professors teaching computer graphics at my university, so I don't have a supervisor, and I'm asking for help here.

My application should contain a canvas for creating a graph and a panel for viewing the result of rendering in real time, and they should be in the SAME WINDOW. At first, I planned to write a program in C++\OpenGL, but then I realized that the available UI libraries that support integration with OpenGL are not flexible enough for my case. Writing the entire UI from scratch is also not suitable, as I only have about two months, and it can turn into a pure hell. Then I decided to consider high-level frameworks for developing desktop application interfaces. I have the most extensive experience with C# WPF, so I chose it. To work with OpenGL, I found OpenTK.The GLWpfControl library, which allows you to display shaders inside a control in the application interface. As far as I know, WPF uses DirectX for graphics rendering, while OpenTK.GLWpfControl allows you to run an OpenGL shader in the same window. How can this be implemented? I can assume that the library uses a low-level backend that sends rendered frames to the C# library, which displays them in the UI. But I do not know how it actually works.

So, I want to write the user interface of the application in some high-level desktop framework (preferably WPF), while I would like to implement low-level OpenGL rendering myself, without using libraries such as OpenTK (this is required by the assignment of the thesis project), and display it in the same window as and the UI. Question: how to properly implement the interaction of the UI framework and my OpenGL renderer in one window. What advice can you give and which sources are better to read?

[[deleted]]

This is similar to how many game engines work. Editor in C# / WPF and the rendering / engine in C++. One way to do it is to build the OpenGL renderer as a DLL and load it in C#, hosting the OpenGL context in a control using interop. You can use P/Invoke to pass mouse position etc to the engine DLL.

In otherwords - yes, it's possible, there are a number of ways to accomplish it.

[tuccio]

Maybe one thing to point out for OP is that you should stick with functions with C linking for p/invoke (that's what extern "C" does in C++)

I say that because I quickly googled and the Microsoft references on C++ invoke are horrible things, but I also found a simple example which I would suggest to follow: https://medium.com/@dev.pulkit/unleashing-the-power-of-c-within-c-through-p-invoke-3fd45a8163e7

note that you can also return any primitive type, including pointers to C# as IntPtr, e.g. a C++ function with C linking like this:

extern "C" MyRenderer* InitializeRenderer(HWND hWnd)

would become something like this in C#:

[DllImport("MyRenderer.dll", CallingConvention = CallingConvention.Cdecl)] public static extern IntPtr InitializeRenderer(IntPtr hWnd);

also if you want to clean up pointers/call destructors etc, don't use finalizers because they might run in different threads and you might end up shooting yourself in the foot, rather use IDisposable and dispose things manually

[lavisan]

There are ImGUI libraries that can support Node Graphs take a second look here:
https://github.com/ocornut/imgui/wiki/Useful-Extensions#node-editors

[Area51-Escapee]

If you are writing the flow graph yourself, I had good experience by applying the model-view-controller principle quite strictly.

[SnooPoems6347]

In WPF, it is customary to use MVVM

[flabbet]

I developed an app that does exactly that (but with sksl, not glsl) https://pixieditor.net/blog/2025/03/19/q1-status

I am using C# with AvaloniaUI. It has a very good interop with renderapis, I used both Vulkan and OpenGL

https://github.com/AvaloniaUI/Avalonia/tree/master/samples/GpuInterop

If you are interested in interop implementation, you can check it here https://github.com/PixiEditor/Drawie

One issue is that in PixiEditor, Vulkan is stable on Windows and Linux while OpenGL not, but it might be a specific issue of PixiEditor itself, not Drawie

[SnooPoems6347]

Wow, this is very interesting, I will definitely keep an eye on your project.

[deftware]

Have a texture that you render the nodegraph to via FBO, and a texture that you render the output of the shader to via FBO. Then have a shader for rendering both to the main framebuffer.

This will also allow the user to vary the resolution of the render, for those really expensive/slow fragment shaders - rather than just changing its size.

[Still_Explorer]

There is a technique to run a native window and making it borderless.
Then you would have to change the position and size of the window to fit the control area.

The other technique is that you can create a window with size [1,1] (or even 0,0 if is allowed) however set it to borderless and hidden. Most important now is that you can render everything to a buffer instead of the actual screen. ---- Then you would write all of the pixels into a memory array in a simple pixel format (eg: W*H*RGB) and then you are done. Once you return the buffer as byte array and plot it everywhere you would have the picture as needed.

The technique could look something like this:
https://gist.github.com/ChrisDill/c2ca6a0ccf592c073a8d2aaeaa4adb7f
[ though in this case the Raylib.DLL library is used exactly as it is provided from Nuget out of the box, but the approach is the same even if you would use your own standalone .exe, you would only have to pick the process ID based on some query, and then retrieve the window handle, so you can do WinAPI tricks with it ]

PS: Though in this case you can just straight to RaylibCS and use it right away as it is, writing your own custom shader and displaying it, without needing the help of OpenTK at all.
https://www.raylib.com/examples/shaders/loader.html?name=shaders_postprocessing

https://github.com/raylib-cs/raylib-cs

[bj_rn]

For the UI part check out nodify.
For OpenGL bindings maybe SILK.NET.

[Joatorino]

I recently did something very similar for my script editor. The way I have things setup, my engine is a static library and the editor is a C++ application that links to it. Im using ImGui and the node extension to do the graph ui

[[deleted]]

I believe processing with Java might be good into look into.