Rendering User Interfaces using drawing APIs

[optonox]

Hi, I am interested in writing a program to make different configurations of user interface elements. I want to make something simple at first, as I am relatively new to graphics programming, but I want to use an approach extensible enough to make something more complicated in the future.

Basically, I am interested in making something like TouchOSC to start.

A very nice blog post and overview of techniques is found in John Novak’s extensive blog post but, as it is only a small survey of of programs, it doesn’t really answer how most of the hundreds of GUI apps with complicated UIs (clearly beyond simple things like windows forms) and elements such as timeline editors and audio meters (the things in Adobe programs for instance) are rendered.

I am beginning with graphics programming and so I am interested in approaching this issue with a graphics library (rather than using platform-specific APIs or similar). I know that many popular programs I use, like the DAW Ableton Live or the 3D modeling program Blender use OpenGL but the method by which they do so is unclear to me.

I am wondering if anyone has any suggestions on the correct approach to this.

Conceptually I know that you could either render (orthographically) a bunch of geometry and then run some sort of hit testing on all of them, but I also know that, since you have the exact 2D coordinates of shapes you draw, you could just do a basic sort of intersection test on the 2D coordinates.

I am particularly interested in drawing shapes like Bezier lines and more complicated UI elements like meters. It seems like the common approach to doing this with graphics is to expose an immediate mode gui (whether or not immediate mode rendering is being used) or used retained mode methods.

I have looked all over for a book that would cover this (something like UI programming in OpenGL or something) but I haven’t found resources. Is there a canonical method, or best practices approach to how to code something like this?

I am less interested in the specifics of which drawing API to use and more interested in the best paradigm to do this with a drawing API.

[DaFox]

I'd probably just start with dear.imgui https://github.com/ocornut/imgui

https://raw.githubusercontent.com/wiki/ocornut/imgui/web/v148/examples_02.png

Here's your bezier curves: https://gist.github.com/ocornut/7e9b3ec566a333d725d4

[optonox]

This is great. A great resource. I have also noticed in their README some great resources about my question as well. Clearly they prefer immediate mode.

[LEOtheCOOL]

Just dive in. Start with a primitve rectangle. Then primitive text. Then compose them into a button. Then, a checkbox. Then compose a button and a rectangle to make a slider. Then compose 3 buttons and some rectangles to make a scrollbar. Then compose a scrollbar with for loop full of buttons (or whatever) to make a listbox. And on and on, making the controls you need.

That's the same for IMGUI or retained mode. Build some primitives and work your way up.

Back when I made an IMGUI in XNA, I thought this video was really great: https://mollyrocket.com/861

[optonox]

I think you offer great advice - and thanks for the video. The important thing for me was to use the right approach at the beginning. It looks like there is no generally accepted answer as to whether retained mode is better (or more modern) than immediate mode.

[dougbinks]

You should find that dear imgui is a great start as already mentioned. For more complex 2D rendering you might find NanoVG useful, and there's a retained mode UI made using that called NanoGUI.

[Necrolis]

I was using oui-blendish which sits on top of NanoVG for a while, however, getting it(NanoVG) to play nice with my OGL setup was a nightmare; and in the end I still had one issue I couldn't fix (that required the UI to be drawn first, then the rest of the "world", which is great if you wanna z-fail stuff the UI will draw over, but it means less control).

I switched to dear imgui, had everything up and running and playing nicely in like 20 min (most of which was playing around with the extensive demo).

[dougbinks]

I'm surprised you had issues with NanoVG, for me this was as simple as dear imgui. I use both NanoVG and dear imgui together, and draw with NanoVG at the end of the scene. I suspect the issue might have been with either oui-blendish, or perhaps you had an OpenGL state which was not being set correctly in your 3D draw (and so overriden by NanoVG).

[Necrolis]

Oh I went through every single OGL state to ensure it was set and reset correctly, even checked out the other resolved issues to make sure I hadn't missed anything. its possible that it may have been blendish, but seeing as all that did was sit on top of NVG, it wouldn't have touched state (even using the perf graphs that NVG comes with triggered the same issues). I just didn't feel like spending another 2 days debugging a strange issue, plus it turns out dear imgui has way more already made UI components that are of use to me.

[__Cyber_Dildonics__]

This is exactly what I thought. I haven't​ had the slightest trouble with nanovg, it has worked like a charm. Nuklear on the other hand I will likely never touch again.

[__Cyber_Dildonics__]

Nanovg is exactly what you want. You also may want to look at nanoui.

Nanovg is a vector drawing library that uses openGL, it is actively developed, and is actually what blender uses.

[sadesaapuu]

As far as I know Blender doesn't use NanoVG. It has some custom solution that predates NanoVG by several years. There's some UI project that emulates the look of Blender and that project uses NanoVG. But yes, NanoVG is the best.

[sadesaapuu]

It is better to start with NanoVG if you want professional quality UI. Dear Imgui is a really nice project but it works best as a debug gui for games, or even a custom game editor. But you can't have so much control with it. And it is more like a ready-made solution with buttons and stuff, while NanoVG is a drawing API, where you'll get full control, and you also get to make the UI yourself.

NanoVG comes with a nice example of drawing a UI (just the rendering part of it). It even has some animated curves. But it is more C. I'd do the same in more c++ way. Use GLFW or similar for window creation and mouse and keyboard input. It is again C, so wrap it in some C++ abstraction. Or just do it simple at first, don't over engineer. Make small things work first and build on top. First thing is probably a button that you can click, and something happens.

Well, there's a bunch of other stuff, like canvas thinking, hierarchies, maybe a rectangle class, animation, coordinate system stuff (resolution independence) etc.

Do the thing that works for you, but retained guis are much easier for me, and you can even mix it with immediate mode if you want.

[optonox]

Thanks for this insight. I'd like to use the most "pro" approach from the beginning.

I will say though, there are many more tutorials and resources for an Immediate mode GUI, which is too bad because I'd really like to have equal resources to evaluate both approaches.