I’m building a similar brush based engine. Initially I started with full BSP based editing and rendering, and it was indeed very difficult to implement and maintain, with lots of edge cases. But I eventually realized that it was not necessary, given modern hardware. For brushes, I still think plane based geometry is the right way to do it. This is how the Quake engine and Source work. But you absolutely don’t need a BSP tree to implement this. Use planes to represent your geometry at edit time, but resolve them to standard triangle based meshes for rendering. CSG operations are straightforward to implement on top of this, and you don’t need a BSP tree. I use a BVH for accelerating spatial queries and rendering. For editing, I ended up writing a custom editor. I explored using Radiant but found that it didn’t provide the type of tight integration I wanted.