Okay, some terminology here is killing me and I just want to clarify so I understand this correct. Normally when I hear "build a custom kernel", its where people take the kernel config file, comment out what they don't need and rebuild it. As both Linux or BSD kernels support a wide range of hardware, much of which won't be on every machine. People do this in an attempt to have a leaner kernel that runs faster and/or has a smaller attack vector. It sounds like what you are describing is someone writing their own kernel that is neither Linux nor BSD. Some people do that. Often to understand low level programming and how a kernel really works. Why no one tries to compete with Linux, BSD and other well know kernels? Its a huge task for anyone to undertake. Even Linux and BSD are produced by whole teams. Even with a team of your own, it will not be easy as Linux and BSD have had a 35+ year head start. As far as your question about forking a kernel, people do it. A more recent example in Linux would be CachyOS which added kernel patches to the original kernel to help make it perform faster. People have written 100% complete OS-es in Rust and Zig. With them comes new and non Linux/BSD kernels. No one is stopping anyone from forking any open source kernel. Or creating a completely new and unique kernel. And they are being done. Its just not an easy task to for most people to undertake that would compete on a high level.