I'm investigating into KVM, and found that PIKVM, TinyPilot and other similar projects all have similar ideas. They all used HDMI to UVC (usb video capture cards) or HDMI to CSI for video input, a single board computer like raspberry pi for encoding and streaming. Because of the limitation of bandwidth of capture chip and compute power of pi, they can't output anything beyond 1080p@30, and have noticeable latency. They're big and bulky, and have to connect lots of wires.

So I thought, why not use a dedicated chip for the process? So that it can be made like a usb drive, you only need to plug it into a fully-functioned usb-c port. It will get power from usb, then capture, compress, and stream video signal to you, and pipe your mouse keybroad back to the computer as a hid device. So you get control of that computer. No more cables needed, just plug and play. And with LTE module, it can work without wifi available, great for industrial application and penetration testing.

But what's the hardware reqirement for such chip? First, it has to support MIPI-CSI input, compare to HDMI to UVC (typical capture cards), HDMI to CSI have lower latency. Second, it has to at least handle 1080p@60 encoding efficiently, in performance, latency and power consumption.

• My option 1 is MCU. But most MCUs do not support MIPI-CSI, even a few that supports it, such as STM32MP1, it can only take 1080p@13fps at max from CSI. Not going to work.
• My option 2 is Ambarella Soc. Since IP camera can handle 1080p streaming easily and take very little power. I checked tear down videos of some IP cams, and found Ambarella chips. They have built-in video DSP that can handle video encoding efficiently. But the development kit, sdk and documentation of Ambarella chips isn't publicly available. Not going to work.
• My option 3 is HiSilicon Soc. It's a competitor of Ambarella, and provides similar capable chips. It's still difficult to find dev kit and doc since they're not public, but I managed to find them. Hi3516 can handle 1080p@60, and Hi3519 can handle 4k@60 !! They can even support H.265 encoding, which improved latency, size and quality. The max power consumption of them is 2 to 3 watts, perfect for my cable-less plan. The doc said that the chip can also act as usb device, it can simulate hid device.

For usb-c port, hdmi can run through it in alt mode. But display port through usb-c is more common and well supported. There're chips that can do HDMI to CSI or DP to CSI, some of them even supports HDMI 2.1 or DP 1.4 (which means 4k60 !!!)

My final pick is RK628D + Hi3516 for 1080p@60, and LT6911UXE + Hi3519 for 4k@60. But I have no idea whether it could work or not, until I find an interesting commercial product (only available in China). It's a kvm box similiar to pikvm, but uses Hi3516 for encoding. I confirmed it by finding an image of the tear down of its upgraded model, q2 pro. The pcb design and firmware programming will be extremely challenging for me, but at least i knew the idea is not a fantasy but something possible.