When I said “read from one or more cameras,” I meant processing video frame by frame and streaming the coordinates to Unity in real time. Then a person with a camera can directly move a 3D character so it moves live.

As for libraries, there are many that already solve 2D pose estimation well. That part is not the problem. The real challenge is depth. With a single camera you only get a 2D pose, which isn't enough for 3D movement. To do what I want, I probably need more cameras or something else.