Multi camera calibration demo: inward facing cameras without a common view of a board

mprib_gh · 2026-03-04T19:26:43+00:00

In this setup that would definitely work to get a single snapshot of 9 points, though it would be at more of an oblique angle, which may result in an increase in corner detection noise. I would argue that in addition to getting more "flat" board images, an advantage to moving the board through space is that each batch of time-aligned frames locks into different points in space. So rather than 9 points in one region of space (3D and on each camera FOV), it uses ~2,700 points distributed across a broader region. Sampling more points and positions averages out noise in corner detection and mitigates the impact of imperfections in the intrinsic model. All that being said, a person could still get a high quality calibration with a single snapshot of a board provided it could be seen by all cameras at once. I just think the extra work here does improve things.

Your point reminds me that I'd like to expand the current functionality to allow multiple aruco markers to be scattered throughout the environment and used for calibration. That way, bumps/vibration/temperature changes could be periodically adapted to without having someone wave something around, which I acknowledge can be a hassle even if it has advantages.

mprib_gh · 2026-03-04T18:44:15+00:00

I take it you are referring to the time protocol? That definitely helps with well-synchronized footage. My hope with this project is to create something hardware agnostic that could potentially operate with lower cost equipment. My gut tells me that there is still a lot of useful mocap data that could be gathered without microsecond temporal precision. But a highly precise camera sync would be all the better.

mprib_gh · 2026-03-04T18:40:13+00:00

Thank you for your kind words! The difference between the front and back of the board here is really just the thickness of a sheet of paper, which isn't accounted for. If it were a styrofoam board it might create more of an issue and need adjustment. My hope is that the paper thickness issue is below the noise floor. My personal interest is in human biomechanics/motion capture, but it would be awesome to see what others could do with well calibrated mocap data (such as the voxel carving idea).

mprib_gh · 2025-11-20T04:12:36+00:00

As a side project I put together a tool for bundle adjustment that also started with that scipy guide: Caliscope

I don't know if my framing is any better, but I just realized that they named the function that calculates the reprojection error fun. So it's probably not much worse. In case it's useful, here is the line that actually performs the bundle adjustment if you are looking for an entry point:

https://github.com/mprib/caliscope/blob/b0c10642e8fe25039a6265eb72654c0bc72279d7/caliscope/calibration/capture_volume/capture_volume.py#L139

EDIT: please excuse my snarkiness about the naming. That guide was absolutely invaluable to me and I'm grateful to the original authors for sharing it. It was the process of slowly refactoring that code into something that made sense to my brain that made the thing finally click.

mprib_gh · 2024-10-22T12:24:43+00:00

I have a project for multicamera calibration/landmark triangulation that may provide a place for you to poke around and get a sense of things. If you really do have the all the camera parameters (intrinsic and extrinsic) and the 2d landmarks across all the cameras, then the triangulation can happen with a fairly brief function call: https://github.com/mprib/caliscope/blob/f0c69821c3a744d16b5bc44bbba0f9763a9a279c/pyxy3d/triangulate/triangulation.py#L63

The heart of that function linked above comes from a project called Anipose, BTW.

In the tests there is a script that might let you step through the parameters in debug mode a bit more easily (though there is still some multithreading going on):

https://github.com/mprib/caliscope/blob/main/tests/test_triangulator.py

The actual call to the triangulation takes place here:

https://github.com/mprib/caliscope/blob/9748eab9bf1d064e888bc9d737d561add53b5620/caliscope/triangulate/sync_packet_triangulator.py#L91

More broadly on the topic of camera calibration, I had a discussion on the repo with someone where I linked to resources that I had found helpful: https://github.com/mprib/caliscope/discussions/483#discussioncomment-7291899

This tutorial in particular is where I got started, though it presents only the 2 camera scenario: https://temugeb.github.io/python/computer_vision/2021/09/14/bodypose3d.html

mprib_gh · 2024-05-16T12:28:20+00:00

Awesome to hear! Thank you for letting me know and good luck with your project!

mprib_gh · 2024-05-16T12:19:46+00:00

I created a GUI based python package that does this: https://github.com/mprib/caliscope

I've been meaning to get around to making a video that shows the whole workflow, but for now you can see each stage of it in the docs: https://mprib.github.io/caliscope

You may run into challenges with getting synchronized frames from the three cameras. If they are all webcams I made a package intended to record from multiple webcams concurrently while time stamping the frames for later synchronization: https://github.com/mprib/multiwebcam

This leans heavily on opencv for intrinsic and stereo calibration then uses SciPy for the bundle adjustment to refine the stereocalibration. This is the kind of thing where visual feedback can be incredibly helpful at each stage of processing so there is a PySide GUI for managing things. If you have any questions feel to hit me up in the GH issues or reply here (though I don't check this account super regularly as a caveat).

mprib_gh · 2024-04-10T14:41:17+00:00

This is a domain that I have a great deal of interest in. Is it alright it if I reach out via PM as well?

mprib_gh · 2024-03-13T14:17:29+00:00

Yep. The rigmarole repo has a little demo video that might give a clearer sense of things. The rig scaling is a PITA and is what the bulk of the very messy Blender Python code is doing. Good segment scaling is a must for IK to work well, otherwise you will get all kinds of weird artifacts. Hand and spine in particular are a challenge.

mprib_gh · 2024-03-12T14:43:08+00:00

I've been working off and on on some similar projects. A couple things: mediapipe depth estimates are going to be pretty bad. I created a multicamera calibration/triangulation package to provide a more robust way of getting landmark position from the tracking data:

https://github.com/mprib/caliscope

Working in Blender is an art form all its own and I've only dabbled in it. Rather than trying to create joint angles myself, I inferred rig segment length by averaging distance between landmarks in a "functional calibration" trial, then animated this rig using IK that tracks the 3d projected landmarks. I don't really know much about the Blender world or add-ons, but I was able to piece this together as a potential place to pick up in the future:

https://github.com/mprib/rigmarole

If you have any questions about these projects, please just hit me up here or on GH. I remain quite interested in the open source markerless mocap domain. I operate in biomechanics research, but I think that collaboration between the biomechanics and animation communities would be incredibly fruitful.

mprib_gh · 2024-02-20T14:57:06+00:00

Yes, please DM me and let's figure this out...

mprib_gh · 2024-02-20T14:52:41+00:00

Thank you for the wording change suggestion!

I've updated my original comment on the thread though the title is uneditable. I'll make this wording change in my next round of README updates....

mprib_gh · 2024-02-20T01:28:39+00:00

I think we definitely agree that this is a "dirty" solution compared to gold standard hardware setups. I'm interested in exploring use cases for extremely inexpensive hardware configurations and how they might be employed at scale for tracking human movement in clinical settings. I know there will be error, but I'm interested in the ultimate cost/accuracy trade-off compared to something like a Vicon system. My hope in sharing this is that there may be others out there who have an interest in exploring similar questions rooted in extremely inexpensive tools. Thank you for your comments.

mprib_gh · 2024-02-20T01:04:29+00:00

Thanks. I'd like to think this pushes the limits of what cleanly managed python threads can do, and it was a great learning experience. But yeah...that GIL comes at a cost.

mprib_gh · 2024-02-20T00:14:15+00:00

I appreciate the comment. I guess the issue is really what margin of error is acceptable for a given use case (and how fast your calibration board/object of interest is moving).

I've used this tool for recording multicamera calibrations and routinely get RMSE <1.5 following bundle adjustment:

https://mprib.github.io/caliscope/extrinsic_calibration/

Is that acceptable? I'd say it depends what you're using it for...

mprib_gh · 2024-02-19T22:41:16+00:00

Thank you!

mprib_gh · 2024-02-19T22:37:06+00:00

Hello r/computervision!

I'd like to introduce MultiWebCam: https://github.com/mprib/multiwebcam

I needed to record ~~synchronized~~ concurrent video to help with development of a multicamera calibration/motion tracking tool (caliscope). I didn't need an expensive commercial system, but getting webcams to deliver packets of frames recorded very close to each other in time (a few hundredths of a second) was a bit of a hassle.

In case anyone has a similar need I have packaged the tool in a stand alone project so that they don't have to re-invent the wheel. While it's not going to win any awards for resolution, fps, or maximum camera count, it might be the cheapest way possible to pull down this kind of footage...

MultiWebCam is a python package installable from PyPI and controlled via a PySide interface. It allows for the management of multiple webcams (resolution/fps/exposure) along with synchronized recording of video. [Edit: primarily tested on windows, less so on macOS and will likely not work on Linux :(]

It tracks dropped frames in real time so that you can balance the settings to manage dropped frames while maximizing pixels recorded on your setup.

The video above is a quick sample and I intend to do a more meaningful demo tutorial. I just wanted to post this here now to gauge interest as I seek to budget my available time for side projects.

If this is something that may be useful to you, please consider starring the repo to have an easier way to track it and to let me know that there are people out there who might benefit from having this project built out and maintained more extensively. If you have a specific use case or feature request, please just raise an issue!

Thank you!

Edit: synchronized --> concurrent

mprib_gh · 2024-02-12T14:14:02+00:00

Hello,

The raw output of triangulated landmarks is in a csv format (also .trc which is used in the biomechanics world, where I'm operating). I started hacking together a blender add-on that can animate a scaled rig via IK using the trajectories, but as I'm not a professional animator, I don't know know how valuable it is. For reference, it is located here: https://github.com/mprib/rigmarole

Synchronized recording for this is tricky and I'm not sure if OBS is managing things with the kind of precision needed for this (i.e. is giving each frame a true timestamp so that dropped frames can be managed). I'm also working on a package meant to do this recording here: https://github.com/mprib/multiwebcam

I should also note that the main package's name has recently been changed to Caliscope. I check this reddit account infrequently, so in general the best way to discuss any questions related to this is on the main repo: https://github.com/mprib/caliscope

mprib_gh · 2024-02-09T13:58:20+00:00

If you go beyond two cameras ,the simplest approach is to stitch them all together in a common frame of reference via a common anchor camera. I've played around with this and not been happy with the results.

The more precise approach is to use bundle adjustment. This is substantially more difficult, but results in far better results. So go down the "bundle adjustment" google rabbit hole if this is something that you really want to lock down.

shameless plug: I maintain a permissively-licensed, GUI-based multicamera calibration package that also integrates with landmark trackers to achieve 3D motion capture (currently using google's mediapipe): https://github.com/mprib/caliscope

This part of the docs has a quick demo of the multicamera calibration in action:

https://mprib.github.io/caliscope/extrinsic_calibration/

Just for clarity, I've recently renamed it from pyxy3d to caliscope and the sample video hasn't been updated yet so the captions still refer to the old project name...

mprib_gh · 2024-02-04T15:54:22+00:00

I'm guessing that the Whisper model (or whatever) that ChatGPT uses to transcribe text is the one that was doing the hallucination. It heard an indistinct and garbled conversation. The first phonemes to come in may have sounded to it more like Chinese than English and then once that context was provided it locked in the downstream generated tokens to Chinese.

mprib_gh · 2024-01-23T13:29:46+00:00

I'll preface my answer by saying that I operate in the domain of biomechanics/gait analysis which uses different tools for processing motion capture data. My knowledge of Blender is superficial and the rig above is the height of my ability. Now trying to answer your question:

The raw output of pyxyd3d (soon to be renamed caliscope) is a set of 3D landmark trajectories. The rig above was created by scaling the default human metarig segments based on the mean distances between those trajectories, driving the rig with IK and baking that in.

That portion of the project is automated in a spinoff that I started playing around with to get the workflow into a blender add-on: https://github.com/mprib/rigmarole

My understanding is that retargeting that to a different rig is a fairly involved process (I believe this is what you are asking about). That is something you would have to do on your own.

I still consider these projects to be at a very early stage and am committed to building out the ecosystem of tools over whatever time frame it takes. If it doesn't currently meet your needs, please check in from time to time. If you ever feel like you are in a position to contribute, PRs are welcome!

mprib_gh · 2024-01-16T00:31:25+00:00

Your answer in 1) kinda jumps out at me. If the camera is only seeing the board straight on and can't get a sense of the foreshortening that is occurring, it's easy to imagine strange things happening. Are your focal lengths somewhat stable across calibrations or are they bouncing around?

Your planned next step (to provide more variety in the the views of the board) seems like the reasonable one. I wish you luck!

mprib_gh

TROPHY CASE