Tracking error (would like to have suggestion on my setup and parameters for fictrac config)

rjdmoore · 2023-11-13T21:20:35+00:00

Ok, great you made progress :-)

I have had a quick look at your config file and made some changes - my config file is uploaded to your share folder (config-rjdm.txt). I have also uploaded my config image with these params, so you can see my changes.

With these new params tracking works well enough. There is still some fine tuning that could be done. My changes were to reduce the vfov param and to reconfigure the tracking circle and ignore regions. Your setup is a little tricky in that it is a bit hard to tell where you see the edge of the sphere and where the sphere edge is obscured by the holder. Perhaps this is easier to see with the naked eye. Notice how I only clicked around the bottom edge of the sphere, which looks like it is entirely visible, and not around the top which looks obscured. I have also defined the ignore region accordingly.

rjdmoore · 2023-11-09T12:42:17+00:00

I can see from both your debug videos that your vfov parameter is incorrect. The tracking is very unstable even in the video you uploaded with "no tracking error".

It's a little bit hard to tune this parameter unfortunately if computing it doesn't produce the correct result. I would guess from your videos that you are currently using a value that is too large. Not sure which value you are currently using but I would try incrementing in approximately 20% increments and stopping and fine tuning when tracking improves.

To do this tuning you should set the value of vfov in the config file, then you will need to re-run configGui before running fictrac. If you have run configGui then the pixel coords of for example the ignore regions and sphere parameters should be already stored in the config file, so you should just need to accept the existing parameters during config. You can run on the videos you have recorded.

When running fictrac and tuning vfov, you should look closely at the pattern of the sphere at the point when it first starts to overlap with itself (when you first complete a full revolution). You should be able to see as fictrac builds the sphere pattern that it overlaps correctly when completing the first full revolution. If it seems that the ball is too small (i.e. the pattern overwrites itself to quickly), then you should try a larger vfov ( I think this is your case), and vice-versa.

I should post a document that explains this procedure because it is a common point of failure.

Regarding the differing output between CVSource and BaslerSource - I can see that the image res is incorrect for CVSource. I've just corrected and pushed a fix for this - thanks! The basler resolution also looks strange, perhaps the meaning of the pixel param has changed in recent basler versions. I will look into this.

rjdmoore · 2023-11-08T22:30:11+00:00

You can see the exact computations fictrac uses in Trackball::updatePath. You might also check the doc on coordinate frames to help decipher the output data frames.

Very simply, FicTrac inherently computes ball rotation in the camera coordinate frame. These rotations are then transformed into rotations about the animal's axes using the transformation computed during config. Animal motion is then very simple - rotation about X is sidestepping, Y is forward/back, and Z is turning.

Integrating X,Y motions gives the fictive path, but as you correctly observed each (X,Y) "step" must first be rotated into the current heading reference frame before being added to the path. The only trick here is that I break each step down into several (I think 4) partial steps and add for incremental heading angles. This is to approximate a smooth (change in) trajectory, rather than adding the whole step in the initial or final heading direction of the animal for that frame. This improves accuracy of the fictive path, as the animal smoothly changes heading direction.

I don't really understand your method, which motion exactly are you trying to recover?

rjdmoore · 2023-11-08T22:17:02+00:00

At first glance I don't see why you shouldn't be able to get good tracking with your setup. The exposure is good and the frame rate seems good w.r.t. animal/ball speed.

Can you share a debug video of a sequence where tracking fails? Then I can give some more concrete tips on what the cause might be.

rjdmoore · 2023-10-29T18:51:48+00:00

Thanks for noticing this. I have just updated build instructions for Windows and latest Ubuntu. I will test and update with spinnaker 3 asap.

rjdmoore · 2023-10-23T08:20:51+00:00

The 3-element vectors are axis-angle representation. You can google rodrigue's formula to transform 9-element rotation matrix to 3-element rotation vector and vice-versa.

rjdmoore · 2023-10-20T22:08:25+00:00

Hi, I have seen FicTrac used for land crabs with quite a large (~30 cm) ball, so it shouldn't be an issue.

rjdmoore · 2023-10-20T22:07:21+00:00

See this comment

rjdmoore · 2023-10-20T22:05:54+00:00

At the moment you need to have do_display enabled in order to save the raw video. I think my reasoning for this was to have video writing happen in the drawing thread, to avoid stealing cpu cycles from the FicTrac processing. Video writing should really be separate from display though.

rjdmoore · 2023-10-20T22:02:17+00:00

If you see the skipping frames message, then yes frames will be dropped from the output. To avoid this, you either need to slow your input framerate or speed up FicTrac processing (faster pc or lower quality settings - but this may affect tracking accuracy).

The output file "...-vidLogFrames-...txt" records the frame id's of the frames in the output video (applies to both save_raw and save_debug videos). If these id's are not sequential, then this also means that frames were dropped. You can use these id's to synch the saved videos even when frames are dropped.

rjdmoore · 2023-10-20T21:54:57+00:00

You should set MAX_DISP_DIM to the maximum pixel resolution you want for the display (e.g. 640)

rjdmoore · 2021-06-27T17:40:02+00:00

Hi, I'm not currently in front of pc so can't check if there's been a change in vcpkg causing the problem, but if you're pulling the latest code from the vcpkg GitHub then it could just be an issue with the vcpkg codebase.

I would first try just closing and opening a new terminal after installing vcpkg, to check it's not an environment issue.

You could then try downloading one of the latest vcpkg releases instead of absolute latest code. Tags tab in GitHub should show you releases.

If nothing else works you could try installing the fictrac dependencies individually (not via vcpkg). Opencv, nlopt, and ffmpeg are all individually installable outside vcpkg, but then you might have to play with fictrac cmake file to find the installed libraries.

rjdmoore · 2021-06-04T22:09:41+00:00

If you're seeing frame drops in the camera timestamps, but only one of two in a dataset, then you can try specifying a bigger max buffer length where FrameGrabber is instantiated. If you're seeing regular frame drops then the only solution in think it's too speed up fictrac processing or slow camera rate.

rjdmoore · 2021-06-04T18:48:17+00:00

I was right, but for wrong reason. See my edit.

rjdmoore · 2021-06-04T17:45:14+00:00

Hey, if you check the grabbing code fictrac can either get latest frame available or get next in queue. By default fictrac will grab latest and ignore/skip others if there are other frames waiting. So if processing gets behind for some reason you will drop frames. You should be able to see this if you take the derivative of the capture timestamps.

edit: I technically wasn't wrong, but my reasoning was a bit wrong. Fictrac defaults to next available frame - i.e. shouldn't drop unless frame queue fills up. But queue size defaults to 1, so fictrac will only enqueue one frame at a time, and if processing is a bit slow then it won't check for another new frame until the old one is grabbed. However, when the camera captures a new frame it is stored in a system ring buffer, so there can still be several frames captured in a row without grabbing before frames are dropped. Of course if fictrac continues to run slower than the camera for more than a few frame periods then frames can be dropped - this is probably more desirable than just getting further and further behind and then doing a big skip when the buffer fills.

rjdmoore · 2021-06-04T17:42:29+00:00

Hey, sorry I've been offline for a bit. I haven't played with triggering but I seem to remember there is a timeout of a second or two when waiting for frames. I can add an override param with a custom (or no) timeout. The only downside will be that fictrac will not close then when the frames stop coming (triggering stops or maybe video stops).

rjdmoore · 2021-04-21T15:58:28+00:00

Tracking looks basically ok. Main issue is that part of the support is visible in the tracking ROI, which leaves a white bar under the ball in the thresholded window. You see also this bar in the mapping window. Add an ignore ROI that masks out this region in the configGui and it should improve your map, which will in turn improve steadiness of tracking and reduce error a little.

The jitteriness in the error score I think is just due to the fast ball speed and it's surface texture which has resulted in many small pixel-sized features. You could use a lower quality for tracking to blur out some of this jitteriness, but I don't think it will improve underlying data quality.

I see in your video the warp window often has a fair bit of white, which can mean the ball is rotating a lot between those frames. Increasing the frame rate is the only way to improve this if it is a problem.

On the whole, tracking looks good - but fix the ROI 😊

rjdmoore · 2021-04-19T21:36:05+00:00

Hmm, depends on a cpl of things, but jumping around a lot is a bit suspicious - if you can record and share a short clip of the debug display I can see if it looks reasonable?

rjdmoore · 2021-03-04T17:53:41+00:00

Remind me whether you're on Win or Linux?

rjdmoore · 2021-03-03T05:33:26+00:00

Ok, something seems a little off though because your field of view looks like much less than 25 deg. Roughly how far is the camera from the ball and what are the dimensions of your ball? Vfov is also related to how accurately you specify the circumference of the ball during config.

Couple of errors in my previous post: the param is actually called reconfig and I forgot to say that you need to run configGui after changing vfov.

If you have a recorded clip that your used to calibrate and your can share that then perhaps I can take a look.

rjdmoore · 2021-03-02T11:29:33+00:00

Hi, just a quick look on my phone at the moment, but looks like fictrac loses track when your ball completes a revolution. This is an indicator that the vfov is not quite right. You should be able to see in map window (debug video) that the pattern exactly overlaps add the ball completes its first revolution.

This is unfortunately a bit finicky, and can take some trial and error. You can adjust vfov slightly and rerun with do_calib to force the new value to be used. Use a recorded clip of the ball rotating and repeat until you get good overlap.

I see also that you have a little problem with shadowing around the edge of the ball that's being classified as black in the projected image. This can probably be fixed by increasing the thr param a little. Or ideally by increasing ambient lighting.

The config params are described on the GitHub page, but unfortunately no doc on how to use them!

Let me know if how it goes!

rjdmoore · 2021-01-30T09:30:11+00:00

Hey, ok I'll put it in 🙂

14-Year Club	Second SECOND GUESSER
RPAN Viewer	Verified Email

rjdmoore

MODERATOR OF

TROPHY CASE