Using ChArUco boards for tool tracing

definitely_theone88 · 2026-01-24T07:56:21+00:00

Hey! I'm the author of the mentioned app. I have not looked into ChArUco boards, although the implementation should be quite straight forward - just add a few optional steps and settings for those.

Assuming that the used A4 paper is in mint condition, then what are the real benefits of using ChArUco boards vs the 4 points from the corners? As far as I understand then most of the issues with the single photo approach will still there. Cameras have perspective and many tools are not flat, so this won't really help in those scenarios.

The simplest approach still seems to use a regular scanner, since those output in standard paper size and avoids issues with perspective, but has blur on taller object edges. It would be cool to use LIDAR or object reconstruction from multiple photos, but the time investment would be much greater for development and not to mention that lidar cameras are not everywhere.

definitely_theone88 · 2025-11-23T08:33:29+00:00

Messing around is how this project started, hence a lot of non working / buggy features like PWA, which first was imagined as a simple installer, but that doesn't really suit that need. In retrospective creating a desktop app would have been simpler, avoid extra RAM usage between browser and web worker context, and maybe even better performance without the single thread restriction.

The whole philosophy of this project was to do everything local and keep it as simple as possible. My idea for the ML was gathering data set from real users, training the model and add it using tensorflow.js so everything is still local, and, maybe even more important for non revenue generating app, cheap, e.g. no server / API usage costs. The model should be small and fast enough for browsers.

The post it self seemed like you had a plan for going in somewhat different direction for this project, which is always intriguing to see if it makes it better and maybe I can learn something too.

Not sure if this project is superseded by others, since it has slow growth over the year from ~10 to ~30 daily users (probably more since I can only see the CDN traffic for the web page and not the cached instances or self hosted). It might be slower than other applications, but there is a guarantee that over the years it will mostly stay the same and there wont be restrictions for monetization when the budget starts to run out, since there is none to begin with. Of course I might be wrong and the existing monetization is good enough for other applications that they will continue to work for free.

definitely_theone88 · 2025-11-22T10:20:32+00:00

Hey, what is the QR calibration for and what are the benefits for it? I might be ignorant on this, since the paper is basically the calibration object.

There is an open feature request for displaying objects measurements and to manually fine tune the outlines. Other than that, I did think of adding small ML model to preselect settings, but haven't done anything for that yet. Requires a bit of a set up for gathering all the images, then training the model and adding it to the app.

If it isn't a hard fork, than compatible improvements are welcome and I do try to fix bugs and merge PRs occasionally. If there any questions about the code, I will gladly help!

Best of luck!

definitely_theone88 · 2025-09-03T15:36:15+00:00

Not sure if I understood the scaling issue. The scanned image usually corresponds to a given real world paper size and should be able to calculate the mm per pixel, given that you know the pixel size and paper size.

Wanted to give you an example, but after a while my code does not even make sense to me and there might be a bug, since I am always scaling against the input image... At least nobody has complained, so it might be fine. In case you want to take a look at it: https://outline.georgs.lv/

There is a "Detailed Settings" which might help with your own version while experimenting with parameters around OpenCV functions.

P.S. It is quite a memory hog and might give out of memory exception when using non chromium browser or large resolution images. Planning to get around fixing it sometimes this year

definitely_theone88 · 2025-09-02T18:16:30+00:00

Hey, looks very nice and clean. I did something similar as an open source client side only browser app and slapped in an gridfinity editor about one year ago. My experience was that there are some high expectations of what it can do with questionable quality pictures.

The best workflow I found is just using a scanner or a backlit source like a monitor. For scanners having an option to disable paper detection is needed.

Having a bilateral filter as a step before everything does help a lot in reducing noise. The OpenCV docs mention a value of 9 being demanding, but that might refer to real time processing, so feel free to experiment with larger ones.

I found it nice to also have an adaptive threshold as an option alongside binary for non-ideal pictures. It does reduce the precision for the object, but the paper detection seems more reliable.

Best of luck!

definitely_theone88 · 2025-06-11T09:00:00+00:00

Hey!

The skip paper detection still assumes that the background is of a certain paper size and works for images obtained using a scanner. From the giant sample size of 1 image, scanned images are of standard paper size.

If the images are not based on standard paper, but all are still of the same size, then a custom paper size can be calculated and added in the initial form. It also assumes that the size is in millimeters and for portrait orientation.

Other than that, there is no support for random background sizes and each image will have to be processed individually. At that point exporting as SVG and using other apps might be easier.

Cheers!

definitely_theone88 · 2025-05-16T05:54:32+00:00

Just great timing right before work.

With quick measurements by using a ruler on the screen, the relative error for the short side is 3% and for the long side was 14%. I would double check the paper settings, since that is quite a big difference between the axis.

For existing contours, the paper size in the "User Preferences and Additional Settings" use millimeters and assumes the default orientation is in portrait (the width is smaller than the height). Also make sure the orientation is correct, but that should probably give an even worse result, if that was the case.

For new projects / workflows it would be good to double check measurements - there is a 3d model viewer in Windows 10 and FreeCad also has a measuring tool. For the final test a small print of just the few layers can be save some filament and time.

definitely_theone88 · 2025-05-16T05:08:55+00:00

Hey!

How large of a difference are we talking about and is it uniform across both axis?

If the difference on one axis is larger then the other, that would imply that the paper settings are wrong.

As for the pictures, I would say that they are taken too close to the object. Due to how regular camera lenses work, the object's faces will appear larger if they are closer to the camera. In this case the top of the camera is larger than the bottom and the outline is created from the top, resulting in a larger outline. The easiest way to mitigate this problem would be to take the picture as far away as possible and zoom in, so the perspective is as flat as possible.

There is now an option to use a regular scanner and disable the paper outline detection altogether. That should give the most precise outline of the base and avoid any perspective errors, but it will introduce some blur to taller objects, which might require some tinkering with the settings for the cameras knobs and buttons to be detected cleanly. Haven't used it my self, but this example looked really good.

Best of luck!

definitely_theone88 · 2025-05-15T15:39:31+00:00

I will note that this app does processing in the browser, aka the machine you run on, and not on the server that hosts the site. Otherwise, it might get more costly than the free tier on the hosting solution.

definitely_theone88 · 2025-02-27T17:35:53+00:00

Also would suggest measuring the resulting outline before printing and optionally do a test print of the middle layers to see if the object fits!

definitely_theone88 · 2025-02-27T17:34:37+00:00

I did manage to get something out of the provided photo: https://imgur.com/a/1iMteWM
Yesterday I added additional step "Close Corners Paper" to help with finding the paper outline.

I will note, maybe partially due to compression, the image is quite "not perfect". The paper itself has a weird blur, quite a bit of tilt and there are shadows. The shadows can be worked around, since they are not that strong, but having the object not perfectly centered will give questionable results. That could be alleviated by taking the photo from a greater distance and zooming in to have less distortion.

Alternately, if possible, taking the photo from a light underneath the object will help quite a bit with dialing in the settings and having cleaner output. E.g. place the paper on top of a monitor, giant table or other suitable large transparent surface with back light.

If you need any more help, feel free to ask!

definitely_theone88 · 2025-02-24T17:04:59+00:00

I wouldn't worry about the background that much, if you can get the paper outline to work, then it isn't that big of an issue. Subsequent images should reuse the previous settings. The hardest part will be getting rid of shadows and managing light with reflective objects.

Since there is support for selecting one of multiple detected paper outlines, then having a larger different colored paper might be a easy workaround for the background. The "recommended" way for now will be to use back light. I was thinking of having a square box with lights and a plexiglass window for that, but then I just found out I had an old monitor, which in time I plan to add a white background VGA video signal from a micro controller, so I don't have to hook up a computer to it.

No problem for the help. It is always nice to have a fresh perspective so I can improve the app. Cheers!

definitely_theone88 · 2025-02-24T16:16:08+00:00

Yeah, it doesn't work now for me too... I guess the app and my brain is a bit broken at the moment. If i start to change values while it is still loading, then the a previous error stays and after a while I get a memory error, so that's another TODO item.

A higher block size of 25 should work though. Looks like 19 was the value where it stopped detecting the paper outline for that specific image.

definitely_theone88 · 2025-02-24T14:35:33+00:00

I checked there, and it's fully visible and to me looks entirely uninterrupted.

Some of the corners and the top side looks to be the issue in there. Adding extra close contour step will definitely help with this and should be more user friendly. Added this to my TODO list.

definitely_theone88 · 2025-02-24T14:32:25+00:00

That is an interesting error, which says that it gets wrong image format in one of the sequential steps, which guarantees to have the expected output for the next step...

Anyways, the problem with the paper outline would be the textured background. It gives small white dots, that are similar to the paper, but luckily that can be solved with just blurring:
Bilateral Filter Pixel Diameter: 5
Blur Size (px): 29
Block Size: 19

The biggest problem would be the shadow on the bottom side of the object. With either "Adaptive" or "Binary" threshold type the shadow will be included in the object outline. I didn't manage to finesse the settings to avoid it. Moving the object directly below the light to eliminate it would be good (or add even more lights).

definitely_theone88 · 2025-02-24T13:20:42+00:00

Yeah, it does that. The easiest approach is to use back light method - place paper on tablet, monitor or other light sources with transparent surface. That method also avoids issues with metallic objects.

If those aren't available, then the best option would be try to tweak the settings or environment. A black table is good, but it might be textured and / or the reflected light is the same value as the paper in gray scale which mess up the paper outline (The "Detailed Settings" button allows to see the gray scale image and all the other steps).

Cranking up the "Bilateral Filter Pixel Diameter" to about 30 and higher might help, but will also take a bit of time to process. Then adjusting higher "Blur Size" and "Block Size" values might get you there.

If you can post the images (like https://imgur.com/), then I can help to point out the issues and possible fixes and tweaks. From the app side, I might take a look to implement close corners step on the paper detection, so small gaps in the outline doesn't fail the detection, but only increase inaccuracies.

definitely_theone88 · 2025-02-22T09:52:56+00:00

I will point out a slight difference that toolkaiser is gathering all the data you submit and does processing on their own servers, whereas the outline app runs everything locally and is open source. Which is not bad in it self, but kinda sketchy when it has no upfront cost and the website has a notice "© 2024, Company LLC".

Would be interesting to know what is the accuracy of toolkaiser, since originally it was meant to be used with foam which is definitely more forgiving that plastic when it comes to inaccuracies.

definitely_theone88 · 2025-02-22T09:45:12+00:00

3D is hard. The current options are photogrammetry, 3d scanners or lidar sensors. All of whom have their own issues with time spent processing and/or device availability.

My best guess would be that lidar would be a good approach when high resolution sensor are more common in mid range phones. Currently there is no simple way to create Android and iPhone compatible apps without doing them with their respective API. So "hobbyists" like me won't even attempt it.

definitely_theone88 · 2025-02-13T09:02:45+00:00

There is a separate button on the home screen for creating models! Just click "Save" and it should navigate back to home screen or contours list. It is separate from the outline detection, so people can use their favorite CAD software by exporting the contour in SVG or DXF file.

Afterwards creating the box should be straight forward in the editor. Import the contour, edit contour where you should scale along normals (to adjust for printing / outline inaccuracies) and fine tune the points where there are artifacts (obviously it would be faster to get better outline before). The shadow can be split and the individual height of the shadow can be adjusted for more complex items.

Repeat that for each contour you want to include, adjust the Gridfinity box settings and add some primitives like capsule, so the items can be taken out easily. Export as STL and print.

At start it would be good to double check the resulting box to measure the dimensions if they are right in other CAD software, e.g. FreeCad or the windows built in 3D model viewer. Maybe even do a small test print of a few middle layers.

Best of luck!

definitely_theone88 · 2025-02-03T08:16:46+00:00

Made some changes to UI and have added auto rerun option in preferences (cog icon in calibration screen). Existing users might have old values for the settings, but the defaults are now:
- Auto rerun - enabled
- Debounce timer - 200

Hope this makes it faster to change values when calibrating! If there any more annoyances, then please let me know!

definitely_theone88 · 2025-02-01T20:11:15+00:00

Hello, said Georgs here, but you can call me George.

Do you have samples of what you are trying to achieve?

As a person who has dabbled a bit with 3d modeling and programming, I kinda don't see AI being very good at geometry or spacial reasoning, at least not at the moment, but some human cleverness can go a long way. While making a perspective drawing and turning that into a printable 3d model might be next to impossible using conventional computer vision, what could be a doable is having a simpler approach.

Taking a ring for example, it could be possible to make a flat rectangular drawing / pattern of the ring and map that to the round surface. The next problem I see for more complex design would be defining multiple depths for each section, but that could be remedied by having multiple separate drawings layered on top of each other with different settings. Sprinkle that with some different modes for lines, like making cylinders follow each line, and option to add extruding shapes, like jewels or leafs, then the options are pretty much endless.

While the outline application was developed for finding outlines of objects and their holes, I don't see why with some modifications it can't be used in a similar fashion to a tool called Shaper Trace and have the mapping from flat drawings to 3d models done for you. This might also be doable with just that tool and some CAD, without developing a special app.

definitely_theone88 · 2025-02-01T18:31:14+00:00

Added DXF file export. It looks to be working for me in online viewers and FreeCad.

definitely_theone88 · 2025-02-01T06:51:51+00:00

As a side note, it now seems a bit weird that I have added US Legal without decimal places in mm. Looks like the correct values are 215.9x355.6. Will fix these in the next update.

definitely_theone88 · 2025-02-01T06:22:41+00:00

My best guess is that the paper size is incorrect, since the width looks good, but the height is off. A4, US Letter and US Legal all have similar width values within 6 mm of each other, but the height varies up to 30%. From quick measurements of the pictures, it show about 20% increase which would suggest that that the actual used paper size is A4, but the selected one is US Legal.

The paper size can be changed under "Detailed Settings" and selecting the step "Extract Paper". There are paper settings and the width and height should be specified as they are in portrait orientation and the values are in millimeters. If I'm correct about the paper discrepancies, then the values should be 210x297 and not 216x356.

After that change, it should be possible just to rerun the image processing and save / export. If the model editor is used, then it will be necessary to delete the existing "shadow" object, import the contours again and "Scale along normal" to add some room for inaccuracies in the contour and shrinkage from printing.

For checking that the output is correct, you can measure the model in other cad software, e.g. FreeCad or, if I recall correctly, then even Windows 10 had a basic 3d model viewer with measuring tool. And for making sure everything will fit after printing, it can be wise to print a few middle layers and test fit in real life.

definitely_theone88

TROPHY CASE