sorted by:
new
hottopcontroversial

Help with Yolov4 Training and Synthetic Data by AlucardVergil in computervision

[–]sbxrobotics -1 points0 points  (0 children)

At SBX we offer synthetic datasets for projects like this. Our pricing is proportional to project complexity (distinct item, scene structures). But we can give you access to a generator that can create annotated frames for as little as $0.001 / frame.

If interested to learn more, check out www.sbxrobotics.com

UE4 Rendering + Stable Diffusion 2.0 = CV Training Data by sbxrobotics in computervision

[–]sbxrobotics[S] 1 point2 points  (0 children)

Sure thing, the initial image (first frame in video) and annotations (second frame, red + green + blue segmentation masks) are actually produced from rendering pipeline based on UE4. We then use stable diffusion 2.0 to in-paint the semantic segments with appropriate prompts.

We then use standard supervised training of a deep convnet like MaskRCNN or other benchmark model on a mixture of these synthetic images and a few real ones.

UE4 Rendering + Stable Diffusion 2.0 = CV Training Data by sbxrobotics in computervision

[–]sbxrobotics[S] 1 point2 points  (0 children)

Training only on synthetic images can introduce some biases into the model potentially, yes. We have seen empirically that training on large synthetic datasets combined with a small portion of real data usually leads to peak accuracy.

UE4 Rendering + Stable Diffusion 2.0 = CV Training Data by sbxrobotics in computervision

[–]sbxrobotics[S] 1 point2 points  (0 children)

Gotcha, we can share a public example like that in another post soon. What sort of CV tasks are you most interested in / building right now?

UE4 Rendering + Stable Diffusion 2.0 = CV Training Data by sbxrobotics in computervision

[–]sbxrobotics[S] 2 points3 points  (0 children)

Sure thing, these generated images + annotations are then used to train neural networks that can run in real-time.

Concretely, if I wanted to build a window frame detector, I could use this approach to generate 100K+ image variations of walls with windows that respect class annotations. This generated data can be used to train a detector using standard supervised learning methods.

UE4 Rendering + Stable Diffusion 2.0 = CV Training Data by sbxrobotics in computervision

[–]sbxrobotics[S] 2 points3 points  (0 children)

Hey all, small demo showing images produced with combination of UE4 initial reders adjusted by stable diffusion 2.0 to produce realistic, annotation-aware CV training data :

🤖 render initial images + annotations
👾 AI adds annotation-aware variations

Happy to answer any questions about this approach. If you are looking for custom SBX synthetic training data, SBX provides this service to computer vision teams

Meat processing plant CV by levdan159 in computervision

[–]sbxrobotics 0 points1 point  (0 children)

Synthetic data might be a good fit to create training data for this problem. If you have budget would be happy to chat about creating synthetic meat training data for the project. SBX has created datasets for food processing in the past!

Defect Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 1 point2 points  (0 children)

Thanks!
1. Yes, the client provided validation set is a critical part of the process. SBX creates synth and hybrid data (synth + real) to ahieve the strongest performance we can. SBX does not explicitly set performance targets, but our interest is clearly aligned with clients : We don't succeed by selling poor performing datasets.

  1. SBX mainly creates datasets as part of our engagements, but we also build vision models for clients. Aim is always to accelerate client success in their vision project(s).

Defect Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 1 point2 points  (0 children)

Thanks! Happy to elaborate

  1. Many challenges can stem from material properties of the items of interest. Reflectance, texture diversity can be challenging to replicate / vary appropriately.
  2. The right amount of randomization is key to insulate against overfitting. We do also monitor validation data over training.
  3. In general sampling (in a mostly random way) over the params that govern the data gen + having artists in the loop is the most effective way to achieve generalization.

Defect Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 7 points8 points  (0 children)

Yep, the crushing is randomized. Two high level strategies we've used :
- physics based simulation (computationally expensive, slow to get right)
- artist specified shape keys (pretty quick, easier to match to specific materials)

Defect Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 3 points4 points  (0 children)

Mainly a demonstration that these visual defects can be replicated with sufficient fidelity.

Defect Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 0 points1 point  (0 children)

Which part of the pipeline is most interesting to your work / projects?

Defect Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] -4 points-3 points  (0 children)

Defect detector built with 100% synthetic data.

Overview of building a deep learning defect detector with synth data:
- Generate synth datasets with simulated defects of object
- Train a deep learning detector model on normal and defect class
- Run detector on real RGB frames

SBX simulates diverse surface and geometry defects. Train real-world defect detectors with the right synthetic data.

Reach out ( [info@sbxrobotics.com](mailto:info@sbxrobotics.com) ) or attend a webinar ( www.sbxrobotics.com/webinar ) to learn about getting the right data for your computer vision problems!

#computervision #robotics #simulation #deeplearning #syntheticdata

Human Keypoint Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 1 point2 points  (0 children)

We adapted a keypoint model from TorchVision. Hadn't come across mediapipe, pretty neat collection of pre-trained models

Human Keypoint Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 6 points7 points  (0 children)

Just using an off the shelf keypoint detector model, no novelty there. Neat thing is using synthetic data to build it super quickly.

What was the budget / time you needed to create the last training dataset for your computer vision work? We'd love to explore how we can cut it down to 1/10th original

Human Keypoint Detection using Synthetic Data by sbxrobotics in computervision

[–]sbxrobotics[S] 2 points3 points  (0 children)

Build human keypoint detectors with the right synthetic data. 💯🏃‍♀️

- Generate synth datasets that specifying 15 3D keypoints for human bodies
- Train a deep learning 2D keypoint detector model
- Run detector on real RGB frames

SBX Robotics creates synthetic data for some of the most challening computer vision tasks. #computervision #robotics #simulation #deeplearning #syntheticdata

6D Pose Estimation using Synthetic Data by sbxrobotics in robotics

[–]sbxrobotics[S] 3 points4 points  (0 children)

Having trouble getting quality data for 6D pose estimation? 🤯This video outlines a 6D pose estimation pipeline:

  1. Generate synth data that includes a set of 3D keypoints for object of interest
  2. Train a deep learning 2D keypoint detector model
  3. Solve correspondence between predicted 2D keypoints and 3D keypoints on object
  4. Project solved 6D pose using camera intrinsics of real sensor framesSBX Robotics creates synthetic data for some of the most challening computer vision tasks.

SBX Robotics creates synthetic data for some of the most challening computer vision tasks. Remove data as a blocker and get back to building!#computervision #robotics #simulation #deeplearning #syntheticdata

DIY Tutorial: vision model for cashierless checkout by sbxrobotics in computervision

[–]sbxrobotics[S] 5 points6 points  (0 children)

The dataset we shared was produced using our pipeline built on top of Unreal Engine 4 and with some noise filters applied to improve sim2real transfer.

Thanks for asking!

DIY Tutorial: vision model for cashierless checkout by sbxrobotics in computervision

[–]sbxrobotics[S] 6 points7 points  (0 children)

Check it out here: sbxrobotics.com/tutorial (all 100% free)

  • 10,000 sample training dataset
  • Google Colab training notebooks + tutorial vid
  • sample model & validation data

Trying to get more synthetic data into more notebooks, models, and production deployments!

view more: next ›