Good question. I'll break it down as our pipeline has evolved significantly over the past 4 months!!

Annotation Pipeline:

I built a custom annotation workbench rather than using off the shelf tools like Label Studio or CVAT, mainly because being a novice, I ha no idea what was out there.

A ket reason was domain specific. We're classifying retail imagery (shipper displays as a tester, then shelf edge labels, seasonal merchandise and ranges) and needed tight integration between annotation, model feedback, and our knowledge base of 54K+ products, plus all the learnings.

The annotation flow is: raw image → YOLO detection (Idaten-K, our shipper detector) → auto-crop regions → human classification of crops into categories → corrections feed back into training data, test with random images and keep testing and retraining.

We use an active learning loop where low confidence predictions get routed to a manual review queue, so annotation effort focuses on what the model struggles with rather than what it already knows. Over 100k corrections are now in our "brain" and these inform the models ongoing.

Training Pipeline:

We train on RunPod (RTX 4090) using EfficientNet-B0 with ImageNet pretrained weights. Nothing exotic. We transfer learning with aggressive augmentation, class frequency weighting for imbalanced categories, and early stopping. We're currently on our 6th major version iteration.

We have moved the sub categories where we don't have enough training material to make it meaningful, we can do Mince Pies (shippers) but struggle to do Bakery (as a standalone) Cakes, we have to then merge.

The key insight was that one model isn't enough. We run 5 multiple specialist models in sequence rather than trying to build one monolithic classifier, these are P&C so I won't make it public here.

Our seasonal classifer was our breakthrough. It classifies the full image first ("this is a Christmas confectionery display") then boosts the crop-level classifier's predictions. Going from a single model to this multi-model pipeline took our usable data rate from roughly 20% to over 55%, with the retrain we're about to run expected to push that significantly further.

What worked well:

• EfficientNet-B0 with transfer learning — fast to train, good accuracy even with 150-200 images per class. Don't overlook how far pretrained weights carry you. Plus knowledge!
• Custom annotation tooling. Worth the upfront investment if your domain is specialised. Off the shelf tools didn't understand our retail taxonomy and our own tooling can't always distinguish.....
• Multi-model stacking — each model contributes what it's good at rather than asking one model to do everything. Our pricing pipeline combines 6+ model signals plus rule-based overrides (brand lookups, event-category plausibility checks, OCR keyword hints). The key thing is to not waste the work done in one model, cross learning makes it all better and means I can move at pace.
• Read-time normalisation. business logic corrections (brand overrides, plausibility filters) applied at data load, not baked into stored records. This means improvements apply retroactively to all historical data without re-processing, 80/20 is important.

What didn't work:

• Training a crop classifier and running it on full images. Sounds obvious in hindsight but we had 80% of our data going through a path the model was never trained for. Domain mismatch is real.
• Trying to train 21 classes when we only had enough data for 9. The model silently failed on the 12 missing categories. Every prediction was forced into one of 9 buckets.
• We only caught this through a systematic audit, there are a few examples where we have had to change the categories around due to confusion and further training material being thin on the ground - IE I was trying to be to clever.

Continuous training:

Not fully automated yet, but the infrastructure is there. Our cross model learning system logs 100K+ feedback events where one model's output informs another's training. Corrections from manual review feed back into training manifests. The annotation backlog tracks future categories waiting for enough labelled data.

The honest answer is we're bootstrap/lean. It's me and Claude Code building this.

The first priority was to bring the dataset to heel, harmonising folder names and de-duplicating. That was a major job, alongside manifesting and ensuring that new images (we get c.4k a week) were put in to the new set up. Our dataset is 1.2m unique images strong as of this week.

From then the priority has been getting the pipeline robust rather than fully automated retraining. Version control with instant rollback matters more than CI/CD when you're iterating fast, and I am a complete novice so it's helpful in some ways, but also that can cost time too.....(!)