Before ImageNet, many vision papers lived on tens of thousands of images and a handful of classes. Deng and colleagues organized millions of images under WordNet synsets and argued that scale and diversity were necessary for real category recognition. The later ILSVRC challenge (1000 classes, fixed train/val/test) became the gravity well for AlexNet, VGG, GoogLeNet, ResNet. The 2009 paper is the dataset and the ontology, not the 2012 neural net.
If you train "on ImageNet" you inherit a particular 1000-way slice, a particular sense of what a "class" is (often object-centric photos), and a decade of leakage into papers that overfit the val set. Treat it as a standard, not as the world.
Hierarchy is a feature and a trap
WordNet synsets split fine concepts (lots of dog breeds) and lump others. That is why so many models are oddly good at terriers and weak at industrial parts: the ontology is not your warehouse. Hierarchical evaluation was discussed; most deep-learning papers optimized flat top-1. If your product cares about parent categories, a flat 1000-way head is a mismatch even if pretrained on ILSVRC.
Collection via search engines and human annotation does not yield a uniform world prior. Geographic, cultural, and photographer biases are now well documented. Using ImageNet-pretrained features on medical or satellite images is a convenience, not a fairness or domain story.
What the dataset did to engineering
Fixed-size training, heavy augmentation, 224×224 crops, and "1-crop vs 10-crop val" all grew up around this benchmark. Your production images are not 224 center crops of Flickr-like photos. When a pretrained ResNet fails, ask whether the pretrain distribution is the issue before you stack more layers.
ILSVRC is also a lesson in challenge design: a public train set, a hidden test, a yearly clock. That social infrastructure produced progress and produced leaderboard hacking. Replicate the hidden test idea internally.
A worked transfer miss
You take ImageNet-pretrained EfficientNet, fine-tune on 12 factory defect classes with 50 images each. It underperforms a small CNN trained from scratch on those 50. The pretrained features are texture-and-object, not scratch-and-glare. ImageNet was not wrong; it was not your data. Try a domain-closer pretrain or more labels.
Failure modes
Reporting ILSVRC numbers as 'human-level' without the paper's caveats.
Test-set reuse for 15 years of architecture search.
Assuming 1000 classes cover 'objects'.
Ignoring license and consent of the underlying images in a commercial train (a later, real constraint).
The challenge versus the full database
ILSVRC's 1.28M images / 1000 classes is not the full ImageNet-21k. Papers that pretrain on "ImageNet" must say which. 21k is a different label grain and a different compute bill. When you inherit a checkpoint, read the card. Also plan for the fact that several synsets are ugly as product labels (sensitive categories, overlapping animals). A warehouse detector should not start from a 1000-way softmax you never inspect. Use ImageNet as pretrain fuel, then put your ontology on the head.
What you can borrow
- Invest in labeled scale and a stable ontology when you want a community benchmark.
- Keep a true hidden test.
- Read the class taxonomy before you transfer; hierarchy may not match the product.
- Treat ImageNet pretraining as a prior for natural object photos, not as universal vision.
- Build internal datasets with the same seriousness as the model.