When people talk about AI in fashion, the conversation usually jumps straight to models. Multimodal systems, image generation, visual search, virtual try-on. But when I look at the problem as an engineer, I usually start somewhere else: what are we actually going to train it on?
In fashion, that question matters more than it sounds. Clothes are difficult objects to recognize consistently. The same jacket can look completely different in a clean product shot, on a person, in motion, under another layer, or in a random street-style photo. Two nearly identical black jackets can be different SKUs, while the same garment can look noticeably different depending on fit, lighting, pose, or camera angle.
So there is no universal fashion dataset. If I want to match the same product across different photos, I need one kind of data. If I want to understand garment details, I need another. Virtual try-on and 3D clothing require something else entirely.
Looking at how fashion datasets have evolved over the past decade makes that pretty clear.
Anton Borzenkov / Unsplash
DeepFashion: when clothing data started getting serious
One of the best-known datasets in the space is DeepFashion.
It was introduced by researchers at the Chinese University of Hong Kong in 2016 and contains more than 800,000 fashion images, ranging from clean retail photography to ordinary consumer photos. The images include category labels, attributes, garment landmarks, and, importantly, links between different photos of the same product.
That last part is what makes DeepFashion especially interesting to me.
Imagine a retailer has a perfect product image of a jacket: clean lighting, a neutral background, the whole garment visible. Then you have a street photo where the same jacket is partly covered by a bag, twisted around the body, wrinkled, and shot under completely different lighting.
A person can often recognize the match immediately. For a model, that is already a real problem.
DeepFashion made it possible to train systems not only to say, “this is a jacket,” but also to understand that two very different-looking images may contain the same product.
DeepFashion also includes a dedicated In-shop Clothes Retrieval benchmark, with 52,712 images representing 7,982 clothing items photographed across different poses and scales.
That maps directly to a practical use case: upload a photo of a garment and search a catalog for the same or a visually similar item.
But DeepFashion should not be treated as some universal dataset for “fashion AI.” It was built around a specific set of computer vision problems that mattered at the time.
Jhana Ellard / Unsplash
ModaNet and DeepFashion2: real photos turned out to be much messier than catalogs
The next obvious problem was learning to work with clothes outside clean retail photography.
In 2018, eBay researchers introduced ModaNet, a street-fashion dataset with polygon annotations for garments across 13 major categories. The original paper describes 55,176 fully annotated images.
I like ModaNet because it captures a very different environment.
Catalog photography is usually friendly to a model: good lighting, predictable angles, very little visual noise. Street-style photography is the opposite. A coat covers a shirt, a bag blocks part of a blazer, people overlap, and the background contains all kinds of things that have nothing to do with the clothing itself.
A year later, DeepFashion2 pushed the idea further.
It contains 491,000 images and roughly 801,000 individual clothing items across 13 categories. Each garment is annotated with information such as position, viewpoint, scale, occlusion, landmarks, bounding boxes, and pixel-level masks. The dataset also contains around 873,000 commercial-consumer clothing pairs.
So instead of assigning one simple label to a photo, you can work separately with the shirt, trousers, jacket, or other garments inside the same image.
That becomes useful for much more than visual search. If I want to isolate individual pieces from an outfit photo, remove backgrounds, understand which garments are worn together, or prepare the image for another processing step, I need this kind of annotation.
DeepFashion2 also shows how quickly the problem gets more complicated. We start with “is this a jacket?” and very quickly move to “where exactly is the jacket, where does it end, which parts belong to it, and is it the same jacket as the one in our catalog?”
Vasilis Caravitis / Unsplash
Fashionpedia: “jacket” is not nearly enough
Even if a model can already find clothing in an image, there is another problem: how much can it actually say about the garment?
For many business use cases, “this is a jacket” is not enough.
You may want to know the length, sleeve type, collar shape, pocket style, fit, and other product details.
That is where Fashionpedia becomes useful.
Fashionpedia combines images with a much richer fashion structure: 27 main apparel categories, 19 garment parts, and 294 detailed attributes. The annotations connect garments with both their components and their characteristics.
That starts to look much closer to real product information.
There is a big difference between:
“jacket”
and:
“cropped double-breasted jacket with long sleeves and welt pockets.”
But this is also where I would draw a clear line. Even a detailed public fashion vocabulary does not know how a specific company structures its own product catalog.
One brand may use overshirt, another may call the same thing a shirt jacket, while a marketplace might place it under light outerwear.
And inside the same company, the item may sit in one category in the PLM, another in the PIM, and a third on the e-commerce storefront.
So an academic dataset can only take you so far. Once the model needs to speak the language of a real catalog, the work becomes a mapping problem: internal category trees, attributes, product names, and the terminology the business actually uses.
Sou Jest / Unsplash
Deep Fashion3D and 4D-DRESS: clothing stopped being a flat image
The next big shift happened when fashion AI started moving beyond conventional photography.
In 2020, Deep Fashion3D introduced a dataset for reconstructing the 3D shape of garments from images. It includes 2,078 reconstructed models across 10 categories and 563 real garment instances.
Now the model is no longer trying to understand only what the garment looks like in a photo. It also has to understand its shape.
For virtual garments, digital humans, and 3D e-commerce, that is a completely different level of data.
Then, in 2024, 4D-DRESS was presented at CVPR.
It contains 64 outfits captured across more than 520 motion sequences, with roughly 78,000 3D scan frames in total. The dataset also includes semantic labels, extracted garment meshes, and fitted body meshes.
I like 4D-DRESS because it shows just how far the field has moved since DeepFashion.
In 2016, the question was whether a model could understand what was in a photo and find the same item somewhere else.
Now we want to understand how real clothing changes shape when a person walks, turns, or moves.
You cannot get there by throwing another million JPEGs at a model and hoping it somehow learns how fabric behaves. You need a very different kind of data, where the garment exists not as a flat picture but as a shape changing over time.
Fashion-Diffusion and VITON-HD: now the image has to be connected to something else
A different branch evolved in parallel.
With the rise of generative AI, an image paired with a simple label like dress stopped being enough. If you want generation, you need meaningful relationships between the image and a description of what is actually in it.
One interesting example is Fashion-Diffusion, introduced in 2023. Its authors collected more than a million fashion images paired with detailed text descriptions, along with fine-grained annotations around people and garments.
DeepFashion helped answer “What is in this image?” Here the question is closer to “What should a garment described by this text look like?”
If I want to generate images from prompts, search products in natural language, or connect visual information with useful text, image + text pairs become much more valuable than simple category labels.
Virtual try-on needs yet another kind of relationship.
One of the best-known datasets for that is VITON-HD, created for high-resolution virtual try-on at 1024 × 768.
Here you have an image of a person and a separate image of a garment, and the system has to produce a new image that preserves the person and the appearance of the clothing while changing how that garment sits on the body.
So the same fashion photo can become useful in completely different ways depending on what it is connected to: a category, a specific SKU, a text description, another image, a person, or a 3D shape.
It feels logical that the next major fashion dataset should come from the industry itself, with richer product information, materials, fit, and proper links to real catalogs. But so far, I have not seen a new public dataset that has become the kind of industry-wide reference point DeepFashion was in 2016.
