The Challenge
General-purpose vision-language models are good at describing what an image shows in the abstract, and considerably less reliable at the kind of fine-grained attribute extraction, multi-image reasoning, and compliance-relevant detail a marketplace actually needs from a product photo, which meant the company was getting inconsistent results from off-the-shelf models exactly where consistency mattered most.
The Approach
Solvd adapted open-weight vision-language models specifically to e-commerce through large-scale visual instruction tuning, task-specific fine-tuning, and evaluation benchmarks built for this domain rather than borrowed from a general one. The work covered product-attribute prediction, deep fashion understanding, dynamic attribute extraction, and a capability the team called Item Intelligence, which pulls structured product and compliance information out of multiple images of the same listing at once
The Outcome
The project produced an e-commerce instruction-tuning dataset of roughly four million examples and a task-specific dataset of 100,000 multi-image items, both built to support the fine-tuning work directly. For Item Intelligence specifically, a fine-tuned 4-billion-parameter model achieved better F1 performance than a zero-shot 27-billion-parameter baseline seven times its size, while running about 3.8 times faster, improving both accuracy and cost at marketplace scale. The work was built to support downstream production use, though productization itself sat outside the team’s scope. Large language models generated and refined the structured annotations behind the multi-image dataset (including re-annotation using automatically identified high-information image regions) and supported building human-verified examples for dynamic attribute extraction.
