Awesome-Compositional-Zero-Shot

Papers and codes about Compositional Zero Shot Learning(CZSL) for computer vision are present on this page. Besides, the commonly-used datasets for CZSL are also introduced.

Papers

2024

2023

2022

2021

2020

2019

2018

2017

Datasets

Most CZSL papers usually conduct experiments on MIT-States and UT-Zappos datasets. However, as CZSL receives more attention, some new datasets are proposed and used in recent papers, such as C-GQA, AO-CLEVr, etc.

MIT-States

Introduced by Isola et al. in Discovering States and Transformations in Image Collections.

The MIT-States dataset has 245 object classes, 115 attribute classes and ∼53K images. There is a wide range of objects (e.g., fish, persimmon, room) and attributes (e.g., mossy, deflated, dirty). On average, each object instance is modified by one of the 9 attributes it affords.

Source:http://web.mit.edu/phillipi/Public/states_and_transformations/index.html

UT-Zappos

Introduced by Yu et al. in Fine-Grained Visual Comparisons with Local Learning.

UT Zappos50K (UT-Zap50K) is a large shoe dataset consisting of 50,025 catalog images collected from Zappos.com. The images are divided into 4 major categories — shoes, sandals, slippers, and boots — followed by functional types and individual brands. The shoes are centered on a white background and pictured in the same orientation for convenient analysis.

Source:https://vision.cs.utexas.edu/projects/finegrained/utzap50k/

C-GQA

Introduced by Naeem et al. in Learning Graph Embeddings for Compositional Zero-shot Learning.

Compositional GQA (C-GQA) dataset is curated from the recent Stanford GQA dataset originally proposed for VQA. C-GQA includes 413 attribute classes and 674 object classes, contains over 9.5k compositional labels with diverse compositional classes and clean annotations, making it the most extensive dataset for CZSL.

Source:https://github.com/ExplainableML/czsl

AO-CLEVr

Introduced by Atzmon et al. in A causal view of compositional zero-shot recognition.

AO-CLEVr is a new synthetic-images dataset containing images of "easy" Attribute-Object categories, based on the CLEVr. AO-CLEVr has attribute-object pairs created from 8 attributes: { red, purple, yellow, blue, green, cyan, gray, brown } and 3 object shapes {sphere, cube, cylinder}, yielding 24 attribute-object pairs. Each pair consists of 7500 images. Each image has a single object that consists of the attribute-object pair. The object is randomly assigned one of two sizes (small/large), one of two materials (rubber/metallic), a random position, and random lightning according to CLEVr defaults.

Source:https://github.com/nv-research-israel/causal_comp

VAW-CZSL

Introduced by Nirat Saini et al. in Disentangling Visual Embeddings for Attributes and Objects.

VAW-CZSL, a subset of VAW, which is a multilabel attribute-object dataset. Sample one attribute per image, leading to much larger dataset in comparison to previous datasets. The images in the VAW dataset come from the Visual Genome dataset which is also the source of the images in the GQA and the VG-Phrasecut datasets.

Source:https://github.com/nirat1606/OADis.

Compositional PartNet

Introduced by Naeem et al. in 3D Compositional Zero-shot Learning with DeCompositional Consensus.

Compositional PartNet (C-PartNet) is refined from PartNet with a new labeling scheme that relates the compositional knowledge between objects by merging and renaming the repeated labels. The relabelled C-PartNet consists of 96 parts compared to 128 distinct part labels in the original PartNet.

Source:https://github.com/ferjad/3DCZSL

Results

Experimental results of some methods on the two most commonly-used datasets(MIT-States, UT-Zappos) and the most challenging dataset(C-GQA) are collected and presented.

All the results are obtained under the setting of Closed World Generalized Compositional Zero-Shot Learning. The current optimal metrics are in bold.

MIT-States

UT-Zappos

C-GQA

Acknowledgements

This page is made by Yanyi Zhang and Jianghao Li, both of whom are graduate students of Dalian University of Technology.