Overview

Name	Dataset	Description	Scene	Download	Remark
mnist_classification	MNIST	See here	Horizontal FL	Click Here	-
cifar10_classification	CIFAR10	See here	Horizontal FL	Click Here
cifar100_classification	CIFAR100	See here	Horizontal FL	Click Here
svhn_classification	SVHN	See here	Horizontal FL	Click Here	-
fashion_classification	FASHION	See here	Horizontal FL	Click Here	-
domainnet_classification	DomainNet	See here	Horizontal FL	Click Here	Feature Skew
office-caltech10_classification	OfficeCaltech10	See here	Horizontal FL	Click Here	Feature Skew
pacs_classification	PACS	See here	Horizontal FL	Click Here	Feature Skew

Details

mnist_classification

mnist

Federated MNIST classification is a commonly used benchmark in FL. It assumes different virtual clients having non-overlapping samples from MNIST dataset.

model

Model Name	Non-Fed Performance	NumPara	Implementation
cnn	-		-
mlp	-		-

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner		Partitioning according to label diversity
DirichletPartitioner		Partitioning according to dir. distribution of labels

cifar10_classification

cifar10

Federated CIFAR10 classification is a commonly used benchmark in FL. It assumes different virtual clients having non-overlapping samples from CIFAR10 dataset.

model

Model Name	Non-Fed Performance	NumPara	Implementation
cnn	-		-
mlp	-		-

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner		Partitioning according to label diversity
DirichletPartitioner		Partitioning according to dir. distribution of labels

cifar100_classification

Federated CIFAR100 classification is a commonly used benchmark in FL. It assumes different virtual clients having non-overlapping samples from CIFAR100 dataset.

model

Model Name	Non-Fed Performance	NumPara	Implementation
cnn	-		-
mlp	-		-

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner		Partitioning according to label diversity
DirichletPartitioner		Partitioning according to dir. distribution of labels

svhn_classification

Federated SVHN classification is a commonly used benchmark in FL. It assumes different virtual clients having non-overlapping samples from SVHN dataset.

model

Model Name	Non-Fed Performance	NumPara	Implementation
cnn	-		-
mlp	-		-

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner		Partitioning according to label diversity
DirichletPartitioner		Partitioning according to dir. distribution of labels

fashion_classification

Federated Fashion classification is a commonly used benchmark in FL. It assumes different virtual clients having non-overlapping samples from FashionMNIST dataset.

model

Model Name	Non-Fed Performance	NumPara	Implementation
lr	-		-

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner		Partitioning according to label diversity
DirichletPartitioner		Partitioning according to dir. distribution of labels

domainnet_classification

domainnet

DomainNet contains images of the same labels but different styles (i.e. 6 styles), which can be used to investigate the influence of feature skew in FL. The paper is available at link

model

Model Name	Non-Fed Performance	NumPara	Implementation
AlexNet	-		-
resnet18

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner		Partitioning according to label diversity
DirichletPartitioner		Partitioning according to dir. distribution of labels

office-caltech10_classification

office-caltech10

Office-Caltech-10 a standard benchmark for domain adaptation, which consists of Office 10 and Caltech 10 datasets. It contains the 10 overlapping categories between the Office dataset and Caltech256 dataset. SURF BoW historgram features, vector quantized to 800 dimensions are also available for this dataset.

model

Model Name	Non-Fed Performance	NumPara	Implementation
AlexNet	-		-
resnet18

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner
DirichletPartitioner

pacs_classification

pacs PACS is an image dataset for domain generalization. It consists of four domains, namely Photo (1,670 images), Art Painting (2,048 images), Cartoon (2,344 images) and Sketch (3,929 images). Each domain contains seven categories.

model

Model Name	Non-Fed Performance	NumPara	Implementation
AlexNet	-		-
resnet18

supported partitioner

Name	IsDefault	Comments
IIDPartitioner	yes
DiversityPartitioner
DirichletPartitioner