Examples inside this folder show how to train CNN models using SINGA for image classification. The data augmentation is done only once before the training.
data includes the scripts for preprocessing image datasets. Currently, MNIST, CIFAR10 and CIFAR100 are included.
model includes the CNN model construction codes by creating a subclass of Module to wrap the neural network operations of each model. Then computational graph is enabled to optimized the memory and efficiency.
autograd includes the codes to train CNN models by calling the neural network operations imperatively. The computational graph is not created.
train_cnn.py is the training script, which controls the training flow by doing BackPropagation and SGD update.
train_multiprocess.py is the script for distributed training on a single node with multiple GPUs; it uses Python's multiprocessing module and NCCL.
train_mpi.py is the script for distributed training (among multiple nodes) using MPI and NCCL for communication.
benchmark.py tests the training throughput using ResNet50 as the workload.