using make_batch_carbon_reader to read data
PYTHONPATH=~/dev/pycarbon # replace with your pycarbon install path
This creates both a train
and test
carbon datasets in /tmp/mnist_external
:
python generate_external_mnist_carbon.py
This will invoke a training run using MNIST carbondata, for 1 epochs, using a batch size of 100, and log every 10 intervals.
python tf_external_example_carbon.py
python tf_external_example_carbon.py -h usage: tf_external_example_carbon.py [-h] [--dataset-url S] [--num-epochs N] [--batch-size N] [--evaluation-interval N] Pycarbon Tensorflow MNIST Example optional arguments: -h, --help show this help message and exit --dataset-url S hdfs:// or file:/// URL to the MNIST pycarbon dataset(default: file:///tmp/mnist_external) --num-epochs N number of epochs to train (default: 1) --batch-size N input batch size for training (default: 100) --evaluation-interval N how many batches to wait before evaluating the model accuracy (default: 10)