PyTorch-Ignite 0.4.1 - Release Notes

Core

New Features and bug fixes

• Improved docs for custom events (#1179)

Handlers and utils

• Added custom filename pattern for saving checkpoints (#1127)

Distributed helper module

• Improved namings in _XlaDistModel (#1173)
• Minor optimization for idist.get_* methods (#1196)
• Fixed distributed proxy sampler runtime error (#1192)
• Fixes bug using idist with "nccl" backend and torch cuda is not available (#1166)
• Fixed issue with logging XLA tensors (#1207)

Contrib

New Features and bug fixes

• Fixes warning about "TrainsLogger output_handler can not log metrics value" (#1170)
• Improved usage of contrib common methods with other save handlers (#1171)

Examples

• Improved Pascal Voc example (#1193)

Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

@Joel-hanson, @WrRan, @jspisak, @marload, @ryanwongsa, @sdesrozis, @vfdev-5

Core

• Added State repr and input batch as engine.state.batch (#641)
• Adapted core metrics only to be used in distributed configuration (#635)
• Added fbeta metric as core metric (#653)
• Added event filtering feature (e.g. every/once/event filter logic) (#656)
• BC breaking change: Refactor ModelCheckpoint into Checkpoint + DiskSaver / ModelCheckpoint (#673)
  • Added option n_saved=None to store all checkpoints (#703)
• Improved accumulation metrics (#681)
• Early stopping min delta (#685)
• Droped Python 2.7 support (#699)
• Added feature: Metric can accept a dictionary (#689)
• Added Dice Coefficient metric (#680)
• Added helper method to simplify the setup of class loggers (#712)

Engine refactoring

Finally solved the issue #62 to resume training from an epoch or iteration

• Engine refactoring + features (#640)
  • engine checkpointing
  • variable epoch lenght defined by epoch_length
  • two additional events: GET_BATCH_STARTED and GET_BATCH_COMPLETED
  • cifar10 example with save/resume in distributed conf

Contrib

• Improved create_lr_scheduler_with_warmup (#646)
• Added helper method to plot param scheduler values with matplotlib (#650)
• BC Breaking change: with multiple optimizer's param groups (#690)
  • Added state_dict/load_state_dict (#690)
• BC Breaking change: Let the user specify tqdm parameters for log_message (#695)

Examples

• Added an example of hyperparameters tuning with Ax on CIFAR10 (#652)
• Added CIFAR10 distributed example

Reproducible trainings as "References"

Inspired by torchvision/references, we provide several reproducible baselines for vision tasks:

• ImageNet
• Pascal VOC2012

Features:

• Distributed training with mixed precision by nvidia/apex
• Experiments tracking with MLflow or Polyaxon

Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

@anubhavashok, @kagrze, @maxfrei750, @vfdev-5

Core

Various improvements in the core part of the library:

• Add epoch_bound parameter to RunningAverage (#488)

• Bug fixes with Confusion matrix, new implementation (#572) - BC breaking

• Added event_to_attr in register_events (#523)

• Added accumulative single variable metrics (#524)

• should_terminate is reset between runs (#525)

• to_onehot returns tensor with uint8 dtype (#571) - may be BC breaking

• Removable handle returned from Engine.add_event_handler() to enable single-shot events (#588)

• New documentation style 🎉

Distributed

We removed mnist distrib example as being misleading and provide distrib branch to adapt metrics for distributed computation. Code is working and is under testing. Please, try it in your use-case and leave us a feedback.

Now in Contributions module

• Added mlflow logger (#558)
• R-Squared Metric in regression metrics module (#496)
• Add tag field to OptimizerParamsHandler (#502)
• Improved ProgressBar with TerminateOnNan (#506)
• Support for layer freezing with Tensorboard integration (#515)
• Improved OutputHandler API (#531)
• Improved create_lr_scheduler_with_warmup (#556)
• Added "all" option to metric_names in contrib loggers (#565)
• Added GPU usage info as metric (#569)
• Other bug fixes

Notebook examples

• Added Cycle-GAN notebook (#500)
• Finetune EfficientNet-B0 on CIFAR100 (#544)
• Added Fashion MNIST jupyter notebook (#549)

Updated nighlty builds

From pip:

pip install --pre pytorch-ignite

From conda (this suggests to install pytorch nightly release instead of stable version as dependency):

conda install ignite -c pytorch-nightly

Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

@ANUBHAVNATANI, @Bibonaut, @Evpok, @Hiroshiba, @JeroenDelcour, @Mxbonn, @anmolsjoshi, @asford, @bosr, @johnstill, @marrrcin, @vfdev-5, @willfrey

Core

• We removed deprecated metric classes BinaryAccuracy and CategoricalAccuracy and which are replaced by Accuracy.

• Multilabel option for Accuracy, Precision, Recall metrics.

• Added other metrics:

  • ConfusionMatrix, IoU, mean IoU

• Operations on metrics: p = Precision(average=False)

  • apply PyTorch operators: mean_precision = p.mean()
  • indexing: precision_no_bg = p[1:]

• Improved our docs with more examples.

• Added FAQ section with best practices.

• Bug fixes

Now in Contributions module

• added TensorboardLogger
• added VisdomLogger
• added PolyaxonLogger
• improved ProgressBar
• New regression metrics
  • Median Absolute Error
  • Median Relative Absolute Error
  • Median Absolute Percentage Error
  • Geometric Mean Relative Absolute Error
  • Canberra Metric
  • Fractional Absolute Error
  • Wave Hedges Distance
  • Geometric Mean Absolute Error
• added new parameter scheduling classes and improved parameters:
  • PiecewiseLinear
  • LRScheduler
  • other helper methods
• added custom events support: CustomPeriodicEvent

Notebook examples

• VAE on MNIST
• CNN for text classification

Nighlty builds with pytorch-nightly as dependency

We also provide pip/conda nighlty builds with pytorch-nightly as dependency:

pip install pytorch-ignite-nightly

or

conda install -c pytorch ignite-nightly 

Core

• We removed deprecated metric classes BinaryAccuracy and CategoricalAccuracy and which are replaced by Accuracy.

• Multilabel option for Accuracy, Precision, Recall metrics.

• Added other metrics:

  • ConfusionMatrix, IoU, mean IoU

• Operations on metrics: p = Precision(average=False)

  • apply PyTorch operators: mean_precision = p.mean()
  • indexing: precision_no_bg = p[1:]

• Improved our docs with more examples.

• Added FAQ section with best practices.

• Bug fixes

Now in Contributions module

• added TensorboardLogger
• added VisdomLogger
• added PolyaxonLogger
• improved ProgressBar
• New regression metrics
  • Median Absolute Error
  • Median Relative Absolute Error
  • Median Absolute Percentage Error
  • Geometric Mean Relative Absolute Error
  • Canberra Metric
  • Fractional Absolute Error
  • Wave Hedges Distance
  • Geometric Mean Absolute Error
• added new parameter scheduling classes and improved parameters:
  • PiecewiseLinear
  • LRScheduler
  • other helper methods
• added custom events support: CustomPeriodicEvent

Notebook examples

• VAE on MNIST
• CNN for text classification

Nighlty builds with pytorch-nightly as dependency

We also provide pip/conda nighlty builds with pytorch-nightly as dependency:

pip install pytorch-ignite-nightly

or

conda install -c pytorch ignite-nightly 

• Improve and fix bug with binary accuracy, precision, recall
• Metrics arithmetics
• ParamScheduler to support multiple optimizers/multiple parameter groups

Thanks to all our contributors !

What's new in this release:

• Contrib module with
  • Parameter schedule
  • TQDM ProgressBar
  • ROC/AUC, AP, MaxAE metrics
  • TBPTT Engine
• New handlers:
  • Terminate on Nan
• New metrics:
  • RunningAverage
  • Merged Categorical/Binary -> Accuracy
• Refactor of examples
• New examples:
  • Fast Neural Style
  • RL

Thanks to all our contributors !

Introduced Engine, Handlers and Metrics.

Metrics:

• BinaryAccuracy
• CategoricalAccuracy
• Loss
• Precision
• Recall
• etc

Handlers:

• ModelCheckpoint
• EarlyStopping
• Timer

Features:

• PyTorch 0.4 support

Examples:

• mnist.py
• mnist_with_tensorboardx.py
• mnist_with_visdom.py
• dcgan.py