TRAINS - AI的自动神奇实验管理器和版本控制

Features and Bug Fixes

• Add Logger histogram mode (stack/group/relative)
• Add PEP610 support for new pip>=20.1 git reference feature (see pypa/pip#609) #62
• Better support for PyTorch Ignite integration
• Support # and ? in uploaded image file names
• Support ArgumentParser.parse_arg() used before initializing task when auto_connect_arg_parser=False
• Fix upload_object_via_stream() in Azure storage driver
• Fix Task.get_tasks() return type

Features

• Add media (audio) support for both Logger and Tensorboard binding using Logger.report_media() #120
• Add autokeras example (see here) #125
• Add caching of extracted zip artifacts
• Add Task.get_tasks() filtering support
• Add trains.storage.StorageManager
• Add nicer stdout log flush
• Add text logging to TF V1 example
• Improve SSL retries and error handling
• Improve shutdown behavior in case of a user abort (Ctrl-C)
• Improve repository and module detection
• Improve resource monitor
  • Report memory usage for process (and sub-processes)
  • Report GPU memory usage for process (and sub-processes)
  • Stability
• Improve Jupyter support
  • Make sure trains is included in Jupyter requirements
  • Ignore IPython directives in converted Python script (like % and ! lines)
• Update Pytorch/TensorboardX examples

Bug Fixes

• Fix renaming/deleting model file right after saving will break asynchronous upload #123
• Fix shutdown behavior:
  • Logger flush delay and termination
  • Repository/package detection termination
  • Threads termination
• Fix text encoding utf-8 and pr_curve broken in Tensorboard support
• Fix single log request exceeds max packet size
• Fix upload debug assets with series / title including , or / did not add iteration index to the filename
• Fix scikit-learn module detection
• Use UID in message if user name cannot be retrieved (from the Trains Slack channel)
• Fix cast in Tensorflow v2 example

Features and Bug Fixes

• Add OpenMPI/Slurm support, support TRAINS_FORCE_MASTER_NODE environment variable to allow forcefully specifying a node is master (not based on the MPI Rank)
• Add Task.get_models() and Task.models for retrieving stored models for previously executed tasks
• Support multiple EventWriter in TensorFlow eager mode (TF 2.0+) #26
• Support reusing models
• Fix TensorFlow NaN/Inf values support #118
• Fix joblib binding
• Fix metrics/files/models upload on shutdown
• Fix support for Task init/close multiple times
• Update manual_reporting.py example with Logger.report_table() example
• Deprecate Task.set_model_config(), Task.get_model_config_text() and Task.get_model_config_dict()

Features and Bug Fixes

• Add support for reporting tables (pandas, csv etc.) using Logger.report_table()
• Support setting task initial iteration for continuing previous runs using Task.set_initial_iteration() and Task.get_initial_iteration()
• Add multiprocess task support #96
• Add trains-init support for config file override using the TRAINS_CONFIG_FILE environment variable or --file command line argument #105
• Add initial slurm support
• Allow disabling repository detection when calling Task.init()
• Add Jupyter notebook server execution example
• Update examples
• Better warning messages for storage errors and missing storage packages
• Fix HTTP link quoting in stored links

Features and Bug Fixes

• Add binding for tensorboard.summarywriter.addscalars
• Add Logger.tensorboard_single_series_per_graph() configuration option to allow per-series TensorBoard-like graphs
• Add docker image interface using Task.set_base_docker() and Task.get_base_docker()
• Use standard OS environment variables to obtain default credentials for:
  • AWS: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION
  • Azure: AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY
  • Google: GOOGLE_APPLICATION_CREDENTIALS
• Add simple dictionary task parameters interface using Task.get_parameters_as_dict() and Task.set_parameters_as_dict() allowing getting and setting parameters for task instances obtained using Task.get_task()
• Make sure Task.connect() always returns the connected instance passed to it
• Prefer tensorflow_gpu over tensorflow when inspecting installed packages and recording experiment dependencies
• Remove title/series naming restrictions (allow $ and .) when reporting metrics
• Fix incorrect printouts in initialization wizard and upgrade notifications
• Fix debug images URL for uploaded files with % in their name

Features and Bug Fixes

• Allow reporting a pre-uploaded image url in Logger.report_image() using the url parameter
• Add support for git repositories without ".git" suffix (e.g. Azure Repos)
• Improve conda support
• Improve hyper-parameters argparser integration
• Fix savefig() patching in matplotlib binding
• Fix logs, events and jupyter flushing on exit

Features and Bug Fixes

• Add support for pyplot.savefig and pylab.savefig in matplotlib binding
• Add support for SageMaker
• Improved configuration wizard
• Try to make sure tensorboard is available when using torch
• Do not store keras model network design if it cannot be serialized #72
• Fix matplotlib binding support

Features and Bug Fixes

• Add support for trains-server v0.13.0
• Add support for nested (non-main) tasks
• Add warning when automatic argument parser binding cannot be turned off
• Add Task.upload_artifact() support for external URLs (pre-uploaded)
• Add support for special characters in hyper-parameter keys (white-spaces, . and $) #69
• Add support for PyTorch .pt model files
• Calculate data-audit artifact uniqueness by user-criteria #45
• Use an environment variable for setting a default docker image #58
• Improve trains-init configuration wizard
• Update examples for new joblib versions
• Update jupyter example to TensorFlow 2
• Fix task clone to copy only input artifacts
• Fix matplotlib import binding when using Agg backend
• Fix ProxyDictPreWrite and ProxyDictPostWrite so they can be pickled correctly #72
• Fix requests issue in Python 2.7 that can cause a deadlock when importing netrc
• Fix argparser binding sub-parser and type casting support #74
• Fix argparser binding python 2.7 unicode handling
• Fix unsynced connected hyper parameters overridden during remote execution

Features and Bug Fixes

• Add default_output_uri feature #57
• Improve trains-init configuration wizard
• Improve argparser binding
• Fix artifact support in data pipeline #63
• Fix threading issue while querying multiple experiments #64
• Fix uploading large files over slow HTTP connections
• Fix support for Git versions < 2

Breaking Changes

• Do not reuse an experiment with artifacts

Features and Bug Fixes

• Add Task.connect_configuration(), connect your configuration file (or configuration dictionary including nesting support). See example.
• Add Task.connect_label_enumeration(), store model detection classes together with the model & experiment
• Edit & override configuration file (or dictionary) content when executing with trains-agent
• Task.connect() automatically supports nested dictionaries (e.g. param/nested/key) , including overriding values when running with trains-agent. See example.
• Add Artifact.get(), download & load an artifact.
• Add Task.get_tasks(), retrieve previous experiments, get metrics, rename etc.
• Improve Windows support
• Improve Minio support
• Fix Python2 support
• Fix Issue #56

Features and Bug Fixes

• TRAINS Agent support (Full DevOps support)
• Add AutoML and orchestration examples
  Supports TRAINS-server v0.12 or above
• Add Logger x/y/z axis title for: report_surface / report_confusion_matrix / report_scatter3d / report_scatter2d / report_histogram
• Add support for TensorFlow 2.0
• Embed pyhocon into package
• Fix artifacts support on Windows
• Fix example code Windows support

Features and Bug Fixes

• Resource-Monitor will only monitor active GPU devices
  based on environment variables: NVIDIA_VISIBLE_DEVICES or CUDA_VISIBLE_DEVICES
• Fix issue #48

Features and Bug Fixes

• Fix Python 2.7 support
• Improve sample code Windows support

Features and Bug Fixes

• GPU Monitoring is now embedded into trains (removed gpustat dependency)
• Add initial support for Tensorflow v2.0 (tested with v2.0.0rc1)
• Add artifact upload retry on network errors (default: 3)
• Suppress urllib3 retry warnings
• Fix Matplotlib support with Agg backend (multiple plot windows caused repeated graphs to be sent)
• Fix support for tuples in hyper-parameters
• Fix multi processing issues with different task types

Features and Bug Fixes

• Full artifacts support (supported by trains-server >= 0.11.0)
• Artifacts include, Pandas.DataFrame, Numpy, PIL Image, local files, and local folder/wildcard (example)
• Artifacts support for folder/wildcard, selected files will be zipped and uploaded
• Resource monitoring, remove sensor reading failure warnings

Breaking Changes

• Logger info/error/warning/console functions were removed, use Logger.report_text (or python logging or print instead)
• Tensorboard scalars are not grouped into one graph, but are stored on individual graphs (to match Tensorboard behavior). To restore previous behavior call Logger.tensorboard_auto_group_scalars(group_scalars=True)

Features and Bug Fixes

• Artifacts support
• Removed apache-libcloud from requirements
• trains-init now verifies credentials against the trains-server installation

Features and Bug Fixes

• Fix broken (v0.10.5) Keras Binding support

Features and Bug Fixes

• Replace opencv-python with the more standard Pillow package
• Improve matplotlib support (custom axis ticks)
• Improve python package detection

Features and Bug Fixes

• Add scikit-learn support (load/store using joblib) #20
• Add xgboost support #10
• Add loguru support #29
• Add sub-domain support trains.conf #27
• Fix sub-process support
• Fix multiple Tensorboard writers #26

Features and Bug Fixes

• Add Matplotlib SVG support
• Add Seaborn support
• Add TRAINS_LOG_ENVIRONMENT environment logging feature
• Add Microsoft Azure notebook support
• Add Google Colab support
• Fix Tensorboard RGB channel order

Features and Bug Fixes

• Fix Jenkins CI/CD support

• Experiment code execution detection

  • Automatically create package requirements section (including used versions)
  • Automatically detect and store source code uncommitted changes
  • Jupyter notebook support, automatically convert notebook to python script (stored under uncommitted changes)
  • Jupyter notebook support, automatically update used packages in notebook (including used versions)

• Add resource monitoring to experiment metrics

  • Sampled every 500ms, averaged over 30 seconds
  • CPU / Network / IO / Memory etc.
  • For GPU support please install gpustat
    (currently not part of the requirements due to gpustat compatibility issues with Windows)
    $ pip install gpustat

• Automatically stop inactive experiments (default: 2 hours)

• Improved visibility

  • Finer status definitions: Identify successful completion vs. user aborted
  • Experiment plot comparison: Ensure different colors for different experiments
  • Parse newline character in experiment description
  • Show experiment start time in table display
  • Add vertical guide in scalar plots
  • Move Hyper-parameters to designated tab
  • "Admin" section now named "Profile"

Features and Bug Fixes

• Improved Jupyter and inline Matplotlib support
• Allow for insecure SSL connection to trains-server (use with care!)
• Automatically announce new trains version available (at least until we hit version 1.0)
• Fix support for local git branches (i.e. no matching remote branch)
• Verified support for Official TensorFlow 1.14 release and PyTorch with TensorBoard