Ray Reviews

The Microsoft Cognitive Toolkit is an open-source toolkit that allows commercial-grade distributed deep-learning. It describes neural networks using a directed graph, which is a series of computational steps. CNTK makes it easy to combine popular models such as feed-forward DNNs (CNNs), convolutional neural network (CNNs), and recurrent neural network (RNNs/LSTMs) with ease. CNTK implements stochastic grade descent (SGD, error-backpropagation) learning with automatic differentiation/parallelization across multiple GPUs or servers. CNTK can be used in your Python, C# or C++ programs or as a standalone machine learning tool via its own model description language (BrainScript). You can also use the CNTK model assessment functionality in your Java programs. CNTK is compatible with 64-bit Linux and 64-bit Windows operating system. You have two options to install CNTK: you can choose pre-compiled binary packages or you can compile the toolkit using the source available in GitHub.

Uber developed Horovod to make distributed deep-learning fast and easy to implement, reducing model training time from days and even weeks to minutes and hours. Horovod allows you to scale up an existing script so that it runs on hundreds of GPUs with just a few lines Python code. Horovod is available on-premises or as a cloud platform, including AWS Azure and Databricks. Horovod is also able to run on Apache Spark, allowing data processing and model-training to be combined into a single pipeline. Horovod can be configured to use the same infrastructure to train models using any framework. This makes it easy to switch from TensorFlow to PyTorch to MXNet and future frameworks, as machine learning tech stacks evolve.