Comparison of Popular AI Frameworks

 

Introduction

"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.” - Tom M. Mitchell

Alan Turing initially proposed thinking machines in his paper "Computing Machinery and Intelligence". AI and ML evolved from research projects to open-source software packages and libraries. The real-life problems are solved using AI/ML Learning techniques.

 

The goal of supervised learning is to create a model that learns the rules and transforms the inputs into the results. Unsupervised learning is about identifying the hidden patterns in the data. The patterns are found using feature learning. Reinforcement learning is related to interaction with the dynamic environment. In this method,  the feedback is expected for a specific action. Reinforcement learning has a goal like winning the strategy game. Semi-supervised learning is where some of the expected results are not provided. In semi-supervised learning, the superset of the problem instances is given.

In machine learning, classification problems are solved by splitting the inputs to multiple classes. The model is created with unseen inputs to multiple classes. Support Vector machine method is used typically as a classifier. Fraud analysis and Loan default prediction are good examples of where SVM is used. Regression analysis is another supervised learning technique where the results are continuous and not discrete. Clustering is a machine learning technique that splits the unknown inputs to different groups. There are other techniques such as density estimation, dimensionality reduction, and topic modeling. Density estimation is related to the input distribution. Dimensionality reduction helps in mapping of inputs to a lower-dimensional space. Topic modeling is used to identify the documents for a set of input topics. Many applications are using the advanced AI/ML techniques like ANN, CNN, and RNN. A neural network is based on a perceptron with more than one layers. The input for the first layer is filtered through a set of layers in the connected set of layers. The result will be the probability set with feature labels.

 

TensorFlow is a google Open Source Software ML package. Apache MLlib is Spark's machine learning library. Torch is another popular framework with wide support for machine learning algorithms. We will look at various AI frameworks and other features. This article will talk about the comparison of the features to create AI models and train them. These open-source frameworks are used for predictive analysis, risk assessment, and conversational analysis.

 

Google TensorFlow

Tensorflow was created in Google by the same team who created Theano. Tensorflow provides python API and the backend is in C/C++. It has the initial version of java API also. It is an open-source deep-learning package. It has a dashboard Tensorboard for presenting the machine learning models. Tensorflow team member (creator) is now with OpenAI. Tensor Flow provides API to create a convolutional neural network for real-life problems such as image analysis, document analysis, and video analysis. The other features are recurrent neural networks, sequence to sequence, large scale linear models, and linear models. Tensorflow performance is not comparable to CNTK and MxNet. It is better than Theano in compilation speed. Tensorflow has features such as computational graph abstraction, data and model parallelism, and pre-trained models. Tensorflow 2.0 and the other versions provide better features and are efficient compared to the first version.

 

Apache Spark MLib

Apache Spark Mlib is Spark's machine learning package. The techniques supported in this package are classification, regression, clustering, and collaborative filtering. It has features such as pipeline tools, pipeline persistence, graph processing, transformation, selection, dimensionality reduction, and feature extraction. Spark was created at the University of California and now with Apache software foundation. Spark's engine can be executed on Hadoop, Mesos, and cloud. Spark's cluster has in-memory compilation feature for machine learning and graph computation.

 

Torch

Torch is an open-source software framework that has features like neural network models, linear algebra routines, indexing/slicing/transposing routines, numeric optimization routines, GPU support,  embeddable to IOS/Android/FPGA  backends, and C/Lua interfaces. Pytorch is the python api providing Torch features. It has features to handle dynamic computation graphs in RNNs. PyTorch is based on matplotlib and seaborn packages. It has features for plotting and creating graphs. Pytorch was created in Facebook and now an open-source package. Torch is based on Lua API. Lua is a scripting language created in Brazil. PyTorch has features like modular components, Layer types, and pre-trained models.

 

Theano

Theano was created in University de Montreal. It has features to train the deep neural network models.  Theano is a python software package used for data analysis and deep learning modeling. Theano is stopped in around 2017. Keras, Lasagne, and Blocks are based on Theano. Theano had a non-intuitive user interface. It had good features like support for RNN, high-level wrappers, error handling, GPU support, and pre-trained models. Theano is based on Numpy, BLAS, and C++ libraries and its performance is the same as on GPUs and CPUs. Theano has features related to symbolic expressions. It is used for deep learning tasks since it has the capability for symbolical differentiation. Using Then model specifications can be designed instead of model executions. It has GPU support to provide better performance.

 

Caffe

Caffe was created in Berkeley and it is a C++ software framework. Caffe is an acronym for convolutional architecture for fast feature embedding. Caffe provides Python API and can handle  CNNs and Feedforward neural networks. Caffe2 is the second version of Caffe and has features for deep learning models. It is scalable and lightweight.  Caffe is used for computer vision-related tasks. It has a lower level API. Caffe 2 has new features compared to version 1.0. The new features are large scale distributed training support, mobile deployment, hardware support to CUDA and CPU, quantized computation flexibility, and better quality due to stress testing. It has RNN components for machine translation and speech recognition problems. Caffe 2.0 is Pytorch as of today.

 

Keras

Keras is a Python deep learning framework and is based on Tensorflow. Keras in Greek stands for horn. Gate of Horn is in greek literature Odyssey. Dream spirits land on Earth from the gate of ivory. Keras was created as a project Open-ended Neuro Electronic Intelligent Robot Operating system. It has good debugging features and modular neural network blocks. Keras has both GPU and CPU support. Keras was developed in Google and it is very popular. It has Torch features and can support Theano, Tensorflow, and Deeplearning4j engines. Keras can be the defacto standard for neural network frameworks in the future. This framework is used for computer vision and text recognition algorithms. Keras can be deployable on the server with Node.js/TFX/TF, Android/IOS, Raspberry Pi, Edge TPU, and in the browser with TF.js.

 

Scikit-Learn

Scikit-Learn is an open-source package that has features related to classification, clustering, and regression modeling. It has support for preprocessing, dimensionality reduction, and evaluation. This library includes Numpy, SciPy, Matplotlib, IPython, Sympy, and Pandas packages. Numpy is a package for Base n-dimensional arrays. Scipy is a package for scientific analysis. Matplotlib is a plotting package. Ipython is an interactive console. Sympy package supports symbolic mathematics. Pandas package is used for data structures and analysis. Scikit learn has to support clustering, cross-validation, datasets, dimensionality reduction, ensemble methods, feature extraction, feature section, parameter tuning, and manifold learning.

 

Comparison

Tensorflow is the popular one in the listed framework above. It has automatic differentiation to handle back-propagation. Keras is a deep learning API based on Tensorflow. Scikit learn is based on Numpy for data processing. Spark is an open-source package for data analytics.

 

Selection of the machine learning package needs to be done after weighing different options. There is a learning curve in gaining skills in these machine learning frameworks. It is better to start small and experiment with complex features later. It is suggested to pick the framework which fits the project requirements. Many of these frameworks change based on other frameworks. In reality, the popularity of the frameworks is based on the latest version available as the release. Similarly, Deep learning frameworks are chosen based on metrics related to parallel computation, performance, visualization, and inbuilt packages. The factors for selection are neural network type, programming language, budget, functionality, and tools. In the next set of articles, we will look into available  ML services on cloud (Google, Microsoft Azure, Amazon AWS, IBM BlueMix) and their features

 

Package	API Language	Pretrained models support	RNN Support	CNN Support
Tensorflow	Python, Java	Yes	Yes	Yes
Spark MLib	Java, Python	No	No	No
Torch	Lua, Python	Yes	Yes	Yes
Theano	Python	Yes	No	No
Caffe	C++	Yes	Yes	Yes
Keras	Python	Yes	Yes	Yes
Sci-kit Learn	Python	No	No	No