Artificial Intelligence is a broader umbrella under which Machine Learning (ML) and Deep Learning (DL) comes. Diagram shows, ML is subset of AI and DL is subset of ML.

Cousins of AI

Artificial Intelligence

“The study of the modelling of human mental functions by computer programs.” — Collins Dictionary

AI is composed of 2 words Artificial and intelligence. Anything which is not natural and created by humans is artificial. Intelligence means ability to understand, reason, plan etc. So we can say that any code, tech or algorithm that enable machine to mimic, develop or demonstrate the human cognition or behavior is AI.

The concept of AI is very old but it got popularity recently. But why?

The reason being earlier we had a very small amount of data to make accurate predictions. But today, there is tremendous increase in the size of the data which is generated every minute and help us to make more accurate predictions. Along with the enormous amount of data, we also have the support of more advanced algorithms, high end computing power and storage that can deal with that huge data size. Examples include Tesla self-driving car, Apple’s Siri and many more.

Machine Learning

We have seen what AI is but what were the issues which lead to the introduction of machine learning?

Few reasons were:

In field of Statistics the problem was “How to efficiently train large complex models?” and in Computer Science & AI, the problem was “How to train more robust versions of AI systems?

So because these issues machine learning was introduced.

What is machine learning?

“Machine learning is the science of getting computers to act without being explicitly programmed.” — Stanford University

It’s a subset of AI which uses statistical methods to enable machines to improve with experience. It enables a computer to act and take data driven decisions to carry out a certain task. These programs or algorithms are designed in such a way that they can learn and improve over time when exposed to new data.

Example:

Suppose we want to create a system that tells us the expected weight of person based on its height. Firstly, we will collect the data. This is how data looks like (picture below). Each point on graph represents a data point.

To start with, we will draw a simple line to predict weight based on height.

A simple line could be W=H-100

Where

W=Weight in kgs

H=Height in cms

This line can help us to make prediction. Our main goal is to reduce distance between estimated value and actual value. i.e the error. In order to achieve this, will draw a straight line which fits through all the points.

Our main goal is to minimize the error and make them as small as possible. Decreasing the error between actual and estimated value improves the performance of model and also the more data points we collect the better our model will become.

So when we feed new data that is height of a person then it could easily tell us the weight of the person.

Deep Learning

“Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks”. — Machine Learning Mastery

It’s a particular kind of machine learning that is inspired by the functionality of our brain cells called neurons which lead to the concept of artificial neural network(ANN). ANN is modeled using layers of artificial neurons or computational units to receive input and apply an activation function along with threshold.

In simple model the first layer is input layer, followed by a hidden layer, and lastly by an output layer. Each layer contains one or more neurons.

Layers in Artificial neural Network

Simple Example to understand how things happen at conceptual level

How you recognize square from other shapes?

First thing we do is check whether the figure has four lines. If yes, we further check if all are lines are connected and closed. If yes we finally check if all are perpendicular and all sides are equal.

We consider the figure as square if it satisfies all the conditions.

As we saw in the example it’s nothing but nested hierarchy of concepts. So we took a complex task of identifying a square and broken down into simpler tasks. Deep learning also does the same thing but at a larger scale.

For instance, A machine performs a task of identifying an animal. Task of the machine is to identify weather given image is of cat or dog.

Differentiating between dog & cat using Deep Learning algorithm

If we would have asked us to solve this using concept machine learning then we would have defined features such as check if it has whiskers or not, check if tail is straight or curve and many other features. We will define all features and let our system identify which features are more important in classifying a particular animal. Now when it comes to deep learning it takes it to one step ahead. Deep learning automatically finds which features are most important for classifying as compared to machine learning where we had to manually give out the features.

So till now we understood AI is bigger picture and machine learning & deep learning are its subparts of it.

Machine learning (ML) vs Deep learning (DL)

Easiest way to understand difference between machine learning and deep learning is “DL IS ML”. More specifically it’s the next evolution of machine learning.

Let’s take few important parameters and compare machine learning with deep learning.

1. Data Dependency

The most important difference between the two is the performance as the data size increases. From the graph below we can see that as the size of the data is small deep learning doesn't performs that well but why?

This is because deep learning algorithm requires large amount of data to understand it perfectly. On the other hand machine learning works perfectly on smaller datasets.

2. Hardware Dependency

Deep learning algorithms are highly dependent on high end machines while machine learning algorithms can work on low end machines as well. This is because requirement of deep learning algorithms include GPU’s which is an integral part of its working. GPU’s are required as they perform large amount of matrix multiplication operations and these operations are only be efficiently optimized if they use GPU’s.

3. Feature engineering

It’s the process of putting domain knowledge to reduce the complexity of data and make patterns more visible to learning algorithms. This process it’s difficult and expensive in terms of time and expertise. In case of machine learning, most of the features are to need be identified by an expert and then hand coded as per the domain and data type. The performance of machine learning depends upon how accurately features are identified and extracted. But in deep learning it tries to learn high level features from the data and because of this it makes ahead of machine learning.

4. Problem Solving Approach

When we solve problem using machine learning, its recommended that break down the problem into sub parts first, solve them individually and then combine them to get the final result. On the other hand in deep learning it solves the problem end to end.

For instance,

The task is multiple object detection i.e what is the object and where it is present in the image.

So let’s see how this problem is tackled using machine learning and deep learning.

In a machine learning approach, we will divide problem in 2 parts. object detection and object recognition.

We will use an algorithm like bounding box detection as an example to scan through image and detect all objects then use object recognition algorithm to recognize relevant objects. When we combine results of both the algorithms we will get the final result that what is the object and where it is present in the image.

In deep learning it perform the process from end to end. We will pass an image to an algorithm and our algorithm will give out the location along with the name of the object.

5. Execution Time

Deep learning algorithms take a lot of time to train. This is because there are so many parameters in a deep learning algorithm that takes the training longer than usual. Whereas in machine learning the training time is relatively less as compared to deep learning.

Now the execution time is completely reverse when it comes to the testing of data. During testing deep learning algorithms takes very less time to run whereas the machine learning algorithms like KNN test time increases as the size of the data increases.

6. Interpretability

This is the main reason why people think a lot before using it in the industry. Suppose we use deep learning to give automated essay scoring. The Performance it gives is excellent and same as human beings but there are some issues that it doesn’t tell us why it has given that score, indeed mathematically it’s possible to find out which nodes of deep neural network were activated at that time but we don’t know what the neurons were supposed to model and what these layers were doing collectively. So we fail to interpret the result but in machine learning algorithms like decision tree gives us a crisp rule that why it chose what it chose so it is easy to interpret reasoning behind it.

I hope now you have clear idea about all three, what relationship they share and how they are different from each other.

Thanks for reading!