How does data help AI?
Previously we talked a little bit about how AI's need a lot of data to learn patterns from. In this video we'll talk about how this data looks. Data can be in many forms, it can be numbers, images, videos, audio, etc. We will focus on data that can be stored in tables. You might have seen tables and softwares like Excel or Google Sheets. It essentially consists of some rows and some columns. What you see on this slide is that rows are numbered 1,2, 3 and so on, while the columns are labeled A, B, C etc.
This is the basic structure of tables as you'll see in Google Sheets. It consists of several rows and columns. Let's take an example dataset. In this dataset, note that the columns are, number of countries visited, number of years in school, height and, who am I. You see how the two words feature and label appear right about in yellow color. These are really important 2 words. You'll be using them quite a lot when you build an AI.
First thing to note is that there is a single column called label and rest everything is features. Label refers to what you would like your AI to predict and features represent information that you will provide your AI for it to make predictions. For example, in this data set, the features are `number of countries visited`, `number of years` in school and `height` and the label is `Who am I`. This means that you will give your AI information about how many years someone went to school, how many countries someone visited and their height. The AI will take this information and it is supposed to predict if this person is an adult or a child. Another word that you come across is sample. Every row here is called a sample. On this slide, for example, there are three samples,, which are provided to the AI to learn from.
A really good question here is, what is a good number of samples? How many samples do you need to provide your AI, before it can learn to do predictions?.
In the real world, there are sometimes millions of samples. It really depends on the kind of prediction that you're trying to make. The number of features which is the number of columns can also be like hundreds, or 1000s in the real world. For a simple example like this, probably a few 100 samples are enough for the AI to learn the pattern. But what happens if you don't provide enough samples?, what if I provide only these three samples?. The AI will have a really hard time to recognize patterns. So it may not do that great a job, if you don't provide enough samples.
The process of building an AI can be divided into 2 stages. The first stage is called training. Training is the process by which AI learns patterns from data. For example, if you give it three features and one label column, it will try to recognize patterns in your features that will help it to predict the label. The output or result of training is something called a model. You can think about the model as the brain. This brain has learnt a specific task to recognize certain patterns in data and is now capable of making some predictions. Here is another example of providing data to an AI. This is a table which contains only two columns. The label column is the feeling, and the feature is the sentence. The AI will get this information as an input, and it is supposed to predict if someone is happy or sad. You can provide a bunch of these examples as data to your algorithm, and the algorithm will recognize patterns in these sentences and create a model.
And once a model is created, it is capable of making predictions on new data that comes in.
Now you can use this model in a second stage called the prediction stage where you can provide it with any English sentence or even a paragraph, and the AI is going to predict if that sentence is happy or sad. One thing to note is that an AI can do a very specific job, and how well it does that job depends on the quality of data that you provide. If you train an AI to detect if someone is happy or sad, then that's the only thing that AI can do.
What will happen if you provided data about how many years someone went to school to an AI that has learnt to predict mood?. The AI has no idea about that kind of data, it will be super confused and do nothing. So AI's are very specific problem solvers, they learn to solve a specific problem.
We have introduced a lot of new terms in this video. To reiterate, the new words that you have heard are training, which is the stage where you help your AI to learn patterns, prediction stage, which is also popularly called inference which is where your AI answers new questions.
Lets talk about one last new term called AI Service. Going back to the example that we saw, where you can enter a sentence and it's going to predict happy or sad, a corresponding AI service would work something like this. You type in a sentence in english, lets say in a box and the AI service will take this as an input and predict if you're happy or sad. You actually interact with a lot of such AI services in your daily life, like Siri, google home, google translate etc. these are all AI services, where you have a model that has already trained on a lot of examples and is available for you to use. In this course, we are going to be creating such AI services, and depending on what kind of data you train your AI model with, the service is going to do a very specific job for you. So whenever we say AI service, think about it as the inference stage or the prediction stage, where you expect the service to accept new data and provide a prediction.