A basic task for any neural network is classification: separating inputs into two groups, called classes.
For instance, you might want to separate emails into spam or non-spam, patients into those with or without a disease, or images into those of a car or not. A label refers to a data point contained inside a specific class.
As a human, you can recognize and categorize some of these things. In the case of determining spam mail, you can classify a message into spam or not spam just by reading it. If the computer had to figure this out for itself, it wouldn't know what's important and what's not.
The goal is to create a model: a system that can predict which class a new data point is without knowing its label. You do that by breaking our input data into features, the basic information underlying each data point. Features are chosen to help you figure out the label as easily as possible. There are lots of features to choose, and the challenge is to use the ones that will help the most.
For each data point, you'll need to know its features and its label.
- The number of words in an email. Spam is often overly wordy. Spam emails sometimes go on very strange tangents; a lot of spam emails will consistently use run-on sentences that kind of don't get to the point very quickly because they're meant to just fill space instead of having actual information and also they don't use a lot of punctuation if any at all which makes it really difficult to read and will drain your brain pretty hard also it blends together the words into just one big soup of communication that drones on and on an...
- How many WORDS are in ALL-CAPS? Spam can be OVERLY EXCITED about things and over use CAPITAL LETTERS.
- Punctuation, maybe???? Spam sometimes uses too many exclamation marks!!!!
By measuring loss with a specific number, you can compare two models, and see which one is a better problem solver by finding the smaller loss. The Neural Network Playground lets you explore simple neural networks and see how they work.
link
8-layer circle
link
Important: If these don't work first try, play it again. It's completely random. In our Neural Network Playground, you'll deal with the very realistic classes of Blue Points and Orange Points.
The background color is the model's prediction: what color our neural network thinks points in that area are.
How often the prediction disagrees with the actual color of the points is the loss of the model.
You can see the loss of your model right above the image, as a graph of loss over time, and as your current loss. Take a look at what happens when you try using just one neuron to solve the problem.
1. Click here to open a playground with just one neuron.
2. Click the Play button to see how one neuron attempts to solve the problem.
3. Wait for the loss to stop changing and write it down. You'll compare it with later models as you go.
A single neuron can only classify points linearly: splitting the data along a single straight line, just like your neuron did. However, this data isn't linear! What to do, what to do...
4. Click here to open a playground with two neurons, or click the + above the neuron to add a second.
5. Click the play button to run the model.
6. Write down the loss once it stops changing to determine which did better.
In this pair, each neuron splits the data linearly, and then the results are combined. You can mouse over each individual neuron to see how it's splitting the data.
7. Try adding a third and fourth neuron with the plus sign above the neurons.
8. Re-run the model each time, and write down your loss. More neurons seem to do better, so why not just add lots and lots and lots of neurons?
Well, your playground is capped at eight neurons. Go ahead and try the eight neuron model, and record your loss.
Try re-running the one and eight neuron models and note how long it takes each for the loss to stop changing.
Because this is a playground and a pretty simple example, both models finish fast, but the eight neuron model is slower.
While more neurons can reduce a model's loss, as more neurons are added, it takes longer and longer to process all of the information. Each neuron adds information to the model. To get a final model, the neural network needs to combine this information into a single result.
The network uses weights to determine how much influence each neuron should have. A large weight means that neuron is important and adds a lot of information. A small weight means that neuron doesn't add as much information but should still be considered. A negative weight means that the neuron should be inverted (blue becomes orange, orange becomes blue) before being considered.
Weights are one of the main things the model learns as it runs, changing weights on individual neurons a bit at a time to lower the loss. The result of each neuron is examined to see how close it was to the correct answer. If it was correct, the weight is raised; if it is was wrong, the weight is lowered.
In your playground, the lines running from inputs to neurons and neurons to the result are weights: blue is positive, orange is negative, and thickness is how large the weight is.
You can click on individual lines to change their weights.
Try re-running a few models and look at how the weights change as the model is generated. How does the resulting model change if you modify the weights? Neural networks are not limited to having neurons in one straight line. You can create multiple layers of neurons, with each layer feeding information to the next, and the final layer combing the results to produce the overall model.
Layering neurons can create models that can understand more complex information with less total neurons. Each layer helps examine the underlying data, making the task of the next layer easier.
By layering neurons, later layers can examine the results of the work done by earlier layers, rather then all of the neurons examining the underlying data. That way, the model can build up information, processing the data one step at a time.
A multi-layer model is an effective choice for this dataset. When you're ready, try out some of the other datasets on the left side of the page.
The goal is to get a low loss with as few neurons as possible. Keep track of your results as you go. Neural networks are a powerful machine learning method for classifying data.
By combining the simple classifications of single neurons into a larger model using weights, even very complex datasets can be classified efficiently.
Real data scientists make notes and compare models by examining the loss of each model. Because machine learning always has some randomness, models are run multiple times, and the average results are compared.
- Classification is the task of separating data into two or more classes
- Loss measures how well a model accomplished this task
- A single neuron can only split data along one line
- Multiple neurons can be combined by weighing their results
- By combining neurons in layers, we can improve a model without adding too many neurons