{"id":1749,"date":"2021-12-29T12:01:18","date_gmt":"2021-12-29T06:31:18","guid":{"rendered":"https:\/\/blog.nuventure.in\/?p=1749"},"modified":"2026-10-28T12:30:16","modified_gmt":"2026-10-28T07:00:16","slug":"neural-networks-explained-how-they-work","status":"publish","type":"post","link":"https:\/\/nuventure.com\/blog\/2021\/12\/29\/neural-networks-explained-how-they-work\/","title":{"rendered":"How Neural Networks Work: Artificial Neural Networks Explained"},"content":{"rendered":"\n

In machine learning, an algorithm is only as good as the data it was trained on. Learning from experience is the whole point. Without machine learning, we would have to manually program a huge number of “if” and “if-else” conditions just to help a computer tell a car from a human being.<\/p>\n\n\n\n

Neural networks are one of the most important tools machine learning offers, and gradient descent is the method that trains them. In this guide, you will learn what an artificial neural network is, how neural networks work, how loss and cost functions measure mistakes, and how gradient descent fixes them.<\/p>\n\n\n\n

What Is an Artificial Neural Network (ANN)?<\/strong><\/h2>\n\n\n\n

An artificial neural network (ANN) is a machine learning model made of connected artificial neurons, loosely inspired by the human brain. Each neuron takes inputs, multiplies them by weights, adds a bias and passes the result through an activation function. By adjusting those weights during training, the network learns to recognize patterns.<\/p>\n\n\n\n

Neural networks are the basis for everything from digital assistants to self-driving cars<\/a>, edge AI<\/a> and just about everything in AI<\/a>.<\/p>\n\n\n\n

To understand how an ANN works, it helps to look first at the brain it imitates.<\/p>\n\n\n\n

How Neurons in the Brain Work<\/strong><\/h2>\n\n\n\n

The human brain is made of billions of neurons. The connections between them determine how we understand things and make decisions. The brain is incredibly plastic: the number of neurons stays more or less constant through life, but the connections keep changing. New ones form, and old ones weaken or grow stronger.<\/p>\n\n\n\n

For our purposes, picture a neuron as a line segment. It receives signals at one end and sends signals out of the other. A single neuron can receive signals from, and send signals to, many other neurons.<\/p>\n\n\n\n

Not all signals are equal. If several neurons send signals at the same time, the combined signal is stronger. And a neuron does not fire for every signal. It has a minimum signal strength, called the action potential, that it needs before it fires.<\/p>\n\n\n\n

One of the thumb rules for how connections form is: “neurons that fire together, wire together.” Pavlov’s dog shows this well.<\/p>\n\n\n\n

Pavlov’s Dog: How Connections Form<\/strong><\/h2>\n\n\n\n

Ivan Pavlov measured how much saliva his dog produced when offered food<\/a>. Then, before each feeding, he rang a bell. After a while, he rang the bell alone, and the dog still produced saliva. The dog’s brain had learned to associate the bell with food.<\/p>\n\n\n\n

Picture one neuron that identifies food, one that produces saliva and one that identifies the bell. (In reality, each would be a complex neural network.) The food neuron and the saliva neuron were already connected, so one firing triggered the other. During the experiment, the bell neuron fired at the same time as the food neuron, and it developed a connection to it. That is how training works.<\/p>\n\n\n\n

How Neural Networks Solve a Problem<\/strong><\/h2>\n\n\n\n

The brain solves problems in a similar way. Suppose it has to decide whether a picture shows a dog. Many factors hint at it. Four legs suggest a dog, but so do other animals have four legs. Dogs usually have fur, but not always. Dogs have two eyes, but so do most animals. Each factor is only a correlation, and together they lead to a conclusion.<\/p>\n\n\n\n

Each factor gets a neuron, and all of them connect to a final neuron that says “yes, this is a dog.” The strength of each connection depends on how much that factor matters. A thick coat points to a dog more strongly than four legs does, so its connection is stronger. When enough factors fire together, the signal passes the final neuron’s action potential, and the decision is made.<\/p>\n\n\n\n

How Do Artificial Neural Networks Work?<\/strong><\/h2>\n\n\n\n

In an artificial neural network, the strength of a connection between neurons is represented by a weight<\/strong>. Artificial neurons, like real ones, receive inputs from many neurons and send their output to many others. Each input is multiplied by its weight, so important factors count for more.<\/p>\n\n\n\n

Take the dog example. Two input neurons, “has four legs” and “has a thick coat,” feed the neuron that lights up for “dog.” Both give an input of 1. The thick coat input is multiplied by 4 and the four-legs input by 2, because a thick coat is a stronger sign of a dog. The total input is 6.<\/p>\n\n\n\n

Threshold, or Bias<\/strong><\/h2>\n\n\n\n

An artificial neuron does not fire every time it gets a signal either. If the total signal is 6 and the neuron’s threshold is 8, it does not fire, so it doesn’t output “dog.”<\/p>\n\n\n\n

In ANNs, the threshold is called the bias<\/strong>. Technically, the bias is the negative of the threshold. If the threshold is 4, the bias is \u22124.<\/p>\n\n\n\n

So three things decide whether a neuron fires: the inputs it receives, the weights attached to them and the bias.<\/p>\n\n\n\n

The Activation Function of a Neural Network<\/strong><\/h2>\n\n\n\n

The activation function is how a neuron’s output is calculated. In its simplest form, you multiply each input by its weight, add them up and subtract the bias:<\/p>\n\n\n\n

Y = x1w1 + x2w2 + x3w3 + \u2026 \u2212 B<\/p>\n\n\n\n

Here x1, x2, x3 are the inputs, w1, w2, w3 are the weights and B is the bias. The result can range from negative infinity to positive infinity, so it is often squeezed into a fixed range. The simplest version outputs 1 if Y > 0 and 0 otherwise.<\/p>\n\n\n\n

Real networks use other activation functions, such as sigmoid, ReLU and softmax, depending on the problem. Some aspects of this don’t map exactly to biological neurons, but the idea is the same.<\/p>\n\n\n\n

How Neural Networks Are Trained<\/strong><\/h2>\n\n\n\n

A neural network is only as good as its training and the data used to train it. The brain learns from experience by strengthening and weakening connections. An artificial network does the same with data.<\/p>\n\n\n\n

To build a network that detects dogs, you need many pictures, some with dogs and some without. These form the training data set. You feed in the pictures along with the expected output, for example 1 for “dog detected” and 0 for “no dog.” An image goes in as pixel data, so in effect, numbers go in and a number comes out.<\/p>\n\n\n\n

Training means finding the combination of weights and biases for every neuron that gives the right outputs. Since you know the inputs and the expected outputs, it resembles solving an equation, except with far too many variables to solve directly. Trying every combination would take too much time and computing power. So we need a way to measure how wrong the network is, and a systematic way to reduce that error.<\/p>\n\n\n\n

Loss Function vs Cost Function<\/strong><\/h2>\n\n\n\n

To improve a network, we first need to know how far its output is from the expected output. That is the job of the loss and cost functions.<\/p>\n\n\n\n