Ann practical
Ann practical by shantanu
https://github.com/gaikwadshantanu12/adypsoe_aids
By adwait
https://github.com/Adwait197/ANN-Practicals
Practical Code
https://annpractical9.blogspot.com
ANN Practical Text
EX.1
EXPERIMENT NO. 1 (Group A)
Aim: Write a Python program to plot a few activation functions that are being usedin
neural networks.
Outcome: At end of this experiment, student will be able to write a python programto plot
a few activation functions that are being used in neural networks.
Hardware Requirement:
Software Requirement: Ubuntu OS,Python Editor(Python Interpreter)
Theory:
In the process of building a neural network, one of the choices you get to make is what
Activation Function to use in the hidden layer as well as at the output layer of the network.
Elements of a Neural Network
Input Layer: This layer accepts input features. It provides information from the outside
world to the network, no computation is performed at this layer, nodes here just pass on
the information(features) to the hidden layer.
Hidden Layer: Nodes of this layer are not exposed to the outer world, they are part of the
abstraction provided by any neural network. The hidden layer performs all sorts of
computation on the features entered through the input layer and transfers the result to the
output layer.
Output Layer: This layer bring up the information learned by the network to the outer
world.
What is an activation function and why use them?
The activation function decides whether a neuron should be activated or not by calculating
the weighted sum and further adding bias to it. The purpose of the activationfunction is to
introduce non-linearity into the output of a neuron.
Activation functions make the back- propagation possible since the gradients are supplied
along with the error to update the weights and biases.
Why do we need Non-linear activation function?
A neural network without an activation function is essentially just a linear regression
model. The activation function does the non-linear transformation to the input making it
capable to learn and perform more complex tasks.
Mathematical proof
Suppose we have a Neural net like this :-
10
Elements of the diagram are as follows:
Hidden layer i.e. layer 1:
z(1) = W(1)X + b(1) a(1)
Here,
● z(1) is the vectorized output of layer 1
● W(1) be the vectorized weights assigned to neurons of hidden layer i.e. w1, w2,
w3 and w4
● X be the vectorized input features i.e. i1 and i2
● b is the vectorized bias assigned to neurons in hidden layer i.e. b1 and b2
● a(1) is the vectorized form of any linear function.
(Note: We are not considering activation function here)
Layer 2 i.e. output layer :-
Note : Input for layer 2 is output from layer 1
z(2) = W(2)a(1) + b(2)
a(2) = z(2)
Calculation at Output layer
z(2) = (W(2) * [W(1)X + b(1)]) + b(2)
z(2) = [W(2) * W(1)] * X + [W(2)*b(1) + b(2)]
Let,
[W(2) * W(1)] = W
[W(2)*b(1) + b(2)] = b
Final output : z(2) = W*X + b which is again a linear function
This observation results again in a linear function even after applying a hidden layer,
hence we can conclude that, doesn’t matter how many hidden layer we attach in neural
net, all layers will behave same way because the composition of two linear function is a
linear function itself. Neuron can not learn with just a linear function attached to it. A
non-linear activation function will let it learn as per the difference w.r.t error. Hence we
need an activation function.
Variants of Activation Function
Linear Function
● Equation : Linear function has the equation similar to as of a straight line i.e. y = x
● No matter how many layers we have, if all are linear in nature, the final activation
function of last layer is nothing but just a linear function of the input of first layer.
● Range : -inf to +inf
● Uses : Linear activation function is used at just one place i.e. output layer.
For example : Calculation of price of a house is a regression problem. House price may
have any big/small value, so we can apply linear activation at output layer.
Sigmoid Function:
● It is a function which is plotted as ‘S’ shaped graph.
● Equation : A = 1/(1 + e-x)
● Nature : Non-linear. Notice that X values lies between -2 to 2, Y values are very steep.
This means, small changes in x would also bring about large changes in the value of Y.
11
● Value Range : 0 to 1
● Uses : Usually used in output layer of a binary classification, where result is either
0 or 1, as value for sigmoid function lies between 0 and 1 only so, result can be
predicted easily to be 1 if value is greater than 0.5 and 0 otherwise.
Tanh Function
● The activation that works almost always better than sigmoid function is Tanh
function also known as Tangent Hyperbolic function. It’s actually mathematically
shifted version of the sigmoid function. Both are similar and can be derived from
each other.
● Equation :-
● Value Range :- -1 to +1
● Nature :- non-linear
● Uses: - Usually used in hidden layers of a neural network as its values lies
between -1 to 1 hence the mean for the hidden layer comes out be 0 or very close
to it, hence helps in centering the data by bringing mean close to 0. This makes
learning for the next layer much easier.
RELU Function:
● It Stands for Rectified linear unit. It is the most widely used activation function.
Chiefly implemented in hidden layers of Neural network.
● Equation :- A(x) = max(0,x). It gives an output x if x is positive and 0 otherwise.
● Value Range :- [0, inf)
● Nature :- non-linear, which means we can easily backpropagate the errors and
have multiple layers of neurons being activated by the ReLU function.
● Uses :- ReLu is less computationally expensive than tanh and sigmoid because it
involves simpler mathematical operations. At a time only a few neurons are
activated making the network sparse making it efficient and easy for
computation.
In simple words, RELU learns much faster than sigmoid and Tanh function.
12
Softmax Function:
`
The softmax function is also a type of sigmoid function but is handy when we are tryingto
handle multi- class classification problems.
● Nature :- non-linear
● Uses :- Usually used when trying to handle multiple classes. the softmax function
was commonly found in the output layer of image classification problems.The
softmax function would squeeze the outputs for each class between 0 and 1 and
would also divide by the sum of the outputs.
● Output:- The softmax function is ideally used in the output layer of the classifier
where we are actually trying to attain the probabilities to define the class of each
input.
● The basic rule of thumb is if you really don’t know what activation function to
use, then simply use RELU as it is a general activation function in hidden layers
and is used in most cases these days.
● If your output is for binary classification then, sigmoid function is very natural
choice for output layer.
● If your output is for multi-class classification then, Softmax is very useful to
predict the probabilities of each classes.
Conclusion: -
Questions:
Q1. What is the role of the Activation functions in Neural Networks?
Q2. List down the names of some popular Activation Functions used in Neural Networks?
Q3. How to initialize Weights and Biases in Neural Networks?
Q4. How are Neural Networks modelled?
Q5. What is an Activation Function?
Practical 2.
EXPERIMENT NO. 2 (Group A)
Aim: Generate ANDNOT function using McCulloch-Pitts neural net by a python
program.
Outcome: At end of this experiment, student will be able to Generate ANDNOTfunction
using McCulloch-Pitts neural net by a python program.
Hardware Requirement:
Software Requirement: Ubuntu OS,Python Editor(Pyhton Interpreter)
Theory:
The early model of an artificial neuron is introduced by Warren McCulloch and Walter
Pitts in 1943. The McCulloch-Pitts neural model is also known as linear threshold gate.
It is a neuron of a set of inputs and one output . The linear threshold gate simply classifies the set
of inputs into two different classes. Thus the output is binary. Such afunction can be described
mathematically using these equations: W1,w2,w3 are weight values normalized in the range of
either or and associated with each input line, is the weighted sum, and is a threshold constant.
The function is a linear step function at threshold as shown in figure below. The symbolic
representation of the linear threshold gate is shown in figure.
The McCulloch-Pitts model of a neuron is simple yet has substantial computing
potential. It also has a precise mathematical definition. However, this model is so
simplistic that it only generates a binary output and also the weight and threshold
values are fixed. The neural computing algorithm has diverse features for various
applications. Thus, we need to obtain the neural model with more flexible
computational features.
MLP Classifier trains iteratively since at each time step the partial derivative so the loss
functions with respect to the model parameters are computed to update the parameters.
It canalso have a regularization term added to the loss function that shrinks model
parameters to prevent over fitting.
This implementation works with data represented as dense numpy arrays or sparse
scipy arrays of floating point values. Remove error values in the data set and train
the modelaccordingly via various operations. To perform classification on data set and
predict the outcomes, alsoto plot the outcomes of the experiment to get better
understanding of the outcomes.
14
Truth Table of AND NOT Function
x1 x2 Y
1 1 0
1 0 1
0 1 0
0 0 0
After that, we have to assume two weights w1 = w2 = 1 and w1 = 1, w2 = -1 for the inputs.
A NN with two input neurons and a single output neuron can operate as an ANDNOT logic
function if we choose weights
W1 = 1, W2 = -1 and threshold Ɵ = 1.
Yin is a activation value
X1=1, X2=1,
Yin= W1*X1+W2*X2=1*1+(-1)*1=0, Yin< Ɵ , so Y=0
X1=1, X2=0
Yin=1*1+0*(-1)=1, Yin= Ɵ , so Y=1
X1=0, X=1
Yin=0-1=-1, Yin< Ɵ , so Y=0
X1=0, X2=0
Yin=0, Yin< Ɵ , so Y=0
So , Y=[0 1 0 0]
Conclusion: -
Questions:
Q1. Explain MCP Model?
Q2. How to generate AND NOT function using MCP Model ?
Q3. Discuss briefly McCulloch Pitt’s artificial neuron model.
Q4. Give McCulloch Pitt’s artificial neuron model limitations?
Practical 3.
EXPERIMENT NO. 3 (Group A)
Aim: Write a Python Program using Perceptron Neural Network to recognize evenand
odd numbers. Given numbers are in ASCII from 0 to 9
Outcome: At end of this experiment, student will be able to Write a Python Program
using Perceptron Neural Network to recognize even and odd numbers.
Hardware Requirement:
Software Requirement: Open Source Python, Programming tool like Jupyter
Notebook, Pycharm, Spyder, Tensorflow.
Theory:
First introduced by Rosenblatt in 1958, The Perceptron: A Probabilistic Model for Information
Storage and Organization in the Brain is arguably the oldest and most simple of the ANN
algorithms. Following this publication, Perceptron-based techniques were all the rage in
the neural network community. This paper alone is hugely responsible for the popularity and
utility of neural networks today.
But then, in 1969, an “AI Winter” descended on the machine learning community that almost
froze out neural networks for good. Minsky and Papert published Perceptrons: an introduction
to computational geometry, a book that effectively stagnated research in neural networks for
almost a decade — there is much controversy regarding the book (Olazaran, 1996), but the
authors did successfully demonstrate that a single layer Perceptron is unable to separate
nonlinear data points.
Given that most real-world datasets are naturally nonlinearly separable, this it seemed that the
Perceptron, along with the rest of neural network research, might reach an untimely end.
Between the Minsky and Papert publication and the broken promises of neural networks
revolutionizing industry, the interest in neural networks dwindled substantially. It wasn’t until
we started exploring deeper networks (sometimes called multi-layer perceptrons) along with
the backpropagation algorithm (Werbos and Rumelhart et al.) that the “AI Winter” in the 1970s
ended and neural network research started to heat up again.
Perceptron Architecture
Rosenblatt (1958) defined a Perceptron as a system that learns using labeled examples (i.e.,
supervised learning) of feature vectors (or raw pixel intensities), mapping these inputs to their
corresponding output class labels.
In its simplest form, a Perceptron contains N input nodes, one for each entry in the input row of
the design matrix, followed by only one layer in the network with just a single node in that layer
(Figure 2).
16
There exist connections and their corresponding weights w1, w2, …, wi from the input xi’s to
the single output node in the network. This node takes the weighted sum of inputs and applies a
step function to determine the output class label. The Perceptron outputs either a 0 or a 1 — 0
for class #1 and 1 for class #2; thus, in its original form, the Perceptron is simply a binary, two-
class classifier.
Figure 2: Architecture of the Perceptron network.
NN with two input neurons (one being the variable Number, the other being a bias neuron), nine
neurons in the hidden layer and one output neuron using a very simple genetic algorithm: at
each epoch, two sets of weights "fight" against each other; the one with the highest error loses
and it's replaced by a modified version of the winner.
The script easily solve simple problems like the AND, the OR and the XOR operators but get
stuck while trying to categorise odd and even numbers. Right now the best it managed to do is to
identify 53 numbers out of 100 and it took several hours.
Conclusion: -
Questions:
Q1. Is it possible to train a NN to distinguish between odd and even numbers only
using as input the numbers themselves?
Q2. Can Perceptron Generalize Non-linear Problems?
Q3. How to create a Multilayer Perceptron NN?
Q4. Is the multilayer perceptron (MLP) a deep learning method? Explain it?
Comments
Post a Comment