## Introduction to Pytorch with Neural NetworksPosted by Indrajeet on December 19th, 2019 PyTorch is an open-source machine learning library for Python developed by Facebook AI Engineers. Facebook primarily built this for Natural Language Processing. PyTorch is developed with the concept of in-built probabilistic programming. - The developer must be familiar to Object detection, bounding box regression and non-maximum suppression
- User must able to create a simple neural network with ease
- User must know how neural network works
Types of Deep Neural Network - Convolutional Neural Networks
- Recurrent Neural Networks
1. Convolutional Neural Networks Multiple layers of data can be easily processed by CNN. Image recognition and face recognition are some applications of Convolutional Neural Networks.CNN can directly operate on images with the input of two-dimensional array rather than focusing on the extraction of what other neural networks focus on. Problems of recognition can be solved using CNN. Facebook and Google have also invested in research and development for recognition projects to get activities with high speed and accuracy. The convolutional neural network includes three basic ideas − - Local respective fields
- Convolution
- Pooling
Create a Convolutional Neural Network using Pytorch
Import all necessary package from torch.autograd import Variable import torch.nn.functional as F
Convolutional Neural Network for batch representation. Batch Shape for input would be of dimension(3,32,32) class SimpleCNN(torch.nn.Module): def __init__(self): super(SimpleCNN, self).__init__() #Input channels = 3, output channels = 18 self.conv1 = torch.nn.Conv2d(3, 18, kernel_size = 3, stride = 2, padding = 2) self.pool = torch.nn.MaxPool2d(kernel_size = 3, stride = 3, padding = 0) self.fc1 = torch.nn.Linear(18 * 16 * 16, 64) self.fc2 = torch.nn.Linear(64, 10) First Convolution size would change from (3,32,32) to (18,32,32) Dimensions would changes from (18, 32, 32) to (18, 16, 16). The data dimension of the input layer will change from (18, 16, 16) to (1, 4608). def forward(self, x): x = F.relu(self.conv1(x)) x = self.pool(x) x = x.view(-1, 18 * 16 *16) x = F.relu(self.fc1(x)) #Size changes from (1, 64) to (1, 10) x = self.fc2(x) return(x)
Recurrent networks are types of deep learning-oriented algorithm which takes a sequential approach. Input and Output will be considered as independent in neural networks. These types of neural networks are called recurrent because they perform mathematical computations in a sequential manner completing one task after another. The training approach for our model will contain one data point per input time. 20 data points are taken as input for sequence x. Step 1 Import all necessary package import torch from torch.autograd import Variable import numpy as np import pylab as pl import torch.nn.init as init
Set the size of the input layer to 7. In order to generate the target sequence, it takes consists of 6 context neurons and 1 input neuron dtype = torch.FloatTensor input_size, hidden_size, output_size = 7, 6, 1 epochs = 300 seq_length = 20 lr = 0.1 data_time_steps = np.linspace(2, 10, seq_length + 1) data = np.sin(data_time_steps) data.resize((seq_length + 1, 1)) x = Variable(torch.Tensor(data[:-1]).type(dtype), requires_grad=False) y = Variable(torch.Tensor(data[1:]).type(dtype), requires_grad=False)
Weights are initialized in the recurrent neural network using a normal distribution with zero means. w1 = torch.FloatTensor(input_size, hidden_size).type(dtype) init.normal(w1, 0.0, 0.4) w1 = Variable(w1, requires_grad = True) w2 = torch.FloatTensor(hidden_size, output_size).type(dtype) init.normal(w2, 0.0, 0.3) w2 = Variable(w2, requires_grad = True)
Now, we will create a function for feed-forward which uniquely defines the neural network.
We will start training procedure of recurrent neural network’s sine wave implementation. Outer loop with iterate over the inner loop and inner loop with iterate overall element. Prediction of the continuous variable with the help of Mean Square Error. for i in range(epochs): total_loss = 0 context_state = Variable(torch.zeros((1, hidden_size)).type(dtype), requires_grad = True) for j in range(x.size(0)): input = x[j:(j+1)] target = y[j:(j+1)] (pred, context_state) = forward(input, context_state, w1, w2) loss = (pred - target).pow(2).sum()/2 total_loss += loss loss.backward() w1.data -= lr * w1.grad.data w2.data -= lr * w2.grad.data w1.grad.data.zero_() w2.grad.data.zero_() context_state = Variable(context_state.data) if i % 10 == 0: print("Epoch: {} loss {}".format(i, total_loss.data[0])) context_state = Variable(torch.zeros((1, hidden_size)).type(dtype), requires_grad = False) predictions = [] for i in range(x.size(0)): input = x[i:i+1] (pred, context_state) = forward(input, context_state, w1, w2) context_state = context_state predictions.append(pred.data.numpy().ravel()[0]) Oodles Technologies, providing high-end tech solutions. Disrupt the digital future with Python Development Services. Our tech experts help you unlock the untapped and unimagined potential of Python, IoT & Mobile App Development solutions for your business. Drop us a line today at Info@oodlestechnologies.com ## Like it? Share it!## More by this author |