Skip to main content

What happens if gradient is set at 1 in "Gradient descent algorithm"

So, What is gradient descent algorithm ?

 - Given a feed forward netword we apply gradient descent as a fundamental function of Operation :
1. Randomly initialize : b, W1, W2, ... , Wm
2. Repeat untill convergence
3. Predict y(i) for each data point in training
4. Calculate loss J(b,w)
5. Calculate gradient of J(b,w)
     b(new)   =  b(old)   -a.gradient
    w1(new) =  w1(old)-a.gradient.w1
    w2(new) =  w2(old)-a.gradient.w2
    wm(new) =  wm(old) - a.gradient.wm
6. Update b,w1,w2,......,wm .. Simulateneously..

- When "Louis Augustin Cauchy" needed a function to find local minima he used idea of slope to iteratively move in direction guided by slope to reach local minima.
- Using the same idea in feed forward networks leads to convergence of minimum error.

Why gradient anyway ?

So if gradient is set to 1 or in other words if we dont use gradient descent we reach to a point directly on x - axis which will not be the required local minima.

Comments

Popular posts from this blog

Event Sourcing with CQRS.

  The way event sourcing works with CQRS is to have  part of the application that models updates as writes to an event log or Kafka topic . This is paired with an event handler that subscribes to the Kafka topic, transforms the event (as required) and writes the materialized view to a read store.

Procedure to enable wireless LAN adapter in vmware!

I have went to find the way in which wlan in Kali Linux could be made listed my airmon-ng command. Most of the videos and posts suggested me that it is necessary to have another wi-fi adapter to have connected to wifi network. But this is not necessary. The procedure to do this is as follows: 1. Download this package in Virtual Machine(Kali linux) " http://wireless.kernel.org/download/compat-wireless-2.6/compat-wireless-2010-06-26-p.tar.bz2" 2.Unzip this package in terminal i.e navigate to place where you have downloaded this file , say desktop then type : cd Desktop for unzip type : tar -jxvf compat-wireless-2010-06-26-p.tar.bz2 3. Navigate into the uncompressed directory and type: i."make unload"  (only words within the quotes are the commands) ii."make load" Done!  now airmon-ng  should have the list of the required devices.