Skip to main content

What happens if gradient is set at 1 in "Gradient descent algorithm"

So, What is gradient descent algorithm ?

 - Given a feed forward netword we apply gradient descent as a fundamental function of Operation :
1. Randomly initialize : b, W1, W2, ... , Wm
2. Repeat untill convergence
3. Predict y(i) for each data point in training
4. Calculate loss J(b,w)
5. Calculate gradient of J(b,w)
     b(new)   =  b(old)   -a.gradient
    w1(new) =  w1(old)-a.gradient.w1
    w2(new) =  w2(old)-a.gradient.w2
    wm(new) =  wm(old) - a.gradient.wm
6. Update b,w1,w2,......,wm .. Simulateneously..

- When "Louis Augustin Cauchy" needed a function to find local minima he used idea of slope to iteratively move in direction guided by slope to reach local minima.
- Using the same idea in feed forward networks leads to convergence of minimum error.

Why gradient anyway ?

So if gradient is set to 1 or in other words if we dont use gradient descent we reach to a point directly on x - axis which will not be the required local minima.

Comments

Popular posts from this blog

Event Sourcing with CQRS.

  The way event sourcing works with CQRS is to have  part of the application that models updates as writes to an event log or Kafka topic . This is paired with an event handler that subscribes to the Kafka topic, transforms the event (as required) and writes the materialized view to a read store.

GraphQL microservices (GQLMS)

I'm curios of GraphQL !     -  GraphQL is an open-source data query and manipulation language for APIs, and a runtime for fulfilling queries with existing data. GraphQL was developed internally by Facebook in 2012 before being publicly released in 2015. It should be solving a problem in querying data !     -GraphQL lets you ask for what you want in a single query, saving bandwidth and reducing waterfall requests. It also enables clients to request their own unique data specifications. A case study ?!    -https://netflixtechblog.com/beyond-rest-1b76f7c20ef6 So, This is just another database technoloy ?  -  No. GraphQL is often confused with being a database technology. This is a misconception, GraphQL is a   query language   for APIs - not databases. In that sense it’s database agnostic and can be used with any kind of database or even no database at all. Source:   howtographql.com