Skip to main content

Posts

Event Sourcing with CQRS.

  The way event sourcing works with CQRS is to have  part of the application that models updates as writes to an event log or Kafka topic . This is paired with an event handler that subscribes to the Kafka topic, transforms the event (as required) and writes the materialized view to a read store.

GraphQL microservices (GQLMS)

I'm curios of GraphQL !     -  GraphQL is an open-source data query and manipulation language for APIs, and a runtime for fulfilling queries with existing data. GraphQL was developed internally by Facebook in 2012 before being publicly released in 2015. It should be solving a problem in querying data !     -GraphQL lets you ask for what you want in a single query, saving bandwidth and reducing waterfall requests. It also enables clients to request their own unique data specifications. A case study ?!    -https://netflixtechblog.com/beyond-rest-1b76f7c20ef6 So, This is just another database technoloy ?  -  No. GraphQL is often confused with being a database technology. This is a misconception, GraphQL is a   query language   for APIs - not databases. In that sense it’s database agnostic and can be used with any kind of database or even no database at all. Source:   howtographql.com    

Virahanka Numbers : An Application of recursion

Brief History Of Computer. The computer as we know it today had its beginning with a 19th century English mathematics professor name Charles Babbage. He designed the Analytical Engine and it was this design that the basic framework of the computers of today are based on. But recursion seems to be existing much before we thought of a machine for calculation. Here's the Virahanka's problem in brief : Find number of of poetic meters with 8 beats, made of: • Short syllables, each 1 beat duration • Long syllables, each  2 beat duration  And the solution was V(D) = V(D-1)+V(D-2) for D> 2 . Given V(1)=1 and V(2)=2. Credits :  https://www.cse.iitb.ac.in/~cs101/2012.2/resources/VirahankaNumbers.pdf

Extract Translate Load

So, What it's like Extract Translate Load ?   - ETL was a solution to get analytics at scale. Once we have huge data at scale of hundreds of tera bytes or even at peta scale, we may need a HPC to ask questions on such data. Using commodity compute horizontally would be cost effective in most of the businesss cases. Initially Hadoop had its helping hand in the process, however when Spark could do it efficiently the world said "Why not?". For us to get analytics on huge data largely unstructred and from hetrogenous sources, like every other engineering problem we divided the problem so we can conquer it with ease. We made a layer to Extract, this layer would just abstract us different data sources and get us the data. Traslate layer would structure the data for us so that our logical questions would fit into the arena.Load come in where we need to distribute the compute task at hand to large commodity clusters. Here's where big data framework would be a friend at hel...

What happens if gradient is set at 1 in "Gradient descent algorithm"

So, What is gradient descent algorithm ?  - Given a feed forward netword we apply gradient descent as a fundamental function of Operation : 1. Randomly initialize : b, W1, W2, ... , Wm 2. Repeat untill convergence 3. Predict y(i) for each data point in training 4. Calculate loss J(b,w) 5. Calculate gradient of J(b,w)      b(new)   =  b(old)   -a.gradient     w1(new) =  w1(old)-a.gradient.w1     w2(new) =  w2(old)-a.gradient.w2     wm(new) =  wm(old) - a.gradient.wm 6. Update b,w1,w2,......,wm .. Simulateneously.. - When "Louis Augustin Cauchy" needed a function to find local minima he used idea of slope to iteratively move in direction guided by slope to reach local minima. - Using the same idea in feed forward networks leads to convergence of minimum error. Why gradient anyway ? So if gradient is set to 1 or in other words if we dont use gradient descent we reach to a point dir...

Whirlwind Questions : Kubernetes

What is Kubernetes ? Goole Tells :  Kubernetes is an open-source container-orchestration system for automating application deployment, scaling, and management. It was originally designed by Google, and is now maintained by the Cloud Native Computing Foundation.  What is container-orchestration ?  Goole Tells :  Container orchestration is the automatic process of managing or scheduling the work of individual containers for applications based on microservices within multiple clusters. Who are Cloud Native Computing Foundation ?  Google Tells : The Cloud Native Computing Foundation (CNCF) hosts critical components of the global technology infrastructure. CNCF brings together the world's top developers, end users, and vendors and runs the largest open source developer conferences. Why did goole let CNCF let develop Kubernetes ? - Actually its still with Google. Google partners with Linux Foundation to form CNCF. History here :  https://blog.risingsta...

3 Dimensions of a Programming Language

- I believe there are 3 broad areas for mastering any programming language : 1. Language Internals 2. Programming Constructs offerred 3. Standard library written already in that particular language. And these 3 categories are in order of priority to hone mastery in a given programming language. Priorities are always decided based on goals. So, here the premise for setting this priority is having said to have mastered such programming languge we should be able to read a large code corpus with ease and understand what the programmer who originally wrote this code intended to do with it. Why Language Internals take a first priority ? - Because we as a software enginners write a tool chain and compilers only to solve existing issues faced in some other programming language. When assembly was just making computer do its job it was difficult for a programmer to express his algorithms in faster way hence came languages like B,C, Fortran and BASIC. After some time into building this t...

Docker Internals : Demistifying Container

PID namespaces : Every running process sees only a limited set of resources. This limited resource set is identified by notion of namespace. In linux kernel for PID namespaces to be enabled CONFIG_PID_NS has to be made true. Network namespaces :  Network support at operating system level contains network interface and routing table entries.There resources are typicall shared across entire OS. This typical behaviout can be altered by notion of Network Namespaces. With particular namespce we can isolate acess to network interface and routing table created hitherto. IPC namespace :  Message queues, shared memory or socket communication can all be isolated with IPC namespce support provided by operating system MNT namespace : Mount is a process abstration which enables access to storage devices.Notion of mount space enables OS to have isolated filesystems accessible to process that are executing. UTS namespace :  UTS stands for UNIX Tim...

Typical Document Processing Operations

First rule of document pre-processing : Improper pre-processing schemes may lead to losss of lexical content. Hence, pre-processings steps are unique to a problem. Having said that there are few pre-processing steps which applies to most of the application at hand. They are : a. tokenization b. normalization  c. substitution. Other well known pre-processing steps : a. Case folding b. Stemming c. Lemmatization d. Remove misspellings e. Punctuations. What is a stemming operation ? - Process of reducing a inflected word to its root. Where inflected word is a word with extra letter or letters added to nouns,verbs and adjectives in different grammatical forms. What is lemmatization ? - Here also there is reduction in inflected word to its root, however stemming resultant need not to be a proper word in vocabulary but in case of lemmatization word has be part of the given language vocabulary. What is case folding and its usage ? - Case-folding is a part of the Unicode s...

Applications of Vector Space Model

What does vector space model mean ? - It's an algebraic representation of text documents. Why do we need to represent text documents algebraically ?  - Typical algebric operations gets simpler to perform and visualize.  - Information retrival, filtering, indexing, ranking can be made with standard procedures and helps to have standardized measuring metrics around it. So how is vector space model helping in Information retrval ?

Free MPI platform for High Performance Computing

Here is the official website of HPC @ Uni.lu platform, which assembles information about the computing clusters operated by the  University of Luxembourg  and the organization running them :  https://hpc.uni.lu/ Page here :  https://ulhpc-tutorials.readthedocs.io/en/latest/parallel/basics/  , helps you get started for using the computing resources offerred by the platform. For learning OpenMP from scratch there is none better than this :  https://nptel.ac.in/courses/106102163/

On Tabs With Tech : End to End Deep Learning Compiler Stack

With universal approximation theorem growing forward interms of Deep learning. Today we have deticated harware backends for machine learning backends. With many backend options available there are separate tool chains available for myraid deep learning backends. TVM open deep learning compiler stack is an effort in having a common software toolset to build models with harware backend abstracted. Do visit  here for more info : tvm.ai/about.

Microservices In Nutshell

What are Microservices ? A dialect of service oriented architecure where the services are fine grained and loosely coupled. Also protocols for communication are lightweight. Whare are the major projects today in industry which are microservices ? Few companies which use microservices are : Comcast Cable,Uber,Netflix,Amazon,Ebay,Sound Cloud,Karma,Groupon. What are the problems in Service Oriented Architecture which microservices solved ?  - Allows skill set flexibility : Multiple technologies can be used to develop the parts isolated.  - Easy and less risky deployment : Replacing only the changed parts of the program.  - Individual services can be scaled.  - Application lifecycle flexibility : can develop individual services in isolation.  - Overall user experience improvement What is a monolithic architecture ? In monolithic application all ther requirements of the application will be provided by a single application abstraction. So, Is there a...

Smart Pointers in C++

* Smart pointer classes are defined in <memory> header, so we need to include "memory" header to use smart pointers. * The header gives access to three template based smart pointers each with unique features. * We get "unique_ptr", "shared_ptr", "weak_ptr" *  Unique pointer are type of smart pointers that can’t be copied.std::unique_ptr is a smart pointer that owns and manages another object through a pointer and disposes of that object when the unique_ptr goes out of scope. *  Shared pointer is much like “unique_prt” but we can make copies of it. *  Weak pointer is a special case of shared pointer. Weak pointer helps in breaking circular references.Destruction using weak pointer will not care about destruction of other objects referenced from this present object under destruction.

Drive Bot In A Path

#define LEFT 13 #define RIGHT 12 #define PATH_DURATION1 5000 #define PATH_DURATION2 1000 #define PATH_DURATION3 500 // the setup function runs once when you press reset or power the board void setup() {   // initialize digital pin LED_BUILTIN as an output.   pinMode(LEFT, OUTPUT);    pinMode(RIGHT, OUTPUT); } // the loop function runs over and over again forever void loop() {   moveStraight();   delay(PATH_DURATION1);          moveRight();   delay(PATH_DURATION2);        moveStraight();   delay(PATH_DURATION1);   moveLeft();   delay(PATH_DURATION2); } void moveStraight(){   digitalWrite(LEFT, HIGH);   digitalWrite(RIGHT,HIGH);           // turn the LED on (HIGH is the voltage level) } void moveRight(){   digitalWrite(LEFT,HIGH);   digitalWrite(RIGHT,LOW); } void moveLeft(){   digitalWrite(LEFT,LOW);  ...

Learning C from Snippets : Pointers to structures

struct exam{ int a; char *ch; }st[]={10,"Testing",11,"Computer",12,"Science",13,"Engineering",14,"Custo",15,"2018"}; int main() { struct exam *e = st; printf("%s, ",e++->ch); ++e; printf("%s, ",++e->ch);   // -> operator has higher precedence than the ++ opearator. printf("%s, ",++e++->ch); printf("%d, ",e[0].a); printf("%s",++e->ch); return 0; } OUTPUT : Testing, cience, ience, 13, ngineering

Learning C from Snippets : What does pointer +1 means ?

int main(int argc, char** argv) { char p[3][4]={{'R','B','R','a'},{'S','R','K','a'},{'A','P','J','a'}}; printf("%d\n",p); //Prints 7339584 printf("%d",p+1); //Prints 7339588 , i.e there is an addition of 4 to the previous number return 0; } int main(int argc, char** argv) { char p[3][3]={{'R','B','R'},{'S','R','K'},{'A','P','J'}}; printf("%d\n",p); //Prints 7339584 printf("%d",p+1);//Prints 7339583 return 0; } _________________________________________________________________________________ int main(int argc, char** argv) { char p[3][2]={{'R','A'},{'S','R'},{'A','P'}}; printf("%c",*(p+1)); //Prints B return 0; } int main(int argc, char** argv) { char p[3][3]={{'R','B','R...

Ensemble Learning

In ensemble learning we combine the output of multiple classifiers in order to on better prediction on classification accuracy. The classifcation of resultant classifer is better then the resultant classifier. We need a way to combine the output of multiple classifiers. First we generate a group of base learners. These learners will be using different algorithms.(say learner which uses a Dicision Tree,Neural Network, Support Vector Machine etc..), they could be same algorithms with different hyperparameters, could have different representations or may use different training sets. Muliple learner will give different classifications. 

Uniqueness of Scala

* The declared type of a symbol is given after the symbol and a colon. The declared type can often be omitted, because the compiler can infer it from the context. * A mixin is a class that provides certain functionality to be inherited by a subclass and isn’t meant for instantiation by itself. A mixin could also be viewed as an interface with implemented methods. * Array types are written Array[T] rather than T[], and array selections are written a(i) rather than a[i]. * Functions can be nested inside other functions. Nested functions can access parameters and local variables of enclosing functions. For instance, the name of the array xs is visible in functions swap and sort1, and therefore need not be passed as a parameter to them.

Why Scala

1. Type Inference : Uses type inference to allow the compiler to infer the type of variables from the instances bound to them. We would write the equivalent line in Scala as follows: val myInstance = new Example() The Scala compiler infers that myInstance has the Example type at compile time. A lot of the time, it is enough to specify the types of the arguments and of the return value of a function. The compiler can then infer types for all the variables defined in the body of the function. Scala code is usually much more concise and readable than the equivalent Java code, without compromising any of the type safety. 2. Immutable objects : Scala encourages the use of immutable objects. val rollnos = List("720", "729") // List is immutable rollnos(1) = "5" // Compile-time error Knowing that some objects cannot be changed once instantiated reduces the number of places bugs can creep in. 3.Functional Programming : Scala encourages functio...