Sunday, March 13, 2016

Learning Sinus Function Using Neural Network


Recently I stumbled upon an excellent demo of a 2 layer neural network written by Florian Muellerklein: https://github.com/FlorianMuellerklein/Machine-Learning.

It is written in Python using numpy and focuses on digit recognition based on sklearn dataset.
I decided to play around with it and add visualization for learning process of a sinus function.
I used matplotlib for creating an animation.

The neural network implementation is typical. It uses standard gradient descent procedure with some optimizations like momentum and regularization (also random initialization).
If you are interested in understanding how exactly gradient descent works, I highly recommend an article from Matt Mazur: http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example.

The network architecture I used was following:

  • 1 input neuron (x parameter of sinus function)
  • 60 hidden neurons
  • 1 output value (the function result)

Here is a link to source code: https://github.com/rafalrusin/Machine-Learning/tree/master2

It converges pretty well. Here's an animation showing convergence of sinus function during consecutive learning iterations (10 learing iterations per frame):



Machine Learning is a very fascinating domain that has been emerging rapidly over recent years, mainly in visual object recognition. I hope that it keeps this pace in future (or even exceeds it!).

3 comments:

  1. Nice to see your blog post.. I really enjoyed by reading your blog post. Thanks a lot for sharing this with us.. Keep on sharing like this informative post.
    Hadoop Training institute in Chennai

    ReplyDelete


  2. Great and useful article. Creating content regularly is very tough. Your points are motivated me to move on.



    Manual testing training in Chennai

    Selenium training in Chennai

    Software testing training in Chennai

    ReplyDelete
  3. Great post! I am actually getting ready to across this information, It's very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.

    Salesforce Training in Chennai

    ReplyDelete