Sunday, March 13, 2016

Learning Sinus Function Using Neural Network

Recently I stumbled upon an excellent demo of a 2 layer neural network written by Florian Muellerklein:

It is written in Python using numpy and focuses on digit recognition based on sklearn dataset.
I decided to play around with it and add visualization for learning process of a sinus function.
I used matplotlib for creating an animation.

The neural network implementation is typical. It uses standard gradient descent procedure with some optimizations like momentum and regularization (also random initialization).
If you are interested in understanding how exactly gradient descent works, I highly recommend an article from Matt Mazur:

The network architecture I used was following:

  • 1 input neuron (x parameter of sinus function)
  • 60 hidden neurons
  • 1 output value (the function result)

Here is a link to source code:

It converges pretty well. Here's an animation showing convergence of sinus function during consecutive learning iterations (10 learing iterations per frame):

Machine Learning is a very fascinating domain that has been emerging rapidly over recent years, mainly in visual object recognition. I hope that it keeps this pace in future (or even exceeds it!).


  1. Nice to see your blog post.. I really enjoyed by reading your blog post. Thanks a lot for sharing this with us.. Keep on sharing like this informative post.
    Hadoop Training institute in Chennai


  2. Great and useful article. Creating content regularly is very tough. Your points are motivated me to move on.

    Manual testing training in Chennai

    Selenium training in Chennai

    Software testing training in Chennai

  3. Great post! I am actually getting ready to across this information, It's very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.

    Salesforce Training in Chennai

  4. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

    Salesforce Training in Chennai

  5. Really it was an awesome article...very interesting to read..You have provided an nice article....Thanks for sharing..
    Android Training in Chennai
    Ios Training in Chennai