Sunday, March 13, 2016

Learning Sinus Function Using Neural Network


Recently I stumbled upon an excellent demo of a 2 layer neural network written by Florian Muellerklein: https://github.com/FlorianMuellerklein/Machine-Learning.

It is written in Python using numpy and focuses on digit recognition based on sklearn dataset.
I decided to play around with it and add visualization for learning process of a sinus function.
I used matplotlib for creating an animation.

The neural network implementation is typical. It uses standard gradient descent procedure with some optimizations like momentum and regularization (also random initialization).
If you are interested in understanding how exactly gradient descent works, I highly recommend an article from Matt Mazur: http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example.

The network architecture I used was following:

  • 1 input neuron (x parameter of sinus function)
  • 60 hidden neurons
  • 1 output value (the function result)

Here is a link to source code: https://github.com/rafalrusin/Machine-Learning/tree/master2

It converges pretty well. Here's an animation showing convergence of sinus function during consecutive learning iterations (10 learing iterations per frame):



Machine Learning is a very fascinating domain that has been emerging rapidly over recent years, mainly in visual object recognition. I hope that it keeps this pace in future (or even exceeds it!).

15 comments:

  1. Nice to see your blog post.. I really enjoyed by reading your blog post. Thanks a lot for sharing this with us.. Keep on sharing like this informative post.
    Hadoop Training institute in Chennai

    ReplyDelete


  2. Great and useful article. Creating content regularly is very tough. Your points are motivated me to move on.



    Manual testing training in Chennai

    Selenium training in Chennai

    Software testing training in Chennai

    ReplyDelete
  3. Great post! I am actually getting ready to across this information, It's very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.

    Salesforce Training in Chennai

    ReplyDelete
  4. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

    Salesforce Training in Chennai

    ReplyDelete
  5. Really it was an awesome article...very interesting to read..You have provided an nice article....Thanks for sharing..
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  6. Thank you for sharing such great information with us. I really appreciate everything that you’ve done here and am glad to know that you really care about the world that we live in.
    Salesforce Training in Chennai
    German Classes in Chennai
    Salesforce Course in Chennai
    Salesforce developer training in chennai
    German Language Course in Chennai
    German Courses in Chennai

    ReplyDelete
  7. Hi! I have more then 4 Years of experience and I am the Founder of a Digital Marketing Agency in Delhi. I just want to say thank you for posting such a useful, impressive and a wicked article.

    ReplyDelete
  8. Hi! I have more then 4 Years of experience and I am the Founder of a Website Development Company in Rohini, Delhi. I just want to say thank you for posting such a useful, impressive and a wicked article.

    ReplyDelete
  9. Thanks for giving great kind of information. So useful and practical for me. Thanks for your excellent blog, nice work keep it up thanks for sharing the knowledge.
    Epoxy Grout manufacturer in delhi

    ReplyDelete
  10. nice topic which you have choose.
    second is, the information which you have provided is better then other blog.
    so nice work keep it up. And thanks for sharing. Laminated Doors manufacturer in hubli

    ReplyDelete
  11. Thanks for giving great kind of information. So useful and practical for me. Thanks for your excellent blog, nice work keep it up thanks for sharing the knowledge. led lawn lights in delhi

    ReplyDelete
  12. THANKS FOR INFORMATION

    you can search low-cost website with high-quality website functions.
    Today Join Us
    Call: +91 - 8076909847

    web development company in delhi

    levantro
    interior designers delhi


    livewebindia
    web design company

    top seo company in delhi

    Best It Service Provider:

    1. Website Designing And Development.
    2. SEO Services.
    3. Software Development.
    4. Mobile App Development.

    ReplyDelete
  13. Nice blog! Thanks for sharing with us. It was really helpful for me. Visit for
    Logistics Company in India
    Freight Forwarders in India

    ReplyDelete
  14. Thanks for sharing such helpful an informative article. I hope you will keep sharing such an informative articles more and more
    Mobile app development company in Gurgaon

    ReplyDelete