In the last few months I’ve gotten back to working on AI. I was talking to a co-worker of mine about the pleasance project, and doing that got me bit by the bug again. I’ve had my new multicore desktop computer for a while now, and I should use it for parallel processing problems.

Pleasance-jruby

I figured since I’ve been doing a lot of JRuby recently, I should port my single-threaded Common Lisp code to Jruby and avail of the advantages brought on by the JVM. That didn’t work very well.

Functional to Object-Oriented programming

The main issue I had with porting the common lisp code to jruby was that I tried to get the code to work in an object-oriented manner. The act of reading my old common lisp code, in a heavily functional style, to translating it to object-oriented code, was way too much of a cognitive shift. Which led me to trying to figure out the problem I was trying to solve instead of doing a dumb porting.

Forgetting the domain space

I’d forgotten nearly everything about the domain space. I was reading my code, and reading my old blog posts about this topic, and it was all strange to me. I eventually made it to the Toronto Reference Library and found that book that made it click with me in the first place and that helped. So then I started writing the neural network code in jruby.

But lets not implement a neural network

Then I figured that maybe its a bad idea to implement a neural net myself. Code that I write is code that I introduce bugs into. So I began searching for JVM friendly neural-network libraries.

I looked for libraries in Java, Clojure, and Scala. The weird thing was that while there were a few in each language, they all had only a few commits on them with the last commit having been ~3 years ago. The same time that I stopped working on the pleasance project. Weird. Eventually I found Neuroph, which is being maintained by students of the University of Belgrade and has a seemingly regular release cycle. Nice.

I learned how to call it from Jruby and got it crunching on data, when I realised that its not providing usable output. In their forums they say that they keep the training going on indefinitely since they have seen errors build up or stay flat for a long time before they converge to a lower number. So I kept it running for a few hours and still didn’t get results I was happy with. It appears, from the copyright notices, that they haven’t touched the particular Java classes that deal with the multilayer perceptron, the type of NN that I’m using. Thats a problem for me, since I don’t particularly feel like debugging Java code.

Enter the Spark

A friend of mine told me that she was learning about big data and using Apache Spark, from a class at the University of Toronto. I looked it up, and lo and behold, they have a machine learning library! This is great. Its Apache, so I know its good quality work that has a focus on performance. And it seems to be positioning itself as the newer and better Apache MapReduce. Shiny.

It doesn’t have neural nets as part of its machine learning (ML) library. I could just stop here and not use it, and just roll my own NN (Back to square one, wot?). Or I could learn to use this shiny new thing that has a bunch of ML libraries built-into it that are implemented in a mathematically sound manner, and pretty much guaranteed to parallelize processing. I’m going to go with option 2.

In the last week, I’ve been reading the excellent book “Scala for Machine Learning”. Its introducing me to a lot of machine learning principles, and I’m looking forward to using them soon. I also found out about the Kaggle competitions, and I’m very interested in using what I learn to these real-world problems.