>With the reading of the Yu, Wang & Lai book (Foreign-exchange-rate forecasting with artificial neural networks), I decided to implement a Genetic Algorithm (GA) to figure out which indicators are actually useful for predicting the future price. This involved further refactoring of the code.

Genetic algorithms

A GA works by sampling and scoring different possible solutions to a problem, and then mixing and matching different parts of the best solutions. This is done iteratively, and eventually a good solution is found (Though its not necessarily the best possible solution out there).

My GA function’s logic is derived directly from what I’ve read about GAs on the internet. A very useful resource, the internet. I can’t think of improvements on the logic for now. I give the GA-function the number of generations I want, the length of each chromosome, and the size I want the population to be, and it creates random chromosomes and feeds them to the NN-function (See below) and ranks the results and performs crossover. I keep the better half of the population and perform crossovers from them through the generations. Its a very computationally-intensive process, but it seems to work. I’m happy to have learned how to do this. I have decided to not use PCA or other statistical measures to find out which indicators are helpful.

I’ve created more indicators based off what the authors of the forecasting book use. I’ve not made the full list of indicators that they say they evaluate, but I have a total of 9 right now (Including the ones I had earlier). This isn’t a big list, but I haven’t gotten the motivation to work on progrmaming more in. I chose instead to forge ahead and work on creating the GA and functionalizing the NN.

Functionalizing the code

The main requirement from the GA was that different NNs compete and be evaluated against each other and that just wasn’t possible with the program structured the way it was. I had to make the entire NN creation and training encapsulated so that I could run it repeatedly with different inputs (That were randomly selected). This complicated things even further since I found it cumbersome to use macros for define-hidden-nodes and define-input-nodes. I think this is the 4th or 5th iteration of the entire NN-training functionality.

To do all this, I learned about the “labels” utility that allows one to define functions within functions. I used this a lot. I also implemented more closures than earlier. Functional programming isn’t so bind-mending anymore; I like that.

Now, everything is inside one function that is called with a “chromosome” as its only argument. This chromosome is a GA concept. This huge NN-function takes this chromosome to determine which indicators are going to be used for this particular NN. After training is done, the NN-function evaluates the aggregate error in the training and validation sets and returns them as part of a list to the GA.

Limitations

Currently, there is a lot of improvement that can be done for training NNs. I don’t know if all of it is necessary, though. One of the major things to consider is to find out a better way to end training. Presently I’m using a hard-limit of training iterations. I was trying to make it work using the validation error (Stop training when validation error starts climbing), but that wasn’t stopping quickly enough for me.

Other improvements to consider would be to have more adaptable parameters. Currently I don’t have a momentum parameter, and my learning rate is static and arbitrary. The Lean, Yu, and Lai book has a whole list of things that can be done to improve performance of NNs. With my reading of this book, I find only the suggestion to use GAs, and this list of possible tweaks to be useful. Perhaps I will find more useful things after some more experience in this problem-domain.

Code

Mapping the chromosome to indices for specifix indicators

Each chromosome is a list of 1s and 0s (Ones and Zeros). This function takes the chromosome and using the position of the 1s, creates a new list that gives the index of the indicators that are going to be used by this NN. The length of this new list (Which is the same as the number of 1s in the chromosome) automatically tells us how many indicators are being used as inputs for this NN. The output of this function is the input for the define-input-nodes function.

(chromosome-to-input-vectors (chromosome)
  (let ((a ()))
    (loop for i below (length chromosome) do
   (if (= 1 (elt chromosome i))
       (push (1+ i)
      a)))
    a))

Define-hidden-nodes

This is the latest iteration of the define-hidden-nodes function. It does away with macros completely. I haven’t noticed a performance loss.

(define-hidden-nodes ()
  (loop for i below numberofhiddennodes do
 (let ((hidden-node-index i) ;Because we are closing over this variable for each hidden-node.
       (number-of-input-nodes numberofinputnodes))
   (setf (aref hidden-node hidden-node-index)
  (lambda (dataset input-index)
    (tanh
     (loop for input-node-index below number-of-input-nodes sum
    (* (aref weights-1 input-node-index hidden-node-index)
       (funcall (aref input-node input-node-index)
         dataset input-index)))))))))