## Project 2 EP'08Learning Classifier Systems First Draft COSC 6367 Spring 2008

last updated: March 5, 2008
deliverables are due: Sa., April 5, 11p
program demos: Tu., April 8

Problem Specification: Develop a adaptive learning system that learns rule-sets / programs that have been evolved using evolutionary programming, that process binary inputs from 9 binary input channels and take 2 different actions based on the inputs: 0 or 1. Moreover, the classifier might decide not to take any action at all, called "no action", or 'na' for short. The classifier receives a reward / punishment based on its output between 1 and 99 units. Training sequences are specified as a list of input-payoff pairs; for example:

000011111 0=+14 1=-92

means that for the input 000011111 the outputs 0 is reward by 14 units and the output 1 is puhished by 92 units. If the classifier does not take any action, it receives a reward of 0 for this action.

Project Philosopy: Each student only choses one approach of the two approaches outlined below.

Rule-based Approach: Design and implement a classifier system that evolves a single rule-set ( Michigan approach) that consist of rules that are specified using the following syntax:

"string of length 9 consisting of 0,1 or *": "output (0 or 1)"

For example,

1********:1

represents a rule that selects the action 1 in the case that the input symbol starts with '1'.

Design a learning strategy, a conflict resolution strategy, a default action selection strategy, and a learning schedule of a classifier system of your own choice. A conflict resolution strategy decides which decision to make, if more than one action is found in the set of matching classifiers. A learning schedule determines when learning takes place. A default action selection strategy determines which action will be taken in the case that no classifier matches the input.

Genetic Programing Approach: Develop a genetic programming system that evolves a population of programs that have 9 input parameters (which are the values of the 9 input channel). First the architecture of the genetic progamming system has to be determined. The architecture determines the structure of program(s) evolved and how actions are selected by the classifier system. Second, decide how fitness of the evolved programs will be evaluated. Third, a learning strategy and a learning schedule has to be chosen. The learning strategy determines how learning is accomplished: e.g. how programs are evolved, how their fitness is computed, what roles previously learned programs play in the learning process, etc. The learning schedule determines when learning takes place.

General Considerations: Be aware of the fact that your classfier system is only allowed to learn from past experience (violating this constraint will be severely penalized); that is, prescanning or preanalyzing the training file are not allowed in the project. Moreover, the classifier systems is not allowed to use knowledge concerning the feedback of actions that were not selected for the given input. This, however, does not prevent you from analyzing the feedback for taken actions for past inputs.

Traning sequences in benchmarks consist of either 400 or 1200 examples; that is, each training sequence consists of 400/1200 input-payoff pairs. Test your system by giving the same training sequence twice (design your program in such a way that it can process the input file multiple times) to your system for each benchmark, and report the result of running your system for the benchmark. Also maintain a system bankaccount that keeps track of your system's performance which is initialized to 0 after seeing the 200th example (that is, the first 200 examples are ignored when evaluating learning performance). Report the value of this bankaccount after seeing the 300th, 600th, 900th, 1200th, 1500th,... example. There will be several benchmarks: Benchmark0 (is a traning benchmark that you will receive no later than March 11), Benchmark1 (is the benchmark you report the results for), Benchmark2 (is another benchmark, but you do not report any results for it), Benchmark3 (is an unknown benchmark you program will be tested with on April 8, 2008).

There will be program demons for Project2. Moreover, write a short report (approx. three single spaced pages) that outlines the architecture of your learning environment, describes the strategies and algorithms that were used). Additionally, prepare a second document that lists and summarizes the results of running your program for the first benchmark (run your system 4 times for the first benchmark, and report the results of the 4 runs). Your system will also be run for Benchmark2 an Benchmark3 during your program demo. Be also prepared to explain the design and evolution of the program during the program demo.

Add a display option to your program that, everytime the program goes into learning mode, displays the current rule-set, parameters of individual classifiers belonging to the current rule-set (e.g. strength,...), your system's bankaccount, and visualizes the rule-set evolution process (at least in part). You can use any programming language to perform the project. Be also aware of the fact that time is limited, and the project has to be finished in about a month. These requirements have to be modified for the genetic progamming approach!