Dr. Eick's Graduate AI-class (COSC 6368)

Deadlines: problem 6, 8, 9: Tu., Oct. 15 in class; problem 7: Th., October 31 in class.

last updated: September 30, 7:50p

Value: 100 (6:6, 7:70 8:12 9:12)

*Problems 6, 8, 9 are "short" paper and pencil problems, wheras in problem 7 you will
you will get some exposure with respect to "making sense of data" and you will also
use a data mining tool.*

Discuss and summarize your results and findings in a 5-6 page (single spaced) report. Include the 5 decision trees you obtained in Step C as an appendix in the report (you can refer to the figures in your report).

A | B | C | X |
---|---|---|---|

0 | 0 | 0 | 1 |

0 | 0 | 1 | 1 |

0 | 1 | 0 | 0 |

0 | 1 | 1 | 1 |

1 | 0 | 0 | 0 |

1 | 0 | 1 | 0 |

1 | 1 | 0 | 0 |

1 | 1 | 1 | 0 |

Show how your neural network computes the answer for the third and fourth example.

Out= If (w1*X1 + w2*X2 + w3*X3) > 0 then 1 else 0

Note that using the extra input, X3, we can achieve the same effect as changing the perceptron's threshold by changing w3. Thus, we can use the same simple perceptron learning rule presented in our textbook to control this threshold as well.

**A.** We want to teach the perceptron to recognize the function X1 XOR X2 with the
following training set:

X1 | X2 | X3 | Out |
---|---|---|---|

1 | 1 | 1 | 0 |

0 | 1 | 1 | 1 |

1 | 0 | 1 | 1 |

0 | 0 | 1 | 0 |

Show the change in the weights of the perceptron for every presentation of a training instance. Assume the initial weights are: w1=0.2, w2=0.2, w3=0.2 Important: Do the iterations according to the order of the samples in the training set. When you finish the four samples go over them again. You should stop the iterations once you get convergence, or when the values you get indicate that there will be no convergence. In either case explain your decision to stop the iterations. Assume in your computations that the learning rate a is 0.2.

Sample# | X1 | X2 | X3 | Output | True_Out | Error | w1 | w2 | w3 |
---|---|---|---|---|---|---|---|---|---|

0 | 0.2 | 0.2 | 0.2 | ||||||

1 | 1 | 1 | 1 | 0 | |||||

2 | 0 | 1 | 1 | 1 | |||||

3 | 1 | 0 | 1 | 1 | |||||

4 | 0 | 0 | 1 | 0 | |||||

5 | 1 | 1 | 1 | 0 | |||||

6 | ... | ... | ... | ... | ... | ... | ... | ... | |

7 | |||||||||

8 | |||||||||

... |

**B.** This time, instead of being limited to a single perceptron, we will introduce hidden
units and use a different activation function. Our new network is depicted in Fig. 2.
Assume that the initial weights are w14 = 0.3, w15 = 0.1, w24 = 0.2, w25 = 0.6,
w34 = 0.1, w35 = 0.1, w36 = 0.1, w46 = 0.5, and w56 = 0.2. The training set is the
same as in (a). Use g=0.1 as your learning rate.
Show what the new weights would be after using the backpropagation algorithm
for two updates using just the first two training instances. Use g(x) = 1/(1+e**(-x)) as the activation
function; that is g'(x)=(e**(-x))/(1+e**(-x))**2).

S# | X1 | X2 | X3 | Out | True_Out | Error | w14 | w15 | w24 | w25 | w34 | w35 | w36 | w46 | w56 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

0 | 0.3 | 0.1 | 0.2 | 0.6 | 0.1 | 0.1 | 0.1 | 0.5 | 0.2 | ||||||

1 | 1 | 1 | 1 | 0 | |||||||||||

2 | 0 | 1 | 1 | 1 |