159.740 Intelligent Systems

159.740 Intelligent Systems Assignment #2 N.H.Reyes Letter Recognition using Deep Neural Nets with Softmax Units Deadline: 4th of November

Instructions: You are allowed to work in a group of 2 members for this assignment.Your task is to write a program that implements and tests a multi-layer feed-forward network forrecognising characters defined in the UCI machine learning repositoryhttp://archive.ics.uci.edu/ml/datasets/Letter+RecognitionRequirements:

1. Use QT to develop your Neural Network application. A short tutorial on QT, and a start-upcode that will help you get started quickly with the assignment is provided via Stream.

2. You may utilise/consult codes available in books and websites provided that you cite themproperly, explain the codes clearly, and incorporate them with the start-up codes provided.

3. Implement a multi-layer feed-forward network using backpropagation learning and test it on thegiven problem domain using different network configurations and parameter settings. Thereshould be at least 2 hidden layers in your neural network.OFm nputnodeLegend:hidden nodoutput node = softmax unit

 Note that all nodes, except the input nodes have a bias node attached to it. 159.740 Intelligent Systems

Assignment #2

N.H.Reyes Inputs

 16 primitive numerical attributes (statistical moments and edge counts) The input values in the data set have been scaled to fit into a range of integer valuesfrom 0 through 15. It is up to you if you want to normalise the inputs before feedingthem to your network.

1. Data setsUse the data set downloadable from:

http://archive.ics.uci.edu/ml/datasets/Letter+Recognition

Training set: use the first 16,000

Test set/Validation set: use the remaining 4,000

ubmit your training data, validation/test data in separate files.

Performance measure:

 Mean Squared Error (MSE)

 Percentage of Good Classification (PGC)

 Confusion Matrix (only for the best Neural Network configuration found)

1. Training

 Provide a facility for shuffling data before feeding it to the network during training

 Provide a facility for continuing network training after loading weights from file (do notreset the weights).

 Provide a facility for training the network continuously until either the maximumepochs have been reached, or the target percentage of good classification has been met.

 For each training epoch, record the Mean Squared Error and the Percentage of GoodClassification in a text file. You need this to plot the results of training later, tocompare the effects of the parameter settings and the architecture of your network.

1. Testing the Network

Calculate the performance of the network on the Test set in terms of both the MSE andPGC.

Network Architecture

It is up to you to determine the number of hidden layers and number of hidden nodesper hidden layer in your network. The minimum number of hidden layers is 2.

Use softmax units at the output layer

Experiment with ReLU and tanh as the activation functions of your hidden units

Determine the weight-update formulas based on the activation functions used

4. Provide an interface in your program for testing the network using an input string consisting ofthe 16 attributes. The results should indicate the character classification, and the 26 actualnumeric outputs of the network. (the start-up codes partly include this functionality already, for

a simple 3-layer network (1 hidden layer), but you need to modify it to make it work for themultiple hidden layer architecture that you have designed).

5. Provide an interface in your program for:
6. Reading the entire data set
7. Initialising the network
8. Loading trained weights
9. Saving trained weights
10. Training the network up to a maximum number of epochs 159.740 Intelligent SystemsAssignment #2

1. Testing the network on a specified test set (from a file)
2. Shuffling the training set.
3. Set the default settings of the user interface (e.g. learning rate, weights, etc.) to the bestconfiguration that delivered the best experiment results.

7. Use a fixed random seed number (123) so that any randomisation can be replicated empirically.
8. It is up to you to write the main program, 代写159.740 Intelligent Systems  and any classes or data structures that you mayrequire.

9. You may choose to use a momentum term or regularization term, as part of backpropagationlearning. Indicate in your documentation, if you are using this technique.

10. You need to modify the weight-update rules to reflect the correct derivatives of the activationfunction used in your network architecture.

11. Provide graphs in Excel showing the network performance on training data and test data(similar to the graphs discussed in the lecture).

12. Provide the specifications of your best trained network. Fill-up Excel workbook(best_network_configuration.xlsx).

13. Provide a confusion matrix for the best NN classifier system found in your experiments.
14. Provide a short user guide for your program.
15. Fill-up the Excel file, named checklist.xlsx, to allow for accurate marking of your assignment.

Criteria for marking

 Documentation – 30%o Submit the trained weights of your best network (name it as best_weights.txt)

o Generate a graph of the performance of your best performing network (MSE vs.Epochs) on the training set and test set.o Generate a confusion matrix of your best network

o fill-up the Excel file, named checklist.xlsxo fill-up the Excel file, named best_network_configuration.xlsx

o provide a short user guide for your program System implementation – 70%Nothing follows.

N.H.Reyes