Activity 17 | Photometric Stereo

Light when shone upon an object gives us an idea on the reflectance of that object that can be used in obtaining information about the object like color. The information stored in an objects reflectance due to a light source is used in many image processing methods which we have explored in the previous activities. Aside from the information that we have been using, we can also obtain the profile or the structure of the object by using images of the object with different light source positions and reconstruct the 3D structure of that object. This is what we call Photometric Stereo.

When light illuminates an object it produces shadows and color gradients all throughout the illuminated surface. And by getting images of the object at different light source locations, we obtain intensity profile of the whole illuminated surface. Different light sources also affect the intensity profile of the captured image of an object. Figure 1 shows the basic idea of Photometric Stereo where the angle in which light hits the surface of the object affects the captured intensity given by Equation 1 for a light source at infinity for that point P.

Figure 1. Photometric Stereo.

Equation 1. Intensity at point P(x,y).

In this activity, we use four images of an object taken with different light source positions and use it to reconstruct the 3D structure of the illuminated area, as shown by Figure 2.

Figure 2. Images of object at different light source positions.

To calculate for the surface normal of the object we need to get the far away point source locations given by V which was already provided, and the intensity I which is simply the images . The equations needed for calculating the surface normal is shown in Figure 2.

Figure 3. Equations needed for solving the surface normal.

After getting the surface normal, we can now compute for the surface elevation of the object at point (u,v) which is given by z = f(u,v) as shown in the equation below.

Figure 4. Surface elevation at point (u,v).

The resulting reconstructed 3D surface is shown below in Figure 5. We can see that the reconstructed surface is not smooth having jagged peaks and grooves. Also there is a concave dip at the areas where a transition in the intensity is located by comparing it to the four object images shown previously. All in all, the semi-spherical shape obtained matches the expected structure of the object from the images. I give myself a grade of 9 in this activity due to the roughness of the reconstructed surface.

Figure 5. Reconstructed 3D surface of object.

Activity 16 | Neural Networks

We have already investigated and implemented two methods for object classification using object features which are Minimum Euclidean Distance and Linear Discriminant Analysis. Another method that can be used for object classification is Neural Networks. This method imitates the way the human brain process information like a network of neurons thus the name Neural Networks.

The human brain manages information by forming paths for that certain information and thus 'learns' the information unlike LDA and the Minimum Distance which relies heavily on the features of the object. Examples and iterations develops or trains the network thus speeds up recognition. The framework of a Neural Network is shown below.

Figure 1. Neural Network Framework

A Neural Network is composed of three layers which are the input layer, hidden layer, and the output layer. The number of inputs in the input layer is determined by the number of features while the hidden layer can be composed of any desired number of nodes. The input receives the data/signals/features which in turn is forwarded to the hidden layer which acts on the inputs and finally passes the results to the output layer.

In this activity we feed the obtained features of our objects to a neural network composed of 3 input nodes that represent the 3 features used namely, area, perimeter and RGB sum. There are also 3 hidden nodes, and an output node. The learning rate used is 0.5 and the number of iterations is 1000.

To train the Neural Network we first entered into the network the training set and then followed by the test set. Figure 2 below shows the results obtained from the Neural Network. To implement this method, the code used by the previous AP186 class, made by Mr. Jeric Tugaff, was utilized. The classification showed 100% accuracy. Changing the order of the objects is also an interesting direction that can be took which might challenge the accuracy of the method since the Neural Network is not trained enough for recognition. I give myself a 10 for this activity.

Figure 2. Resulting classification obtained using Neural Networks.

Activity 15 | Probabilistic Classification

Previously we have performed object classification by using the Minimum Euclidean Distance of the obtained features of the train and test objects. Another method for classifying objects is the Linear Discriminant Analysis or LDA. This method simply uses the probability that an object belongs to a certain group which depends on the features of that specific object and the mean feature vector for that certain group. This also depends on the number of samples present in a group. The highest probability that an object belongs to a group automatically classifies that object into that group. This is called Bayes Rule. LDA uses this rule but starts by assuming the distribution of each group which a multivariate Normal Distribution and that all groups share the same covariance matrix. The probability that an object belongs to a group is given by,

where, is the mean of features for group i, is the feature data of object/row k in x which is the matrix containing the features data, C is the pooled within group covariance matrix, and p is the prior probability vector for object i. Simply, object k is assigned to group with highest f. The results for the data collected from last activity is shown below.

The yellow highlighted areas are the maximum f values. The test objects were also jumbled to show that the method works no matter what the order of test objects is. It can be seen that there is a 100% accuracy in the classification of the objects. I give myself a grade of 10 for this activity for successfully implementing LDA and getting a 100% accurate object classification.

References: Kardi Teknomo's Page - Discriminant Analysis Tutorial http://people.revoledu.com/kardi/tutorial/LDA/index.html