Activity 18 | Noise Models and Basic Image Restoration

Image restoration is a basic image processing application where an image that has been degraded or damaged is being recovered. To recover the image, the degradation must be known and thus can be used in restoring the image.

A common source of degradation is the presence of noise in the image. Noise can hide useful information in an image. It is acquired mostly during image acquisition and transmission. Noise is composed of random values that follows a distribution which is expressed by a Probability Density Function of PDF. Figure 1 shows the respective PDFs of the different noise types to be used in this activity.

Figure 1. PDFs of different noise types.

In this activity we implement basic image restoration of noisy images using different filters. We use a test pattern shown by Figure 3 along with its PDF and then add noise to the image which will be used as the image to be restored. In restoring the noisy images, four different types of filtering, shown by Figure 2. are investigated for each type of noise.

Figure 2. Types of filtering used.

Figure 3. Test image and its PDF.

It can be seen that each peak in the PDF of the original test image has spread into the shape of the distribution added noise of the such that the three gray levels are almost coinciding with each other. This is the degradation that is done when adding noise. The same goes for the other distributions. The PDFs of the restored images simply shows how the filters restore the image which is by filtering out the unwanted gray values by thus thinning the fattened distribution for each peak gray value.

Figure 4 shows the noisy image for the added Gaussian noise with mean = 0.5 and standard deviation = 0.8. The resulting restored images using the five filters were almost the same achieving the three distinct gray value peaks which is close enough to the original.

Figure 4. Image with gaussian noise and restored images.

Figure 5 shows the noisy image for the Rayleigh Distribution. From the PDFs of the restored images, it can be said that the restored images using the arithmetic and geometric mean filter were much better than the other filters.

Figure 5. Image with rayleigh noise and restored images.

For the image with Gamma Noise shown in Figure 6, all the filters produced almost the same restored images except for the +Q Contraharmonic filter where the spread of the gray values is still visible.

Figure 6. Image with gamma noise and restored images.

The test image added with Exponential Noise is shown in Figure 7 and the restored image using the +Q Contraharmonic is clearly not a good result which is obvious from the image itself. Looking at the PDF for this restored image shows that the peaks were just shifted and broadened thus resulting to the noisy image.

Figure 7. Image with exponential noise and restored images.

Uniform Noise added image shown in Figure 8 shows great degradation due to the shape of the distribution itself where the is no defined peak value. Here, the arithmetic mean filter seems to have produced the best results although the peak gray values are shifted resulting to a lighter gray value for the area that is supposed to be black.

Figure 8. Image with uniform noise and restored images.

For the Salt and Pepper Noise it can be seen that the arithmetic mean filter just blurs the salt and pepper noise thus smoothening it out. It can also be noted that the +Q contraharmonic filter filters out the pepper noise while the geometric, harmonic and -Q contraharmonic filters can filter out the salt noise.

Figure 9. Image with impulse or salt and pepper noise and restored images.

The arithmetic mean filter basically blurs the image to smooth out the noise. The noise models and image restoration filters are also applied to a grayscale image which was used in a previous activity. The following series of images show the image with the different added noise types and the corresponding restored images using the five filters.

I give myself a grade of 10 for this activity.

Activity 17 | Photometric Stereo

Light when shone upon an object gives us an idea on the reflectance of that object that can be used in obtaining information about the object like color. The information stored in an objects reflectance due to a light source is used in many image processing methods which we have explored in the previous activities. Aside from the information that we have been using, we can also obtain the profile or the structure of the object by using images of the object with different light source positions and reconstruct the 3D structure of that object. This is what we call Photometric Stereo.

When light illuminates an object it produces shadows and color gradients all throughout the illuminated surface. And by getting images of the object at different light source locations, we obtain intensity profile of the whole illuminated surface. Different light sources also affect the intensity profile of the captured image of an object. Figure 1 shows the basic idea of Photometric Stereo where the angle in which light hits the surface of the object affects the captured intensity given by Equation 1 for a light source at infinity for that point P.

Figure 1. Photometric Stereo.

Equation 1. Intensity at point P(x,y).

In this activity, we use four images of an object taken with different light source positions and use it to reconstruct the 3D structure of the illuminated area, as shown by Figure 2.

Figure 2. Images of object at different light source positions.

To calculate for the surface normal of the object we need to get the far away point source locations given by V which was already provided, and the intensity I which is simply the images . The equations needed for calculating the surface normal is shown in Figure 2.

Figure 3. Equations needed for solving the surface normal.

After getting the surface normal, we can now compute for the surface elevation of the object at point (u,v) which is given by z = f(u,v) as shown in the equation below.

Figure 4. Surface elevation at point (u,v).

The resulting reconstructed 3D surface is shown below in Figure 5. We can see that the reconstructed surface is not smooth having jagged peaks and grooves. Also there is a concave dip at the areas where a transition in the intensity is located by comparing it to the four object images shown previously. All in all, the semi-spherical shape obtained matches the expected structure of the object from the images. I give myself a grade of 9 in this activity due to the roughness of the reconstructed surface.

Figure 5. Reconstructed 3D surface of object.

Activity 16 | Neural Networks

We have already investigated and implemented two methods for object classification using object features which are Minimum Euclidean Distance and Linear Discriminant Analysis. Another method that can be used for object classification is Neural Networks. This method imitates the way the human brain process information like a network of neurons thus the name Neural Networks.

The human brain manages information by forming paths for that certain information and thus 'learns' the information unlike LDA and the Minimum Distance which relies heavily on the features of the object. Examples and iterations develops or trains the network thus speeds up recognition. The framework of a Neural Network is shown below.

Figure 1. Neural Network Framework

A Neural Network is composed of three layers which are the input layer, hidden layer, and the output layer. The number of inputs in the input layer is determined by the number of features while the hidden layer can be composed of any desired number of nodes. The input receives the data/signals/features which in turn is forwarded to the hidden layer which acts on the inputs and finally passes the results to the output layer.

In this activity we feed the obtained features of our objects to a neural network composed of 3 input nodes that represent the 3 features used namely, area, perimeter and RGB sum. There are also 3 hidden nodes, and an output node. The learning rate used is 0.5 and the number of iterations is 1000.

To train the Neural Network we first entered into the network the training set and then followed by the test set. Figure 2 below shows the results obtained from the Neural Network. To implement this method, the code used by the previous AP186 class, made by Mr. Jeric Tugaff, was utilized. The classification showed 100% accuracy. Changing the order of the objects is also an interesting direction that can be took which might challenge the accuracy of the method since the Neural Network is not trained enough for recognition. I give myself a 10 for this activity.

Figure 2. Resulting classification obtained using Neural Networks.

Activity 15 | Probabilistic Classification

Previously we have performed object classification by using the Minimum Euclidean Distance of the obtained features of the train and test objects. Another method for classifying objects is the Linear Discriminant Analysis or LDA. This method simply uses the probability that an object belongs to a certain group which depends on the features of that specific object and the mean feature vector for that certain group. This also depends on the number of samples present in a group. The highest probability that an object belongs to a group automatically classifies that object into that group. This is called Bayes Rule. LDA uses this rule but starts by assuming the distribution of each group which a multivariate Normal Distribution and that all groups share the same covariance matrix. The probability that an object belongs to a group is given by,

where, is the mean of features for group i, is the feature data of object/row k in x which is the matrix containing the features data, C is the pooled within group covariance matrix, and p is the prior probability vector for object i. Simply, object k is assigned to group with highest f. The results for the data collected from last activity is shown below.

The yellow highlighted areas are the maximum f values. The test objects were also jumbled to show that the method works no matter what the order of test objects is. It can be seen that there is a 100% accuracy in the classification of the objects. I give myself a grade of 10 for this activity for successfully implementing LDA and getting a 100% accurate object classification.

References: Kardi Teknomo's Page - Discriminant Analysis Tutorial http://people.revoledu.com/kardi/tutorial/LDA/index.html

Activity 14 | Pattern Recognition

Sorting or classifying objects using visual information depends on the features of the objects. The human brain can draw, process and manipulate these features from a previous encounter to identify objects. This ability is being implemented in computer vision although it is not on the same level as the human brain.

In this activity, we implemented pattern recognition using images of different groups of objects and obtained sets of features to describe the objects and to use for recognizing new sets of objects. The groups of objects I used in this activity are 10 coins, 10 fasteners, and 10 leaves as shown by Figures 1-3. The features that were obtained are the area, perimeter and the sum of the RGB values of the object images.

Figure 1. Object group 1: coins.


Figure 2. Object group 2: fasteners.


Figure 3. Object group 3: leaves.

The first five objects of each group is used as the training set or the first set of examples where we get the basis for pattern recognition. The last five objects of each group is used as the testing set or the prediction set which will be used for testing the basis features. The same features was obtained for the test set and will be used for predicting which group of objects it belongs.

The features were obtained using basic image processing techniques such as segmentation by thresholding then counting of the number of pixels of the segmented image to get the area. The perimeter was obtained by getting the length of the contour of the segmented image while the RGB sum is simply the sum of the RGB values of the object image. The basis for pattern recognition is the mean of the features of the five sample objects for each group.

An object is classified into a group by computing the Minimum Distance of its features with respect to the basis vector or the mean feature vector. The test object is then classified into the group with the smallest feature distance. The figures below show the test objects and the Euclidean Distance for each feature. The highlighted areas represent the group with the minimum distance and thus the group where the test object is classified.

Figure 4. Euclidean Distances for test objects composed of coins.


Figure 5. Euclidean Distances for test objects composed of fasteners.


Figure 6. Euclidean Distances for test objects composed of leaves.

All in all the accuracy of the pattern recognition is 100%. I give myself a grade of 10 for this activity for the successful implementation of the Minimum Euclidean Distance for object classification.

Activity 12 | Color Image Segmentation

It is a common problem in image processing when the region of interest is more than a problem of separating it from a background, or an application of threshold especially when the image involves a great variety of colors. In this activity the region of interest is isolated from the image by image segmentation using color in such a way that different brightness levels are not a problem by converting the RGB color space of the image to the Normalized Chromaticity Coordinates or NCC which is shown in Figure 1.

Figure 1. Normalized Chromaticity Coordinates

Image segmentation can be done in two ways. First is the parametric segmentation and second is non-parametric segmentation. Parametric segmentation involves the use of image patches of the color of the ROI and its histogram. The ROI can be isolated by doing histogram backprojection to the image using the histogram of the patch of the ROI. Figure 2 shows the original image used for this activity while Figure 3 shows the patches used and the resulting image after applying histogram backprojection to the image using the histogram of the color patch.

Figure 2. Original image. Hibiscus.

Activity 11 | Color Camera Processing

One of the major factors used in judging the quality of digital cameras is its ability to capture colors satisfactorily. Most cameras nowadays give a set of options on the lighting conditions for capturing digital images. These set of options is often called White Balance Setting that gives a set of white balancing constants needed for different conditions.

In this activity, the digital camera of a Sony Ericsson phone was used to take pictures of objects using the different White Balancing Options which are bulb, cloudy, daylight, and fluorescent. Figure 1 below shows the images captured.

Figure 1. Different white balancing settings for differently colored objects (left) and objects of the same hue (right) like objects that have the color green. The white balancing settings are (top to bottom) bulb, cloudy, daylight, and fluorescent.

From the images obtained, it is observed that the bulb and fluorescent settings produced bluish images although the bulb setting images are more bluish and the fluorescent setting images are dimmer, while the cloudy setting produced images with slightly yellowish images and the daylight setting produced brighter colors and less yellowish images than the cloudy setting.

Now we apply two different White Balancing algorithms to these images namely the White Patch Algorithm and the Gray World Algorithm. The White Patch Algorithm is done by dividing the RGB channels of the original image by the RGB value of the white patch in the original image. Figure 2 shows the white patches used for each image in Figure 1 respectively and Figure 3 shows the resulting images after the White Patch Algorithm.

Figure 2. White patches used for White Patch Algorithm for the respective images in Figure 1.

Figure 3. White Balanced images using White Patch Algorithm.

As observed from the first row of images in Figure 3 it seemed that the white balancing went wrong. By examining the RGB channels (see Figure 4) of the original image of one of the wrongly white balanced images we see that there are areas in the image where the R values are very small and this includes the white patch used for the White Patch Algorithm. Performing the balancing on the red channel will result to very high values for the areas having high red values and very low values for the areas with very low red values thus resulting to the first row of images in Figure 3. It can also be noted that these images were captured with the White Balance setting turned to 'Bulb' which may be the source of error for the resulting image.

Figure 4. Respective R, G, and B channels of the original image for the resulting image (first row) in Figure 3.

Now we examine the second white balancing algorithm which is the Gray World Algorithm. In this algorithm the balancing constant is based on all the respective RGB values of the image and not just a white patch in the image. Figure 5 shows the resulting white balanced images of the original images in Figure 1.

Figure 5. White Balanced images using White Patch Algorithm.

Summary of Images

From the obtained white balanced images, we can see that the White Patch Algorithm gives better results but has a disadvantage when the RGB values of the original image has extreme values that could be very low or very high for particular areas. The Gray World Algorithm did not give a good white balancing result because it depends on the most dominant color on the image although extreme RGB values is not a problem with it.

I give myself a 9 for this activity since I was able to implement correctly the two white balancing algorithms and was able to identify possible sources of error, although I was not able to adjust the unsatisfactory results obtained for the images with 'Bulb' white balance setting.

I thank Kaye Vergel for lending me her Sony Ericsson phone with digital camera, Gilbert Gubatan for the red umbrella, Cherry Palomero for the pink clear folder and yellow hand fan, Luis Buno for the blue logbook, Shamandura Cabato for modeling her green shirt, Winsome Rara for the green eyeglass case, Jica Monsanto for the green box and the white piece of paper, NIP for the green floor paint and green colored chalk, and lastly Miguel Sison for the comments and recommendations on the unsatisfatory results for the white balancing of the images with 'Bulb' white balance setting.