Naïve Bayes vs. SVM for Image Classification

A comparison of the two most popular image classifiers

Source

Background

Naïve Bayes is the simplest classifier, which used the language of graphical models. This method assumes that each category has its own distribution over the codebook, and the distribution of each category is observably different from those of others. For instance, a building category might emphasize codewords that represent window, door, or floor, while a face category might show a significant representation of codewords for eye, nose, and mouth.

On the other hand, the SVM classifier searches for a hyperplane that maximizes the margin between two classes of data, which can be used for classification, regression, and outlier detection. In order to use SVM to classify images of multiple tasks, we use the one-against-all approaches. Given n categories, we train n SVM classifiers, each of which is responsible for differentiating a category i against the remaining n – 1 categories. In the end, we assign an image to the SVM category with the largest computed output.

It has been generally observed that SVM performs better and return higher accuracy than Naïve Bayes. In this article, we will perform image classification on the fine-tuned dataset by Caltech to see whether this assumption regarding SVM and Naïve Bayes is correct.

Data Collection

After downloading the dataset mentioned above, we write the following code to partition images into training and testing set:

Feature Extraction

Both classifiers require input to be features extracted from images, thus we use the Bag-of-Visual-Word approach:

Training

After extracting features and constructing codebook from the dataset, we can feed them into SVM and Naïve Bayes classifiers for training:

You might notice that in the code snippet above, I use a function called save_model to serialize and store trained models in pickle files for later use:

Evaluation

After obtaining the training models, we evaluate their accuracies against the testing data:

Key Observations

We run classification on images of classes Airplane , Face , Motorbike and get the following results:

• When we partition the training and testing sets with a 50:50 ratio, each set of 100 images for each class, the result is about 78% and 74% accuracy for SVM and Naïve Bayes respectively, signaling a small difference.
• However, when we increase the number of images in the training and testing sets to 800 images for each class, the result becomes about 91% and 81% accuracy for SVM and Naïve Bayes respectively, showing a wider gap.
• When we partition the dataset with a ratio of 80:20, which means 80% of the data goes to training and the remaining 20% is for testing, the accuracy rates even increase: 94% for SVM and 83% for Naïve Bayes.

Conclusion

From this experiment, we can conclude that SVM outperforms Naïve Bayes for image classification and the difference gets larger as we increase the dataset size. The code repo for this article can be found here.

Want to Connect?
Check out my personal blog.