ANN (Artificial Neural Networks) and SVM (Support Vector Machines) are two popular strategies for supervised machine learning and classification. It's not often clear which method is better for a particular project, and I'm certain the answer is always "it depends." Often, a combination of both along with Bayesian classification is used.
These questions on Stackoverflow have already been asked regarding ANN vs SVM:
what the difference among ANN, SVM and KNN in my classification question
Support Vector Machine or Artificial Neural Network for text processing?
In this question, I'd like to know specifically what aspects of an ANN (specifically, a Multilayer Perceptron) might make it desirable to use over an SVM? The reason I ask is because it's easy to answer the opposite question: Support Vector Machines are often superior to ANNs because they avoid two major weaknesses of ANNs:
(1) ANNs often converge on local minima rather than global minima, meaning that they are essentially "missing the big picture" sometimes (or missing the forest for the trees)
(2) ANNs often overfit if training goes on too long, meaning that for any given pattern, an ANN might start to consider the noise as part of the pattern.
SVMs don't suffer from either of these two problems. However, it's not readily apparent that SVMs are meant to be a total replacement for ANNs. So what specific advantage(s) does an ANN have over an SVM that might make it applicable for certain situations? I've listed specific advantages of an SVM over an ANN, now I'd like to see a list of ANN advantages (if any).
This question is related to
machine-learning
neural-network
classification
svm
We should also consider that the SVM system can be applied directly to non-metric spaces, such as the set of labeled graphs or strings. In fact, the internal kernel function can be generalized properly to virtually any kind of input, provided that the positive definiteness requirement of the kernel is satisfied. On the other hand, to be able to use an ANN on a set of labeled graphs, explicit embedding procedures must be considered.
One obvious advantage of artificial neural networks over support vector machines is that artificial neural networks may have any number of outputs, while support vector machines have only one. The most direct way to create an n-ary classifier with support vector machines is to create n support vector machines and train each of them one by one. On the other hand, an n-ary classifier with neural networks can be trained in one go. Additionally, the neural network will make more sense because it is one whole, whereas the support vector machines are isolated systems. This is especially useful if the outputs are inter-related.
For example, if the goal was to classify hand-written digits, ten support vector machines would do. Each support vector machine would recognize exactly one digit, and fail to recognize all others. Since each handwritten digit cannot be meant to hold more information than just its class, it makes no sense to try to solve this with an artificial neural network.
However, suppose the goal was to model a person's hormone balance (for several hormones) as a function of easily measured physiological factors such as time since last meal, heart rate, etc ... Since these factors are all inter-related, artificial neural network regression makes more sense than support vector machine regression.
One answer I'm missing here: Multi-layer perceptron is able to find relation between features. For example it is necessary in computer vision when a raw image is provided to the learning algorithm and now Sophisticated features are calculated. Essentially the intermediate levels can calculate new unknown features.
If you want to use a kernel SVM you have to guess the kernel. However, ANNs are universal approximators with only guessing to be done is the width (approximation accuracy) and height (approximation efficiency). If you design the optimization problem correctly you do not over-fit (please see bibliography for over-fitting). It also depends on the training examples if they scan correctly and uniformly the search space. Width and depth discovery is the subject of integer programming.
Suppose you have bounded functions f(.) and bounded universal approximators on I=[0,1] with range again I=[0,1] for example that are parametrized by a real sequence of compact support U(.,a) with the property that there exists a sequence of sequences with
lim sup { |f(x) - U(x,a(k) ) | : x } =0
and you draw examples and tests (x,y) with a distribution D on IxI.
For a prescribed support, what you do is to find the best a such that
sum { ( y(l) - U(x(l),a) )^{2} | : 1<=l<=N } is minimal
Let this a=aa which is a random variable!, the over-fitting is then
average using D and D^{N} of ( y - U(x,aa) )^{2}
Let me explain why, if you select aa such that the error is minimized, then for a rare set of values you have perfect fit. However, since they are rare the average is never 0. You want to minimize the second although you have a discrete approximation to D. And keep in mind that the support length is free.
One thing to note is that the two are actually very related. Linear SVMs are equivalent to single-layer NN's (i.e., perceptrons), and multi-layer NNs can be expressed in terms of SVMs. See here for some details.
Source: Stackoverflow.com