
Shashank D Shastry
Shashank is an SEO Content Specialist at G2 with over six years of experience in the B2B SaaS space. He tests and reviews tools across different software categories to create a diverse mix of content. His goal is to demystify G2 Data into clear, actionable insights for buyers navigating their software purchase decisions. When he's not working, he's likely checking out a critically acclaimed piece of pop culture or trying a new health fad.
TL;DR: What Is a Support Vector Machine (SVM)? A Simple Guide
- What is a support vector machine? A supervised machine learning algorithm that classifies data by drawing a boundary, called a hyperplane, with the widest margin between two classes.
- How does an SVM work? It trains on labeled examples, finds the hyperplane with the largest gap to the nearest points of each class, and uses that boundary to label new data.
- What are support vectors? The training points closest to the boundary. Only these points define the hyperplane, so removing a point that sits well beyond the margin doesn’t change the model.
- What is the kernel trick? A shortcut that lets an SVM separate data that no straight line can split by comparing points as if they sat in a higher-dimensional space without computing that space directly.
- What are the types of SVM? Linear SVMs for data a straight boundary can split and non-linear (kernel) SVMs for everything else. Regression (SVR) and one-class versions extend the method to predicting values and detecting outliers.
- Where are SVMs used? Text categorization, handwritten character recognition, human detection in images, and anomaly detection.
- What are the limits of SVMs? Training time climbs quickly as the number of samples grows, features need scaling, and SVMs don’t output probabilities directly.
- How do you build an SVM? Split and scale your data, pick a kernel, train, evaluate with precision, recall, and F1, then tune C and gamma.
Support vector machines (SVMs) have been a standard tool for classification since the 1990s. An SVM learns from labeled examples and draws the boundary that keeps two classes as far apart as possible, which helps it label data it hasn’t seen before.
The idea is easy to state, but the details trip people up: what a margin is, why only a few points matter, and how an SVM handles data that no straight line can split. This guide covers each of those, plus kernels, types, applications, and trade-offs. SVMs are available in most machine learning software, and the later sections show how to build one.
What is a support vector machine?
A support vector machine (SVM) is a supervised machine learning algorithm that classifies data by finding the hyperplane that separates two classes with the widest possible margin. The data points closest to that boundary are called support vectors, and they alone define where the boundary sits.
SVMs belong to supervised learning, so they need training data where every example already has a label, such as "spam" or "not spam." Each example is described by its features, and each feature is one dimension. A data point with 10 features sits in 10-dimensional space.
The method builds on Vladimir Vapnik’s work on optimal hyperplanes and the kernel approach of Boser, Guyon and Vapnik. Corinna Cortes and Vladimir Vapnik then extended it to data that can’t be separated without errors in their 1995 paper, "Support-Vector Networks".
Four terms come up in every SVM explanation:
- Hyperplane: The decision boundary. In two dimensions it’s a line, in three it’s a flat plane, and in higher dimensions it’s a flat surface with one dimension fewer than the space.
- Margin: The distance between the hyperplane and the nearest training points of each class. An SVM picks the hyperplane that makes this distance as large as possible.
- Support vectors: The training points that sit on the margin boundaries, or inside them when classes overlap. Only these points define the hyperplane.
- Features: The measurable properties of each example, such as pixel values in an image or word counts in a document.
Did you know? SVMs aren’t limited to classification. The scikit-learn user guide lists three uses: classification, regression (support vector regression, or SVR) and outlier detection.
How does a support vector machine work?
An SVM turns classification into an optimization problem. During training, it searches for the hyperplane that maximizes the margin while keeping each class on its own side. Because the final boundary depends only on the support vectors, the trained model stays compact, which is why scikit-learn lists memory efficiency as an advantage.
How does an SVM find the best hyperplane?
- Train on labeled points. Each point has feature values and a class label, usually written as +1 or -1.
- Search candidate hyperplanes. The solver looks for the one that classifies the training data correctly with the largest margin.
- Keep the support vectors. Points that sit beyond the margin get a weight of zero, so they drop out of the decision function.
- Classify new points. A new point gets the label of the side of the hyperplane it falls on.
Under the hood, the search is a quadratic programming problem. That is the reason training cost rises quickly with more data, which the limitations section covers.
What is a soft margin, and what does C do?
Real data rarely separates cleanly. A soft margin lets some points fall inside the margin or on the wrong side and charges a penalty for each one. The parameter C sets that penalty.
A low C gives a smoother boundary that tolerates more errors. A high C pushes the model to classify every training example correctly. If your data has a lot of noisy observations, the scikit-learn docs advise decreasing C.
How does an SVM handle more than two classes?
SVMs are built for two classes, so multiclass problems are split into several binary ones. scikit-learn’s SVC uses a one-versus-one approach, which trains one classifier for every pair of classes, or n × (n - 1) / 2 classifiers for n classes. LinearSVC uses one-versus-rest and trains one model per class.
Want to learn more about Image Recognition Software? Explore Image Recognition products.
What is the kernel trick in SVM?
Some data can’t be split by any straight line, such as one class forming a ring around another. The kernel trick handles this by implicitly mapping the points into a higher-dimensional space, where a flat hyperplane can separate them. The scikit-learn docs note that the mapping can reach a higher or even infinite-dimensional space.
The trick is that the SVM never computes the new coordinates. It only needs a kernel function, which returns how similar two points would be after the mapping. That keeps training practical even when the implied space is far too large to store.
What are the SVM kernel types?
The four primary SVM kernel types are linear, polynomial, RBF and sigmoid. RBF is the default kernel in scikit-learn’s SVC class.
| Kernel | What it computes | Parameters to tune |
|---|---|---|
| Linear | The dot product of two points, so the boundary stays a straight hyperplane | C |
| Polynomial | A polynomial of the dot product, which captures interactions between features | C, degree, gamma, coef0 |
| RBF (radial basis function) | A similarity score that falls as the distance between two points grows | C, gamma |
| Sigmoid | A tanh function applied to the dot product | C, gamma, coef0 |
Tip: SVMs aren’t scale invariant, so features measured on larger scales dominate the margin. The scikit-learn docs recommend scaling each feature to a range such as 0 to 1, or standardizing it to mean 0 and variance 1, and applying the same scaling to your test data.
What are the types of SVM?
SVMs are usually grouped by the kind of boundary they learn and the task they solve.
What is a linear SVM?
A linear SVM uses a straight hyperplane with no kernel transformation. It fits data that a straight line or flat plane can separate, or nearly separate. In scikit-learn, LinearSVC is a faster implementation for this case.
What is a non-linear (kernel) SVM?
A non-linear SVM applies a kernel such as polynomial or RBF so the boundary can curve around the data. It suits classes that overlap in complicated ways, at the cost of slower training.
What is support vector regression (SVR)?
SVR applies the same idea to predicting continuous values. Instead of a margin between classes, it fits a tube around the predictions and ignores training points that fall close enough to their targets.
What is a one-class SVM?
A one-class SVM learns what normal data looks like and flags points that fall outside it, which makes it useful for outlier and novelty detection. scikit-learn implements it as OneClassSVM.
How is an SVM different from other classification algorithms?
SVM is one of several supervised algorithms for classification. The differences show up in what each one stores and when it does its work.
- SVM vs. K-nearest neighbor (KNN): KNN stores the whole training set and compares each new point to its neighbors at prediction time. An SVM trains once and keeps only the support vectors in its decision function.
- SVM vs. logistic regression: Logistic regression estimates the probability of a class. An SVM maximizes the margin and returns a score, so getting probabilities from an SVM takes an extra calibration step.
- SVM vs. random forest: A random forest combines many decision trees and picks the class with the most votes. An SVM relies on one boundary shaped by a kernel.
For a wider view of how these algorithms fit together, see G2’s guide to machine learning models.
What are the advantages and limitations of support vector machines?
While Support Vector Machines excel at finding optimal decision boundaries in complex datasets, their computational trade-offs make them better suited for certain problems than others.
What are the advantages of SVMs?
Support Vector Machines (SVMs) offer several distinct strengths that make them a staple for high-dimensional and complex classification tasks:
- High-dimensional data: SVMs stay effective in high-dimensional spaces, including cases with more features than samples.
- Memory efficiency: The decision function uses only the support vectors, not the full training set.
- Flexibility: You can pick a built-in kernel or supply a custom one.
What are the limitations of SVMs?
Despite their strengths in high-dimensional spaces, Support Vector Machines have several practical limitations that restrict when they should be used:
- Training cost rises with data size: In scikit-learn’s libsvm-based implementation, training time scales between O(n_features × n_samples²) and O(n_features × n_samples³), depending on how well the cache is used. LinearSVC is the faster option for linear problems.
- No direct probabilities: Probability estimates come from Platt scaling, which adds an expensive cross-validation step on large datasets.
- Sensitivity to scaling and settings: You need to scale features and choose the kernel, C and gamma with care.
- Overfitting risk with many features: When features far outnumber samples, choosing the kernel and the regularization term carefully is crucial.
Where are support vector machines used?
Published work shows SVMs in several areas:
- Text categorization: In his study of text categorization with SVMs, Thorsten Joachims found that text data has more than 10,000 features, a property that suits SVMs.
- Handwritten character recognition: Cortes and Vapnik compared support-vector networks with classical learning algorithms in a benchmark study of optical character recognition.
- Human and object detection in images: Navneet Dalal and Bill Triggs fed histogram of oriented gradients features to a linear SVM to classify image windows as person or not person, in their CVPR paper. SVMs are one of several classifiers used in image recognition.
- Anomaly detection: One-class SVMs flag data that doesn’t match the training set, such as unusual financial transactions.
Which software supports support vector machines?
You can train SVMs with code libraries or inside analysis environments. Two products on G2 that support them:
Machine learning software on G2 with SVM support
-
scikit-learn
Best for: Python developers and data scientists who want ready-made classification, regression and clustering algorithms, including support vector machines, that work with NumPy and SciPy.
G2 rating: 4.8/5 -
MATLAB
Best for: Engineers and scientists who want a numeric computing environment for data analysis and algorithm development. You can train SVM classifiers with the fitcsvm function.
G2 rating: 4.5/5
These are products listed on G2 that support SVM training. Ratings reflect G2 data at the time of writing.
While code libraries like scikit-learn and MATLAB handle algorithm training and tuning, data science teams moving models into production often pair them with broader machine learning platforms such as Databricks, Amazon SageMaker, or Dataiku that support end-to-end workflows from raw data to inference.
These platforms help teams process million-row datasets without extreme compute costs, use AutoML to speed up model selection without deep tuning expertise, inspect explainability metrics to see why a prediction was made, and monitor production models for data drift and automated retraining.
How do you build a support vector machine classifier?
The workflow is similar in most tools. The steps and the code below use scikit-learn in Python.
- Split your data. Hold out a test set so you can check the model on data it hasn’t seen.
- Scale the features. Standardize or normalize each feature, fitting the scaler on the training data only.
- Choose a kernel. Start with linear for text and other high-dimensional data, or RBF (the default) for general problems. Use LinearSVC for large linear problems.
- Train the model. Fit the classifier on the training set.
- Evaluate it. Compare predictions with the test labels using precision, recall, and F1. Precision is the share of predicted positives that are correct, recall is the share of actual positives the model finds, and F1 is the harmonic mean of the two.
- Tune the hyperparameters. Search over C, gamma, and the kernel. The scikit-learn docs suggest GridSearchCV with C and gamma values spaced exponentially far apart.
Tip: If one class is much smaller than the other, set class_weight="balanced" in SVC and try different values of C.
Frequently asked questions about support vector machines
Got more questions? Here are the answers.
Q. Is SVM supervised or unsupervised learning?
Standard SVMs are supervised because they train on labeled examples. The one-class SVM is the exception: it trains without class labels, and scikit-learn lists it under unsupervised novelty and outlier detection.
Q. What is the main goal of a support vector machine?
The goal is to find the hyperplane that separates classes with the largest margin. In general, a larger margin lowers the generalization error of the classifier, according to the scikit-learn docs.
Q. When should you use a support vector machine?
Consider an SVM when you have labeled data with many features, such as text, and a dataset small enough that training time stays manageable. Avoid kernel SVMs when you have very large sample counts, or when you need probabilities without the extra calibration step.
Q. Are SVMs still used?
Yes. The current scikit-learn documentation still covers SVC, SVR and OneClassSVM. For very large datasets, linear models such as LinearSVC or SGDClassifier are the practical alternatives.
Q. Is SVM better than random forest?
Neither wins every time. SVMs handle high-dimensional data well, including cases with more features than samples, but they need scaled inputs and tuned kernels. Random forests combine many decision trees and use majority voting. Compare both with cross-validation on your own data.
Q. What is the difference between SVM and logistic regression?
Logistic regression models the probability of a class, while an SVM maximizes the margin between classes. Logistic regression returns probabilities directly. An SVM returns a score, and probabilities need an extra calibration step.
Q. What does the C parameter do in an SVM?
C sets the penalty for training points that fall inside the margin or on the wrong side. Lower C means more regularization and a smoother boundary. Higher C fits the training data more tightly and trains more slowly, according to the scikit-learn docs.
Q. What tools help teams scale and monitor machine learning models like SVMs in production?
For training and experimenting with SVMs, data scientists commonly use scikit-learn and MATLAB. When moving classification models into production on million-row datasets, end-to-end machine learning platforms such as Databricks, Amazon SageMaker, and Dataiku add AutoML tuning, prediction explainability, and automated monitoring to catch data drift before accuracy drops.
Choosing an SVM for your next model
A support vector machine classifies data by finding the hyperplane with the widest margin, and the kernel trick extends that idea to boundaries that curve. SVMs suit labeled data with many features and moderate sample counts. They need scaled inputs, tuned parameters and extra work to produce probabilities.
If you’re building image-based models, browse image recognition software on G2 to compare tools by user ratings and reviews.
This was originally written by Sagar Joshi in 2024. It has been updated with new information.
