I'm studying SVMs and wrote a demo in MATLAB (because I couldn't get a quadratic programming package to work correctly in Python). Right now it's simple and can only do linearly-separable cases (nonlinear kernels will be implemented later). This is the first MATLAB program I've written outside of a MATLAB seminar I'm taking, so any thoughts would be appreciated.
% simple SVM demo clc;clear all; % input data; y's are labels for data y = [ -1; -1; -1; -1; -1; 1; 1; 1; 1; 1; 1 ]; data = [ 0 3; 1 2; 2 1; 3 3; 5 1; 5 5; 6 5; 6 7; 7 3; 8 6; 8 1; ]; % separate positive and negative datapoints posDps = data(find(y == 1), :); negDps = data(find(y == -1), :); % space is number of dimensions (n-space) % right now, b/c being plotted in 2D, can only be 2-space space = size(data, 2); % n is number of input variables n = length(data); [ y1, y2 ] = meshgrid(y, y); [ i, j ] = meshgrid(1:n, 1:n); % generate matrices for minimization P(i, j) = y1(i, j) .* y2(i, j) .* (data(i,:) * data(j,:)'); q = -1 * ones(n, 1); % generate matrices for inequality constraint (alpha >= 0) % also can use LB parameter A = -1 * eye(n); b = zeros(n, 1); % generate matrices for equality constraint Aeq = y'; beq = [ 0 ]; % use quadprog package to generate alphas alpha = quadprog(P, q, A, b, Aeq, beq, , , , optimoptions('quadprog', 'Display', 'off')); % calculate w from alpha w = (data' .* repmat(y', [space 1])) * alpha; % finding b parameter % (y_n)(x_n * w + b) = 1 for support vector, so b = y_n - w * x_n threshold = 1e-5; svIndices = find(alpha > threshold); b = y(svIndices(1)) - data(svIndices(1),:) * w; % display points figure; hold on; scatter(posDps(:,1), posDps(:,2)); scatter(negDps(:,1), negDps(:,2)); % draw line (only 2D for now) margin = 1; domain = (min(data(:,1)) - margin):(max(data(:,1)) + margin); plot(domain, (w(1) .* domain + b)/(-w(2))); % plotting gutters plot(domain, (-1 + w(1) .* domain + b)/(-w(2)), 'g:'); plot(domain, (1 + w(1) .* domain + b)/(-w(2)), 'g:');
I'd like any feedback, but here are some specific questions I had in mind:
- For storing one-dimensional data (arrays), is there usually a preference between column and row matrices?
Is there an easy way to deal with nested matrices? I tried to combine the labels and attribute vectors like so:
data = [ -1 [ 0 3 ]; -1 [ 1 2 ]; % ... ]
and use more indices (e.g.,
data(1,2,2)) but that didn't work.
- There are a ton of different ways to create matrices, and I'm not sure if the ones I am using are the most appropriate (e.g., via literals,
- What is there to do about arbitrary imprecision? For example, I had to put in an arbitrary value
thresholdto find non-zero values because using
find(A > 0)wasn't working. (Since the values were on the magnitude of about 1e-12, this was larger than the value of
eps()so that function didn't seem very helpful.)
- Naming conventions -- is there one for MATLAB? I don't believe I've seen a consistent style.