nfoldxval

PURPOSE ^

Runs n-fold cross validation on data with a given classifier.

SYNOPSIS ^

function CM=nfoldxval( data, IDX, clfInit, clfparams,types, ignoreT, fname, show )

DESCRIPTION ^

 Runs n-fold cross validation on data with a given classifier.

 Given n separate labeled data sets, trains classifier using n-1 data
 sets, test on remaining one.  Average results over all n such runs.
 Shows overall results in average confusion matrix.

 The classifier is passed in as a parameter.  For this to work the
 classifier (clf) must follow certain conventions.  The conventions are:
  1) To initialize the clf ('p' is the dimension of the data):
     clf = clfInit( p, clfparams{:} )
  2) clf must point to 2 functions for training and applying it:
     clf.funTrain    and   clf.funFwd
  3) For training the following will be called:
     clf = clf.funTrain( clf, X, Y );
  4) For testing the following will be called:
     pred = clf.funFwd( clf, Xtest );
 The format for X is nxp where there are n data points and p is their
 dimension. The format for Y is nx1.

 Given data in a cell array, to string out into single array:
  IDX = cell2mat(permute(IDX,[2 1]));
  data = cell2mat(permute(data,[2 1]));
 For a simple, small dataset, can do leave one out clf as follows:
  [n,p]=size(data); IDX=mat2cell(IDX,ones(1,n),1);
  data=mat2cell(data,ones(1,n),p);
 Overall error can be calculated via:
   er = 1-sum(diag(CM))/sum(CM(:))
 Normalized confusion matrix can be calculated via:
   CMn = CM ./ repmat( sum(CM,2), [1 size(CM,2)] );

 USAGE
  CM=nfoldxval( data, IDX, clfInit, clfparams, ...
                [types], [ignoreT], [fname], [show] )
 INPUTS
  data        - cell array of (n x p) arrays each of n samples of dim p
  IDX         - cell array of (n x 1) arrays each of n labels
  clfInit     - classifier initialization function
  clfparams   - classifier parameters
  types       - [] cell array of string labels for types
  ignoreT     - [] array of types to ignore {eg: [1 4 5]}.
  fname       - [] specify a file to save CM to, as well as image
  show        - [] will display results in figure(show)

 OUTPUTS
  CM          - confusion matrix

 EXAMPLE
  load clfData;
  %%% 2 class
  nfoldxval( data, IDX, @clfLda,{'linear'}, [],[],[],1 );   % LDA
  nfoldxval( data, IDX, @clfKnn,{4},[],[],[],2 );           % 4 kNN
  nfoldxval( data, IDX, @clfSvm,{'poly',2},[],[],[],3 );    % poly SVM
  nfoldxval( data, IDX, @clfSvm,{'rbf',2^-12},[],[],[],4 ); % rbf SVM
  nfoldxval( data, IDX, @clfDecTree,{},[],[],[],5 );        % dec. tree
  %%% multiclass
  clfparams = {@clfSvm,{'rbf',2^-12},nclasses};
  nfoldxval( data, IDX, @clfEcoc,clfparams,[],[],[],6 );    % ECOC

 See also CLFKNN, CLFLDA, CLFSVM, CLFECOC, CLFDECTREE, DEMOCLASSIFY

 Piotr's Image&Video Toolbox      Version 2.30
 Copyright 2009 Piotr Dollar.  [pdollar-at-caltech.edu]
 Please email me if you find bugs, or have suggestions or questions!
 Licensed under the Lesser GPL [see external/lgpl.txt]

CROSS-REFERENCE INFORMATION ^

This function calls: This function is called by:
Generated by m2html © 2003