runDSKCF.m is a script to test the DS-KCF RGBD tracker described in [1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. Burghardt, A.Paiement, L. Tao, DS-KCF: A ~real-time tracker for RGB-D data, Journal of Real-Time Image Processing the script shows how to: -to set DS-KCF parameters -to select folder(s) containing depth and color images -to run the DS-KCF tracking core in this example the RGBD princeton data [2] (validation set) presented in is used (see the data folder of this package). The data folder structure has to be as in the example TOPFOLDER VideoSequence1 . . VideoSequenceK depth (folder Containing Depth Data) rgb (folder Containing Color Data) init.txt (tracker BB for the first frame) frames.mat* (info about frames order) framesNEW.mat* (info about frames order) . . VideoSequenceN *Please read the DS-KCF paper about depth and color depth stream alignement. framesNEW.mat is the more accurate selection of aligned depth and color frames proposed in our paper for the dataset presented in [2]. Note that to load other datasets or sequences framesNEW.mat could not be required (i.e. simple sequential images ID) About the meaning of the various parameters please read [1] and [3] or the help of WRAPPERDSKCF functions [1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. Burghardt, A.Paiement, L. Tao, DS-KCF: A ~real-time tracker for RGB-D data, Journal of Real-Time Image Processing [2] S. Song and J. Xiao. Tracking revisited using RGBD camera: Unified benchmark and baselines. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 233–240, 2013. [3] J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. High-speed tracking with kernelized correlation filters. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2015. University of Bristol Massimo Camplani and Sion Hannuna massimo.camplani@bristol.ac.uk hannuna@compsci.bristol.ac.uk
0001 % runDSKCF.m is a script to test the DS-KCF RGBD tracker described in 0002 % 0003 %[1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. Burghardt, 0004 % A.Paiement, L. Tao, DS-KCF: A ~real-time tracker for RGB-D data, Journal 0005 % of Real-Time Image Processing 0006 % 0007 % the script shows how to: 0008 % -to set DS-KCF parameters 0009 % -to select folder(s) containing depth and color images 0010 % -to run the DS-KCF tracking core 0011 % 0012 % in this example the RGBD princeton data [2] (validation set) presented in 0013 % is used (see the data folder of this package). The data folder structure 0014 % has to be as in the example 0015 % 0016 % TOPFOLDER 0017 % VideoSequence1 0018 % . 0019 % . 0020 % VideoSequenceK 0021 % depth (folder Containing Depth Data) 0022 % rgb (folder Containing Color Data) 0023 % init.txt (tracker BB for the first frame) 0024 % frames.mat* (info about frames order) 0025 % framesNEW.mat* (info about frames order) 0026 % . 0027 % . 0028 % VideoSequenceN 0029 % 0030 % *Please read the DS-KCF paper about depth and color depth stream 0031 % alignement. framesNEW.mat is the more accurate selection of aligned 0032 % depth and color frames proposed in our paper for the dataset presented 0033 % in [2]. Note that to load other datasets or sequences framesNEW.mat could 0034 % not be required (i.e. simple sequential images ID) 0035 % 0036 % About the meaning of the various parameters please read [1] and [3] or 0037 % the help of WRAPPERDSKCF functions 0038 % 0039 % [1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. 0040 % Burghardt, A.Paiement, L. Tao, DS-KCF: A ~real-time tracker for RGB-D 0041 % data, Journal of Real-Time Image Processing 0042 % 0043 % [2] S. Song and J. Xiao. Tracking revisited using RGBD camera: Unified benchmark and 0044 % baselines. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 0045 % 233–240, 2013. 0046 % 0047 % [3] J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. High-speed 0048 % tracking with kernelized correlation filters. Pattern Analysis and 0049 % Machine Intelligence, IEEE Transactions on, 2015. 0050 % 0051 % University of Bristol 0052 % Massimo Camplani and Sion Hannuna 0053 % 0054 % massimo.camplani@bristol.ac.uk 0055 % hannuna@compsci.bristol.ac.uk 0056 0057 clc 0058 clear all 0059 currentFolder=cd(); 0060 %%add the DS-KCFresults 0061 0062 dskcfPath{1}='../'; 0063 dskcfPath{2}='../functionsDepthSeg'; 0064 dskcfPath{3}='../functionsIO'; 0065 dskcfPath{4}='../functionsOcclusions'; 0066 dskcfPath{5}='../functionsScaleChange'; 0067 dskcfPath{6}='../functionsTracking'; 0068 dskcfPath{7}='../functionsShape'; 0069 0070 0071 for i=1:length(dskcfPath) 0072 cd(dskcfPath{i}); 0073 tmpPath=cd(); 0074 %addpath(genpath(tmpPath)); 0075 addpath(tmpPath); 0076 cd(currentFolder); 0077 end 0078 0079 cd(currentFolder) 0080 0081 %insert here the absolute path here you want to save your results or use 0082 %the relative path DS-KCFresults 0083 rootDestFolder=('../../DS-KCFresults'); 0084 0085 mkdir(rootDestFolder); 0086 cd(rootDestFolder); 0087 %take absolute value and create the results folder 0088 rootDestFolder=cd(); 0089 0090 0091 cd(currentFolder) 0092 0093 %now select the data folder 0094 rootSourceFolder=('../../data/ValidationSet'); 0095 cd(rootSourceFolder); 0096 rootSourceFolder=cd(); 0097 0098 0099 %select all the videos in the folder 0100 dirInfo = dir(); 0101 isDir = [dirInfo.isdir]; 0102 listAllVideos = {dirInfo(isDir).name}; 0103 listAllVideos = listAllVideos(3:end); 0104 0105 %If you don't want to precess all the video set this to false 0106 processAllVideos=true; 0107 0108 %eventually select your subset of videos 0109 if(processAllVideos==false) 0110 %insert video names manually!!!! 0111 %listVideos{1}='bear_front'; 0112 %listVideos{1}='new_ex_occ4'; 0113 %listVideos{2}='zcup_move_1'; 0114 listVideos{1}='child_no1'; 0115 %listVideos{1}='face_occ5'; 0116 else 0117 listVideos=listAllVideos; 0118 end 0119 0120 %Saving info struct saveParameters contains some flags about saving tracker 0121 %results 0122 saveParameters.savingImage=false; %save images with Bounding Box overlapped 0123 saveParameters.savingTrackBool=true; %save tracker output 0124 saveParameters.savingDSKCFParamMat=true; %save in a matlab file the parameters used 0125 saveParameters.overwriteFolderBool=false; %overwrite results folder or generate new ones 0126 saveParameters.show_visualization=false; %show the tracking results live in a matlab figure 0127 saveParameters.noBitShift=false; %for some dataset the depth data need a bitshift (see wrapperDSKCF()) 0128 0129 0130 %% SETTING TRACKER'S PARAMETERS 0131 % the struct "DSKCFparameters" is built to contains all the parameters it 0132 % will be created at the end of the section 0133 kernel_type='gaussian'; 0134 0135 %change only this flag for feature selection, the rest is automatic!!!! 0136 feature_type = 'hog_concatenate'; 0137 kernel.type = kernel_type; 0138 0139 %Different features that can be used 0140 features.rawDepth= false; 0141 features.rawColor=false; 0142 features.rawConcatenate=false; 0143 features.rawLinear=false; 0144 features.hog_color = false; 0145 features.hog_depth = false; 0146 features.hog_concatenate = false; 0147 features.hog_linear = false; 0148 0149 0150 padding = 1.5; %extra area surrounding the target 0151 lambda = 1e-4; %regularization 0152 output_sigma_factor = 0.1; %spatial bandwidth (proportional to target) 0153 0154 %Set the scale Sq in [1] 0155 scales = 0.4:0.1:2.2; 0156 0157 0158 %Note this switch is not necessary, you can eventually 0159 switch feature_type 0160 case 'rawDepth', 0161 interp_factor = 0.075; %linear interpolation factor for adaptation 0162 0163 kernel.sigma = 0.2; %gaussian kernel bandwidth 0164 0165 kernel.poly_a = 1; %polynomial kernel additive term 0166 kernel.poly_b = 7; %polynomial kernel exponent 0167 0168 features.rawDepth = true; 0169 cell_size = 1; 0170 case 'rawColor', 0171 interp_factor = 0.075; %linear interpolation factor for adaptation 0172 0173 kernel.sigma = 0.2; %gaussian kernel bandwidth 0174 0175 kernel.poly_a = 1; %polynomial kernel additive term 0176 kernel.poly_b = 7; %polynomial kernel exponent 0177 0178 features.rawColor = true; 0179 cell_size = 1; 0180 0181 case 'rawConcatenate', 0182 interp_factor = 0.075; %linear interpolation factor for adaptation 0183 0184 kernel.sigma = 0.2; %gaussian kernel bandwidth 0185 0186 kernel.poly_a = 1; %polynomial kernel additive term 0187 kernel.poly_b = 7; %polynomial kernel exponent 0188 0189 features.rawConcatenate = true; 0190 cell_size = 1; 0191 case 'rawLinear', 0192 interp_factor = 0.075; %linear interpolation factor for adaptation 0193 0194 kernel.sigma = 0.2; %gaussian kernel bandwidth 0195 0196 kernel.poly_a = 1; %polynomial kernel additive term 0197 kernel.poly_b = 7; %polynomial kernel exponent 0198 0199 features.rawLinear = true; 0200 cell_size = 1; 0201 0202 case 'hog_color' 0203 interp_factor = 0.02; 0204 0205 kernel.sigma = 0.5; 0206 0207 kernel.poly_a = 1; 0208 kernel.poly_b = 9; 0209 0210 features.hog_color = true; 0211 features.hog_orientations = 9; 0212 cell_size = 4; 0213 case 'hog_depth' 0214 interp_factor = 0.02; 0215 0216 kernel.sigma = 0.5; 0217 0218 kernel.poly_a = 1; 0219 kernel.poly_b = 9; 0220 0221 features.hog_depth = true; 0222 features.hog_orientations = 9; 0223 cell_size = 4; 0224 case 'hog_concatenate' 0225 interp_factor = 0.02; 0226 0227 kernel.sigma = 0.5; 0228 0229 kernel.poly_a = 1; 0230 kernel.poly_b = 9; 0231 0232 features.hog_concatenate = true; 0233 features.hog_orientations = 9; 0234 cell_size = 4; 0235 case 'hog_linear' 0236 interp_factor = 0.02; 0237 0238 kernel.sigma = 0.5; 0239 0240 kernel.poly_a = 1; 0241 kernel.poly_b = 9; 0242 0243 features.hog_linear = true; 0244 features.hog_orientations = 9; 0245 cell_size = 4; 0246 0247 otherwise 0248 error('Unknown feature.') 0249 end 0250 0251 %copy the parameters to the struct 0252 DSKCFparameters.features=features; %feature selection for tracking 0253 DSKCFparameters.kernel=kernel; %kernel size and type 0254 DSKCFparameters.interp_factor=interp_factor; %interpolation factor 0255 DSKCFparameters.cell_size=cell_size; %HOG parameters 0256 DSKCFparameters.padding=padding; 0257 DSKCFparameters.lambda=lambda; 0258 DSKCFparameters.output_sigma_factor=output_sigma_factor; 0259 DSKCFparameters.scales=scales; % fixed scales 0260 0261 %% PROCESSING LOOP 0262 0263 numVideo=length(listVideos); 0264 0265 % For the Princeton dataset 0266 loadFileOrder='newOrder'; 0267 %loadFileOrder='princetonMAT'; 0268 0269 %Please read the DS-KCF paper about depth and color depth stream 0270 % alignement. framesNEW.mat is the more accurate selection of aligned 0271 % depth and color frames proposed in our paper for the dataset presented 0272 % in [2]. Note that to load other datasets or sequences framesNEW.mat 0273 % could not be required (i.e. simple sequential images ID) 0274 0275 %For each selected sequence start to process!!!!!! 0276 for i=1:numVideo 0277 0278 %Generate a tmpResults folder with this name 0279 %DS-KCF_VIDEONAME_featureSelected_#ID. The folder is generated with a 0280 %different #ID if run multiple times and 0281 %saveParameters.overwriteFolderBool has been set to false. See the help 0282 %of generateFolderResults 0283 tmpDestFolder=generateFolderResults(rootDestFolder,listVideos{i},feature_type); 0284 0285 %save in the folder input parameters and the current WORKSPACE!!!!! 0286 save([tmpDestFolder '/inputParam.mat']); 0287 0288 boolPrinceton=strcmp(loadFileOrder,'princetonMAT'); 0289 boolNewOrder=strcmp(loadFileOrder,'newOrder'); 0290 0291 %Load Image information as path etc. Read also the inititial target 0292 %position 0293 0294 if(boolNewOrder) 0295 [img_files, depth_files, pos, target_sz, ground_truth, video_path, depth_path] = ... 0296 load_video_info_depthFROMMAT(rootSourceFolder, listVideos{i},1); 0297 else 0298 [img_files, depth_files, pos, target_sz, ground_truth, video_path, depth_path] = ... 0299 load_video_info_depthFROMMAT(rootSourceFolder, listVideos{i},0); 0300 end 0301 0302 %call tracker wrapper function with all the relevant parameters 0303 [dsKCFoutputSr,dsKCFoutputSq,dsKCFsegmentationOut, avTime,totalTime,timeMatrix] = ... 0304 wrapperDSKCF(video_path, depth_path,img_files, depth_files, pos, ... 0305 target_sz, DSKCFparameters,saveParameters.show_visualization,... 0306 saveParameters.savingImage,tmpDestFolder,saveParameters.noBitShift ); 0307 0308 %save tracking results and processing time 0309 save([tmpDestFolder '/procTime.mat'], 'totalTime'); 0310 save([tmpDestFolder '/procTime.txt'], 'totalTime','-ascii'); 0311 0312 save([tmpDestFolder '/procTimeFrames.mat'], 'avTime'); 0313 save([tmpDestFolder '/procTimeFrames.txt'], 'avTime','-ascii'); 0314 0315 save([tmpDestFolder '/modulesTimeFrames.mat'], 'timeMatrix'); 0316 save([tmpDestFolder '/modulesTimeFrames.txt'], 'timeMatrix','-ascii'); 0317 0318 0319 %Note we are saving the results as y x height width to be consistent 0320 %with the notation presented in the princeton RGB-D [2] 0321 0322 %Results using Sr in [1] use this for your comparison 0323 trackRes=[dsKCFoutputSr]; 0324 save([tmpDestFolder '/' listVideos{i} '.mat'], 'trackRes'); 0325 save([tmpDestFolder '/' listVideos{i} '.txt'], 'trackRes','-ascii'); 0326 0327 %Results using Sq in [1] 0328 trackResTargetSize=[dsKCFoutputSq]; 0329 save([tmpDestFolder '/' listVideos{i} 'TargetSize.mat'], 'trackResTargetSize'); 0330 save([tmpDestFolder '/' listVideos{i} 'TargetSize.txt'], 'trackResTargetSize','-ascii'); 0331 0332 %Results using segmentation 0333 save([tmpDestFolder '/' listVideos{i} 'dsKCFsegmentationOut.mat'], 'dsKCFsegmentationOut'); 0334 save([tmpDestFolder '/' listVideos{i} 'dsKCFsegmentationOut.txt'], 'dsKCFsegmentationOut','-ascii'); 0335 end