CHECKOCCLUSIONSDSKCF_NOISEMODEL function for detecting occlusions CHECKOCCLUSIONSDSKCF_NOISEMODEL.m is the function that detects occlusions in the DSKCF tracker framework. For more information about how DSKCF handles occlusions see [1]. Please note that this function was partially built extending the RGBD tracker code presented in [2] and available under under Open Source MIT License at http://tracking.cs.princeton.edu/code.html INPUT: - depthMapCurr current depth image - noDataCurrent mask marking missing depth data - trackerDSKCF_struct DS-KCF tracker data structure - bb tracked region bounding box in the format [topLeftX, topLeftY, bottomRightX, bottomRightY] read as [columnIndexTopLeft, rowIndexTopLeft, columnIndexBottomRight, rowIndexBottomRight] OUTPUT - p fraction of pixel belonging to the occluding object - depthEstimatedd estimated mean depth of the closest object to the camera - stEstimated estimated variance depth of the closest object to the camera - depthCurr estimated mean depth value of the target (assigned even if it is not the closest object with respect to the camera) - stdNew estimated depth standard deviation of the target (assigned even if it is not the closest object with respect to the camera) - LabelReg label image of the same size as the input image. For example, LabelReg==i represents the region associated with prototype C(i), where i=[1,k] (k = number of clusters). - Centers 1-by-k array of cluster centroids. - LUTCC L-by-1 array that specifies the intensity-class relations, where L is the dynamic intensity range of the input image. Specifically, LUT(1) corresponds to class assigned to min(im(:)) and LUT(L) corresponds to the class assigned to max(im(:)). -regionIndex label of the closest object's cluster -minIndexReduced index of the clusters after area small filtering -regionIndexOBJ label of the target object's cluster See also ENLARGEBB, FASTDEPTHSEGMENTATIONDSKCF_NOISEMODEL, CALCULATENOISEVAR, ROIFROMBB, SINGLEFRAMEDSKCF [1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. Burghardt, A.Paiement, L. Tao, DS-KCF: A ~real-time tracker for RGB-D data, Journal of Real-Time Image Processing [2] Shuran Song and Jianxiong Xiao. Tracking Revisited using RGBD Camera: Baseline and Benchmark. 2013. University of Bristol Massimo Camplani and Sion Hannuna massimo.camplani@bristol.ac.uk hannuna@compsci.bristol.ac.uk
0001 function [p, depthCurr,stdNew,depthEstimated,stEstimated,... 0002 minIndexReduced,LabelReg,Centers,regionIndex,LUTCC,regionIndexOBJ] ... 0003 = checkOcclusionsDSKCF_noiseModel(depthMapCurr,noDataCurrent,... 0004 trackerDSKCF_struct, bb) 0005 0006 %CHECKOCCLUSIONSDSKCF_NOISEMODEL function for detecting occlusions 0007 % 0008 %CHECKOCCLUSIONSDSKCF_NOISEMODEL.m is the function that detects occlusions 0009 %in the DSKCF tracker framework. For more information about how DSKCF 0010 %handles occlusions see [1]. Please note that this function was partially 0011 %built extending the RGBD tracker code presented in [2] and available under 0012 %under Open Source MIT License at 0013 % http://tracking.cs.princeton.edu/code.html 0014 % 0015 % 0016 % INPUT: 0017 % - depthMapCurr current depth image 0018 % - noDataCurrent mask marking missing depth data 0019 % - trackerDSKCF_struct DS-KCF tracker data structure 0020 % - bb tracked region bounding box in the format [topLeftX, topLeftY, 0021 % bottomRightX, bottomRightY] read as [columnIndexTopLeft, 0022 % rowIndexTopLeft, columnIndexBottomRight, rowIndexBottomRight] 0023 % 0024 % OUTPUT 0025 % - p fraction of pixel belonging to the occluding object 0026 % - depthEstimatedd estimated mean depth of the closest object to the camera 0027 % - stEstimated estimated variance depth of the closest object to the camera 0028 % - depthCurr estimated mean depth value of the target (assigned even if it is not 0029 % the closest object with respect to the camera) 0030 % - stdNew estimated depth standard deviation of the target (assigned even if it is not 0031 % the closest object with respect to the camera) 0032 % - LabelReg label image of the same size as the input image. For example, 0033 % LabelReg==i represents the region associated with prototype C(i), 0034 % where i=[1,k] (k = number of clusters). 0035 % - Centers 1-by-k array of cluster centroids. 0036 % - LUTCC L-by-1 array that specifies the intensity-class relations, 0037 % where L is the dynamic intensity range of the input image. 0038 % Specifically, LUT(1) corresponds to class assigned to 0039 % min(im(:)) and LUT(L) corresponds to the class assigned to 0040 % max(im(:)). 0041 % -regionIndex label of the closest object's cluster 0042 % -minIndexReduced index of the clusters after area small filtering 0043 % -regionIndexOBJ label of the target object's cluster 0044 % See also ENLARGEBB, FASTDEPTHSEGMENTATIONDSKCF_NOISEMODEL, 0045 % CALCULATENOISEVAR, ROIFROMBB, SINGLEFRAMEDSKCF 0046 % 0047 % 0048 % [1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. 0049 % Burghardt, A.Paiement, L. Tao, DS-KCF: A ~real-time tracker for RGB-D 0050 % data, Journal of Real-Time Image Processing 0051 % 0052 % [2] Shuran Song and Jianxiong Xiao. Tracking Revisited using RGBD 0053 % Camera: Baseline and Benchmark. 2013. 0054 % 0055 % University of Bristol 0056 % Massimo Camplani and Sion Hannuna 0057 % 0058 % massimo.camplani@bristol.ac.uk 0059 % hannuna@compsci.bristol.ac.uk 0060 0061 bbPrev = trackerDSKCF_struct.previousTarget.bb; 0062 depthPrev = trackerDSKCF_struct.previousTarget.meanDepthObj; 0063 0064 p=999; 0065 depthCurr=depthPrev; 0066 0067 0068 stdOLD=trackerDSKCF_struct.previousTarget.stdDepthObj; 0069 regionIndexOBJ=0; 0070 if isempty(bb), 0071 stdNew=stdOLD; 0072 depthEstimated=depthPrev; 0073 stEstimated=stdOLD; 0074 minIndexReduced=1; 0075 LabelReg=[]; 0076 Centers=[]; 0077 regionIndex=0; 0078 LUT=[]; 0079 0080 return; 0081 end 0082 0083 bbIn=bb; 0084 bb=enlargeBB(bb ,0.05,size(depthMapCurr)); 0085 0086 %caluclate area of the current bounding box 0087 bbFinalArea=(trackerDSKCF_struct.currentTarget.w)*(trackerDSKCF_struct.currentTarget.h); 0088 0089 0090 %extract the target roi, from the depth and the nodata mask 0091 front_depth=roiFromBB(depthMapCurr,bb); 0092 depthNoData=roiFromBB(noDataCurrent,bb); 0093 0094 %hard coded quadratic noise model of the Kinect according to 0095 %M. Camplani, T. Mantecon, and L. Salgado. Depth-color fusion strategy for 0096 %3-D scene modeling with Kinect. Cybernetics, IEEE Transactions on, 0097 %43(6):1560–1571, 2013 0098 noiseModelVector=[2.3,0.00055,0.00000235]; 0099 0100 [LabelReg,Centers,LUT,H,I,LUTCC]=fastDepthSegmentationDSKCF_noiseModel... 0101 (front_depth,3,depthNoData,1,[-1,-1,-1],1,depthPrev,stdOLD,noiseModelVector); 0102 0103 %wrong segmentation....you must exit 0104 if(isempty(LabelReg)) 0105 p=0; 0106 depthCurr=depthPrev; 0107 stdNew=stdOLD; 0108 depthEstimated=depthPrev; 0109 stEstimated=stdOLD; 0110 minIndexReduced=0; 0111 regionIndex=0; 0112 return 0113 end 0114 0115 %%clean very smallRegions.... 0116 tmpProp=regionprops(LabelReg,'Area'); 0117 areaList= cat(1, tmpProp.Area); 0118 widthTarget=bbIn(4)-bbIn(2); 0119 heightTarget=bbIn(3)-bbIn(1); 0120 minArea=widthTarget*heightTarget*0.09; 0121 0122 areaSmallIndex=areaList<minArea; 0123 if(sum(areaSmallIndex)==length(areaList)) 0124 areaSmallIndex=[]; 0125 end 0126 0127 %exclude the small area index setting a super high depth!!!!!!!! 0128 %it will never be used 0129 Centers(:,areaSmallIndex)= 1000000; 0130 0131 %%%% 0132 [targetDepth,regionIndex]=min(Centers); 0133 0134 depthVector=double(front_depth(LabelReg==regionIndex)); 0135 0136 targetStd=std(depthVector); 0137 targetStd=max(2.5*calculateNoiseVar(targetDepth,noiseModelVector(1),noiseModelVector(2),noiseModelVector(3)),targetStd); 0138 if(targetStd<5) 0139 targetStd=stdOLD; 0140 end 0141 0142 0143 %find closest peak to the old depth distribution 0144 depthEstimated=[]; 0145 stEstimated=[]; 0146 0147 selectionIndex=I>-10000; 0148 %%%AGAIN THIS CONDITION SHOULD BE CHECKED.....IF IT IS REACHED SOMEHOW 0149 if isnan(depthPrev), 0150 depthCurr=targetDepth; 0151 p=0; 0152 else 0153 peakDistances=abs(Centers-depthPrev); 0154 [minDist, minIndex]=min(peakDistances); 0155 0156 %register the plane index when you filtered out some small 0157 %regions.... 0158 CentersReduced=Centers(Centers<1000000); 0159 peakDistancesReduced=abs(CentersReduced-depthPrev); 0160 [minDistReduced, minIndexReduced]=min(peakDistancesReduced); 0161 0162 depthEstimated=targetDepth; 0163 stEstimated=targetStd; 0164 stEstimated=max(2.5*calculateNoiseVar(depthEstimated,noiseModelVector(1),noiseModelVector(2),noiseModelVector(3)),stEstimated); 0165 %check first if the main mode in previous frame is the first peak of the actual distribution 0166 if((minIndexReduced==1 && minDist<3*stdOLD)) 0167 %%everything seems ok....no occluding object, just a movement 0168 %%of the object....update the depth!!! 0169 depthCurr=targetDepth; 0170 stdNew=mean([targetStd stdOLD]); 0171 selectionIndex=LUT~=LUTCC(minIndex); 0172 regionIndexOBJ=minIndex; 0173 else 0174 %%%% THERE IS AN OCCLUSION......WHAT TO DO? 0175 %%find the new corresponding region (if exist) and calculate 0176 if(minDist<2.5*stdOLD) 0177 depthCurr=Centers(minIndex); 0178 depthVector=double(front_depth(LabelReg==minIndex)); 0179 stdNew=mean([std(depthVector) stdOLD]); 0180 selectionIndex=LUT~=LUTCC(minIndex); 0181 if(stdNew<5) 0182 stdNew=stdOLD; 0183 end 0184 regionIndexOBJ=minIndex; 0185 else 0186 depthCurr=depthPrev; 0187 stdNew=stdOLD; 0188 end 0189 end 0190 0191 normN=H/bbFinalArea; 0192 %not only consider the closest points, but also remove the pixels 0193 %beloning to the target region....only in this way you can really 0194 %estimate p 0195 validIndex=I<(depthCurr-1.5*stdNew); 0196 validIndex=validIndex & selectionIndex; 0197 p=sum(normN(validIndex)); 0198 end 0199 end 0200 0201