CHECKOCCLUSIONSDSKCF_SECONDPLANE function to detect target candidate in the occluded area CHECKOCCLUSIONSDSKCF_SECONDPLANE.m is the function that estimate the degree of the current occlusion in the DSKCF tracker framework. For more information about how DSKCF handles occlusions see [1]. Please note that this function was partially built extending the RGBD tracker code presented in [2] and available under under Open Source MIT License at http://tracking.cs.princeton.edu/code.html INPUT: - depthMapCurr current depth image - noDataCurrent mask marking missing depth data - trackerDSKCF_struct DS-KCF tracker data structure - bb tracked region bounding box in the format [topLeftX, topLeftY, bottomRightX, bottomRightY] read as [columnIndexTopLeft, rowIndexTopLeft, columnIndexBottomRight, rowIndexBottomRight] OUTPUT - p fraction of pixel belonging to the occluding object - secondPlaneDepth estimated mean depth of the object in the second plane - secondPlaneDepthStd estimated variance depth of the object in the second plane - depthCurr estimated mean depth value of the target (assigned even if it is not the closest object with respect to the camera) - stdNew estimated depth standard deviation of the target (assigned even if it is not the closest object with respect to the camera) - LabelReg label image of the same size as the input image. For example, LabelReg==i represents the region associated with prototype C(i), where i=[1,k] (k = number of clusters). - Centers 1-by-k array of cluster centroids. - LUTCC L-by-1 array that specifies the intensity-class relations, where L is the dynamic intensity range of the input image. Specifically, LUT(1) corresponds to class assigned to min(im(:)) and LUT(L) corresponds to the class assigned to max(im(:)). -regionIndex label of the closest object's cluster -minIndexReduced index of the clusters after area small filtering See also ENLARGEBB, FASTDEPTHSEGMENTATIONDSKCF_NOISEMODEL, CALCULATENOISEVAR, ROIFROMBB, SINGLEFRAMEDSKCF [1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. Burghardt, A. Paiement, L. Tao, DS-KCF: A real-time tracker for RGB-D data, Journal of Real-Time Image Processing [2] Shuran Song and Jianxiong Xiao. Tracking Revisited using RGBD Camera: Baseline and Benchmark. 2013. University of Bristol Massimo Camplani and Sion Hannuna massimo.camplani@bristol.ac.uk hannuna@compsci.bristol.ac.uk
0001 function [p, depthCurr,stdNew,LabelReg,Centers,regionIndex,LUTCC... 0002 secondPlaneDepth,secondPlaneDepthStd] = checkOcclusionsDSKCF_secondPlane... 0003 (depthMapCurr,noDataCurrent,trackerDSKCF_struct, bb) 0004 %CHECKOCCLUSIONSDSKCF_SECONDPLANE function to detect target candidate in 0005 %the occluded area 0006 % 0007 %CHECKOCCLUSIONSDSKCF_SECONDPLANE.m is the function that estimate the 0008 %degree of the current occlusion in the DSKCF tracker framework. For more 0009 %information about how DSKCF handles occlusions see [1]. Please note that 0010 %this function was partially built extending the RGBD tracker code 0011 %presented in [2] and available under under Open Source MIT License at 0012 % http://tracking.cs.princeton.edu/code.html 0013 % 0014 % 0015 % INPUT: 0016 % - depthMapCurr current depth image 0017 % - noDataCurrent mask marking missing depth data 0018 % - trackerDSKCF_struct DS-KCF tracker data structure 0019 % - bb tracked region bounding box in the format [topLeftX, topLeftY, 0020 % bottomRightX, bottomRightY] read as [columnIndexTopLeft, 0021 % rowIndexTopLeft, columnIndexBottomRight, rowIndexBottomRight] 0022 % 0023 % OUTPUT 0024 % - p fraction of pixel belonging to the occluding object 0025 % - secondPlaneDepth estimated mean depth of the object in the second 0026 % plane 0027 % - secondPlaneDepthStd estimated variance depth of the object in the second 0028 % plane 0029 % - depthCurr estimated mean depth value of the target (assigned even if it is not 0030 % the closest object with respect to the camera) 0031 % - stdNew estimated depth standard deviation of the target (assigned even if it is not 0032 % the closest object with respect to the camera) 0033 % - LabelReg label image of the same size as the input image. For example, 0034 % LabelReg==i represents the region associated with prototype C(i), 0035 % where i=[1,k] (k = number of clusters). 0036 % - Centers 1-by-k array of cluster centroids. 0037 % - LUTCC L-by-1 array that specifies the intensity-class relations, 0038 % where L is the dynamic intensity range of the input image. 0039 % Specifically, LUT(1) corresponds to class assigned to 0040 % min(im(:)) and LUT(L) corresponds to the class assigned to 0041 % max(im(:)). 0042 % -regionIndex label of the closest object's cluster 0043 % -minIndexReduced index of the clusters after area small filtering 0044 % 0045 % See also ENLARGEBB, FASTDEPTHSEGMENTATIONDSKCF_NOISEMODEL, 0046 % CALCULATENOISEVAR, ROIFROMBB, SINGLEFRAMEDSKCF 0047 % 0048 % 0049 % [1] S. Hannuna, M. Camplani, J. Hall, M. Mirmehdi, D. Damen, T. 0050 % Burghardt, A. Paiement, L. Tao, DS-KCF: A real-time tracker for RGB-D 0051 % data, Journal of Real-Time Image Processing 0052 % 0053 % [2] Shuran Song and Jianxiong Xiao. Tracking Revisited using RGBD 0054 % Camera: Baseline and Benchmark. 2013. 0055 % 0056 % University of Bristol 0057 % Massimo Camplani and Sion Hannuna 0058 % massimo.camplani@bristol.ac.uk 0059 % hannuna@compsci.bristol.ac.uk 0060 % 0061 0062 0063 bbPrev = trackerDSKCF_struct.previousTarget.bb; 0064 depthPrev = trackerDSKCF_struct.previousTarget.meanDepthObj; 0065 0066 p=999; 0067 depthCurr=depthPrev; 0068 0069 0070 secondPlaneDepth=[]; 0071 secondPlaneDepthStd=[]; 0072 0073 stdOLD=trackerDSKCF_struct.previousTarget.stdDepthObj; 0074 if isempty(bb), 0075 stdNew=stdOLD; 0076 minIndexReduced=1; 0077 LabelReg=[]; 0078 Centers=[]; 0079 regionIndex=0; 0080 LUT=[]; 0081 return; 0082 end 0083 0084 bbIn=bb; 0085 bb=enlargeBB(bb ,0.05,size(depthMapCurr)); 0086 0087 %extract the target roi, from the depth and the nodata mask 0088 front_depth=roiFromBB(depthMapCurr,bb); 0089 depthNoData=roiFromBB(noDataCurrent,bb); 0090 0091 %hard coded quadratic noise model of the Kinect according to 0092 %M. Camplani, T. Mantecon, and L. Salgado. Depth-color fusion strategy for 0093 %3-D scene modeling with Kinect. Cybernetics, IEEE Transactions on, 0094 %43(6):1560–1571, 2013 0095 noiseModelVector=[2.3,0.00055,0.00000235]; 0096 0097 [LabelReg,Centers,LUT,H,I,LUTCC]=fastDepthSegmentationDSKCF_noiseModel... 0098 (front_depth,3,depthNoData,1,[-1,-1,-1],1,depthPrev,stdOLD,noiseModelVector); 0099 0100 %wrong segmentation....you must exit 0101 if(isempty(LabelReg)) 0102 p=0; 0103 stdNew=stdOLD; 0104 minIndexReduced=1; 0105 LabelReg=[]; 0106 Centers=[]; 0107 regionIndex=0; 0108 LUT=[]; 0109 return 0110 end 0111 0112 %%clean very smallRegions.... 0113 tmpProp=regionprops(LabelReg,'Area'); 0114 areaList= cat(1, tmpProp.Area); 0115 widthTarget=bbIn(4)-bbIn(2); 0116 heightTarget=bbIn(3)-bbIn(1); 0117 minArea=widthTarget*heightTarget*0.05; 0118 areaSmallIndex=areaList<minArea; 0119 0120 %exclude the small area index setting a super high depth!!!!!!!! 0121 %it will never be used 0122 Centers(:,areaSmallIndex)= 1000000; 0123 0124 %%%% 0125 0126 [targetDepth,regionIndex]=min(Centers); 0127 0128 depthVector=double(front_depth(LabelReg==regionIndex)); 0129 targetStd=std(depthVector); 0130 targetStd=max(2.5*calculateNoiseVar(targetDepth,noiseModelVector(1),noiseModelVector(2),noiseModelVector(3)),targetStd); 0131 %targetStd=std(depthVector); 0132 if(targetStd<5) 0133 targetStd=stdOLD; 0134 end 0135 0136 0137 selectionIndex=I>-10000; 0138 %%%AGAIN THIS CONDITION SHOULD BE CHECKED.....IF IT IS REACHED SOMEHOW 0139 if isnan(depthPrev), 0140 depthCurr=targetDepth; 0141 p=0; 0142 else 0143 peakDistances=abs(Centers-depthPrev); 0144 [minDist, minIndex]=min(peakDistances); 0145 0146 %register the plane index when you filtered out some small 0147 %regions.... 0148 CentersReduced=Centers(Centers<1000000); 0149 peakDistancesReduced=abs(CentersReduced-depthPrev); 0150 [minDistReduced, minIndexReduced]=min(peakDistancesReduced); 0151 if(isempty(minDistReduced)) 0152 minDist=200000; 0153 minIndexReduced=0; 0154 end 0155 %check firs if the main mode in previous frame is the first peak of the actual distribution 0156 if(minIndexReduced==1 && minDist<3*stdOLD) 0157 %%everything seems ok....no occluding object, just a movement 0158 %%of the object....update the depth!!! 0159 depthCurr=targetDepth; 0160 stdNew=mean([targetStd stdOLD]); 0161 selectionIndex=LUT~=LUTCC(minIndex); 0162 else 0163 0164 if(minDist<2.5*stdOLD) 0165 depthCurr=Centers(minIndex); 0166 depthVector=double(front_depth(LabelReg==minIndex)); 0167 stdNew=mean([std(depthVector) stdOLD]); 0168 if(stdNew<5) 0169 stdNew=stdOLD; 0170 end 0171 selectionIndex=LUT~=LUTCC(minIndex); 0172 else 0173 depthCurr=depthPrev; 0174 stdNew=stdOLD; 0175 0176 %%output anyway the depth of the belonging region 0177 %%especially if it is the second depth plane..... 0178 %if(LUTCC(minIndex)==2) 0179 if(minIndexReduced==2) 0180 secondPlaneDepth=Centers(minIndex); 0181 depthVector=double(front_depth(LabelReg==minIndex)); 0182 secondPlaneDepthStd=std(depthVector); 0183 if(secondPlaneDepthStd<5) 0184 secondPlaneDepthStd=stdOLD; 0185 end 0186 selectionIndex=LUT~=LUTCC(minIndex); 0187 end 0188 end 0189 end 0190 normN=H/sum(H(:)); 0191 %not only consider the closest points, but also remove the pixels 0192 %beloning to the target region....only in this way you can really 0193 %estimate p 0194 validIndex=I<(depthCurr-1.5*stdNew); 0195 %validIndex=validIndex & (LUT~=LUTCC(minIndex)); 0196 validIndex=validIndex & selectionIndex; 0197 p=sum(normN(validIndex)); 0198 end 0199 end