How to get the 4 pts for mouth in face parts detection????

This is the pgm for face parts detection...

img = imread('f5.jpg');
detector = buildDetector();
[bbox bbimg faces bbfaces] = detectFaceParts(detector,img,3);

figure;imshow(bbimg);

for i=1:size(bbfaces,2)
figure;imshow(bbfaces{i});
end

leye = imcrop(img,bbox(:,5:8));
figure;imshow(leye);title ('Left Eye');
imsave;
reye = imcrop(img,bbox(:,9:12));
figure;imshow(reye); title ('Right Eye');
imsave;
mouth = imcrop(img,bbox(:,13:16));
figure;imshow(mouth); title ('Mouth');
imsave;
nose = imcrop(img,bbox(:,17:20));
figure;imshow(nose); title ('Nose');
imsave;

Using this pgm m extracting the left eye, right eye, mouth & nose. from tis i wanted to find 4 coordinate points(AU1, AU2, AU3, AU4) for mouth. how do i get those 4 pts?? can anyone plz help me out..

Answer 1

[Image Analyst]

I suggest you look at detail at the function detectFaceParts().

[sudha]

function [bbox,bbX,faces,bbfaces] = detectFaceParts(detector,X,thick)

if( nargin < 3 )
  thick = 1;
end

%%%%%%%%%%%%%%%%%%%%%%%detect face %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Detect faces
bbox = step(detector.detector{5}, X);

bbsize = size(bbox);
partsNum = zeros(size(bbox,1),1);

%%%%%%%%%%%%%%%%%%%%%%%detect parts %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
nameDetector = {'LeftEye'; 'RightEye'; 'Mouth'; 'Nose'; };
mins = [[12 18]; [12 18]; [15 25]; [15 18]; ];

stdsize = detector.stdsize;

for k=1:4
if( k == 1 )
  region = [1,int32(stdsize*2/3); 1, int32(stdsize*2/3)];
elseif( k == 2 )
  region = [int32(stdsize/3),stdsize; 1, int32(stdsize*2/3)];
elseif( k == 3 )
  region = [1,stdsize; int32(stdsize/3), stdsize];
elseif( k == 4 )
  region = [int32(stdsize/5),int32(stdsize*4/5);     int32(stdsize/3),stdsize];
else
region = [1,stdsize;1,stdsize];
end

bb = zeros(bbsize);
for i=1:size(bbox,1)
XX = X(bbox(i,2):bbox(i,2)+bbox(i,4)-1,bbox(i,1):bbox(i,1)+bbox(i,3)-1,:);
XX = imresize(XX,[stdsize, stdsize]);
XX = XX(region(2,1):region(2,2),region(1,1):region(1,2),:);

b = step(detector.detector{k},XX);

if( size(b,1) > 0 )
 partsNum(i) = partsNum(i) + 1;

   if( k == 1 )
    b = sortrows(b,1);
   elseif( k == 2 )
    b = flipud(sortrows(b,1));
   elseif( k == 3 )
    b = flipud(sortrows(b,2));
   elseif( k == 4 )
    b = flipud(sortrows(b,3));
   end

   ratio = double(bbox(i,3)) / double(stdsize);
   b(1,1) = int32( ( b(1,1)-1 + region(1,1)-1 ) * ratio + 0.5 ) + bbox(i,1);
   b(1,2) = int32( ( b(1,2)-1 + region(2,1)-1 ) * ratio + 0.5 ) + bbox(i,2);
   b(1,3) = int32( b(1,3) * ratio + 0.5 );
   b(1,4) = int32( b(1,4) * ratio + 0.5 );

   bb(i,:) = b(1,:);
  end
 end
 bbox = [bbox,bb];

 p = ( sum(bb') == 0 );
 bb(p,:) = [];
end

%%%%%%%%%%%%%%%%%%%%%%%draw faces %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
bbox = [bbox,partsNum];
bbox(partsNum<=2,:)=[];

if( thick >= 0 )
t = (thick-1)/2;
t0 = -int32(ceil(t));
t1 = int32(floor(t));
else
t0 = 0;
t1 = 0;
end

bbX = X;
boxColor = [[0,255,0]; [255,0,255]; [255,0,255]; [0,255,255]; [255,255,0]; ];
for k=5:-1:1
shapeInserter = vision.ShapeInserter('BorderColor','Custom','CustomBorderColor',boxColor(k,:)); 
for i=t0:t1
bb = int32(bbox(:,(k-1)*4+1:k*4));
bb(:,1:2) = bb(:,1:2)-i;
bb(:,3:4) = bb(:,3:4)+i*2;
bbX = step(shapeInserter, bbX, bb);
end
end

%%%%%%%%%%%%%%%%%%%%%%%faces %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if( nargout > 2 )
faces = cell(size(bbox,1),1);
boxfaces = cell(size(bbox,1),1);
for i=1:size(bbox,1)
faces{i,1} = X(bbox(i,2):bbox(i,2)+bbox(i,4)-1,bbox(i,1):bbox(i,1)+bbox(i,3)-1,:);
bbfaces{i,1} = bbX(bbox(i,2):bbox(i,2)+bbox(i,4)-1,bbox(i,1):bbox(i,1)+bbox(i,3)-1,:);
end
end

This is the code for detectFaceParts(). could u plz help me out. m nt able to find it out.

[Image Analyst]

I said "I suggest you look..." - so what did you learn? I really have no idea what that code does and don't have the time to figure it out and explain it to you. Sorry but I hope you understand. You can use the debugger to step through it and understand what it does.

Answer 2

[Anand]

img = imread('f5.jpg');
mouthDetector = vision.CascadeObjectDetector('Mouth');
bbox = step(mouthDetector,I);

[sudha]

Thank you for ur help... actually m able to detect eyes,nose and mouth. i want four points for mouth(corners and outer mid points of the lips), corners of the eyes and tip of the nose. how do i get these points??

[Anand]

You would have to do some post-processing to get that information. For example, chances are the strongest corner points (using the corner function) at a symmetrical distance from the eye-ball (which can be found using imfindcircles). This is code you would need to develop yourself.