Answer by myself
I just review the webpage https://www.mathworks.com/help/optim/ug/iterations-can-violate-constraints.html . It says that iterations in fmincon active-set algorithm can violate constraints, which seems to appear in the figures above, especially the last two. So I change the algorithm from active-set to interior-point and the optimized result for Case (1)-(4) under b=2,5 and 10 are the same. The plots for 'fval' and 'constrviolation' under b=2,5 and 10 are the same, as below:
which looks more reasonable than that of 'active-set'.
Am I right by changing the algorithm to solve the problem here?