How to import segment anything model into matlab?

Hello,

I have played around with segment anything model by metaAI and it is great! I have been using it in python, however, I am wondering, is there a way to import it into matlab? I would like to use the matlab App desginer to design a GUI that uses the SAM

Thanks

[Mohamed Eltabbal]

Did you manage to use the segment anything ONNX model in matlab for image segmentation

Answer 1

[Surya]

Hi,

If you have knowledge of Python, you have the option to utilize the MATLAB Python Interface. You can find details on calling Python functions from MATLAB on link.

However, to use this feature, you must have a MATLAB-compatible version of Python installed on your computer. check

With the interface, you can import SAM into MATLAB.

Example: (MATLAB code)

sam = py.segment_anything.sam_model_registry["<model_type>"](checkpoint="<path/to/checkpoint>")

Make sure that you've installed "segment_anything" in Python using pip.

Answer 2

[xingxingcui]

Hi,@Layla Bitar

In Matlab R2024a, have introduce Image Processing Toolbox Model for Segment Anything Model,no dependent on Python.

Note: currently this model internal function segmentAnythingModel.m have a bug,you should modify from line 303 like follow.

if image_size(1) > image_size(2)

Iresized = imresize(I,[input_size,nan], "bilinear");

scale = input_size / size(I,1);

% Iresized = padarray(Iresized,0,input_size-size(Iresized,2),'post');

Iresized = padarray(Iresized,[0,input_size-size(Iresized,2)],0,'post'); % modify use this!

else

Iresized = imresize(I,[nan,input_size], "bilinear");

scale = input_size / size(I,2);

Iresized = padarray(Iresized,[input_size-size(Iresized,1),0],0,'post'); % modify use this!

end

Here link I give a Example.

-------------------------Off-topic interlude, 2024-------------------------------

I am currently looking for a job in the field of CV algorithm development, based in Shenzhen, Guangdong, China,or a remote support position. I would be very grateful if anyone is willing to offer me a job or make a recommendation. My preliminary resume can be found at: https://cuixing158.github.io/about/ . Thank you!

Email: [e-mail]

[France]

When the new MATLAB version will be released?

[xingxingcui]

@France ，MATLAB version a is released in March and version b in September each year.