In my experience so far, Fooocus is the best open-sourced image generation software. It gives high quality results without much extra efforts for casual use while providing lots of options to tweak for demanding users.
FOOOCUS has an easy-to-use inpainting UI that one can brush over the image to get the mask for inpainting. But sometimes I do want to quickly select a full object.
I spend some time on Saturday to play with the Gradio widgets, which is actually quite interesting that I think I will use it for future projects, to add EfficientSAM (a variant of Segment-Anything Model) to FOOOCUS.
I chose EfficientSAM for its compactness, in terms of both model and code. It should be quite intuitive to replace the segmentation model if needed. Just grep “esam” for the relevant lines. There are surely things to be done to make it faster, e.g., by caching the image features for multiple rounds of prompting. The modified version is at pppoe/FOOOCUS-SAM.
In addition to get the segmentation mask, there is a few more operations to clear and invert the mask. Uploading a mask from file is also a choice.
It is a little bit tricky to sample points from the brushed area. I posted a gist for this part to get uniformly distributed 2d points to cover the brushed area as much as possible (link to git).
Replace the background with “A table in the park”
I didn’t find an easy to use Web app to edit masks, which is a little surprising to me. Hope the open source community to have something on this in the future.