Woooooooo! Using the eraser model LaMa, which was able to be converted to iOS format before you knew it, on iPhone

3 min readJan 3, 2024

That eraser feature can finally be converted to iOS format.

LaMa inpainting models that were previously not available in iOS format.
The Fourier transform or something could not be converted.
It seems that conversion is now possible with an update to CoreMLTools.
I found someone on GitHub who provided a conversion script.

Thank you, Maruman-san.

Conversion method

git clone https://github.com/mallman/CoreMLaMa.git
cd CoreMLaMa
pip install -r requirements.txt
python convert_lama.py

You can get LaMa.mlpackage just by doing this.

The size is about 200MB and the execution time is about 2 seconds (iPhone13pro).
Is it the best?

By the way, the conversion script creates a wrapper model for LaMa and denormalizes the output to the pixel values of the image.

class CoreMLaMa(torch.nn.Module):
    def __init__(self, lama):
        super(CoreMLaMa, self).__init__()
        self.lama = lama

    def forward(self, image, mask):
        normalized_mask = ((mask > 0) * 1).byte()
        lama_out = self.lama(image, normalized_mask)
        output = torch.clamp(lama_out * 255, min=0, max=255)
        return output

Hmm, I’ll learn a lot.

Run on iOS

let config = MLModelConfiguration()
let model = try LaMa(configuration: config)
let input = try! LaMaInput(imageWith: inputCGImage, maskWith: maskCGImage)
let out = try! model.prediction(input: input!)
let pixelBuffer = out.output

That’s all. Just enter the input image and a black and white image where the area you want to erase is white.
The result is also displayed as an image (PixelBuffer).
Isn’t it the best?

input