If you want to use AI, you have to use GPU.
When using an AI model on Android, by default it will run only on the CPU.
If your device has a GPU (Graphics Processing Unit), you can significantly speed up processing by using the GPU .
For example, when running Yolov8 on a Google Pixel 7a, it runs in
0.9 seconds on the CPU
0.2 seconds on the GPU
GPUDelegate
To use the device’s GPU, set TensorFlowLite’s GPUDelegate
. Just add GPUDelegate to the options when initializing the TensorFlowLite Interpreter.
(By the way, there is also NNAPIDelegate (NeuralNetworkAPI), but this was no different from the CPU in my environment (Google Pixel 7a).)
Method
Installing the required libraries
build.gradle
implementation("org.tensorflow:tensorflow-lite:+")
implementation("org.tensorflow:tensorflow-lite-gpu:+")
implementation("org.tensorflow:tensorflow-lite-gpu-api:+")
implementation("org.tensorflow:tensorflow-lite-gpu-delegate-plugin:+")
implementation("org.tensorflow:tensorflow-lite-support:+")
It seems that the four libraries from the top may crash if the versions do not match.
If you set it to :+, the latest version will be installed, so it’s OK.
GPUDelegate settings
module import
import org.tensorflow.lite.Interpreter
import org.tensorflow.lite.gpu.CompatibilityList
import org.tensorflow.lite.gpu.GpuDelegate
Delegate settings.
Set GPUDelegate in the model’s Interpreter options.
val compatList = CompatibilityList()
val options = Interpreter.Options().apply{
if(compatList.isDelegateSupportedOnThisDevice){
// if the device has a supported GPU, add the GPU delegate
val delegateOptions = compatList.bestOptionsForThisDevice
this.addDelegate(GpuDelegate(delegateOptions))
} else {
// if the GPU is not supported, run on 4 threads
this.setNumThreads(4)
}
}
interpreter = Interpreter(model, options)
The model will now run using the GPU if your device has one available.
🐣
I’m a freelance engineer.
Work consultation
Please feel free to contact us with a brief development description.
rockyshikoku@gmail.com
I am creating applications using machine learning and AR technology.
I send machine learning / AR related information.