如何使用 Java 将图像提供给 android 中的 TFLite 预训练模型？

Question

我是安卓开发新手， 我需要获得一个预训练的图像分类模型并使用我选择的图像运行它， 除了加载 TFLite 模型，我还设法从图库中选择图像并从 URI 中检索图像

我还看到了有关 TFLite 模型转换的教程，并设法运行了一个将 C 度转换为 F 度的简单模型

我无法对图像执行此操作，有什么帮助吗？

用于图像选择和模型加载的 Android 代码：

import android.annotation.SuppressLint;
import android.app.Activity;
import android.content.Intent;
import android.content.res.AssetFileDescriptor;
import android.net.Uri;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;

import androidx.appcompat.app.AppCompatActivity;

import org.tensorflow.lite.Interpreter;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class MainActivity extends AppCompatActivity {

    Button btnChoose;
    ImageView imageView;
    Interpreter tflite;
    TextView textView;

    String[] classes;

    private static int GALLERY_REQUEST_CODE = 35;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        imageView = findViewById(R.id.imageView);
        textView = findViewById(R.id.textView);

        btnChoose = findViewById(R.id.btnChoose);

        btnChoose.setOnClickListener(new View.OnClickListener() {
            @SuppressLint("IntentReset")
            @Override
            public void onClick(View v) {
                Intent intent=new Intent(Intent.ACTION_PICK);
                intent.setType("image/*");
                String[] mimeTypes = {"image/jpeg", "image/png"};
                intent.putExtra(Intent.EXTRA_MIME_TYPES,mimeTypes);
                startActivityForResult(intent,GALLERY_REQUEST_CODE);
            }
        });

        try{
            tflite = new Interpreter(loadModelFile());
        } catch (Exception ex){
            ex.printStackTrace();
        }

        try{
            InputStream is = getAssets().open("labels_mobilenet_quant_v1_224.txt");
            int size = is.available();
            byte[] buffer = new byte[size];
            is.read(buffer);
            is.close();
            classes = new String(buffer).split("\n");
            textView.setText(classes[1]);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }


    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        super.onActivityResult(requestCode, resultCode, data);

        if (requestCode == GALLERY_REQUEST_CODE && resultCode == Activity.RESULT_OK && null != data) {
            Uri selectedImage = data.getData();
            imageView.setImageURI(selectedImage);
        }
    }

    // TODO:
    /*
    doInf Function
     */

    private MappedByteBuffer loadModelFile() throws IOException {
        AssetFileDescriptor fileDescriptor = this.getAssets().openFd("mobilenet_v1_1.0_224_quant.tflite");
        FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
        FileChannel fileChannel = inputStream.getChannel();
        long startOffset = fileDescriptor.getStartOffset();
        long declaredLength = fileDescriptor.getDeclaredLength();
        return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
    }

}

我之前做过的 TFLite 简单模型代码

Button btn;
TextView tvOutput;
EditText etInput;
Interpreter tflite;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    btn = findViewById(R.id.button);
    etInput = findViewById(R.id.etInput);
    tvOutput = findViewById(R.id.tvOutput);

    try{
        tflite = new Interpreter(loadModelFile());
    } catch (Exception ex){
        ex.printStackTrace();
    }

    btn.setOnClickListener(new View.OnClickListener() {
        @SuppressLint("SetTextI18n")
        @Override
        public void onClick(View v) {
            float pred = doInf(etInput.getText().toString());
            tvOutput.setText(Float.toString(pred));
        }
    });
}

public float doInf(String inputString){
    float[] inputVal = new float[1];
    inputVal[0] = Float.parseFloat(inputString);

    float[][] outputVal = new float[1][1];

    tflite.run(inputVal, outputVal);

    return outputVal[0][0];
}

private MappedByteBuffer loadModelFile() throws IOException{
    AssetFileDescriptor fileDescriptor = this.getAssets().openFd("linear.tflite");
    FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
    FileChannel fileChannel = inputStream.getChannel();
    long startOffset = fileDescriptor.getStartOffset();
    long declaredLength = fileDescriptor.getDeclaredLength();
    return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}

Answer 1

似乎您正在尝试集成 mobilenet 的图像分类模型。试试 TFLite 任务库，它只需要 5 行代码即可运行，它会自动为您封装图像处理和输出处理。请参阅instruction 和Android app example。