张量流中的对象计数

Question

我有一个问题。我想在 tensorflow 中创建一个深度学习图像分类模型以进行忽略检测 就像我有一个图像，右侧为空或左侧为空或无侧为空。 我会给模型输入一个图像，模型会将图像分类为左侧情感或左侧空或无侧空。

左侧为空表示左侧没有物体 右侧空意味着右侧没有物体 no side empty 表示两边都有对象。

那个对象可以是任何东西，无论是圆形还是三角形或其他任何东西，只要它是对象。 我怎么能在张量流中做到这一点？任何指南？ 意思是如何计算图像左右两侧的物体

谢谢

Answer 1

您正在尝试做的是图像分类。在这种情况下，您有 3 个类让我们称它们为 rempty、lempty 和 noempty。现在创建如下所示形式的目录结构

image_dir
---rempty
   ----image 0  # first image where right side is empty
   ----image 1  # second image where right side is empty
   ----
   --- image N  # Nth images where right side is empty
---lempty
   ----image 0  # first image where left side is empty
   ----image 1  # second image where left side is empty
   ----
   --- image K  # Kth images where left side is empty
---noempty
   ----image 0  # first image where no side is empty
   ----image 1  # second image where no side is empty
   ----
   --- image J  # Jth images where left side is empty

所以现在我们将创建一个训练集、一个测试集和一个验证集。我们将使用数据框来创建它们

def preprocess (sdir, trsplit, vsplit, random_seed):
    filepaths=[]
    labels=[]    
    classlist=os.listdir(sdir)
    for klass in classlist:
        classpath=os.path.join(sdir,klass)
        flist=os.listdir(classpath)
        for f in flist:
            fpath=os.path.join(classpath,f)
            filepaths.append(fpath)
            labels.append(klass)
    Fseries=pd.Series(filepaths, name='filepaths')
    Lseries=pd.Series(labels, name='labels')
    df=pd.concat([Fseries, Lseries], axis=1)       
    # split df into train_df and test_df 
    dsplit=vsplit/(1-trsplit)
    strat=df['labels']    
    train_df, dummy_df=train_test_split(df, train_size=trsplit, shuffle=True, random_state=random_seed, stratify=strat)
    strat=dummy_df['labels']
    valid_df, test_df=train_test_split(dummy_df, train_size=dsplit, shuffle=True, random_state=random_seed, stratify=strat)
    print('train_df length: ', len(train_df), '  test_df length: ',len(test_df), '  valid_df length: ', len(valid_df))
    print(list(train_df['labels'].value_counts()))
    return train_df, test_df, valid_df

上述函数创建训练、测试和有效数据帧。现在调用该函数并使用它们来创建训练生成器、测试生成器和验证生成器

sdir=img_dir
train_df, test_df, valid_df= preprocess(sdir, .8,.1, 123)
img_size=(224,224) # set this to the desired image size
channels=3 # set to 3 for rgb images
length=len(test_df)
test_batch_size=sorted([int(length/n) for n in range(1,length+1) if length % n ==0 and length/n<=80],reverse=True)[0]  
test_steps=int(length/test_batch_size)
batch_size=30 # set this to the desired batch size
def scalar(img):    
    return img  # EfficientNet expects pixelsin range 0 to 255 so no scaling is required
gen=ImageDataGenerator(preprocessing_function=scalar)
train_gen=gen.flow_from_dataframe( train_df, x_col='filepaths', y_col='labels', target_size=img_size, class_mode='categorical',
                                    color_mode='rgb', shuffle=True, batch_size=batch_size)
test_gen=gen.flow_from_dataframe( test_df, x_col='filepaths', y_col='labels', target_size=img_size, class_mode='categorical',
                                    color_mode='rgb', shuffle=False, batch_size=test_batch_size)

valid_gen=gen.flow_from_dataframe( valid_df, x_col='filepaths', y_col='labels', target_size=img_size, class_mode='categorical',
                                    color_mode='rgb', shuffle=True, batch_size=batch_size)
classes=list(train_gen.class_indices.keys())
class_count=len(classes)

现在可以在 model.fit 中使用 train_gen 和 valid_gen。 test_gen 可用于 model.evaluate 或 model.predict。如果如下所示，这是一个很好的模型

base_model=tf.keras.applications.EfficientNetB2(include_top=False, weights="imagenet",input_shape=img_shape, pooling='max') 
x=base_model.output
x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
x = Dense(256, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
                bias_regularizer=regularizers.l1(0.006) ,activation='relu')(x)
x=Dropout(rate=.45, seed=123)(x)        
output=Dense(class_count, activation='softmax')(x)
model=Model(inputs=base_model.input, outputs=output)
model.compile(Adamax(lr=.001), loss='categorical_crossentropy', metrics=['accuracy']) 

现在训练模型并在测试集上评估性能

epochs=10 # set this to desired epochs
history=model.fit(x=train_gen,  epochs=epochs, verbose=1, validation_data=valid_gen,
               validation_steps=None,  shuffle=False,  initial_epoch=0)
acc=model.evaluate( test_gen, verbose=1, steps=test_steps, return_dict=False)[1]*100
msg=f'accuracy on the test set is {acc:5.2f} %'
print(msg)

你需要导入这些模块

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import backend as K
from tensorflow.keras.layers import Dense, Activation, Dropout, BatchNormalization
from tensorflow.keras.optimizers import Adam, Adamax
from tensorflow.keras.metrics import categorical_crossentropy
from tensorflow.keras import regularizers
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Model
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
import os

Answer 2

如果是分类模型，则无需担心如何检测或计算图像中的对象。这就是深度学习所做的（在您的情况下可能是神经网络）。阅读this，了解使用 tensorflow 进行图像分类。