Pytorch_模型转Caffe（一）解析caffemodel与prototxt

Pytorch_模型转Caffe（一）

Pytorch_模型转Caffe（一）

1.Caffe简介

2.Caffe进行目标检测任务

利用ssd进行目标检测任务，主要步骤如下（重点是模型的移植）

3.Caffe五大组件

4.caffemodel

包含了prototxt(除了solver.prototxt) 和 weights bias
prototxt 以文本的方式存储网络结构
通过创建caffe_pb2.NetParameter()对象，获取caffemodel内容

model = caffe_pb2.NetParameter()        
f = open(caffemodel_filename, \'rb\')
model.ParseFromString(f.read())

循环获取每个layer下的参数
model.layer是每层的信息

## 逐个解析prototxt 内容 但有点复杂
for i,layer in enumerate(Tarpa_model.layer):
    tops = layer.top
    bottoms = layer.bottom
    top_str = \'\'
    bottom_str =\'\'
    transform_param_str = \'\'
    data_param_str    = \'\'
    annotated_data_param_str=\'\'
    for top in layer.top:
        top_str += \'\ttop:"{}"\n\'.format(top)
    for bottom in layer.bottom:
        bottom_str += \'\tbottom:"{}"\n\'.format(bottom)
    # transform 
    if str(layer.transform_param)!=\'\':
        transform_param_str = str(layer.transform_param).split(\'\n\')
        new_str_trans =\'\'
        for item in transform_param_str:
            new_str_trans += \'\t\t\'+str(item) + \'\n\' if item!=\'\' else \'\'
        # print(new_str_trans)
        transform_param_str = \'\t\' +\'transform_param {\n\'+ new_str_trans +\'\t}\'+\'\n\'
    # data_param
    if str(layer.data_param) != \'\':
        data_param_str = str(layer.data_param).split(\'\n\')
        new_str_data_param =\'\'
        for item in data_param_str:
            new_str_data_param += \'\t\t\'+str(item) + \'\n\' if item!=\'\' else \'\'

        data_param_str = \'\t\' +\'data_param {\n\'+ new_str_data_param +\'\t}\'+\'\n\'
    # annotated_data_param
    if str(layer.annotated_data_param) != \'\':
        annotated_data_param_str = str(layer.annotated_data_param).split(\'\n\')
        new_str_annotated_data_param =\'\'
        for item in annotated_data_param_str:
            new_str_annotated_data_param += \'\t\t\'+str(item) + \'\n\' if item!=\'\' else \'\'
        annotated_data_param_str = \'\t\' +\'annotated_data_param {\n\'+ new_str_annotated_data_param +\'\t}\'+\'\n\'

解析后的部分结果

### train.prototxt 卷积层
layer {
  name: "conv1_2"
  type: "Convolution"
  bottom: "conv1_1"
  top: "conv1_2"
  param {
    lr_mult: 1.0
    decay_mult: 1.0
  }
  param {
    lr_mult: 2.0
    decay_mult: 0.0
  }
  convolution_param {
    num_output: 64
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.0
    }
  }
}

5.通过caffemodel解析train.prototxt

旨在学习了解caffemodel中的数据存储结构
采用剔除法，先保存所有layer，之后删除blobs和其他无用信息

import caffe.proto.caffe_pb2 as caffe_pb2
caffemodel_filename = src_path + \'/***.caffemodel\'
Tarpa_model = caffe_pb2.NetParameter()        
f = open(caffemodel_filename, \'rb\')
Tarpa_model.ParseFromString(f.read())
f.close()

print(Tarpa_model.name)
print(Tarpa_model.input)
# print(Tarpa_model.layer)
# print(type(Tarpa_model.layer))
f = open(\'_caffemodel_.log\',\'w\')
f.write(\'name: "{}"\'.format(Tarpa_model.name)+\'\n\')
for i,layer in enumerate(Tarpa_model.layer):
    transform_param_str = str(layer).split(\'\n\')
    new_str_trans =\'\'
    comtinue_flag = 0
    for item in transform_param_str:
        if item == \'phase: TRAIN\':
            continue
        if comtinue_flag and \'}\'in item:
            continue
        comtinue_flag = 0
        if \'blobs\' in item or \'data:\'in item or \'shape\'in item or \'dim:\'in item:
            comtinue_flag = 1
            continue
        new_str_trans += \'\t\'+str(item) + \'\n\' if item!=\'\' else \'\'
    layer_str = \'layer {\' +\'\n\'+\
    new_str_trans+\
    \'}\'+\'\n\'
    f.write(str(layer_str))
    print(i)
    # if i ==2:
    #     break
f.close()

6.caffemodel解析现存问题

在生成.prototxt后可以看出有很多split字段，暂未得到解决

layer {
	name: "data_data_0_split"
	type: "Split"
	bottom: "data"
	top: "data_data_0_split_0"
	top: "data_data_0_split_1"
	top: "data_data_0_split_2"
	top: "data_data_0_split_3"
	top: "data_data_0_split_4"
	top: "data_data_0_split_5"
	top: "data_data_0_split_6"
	top: "data_data_0_split_7"
}