参考论文：ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
作者：Xiangyu Zhang ，Xinyu Zhou，Mengxiao Lin，Jian Sun
1、论文摘要
我们引入了一个高效计算的CNN结构名字叫做shuffleNet，这个结构被设计用来解决部署算力非常有限的移动设备问题，这个新的结构使用了两个新的操作,pointwise group convolution 和 channel shuffle能够在极大减少计算量的同时保持一定的精度。我们在ImageNet classification和MS COCO目标检测数据集上做实验论证了ShuffleNet和其他的结构相比有着很好的性能。比如,相比于mobilenet，shufflenet在ImageNet 分类任务上有着更低的top-1错误率(错误率是7.8%)需要的计算量为40MFLOPs。在一个ARM-based移动设备,ShuffleNet相比于AlexNet实现了保持一定的精度的同时，实现了13A的速度。
2 Group Convolution（分组卷积）
第一个创新点就是分组1*1卷积
简单来说分组卷积就是将特征图分为不同的组，再对每组特征图分别进行卷积。这里的分组一般都是分为 n 个等份，理论上其实不是等份也可以，不 过一般为了实现方便都是分为等份。分组卷积的好处主要是可以减少模型的计算量和训练参 数，同时对模型准确率影响不大，甚至有可能会提高模型准确率。
在分组卷积中，每个卷积核只处理部分通道，比如上图中，红色卷积核只处理红色的通道，绿色卷积核只处理绿色通道，黄色卷积核只处理黄色通道。此时每个卷积核有2个通道，每个卷积核生成一张特征图。
下面我们通过几个图来详细了解一下，下图为普通卷积
图中的 Conv 表示卷积。
这里特征图的大小和卷积和的大小都不是重点内容，所以图中没有标出，我们只要能看 出 6 个特征图卷积后得到 12 个特征图就可以了。不过为了让大家理解分组卷积的计算量和 权值数量这里我们举例计算一下，假设特征图大小是 28×28，卷积核大小为 5×5，Same Padding。卷积层权值数量为 5×5×6×12+12=1812，乘法计算量为 5×5×28×28×6× 12=1411200。
下面我们看一下分组卷积，分组卷积一般都是把特征图分为 n 个等份，然后再对 n 个等 份的特征图分别卷积，这里的 n 可以人为设置，如图所示。
图中的 Conv 表示卷积。
为了跟普通卷积对比，所以这里分组卷积的例子输入也是 6 个特征图，输出也是 12 个特 征图。这里我们可以看到把 6 个特征图分为了 3 组，每组 2 个特征图，每组分别进行卷积， 卷积后得到 4 个特征图。最后再把 3 个组共 12 个特征图组合起来。假设特征图大小是 28× 28，卷积核大小为 5×5，Same Padding。这里卷积层权值数量为 5×5×2×4× 3+12=612，乘法计算量为 5×5×28×28×2×4×3=470400。权值数量和计算量都约为普通 卷积的 1/3。
分组卷积过程也可以描述如下：
1、假设输入的形状为 H × W × C ，用 k 个 h × w 的卷积核对其进行卷积操作；
2、把输入分为g组，每组形状为H × W × ( C / g )(假设可以整除)
3、把卷积核也分为g组，每组为k/g(假设可整除)个h × w 卷积核
4、按顺序，每组的输入和该组内的卷积核分别做标准卷积操作，输出 g 组形状为H ′ × W ′ × ( k / g ) ；
5、将这 g 组特征合并起来,得到最终形状为H ′ × W ′ × k 的特征;
下图为分两组时的举例：
或者看下图
左边标准卷积，每个卷积核处理12个通道
右边分组卷积，假设输入的12个通道分为3组，每个卷积核只处理4个通道
3、Channel Shuffle(通道重排)
图1
图1（a）说明了两个堆叠组卷积层的情况。很明显，某个组的输出只与组内的输入有关。此属性会阻塞通道组之间的信息流并削弱表示。
如果我们允许组卷积从不同组中获取输入数据（如图 1（b）所示），输入和输出通道将完全相关。
对于上一层生成的特征图，我们可以先将每组中的通道划分为几个子组，然后将不同的子组馈入下一层中的每个组。这可以通过通道混洗操作有效而优雅地实现（图 1（c））：假设一个卷积层具有 g 个组，其输出有 g × n 个通道；我们首先将输出通道维度重塑为 (g, n)，转置然后将其展平作为下一层的输入。请注意，即使两个卷积的组数不同，该操作仍然有效。此外，channel shuffle 也是可微分的，这意味着它可以嵌入到网络结构中进行端到端训练。
举个例子来说，如下图。分组卷积生成的三组特征图，第一组1~4；第二组5~8；第三组9~12。先将特征图重塑，为三行N列的矩形。然后进行转置，变成N行三列。最后压平，从二维tensor变成一维tensor，每一组的特征图交叉组合在一起。实现各组之间的信息交融。
4、ShuffleNet Unit
图2
图 2. ShuffleNet 单元
a) 具有深度卷积 (DWConv) [3, 12] 的残差单元 [9]；
b) 具有逐点组卷积 (GConv) 和通道混洗的 ShuffleNet 单元；
c) ShuffleNet 单元，stride = 2.
这里需要注意(b)中第一个分组卷积降维，然后通道重排，在进行3*3的DepthWise卷积，最后使用1*1的分组卷积升维，目的还是让残差前后的shape一致，保证可以进行Add操作。
©图是ShuffleNet的下采样模块。注意，这里左右分支不是使用Add操作，因为最后左右分支的shape是不一致的。右分支的通道数和左分支的通道数叠加 == 输出特征图的通道数out_channel(重点，和上面的残差是不一样的)，残差边上使用了池化窗口为3*3，stride=2的平均池化Add是逐元素求个，Concat是某个方向的叠加，这里是再通道方向叠加。
5、ShuffleNet网络结构设计
上图为不同分组数的ShuffleNet网络结构，通常我们将g=3的那一个网络作为baseline Network
我们在表 1 中展示了整体 ShuffleNet 架构。所提出的网络主要由一组 ShuffleNet 单元组成，分为三个阶段。每个阶段的第一个构建块应用 stride = 2。一个阶段内的其他超参数保持不变，下一个阶段的输出通道加倍。与 [9] 类似，我们将每个 ShuffleNet 的瓶颈通道数设置为输出通道的 1/4
上表中stage2的第一个block上不用GConv,用普通的1*1卷积，因为此时输入通道数只有24，太少了。且每个stage中的第一个block的stride=2（对应ShuffleNet Unit中的下采样模块,c图），其他block的stride=1(对应ShuffleNet基本模块，图b)
6、代码复现
这里只是简单复现下，细节问题并没有管，比如那个stage2的第一个block上不用GConv,用普通的1*1卷积就没管。
1 import tensorflow as tf 2 from tensorflow.keras.layers import concatenate, Conv2D, Activation, BatchNormalization, DepthwiseConv2D 3 from tensorflow.keras.layers import add, AvgPool2D,MaxPool2D,GlobalAveragePooling2D,Dense 4 from tensorflow.keras.models import Model 5 from plot_model import plot_model
6.1 Channel Shuffle模块
1 # 通道重排,跨组信息交互 2 def channel_shuffle(inputs, num_groups): 3 # 先得到输入特征图的shape，b:batch size，h,w:一张图的size，c:通道数 4 b, h, w, c = inputs.shape 5 6 # 确定shape = [b, h, w, num_groups, c//num_groups]。通道维度原来是一个长为c的一维tensor，变成num_groups行n列的矩阵 7 # 在通道维度上将特征图reshape为num_groups行n列的矩阵 8 x_reshaped = tf.reshape(inputs, [-1, h, w, num_groups, c // num_groups]) 9 10 # 确定转置的矩形的shape = [b, h, w, c//num_groups, num_groups] 11 # 矩阵转置，最后两个维度从num_groups行n列变成n行num_groups列 12 x_transposed = tf.transpose(x_reshaped, [0, 1, 2, 4, 3]) 13 14 # 重新排列，shotcut和x的通道像素交叉排列，通道维度重新变成一维tensor 15 output = tf.reshape(x_transposed, [-1, h, w, c]) 16 # 返回通道维度交叉排序后的tensor 17 return output
6.2 分组卷积模块
我这里tensorflow版本为2.0，我看官网API中高版本的Conv2D参数中已经有了groups属性了。
1 def group_conv(inputs, filters, kernel, strides, num_groups): 2 conv_side_layers_tmp = tf.split(inputs, num_groups, axis=3) 3 conv_side_layers = [] 4 for layer in conv_side_layers_tmp: 5 conv_side_layers.append(tf.keras.layers.Conv2D(filters // num_groups, kernel, strides, padding='same')(layer)) 6 x = concatenate(conv_side_layers, axis=-1) 7 8 return x
6.3 普通卷积模块
1 # 普通卷积：卷积+批标准化+ReLU激活
2 def conv(inputs, filters, kernel_size, stride=1):
3 x = Conv2D(filters, kernel_size, stride, padding='same', use_bias=False)(inputs)
4 x = BatchNormalization()(x)
5 x = Activation('relu')(x)
6 return x
6.4 DepthWise卷积
1 # DWConv:深度可分离卷积块(论文中DWConv卷积核全是3*3,步长有1和2两种) 2 def depthwise_conv_bn(inputs, kernel_size, stride=1): 3 x = DepthwiseConv2D(kernel_size=kernel_size, 4 strides=stride, 5 padding='same', 6 use_bias=False)(inputs) 7 x = BatchNormalization()(x) 8 return x
注意，这里没有用ReLU
6.5 ShuffleNetV1基本模块
1 # ShuffleNetV1基本模块(Add)
2 def shuffleNetUnitA(inputs, num_groups):
3 in_channels = inputs.shape[-1]
4 out_channels = in_channels
5 bottleneck_channels = out_channels // 4
6
7 # 1*1分组卷积降维
8 x = group_conv(inputs, bottleneck_channels, kernel=1, strides=1, num_groups=num_groups)
9 x = BatchNormalization()(x)
10 x = Activation('relu')(x)
11 # Channel Shuffle
12 x = channel_shuffle(x, num_groups)
13 # 3*3 DWConv
14 x = depthwise_conv_bn(x, kernel_size=3, stride=1)
15 # 1*1分组卷积升维(要保证残差连接前后的shape一致)
16 x = group_conv(x, out_channels, kernel=1, strides=1, num_groups=num_groups)
17 x = BatchNormalization()(x)
18 x = add([inputs, x])
19 x = Activation('relu')(x)
20 return x
6.6 ShuffleNetV1下采样模块
1 # ShuffleNetV1下采样模块(下采样模块，concat)
2 def shuffleNetUnitB(inputs, out_channels, num_groups):
3 in_channels = inputs.shape[-1]
4 # 右分支的通道数和左分支的通道数叠加 == 输出特征图的通道数out_channel(重点，和上面的残差是不一样的)
5 out_channels -= in_channels
6 bottleneck_channels = out_channels // 4
7 # （1）右分支
8 # 1*1 GConv
9 x = group_conv(inputs, bottleneck_channels, kernel=1, strides=1, num_groups=num_groups)
10 x = BatchNormalization()(x)
11 x = Activation('relu')(x)
12 # Channel Shuffle
13 x = channel_shuffle(x, num_groups)
14 # 3*3 DWConv,stide=2
15 x = depthwise_conv_bn(x, kernel_size=3, stride=2)
16 # 1*1 GConv
17 x = group_conv(x, out_channels, kernel=1, strides=1, num_groups=num_groups)
18 x = BatchNormalization()(x)
19
20 # (2)左分支:3*3 AVG Pool,stride=2
21 y = AvgPool2D(pool_size=3, strides=2, padding='same')(inputs)
22 # 在通道维度上堆叠
23 x = concatenate([y, x], axis=-1)
24 x = Activation('relu')(x)
25 return x
6.5和6.6的通道数需要结合论文好好看，要不你看不懂为什么会这样设计，原论文中都有解释。
6.6 stage
每个stage中的第一个block的stride=2(即下采样模块)，其他block的stride=1(即基本模块)
1 def stage(inputs, out_channels, num_groups, n): 2 # 每个stage中的第一个block的stride=2(即下采样模块)，其他block的stride=1(即基本模块) 3 # 都是按照论文搭建的，要去看论文原文，要不你绝对不理解为什么这样搭建，嘿嘿。 4 x = shuffleNetUnitB(inputs, out_channels, num_groups) 5 6 for _ in range(n): 7 x = shuffleNetUnitA(x, num_groups) 8 return x
6.7 网络搭建
1 # first_stage_channels为第一个stage的输出通道数 2 # num_groups为分组数量 3 def ShuffleNet(inputs, first_stage_channels, num_groups,num_classes): 4 # 构建网络输入tensor 5 inputs = tf.keras.Input(shape=inputs) 6 # 论文中先用了一个普通卷积和池化 7 x = Conv2D(filters=24, 8 kernel_size=3, 9 strides=2, 10 padding='same')(inputs) 11 x = MaxPool2D(pool_size=3, strides=2, padding='same')(x) 12 # 三个stage,每个stage的第一个block的stride=2 13 # 同一个stage内的其他超参数不变，下一个stage的输出通道数加倍(这个可以通过论文中的表格看出，原文也给了) 14 # n为分组卷积的分组数量，论文中用g表示 15 x = stage(x, first_stage_channels, num_groups, n=3) 16 x = stage(x, first_stage_channels * 2, num_groups, n=7) 17 x = stage(x, first_stage_channels * 4, num_groups, n=3) 18 19 x = GlobalAveragePooling2D()(x) 20 # 我看过其他大佬的文章说compile的时候再用softmax，那样更稳定，有时间再试试吧 21 x = Dense(num_classes, activation='softmax')(x) 22 23 # 完整网络架构 24 model = Model(inputs=inputs, outputs=x) 25 return model
6.8 自定义数据集测试
1 # 类别数 2 num_classes = 17 3 # 批次大小 4 batch_size = 32 5 # 周期数 6 epochs = 100 7 # 图片大小 8 image_size = 224
查看模型摘要
1 model=ShuffleNet(inputs[224,224,3],first_stage_channels=240,num_groups=3,num_classes=17) 2 model.summary()
1 Model: "functional_1" 2 __________________________________________________________________________________________________ 3 Layer (type) Output Shape Param # Connected to 4 ================================================================================================== 5 input_1 (InputLayer) [(None, 224, 224, 3) 0 6 __________________________________________________________________________________________________ 7 conv2d (Conv2D) (None, 112, 112, 24) 672 input_1[0][0] 8 __________________________________________________________________________________________________ 9 max_pooling2d (MaxPooling2D) (None, 56, 56, 24) 0 conv2d[0][0] 10 __________________________________________________________________________________________________ 11 tf_op_layer_split (TensorFlowOp [(None, 56, 56, 8), 0 max_pooling2d[0][0] 12 __________________________________________________________________________________________________ 13 conv2d_1 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][0] 14 __________________________________________________________________________________________________ 15 conv2d_2 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][1] 16 __________________________________________________________________________________________________ 17 conv2d_3 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][2] 18 __________________________________________________________________________________________________ 19 concatenate (Concatenate) (None, 56, 56, 54) 0 conv2d_1[0][0] 20 conv2d_2[0][0] 21 conv2d_3[0][0] 22 __________________________________________________________________________________________________ 23 batch_normalization (BatchNorma (None, 56, 56, 54) 216 concatenate[0][0] 24 __________________________________________________________________________________________________ 25 activation (Activation) (None, 56, 56, 54) 0 batch_normalization[0][0] 26 __________________________________________________________________________________________________ 27 tf_op_layer_Reshape (TensorFlow [(None, 56, 56, 3, 1 0 activation[0][0] 28 __________________________________________________________________________________________________ 29 tf_op_layer_Transpose (TensorFl [(None, 56, 56, 18, 0 tf_op_layer_Reshape[0][0] 30 __________________________________________________________________________________________________ 31 tf_op_layer_Reshape_1 (TensorFl [(None, 56, 56, 54)] 0 tf_op_layer_Transpose[0][0] 32 __________________________________________________________________________________________________ 33 depthwise_conv2d (DepthwiseConv (None, 28, 28, 54) 486 tf_op_layer_Reshape_1[0][0] 34 __________________________________________________________________________________________________ 35 batch_normalization_1 (BatchNor (None, 28, 28, 54) 216 depthwise_conv2d[0][0] 36 __________________________________________________________________________________________________ 37 tf_op_layer_split_1 (TensorFlow [(None, 28, 28, 18), 0 batch_normalization_1[0][0] 38 __________________________________________________________________________________________________ 39 conv2d_4 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][0] 40 __________________________________________________________________________________________________ 41 conv2d_5 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][1] 42 __________________________________________________________________________________________________ 43 conv2d_6 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][2] 44 __________________________________________________________________________________________________ 45 concatenate_1 (Concatenate) (None, 28, 28, 216) 0 conv2d_4[0][0] 46 conv2d_5[0][0] 47 conv2d_6[0][0] 48 __________________________________________________________________________________________________ 49 average_pooling2d (AveragePooli (None, 28, 28, 24) 0 max_pooling2d[0][0] 50 __________________________________________________________________________________________________ 51 batch_normalization_2 (BatchNor (None, 28, 28, 216) 864 concatenate_1[0][0] 52 __________________________________________________________________________________________________ 53 concatenate_2 (Concatenate) (None, 28, 28, 240) 0 average_pooling2d[0][0] 54 batch_normalization_2[0][0] 55 __________________________________________________________________________________________________ 56 activation_1 (Activation) (None, 28, 28, 240) 0 concatenate_2[0][0] 57 __________________________________________________________________________________________________ 58 tf_op_layer_split_2 (TensorFlow [(None, 28, 28, 80), 0 activation_1[0][0] 59 __________________________________________________________________________________________________ 60 conv2d_7 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][0] 61 __________________________________________________________________________________________________ 62 conv2d_8 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][1] 63 __________________________________________________________________________________________________ 64 conv2d_9 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][2] 65 __________________________________________________________________________________________________ 66 concatenate_3 (Concatenate) (None, 28, 28, 60) 0 conv2d_7[0][0] 67 conv2d_8[0][0] 68 conv2d_9[0][0] 69 __________________________________________________________________________________________________ 70 batch_normalization_3 (BatchNor (None, 28, 28, 60) 240 concatenate_3[0][0] 71 __________________________________________________________________________________________________ 72 activation_2 (Activation) (None, 28, 28, 60) 0 batch_normalization_3[0][0] 73 __________________________________________________________________________________________________ 74 tf_op_layer_Reshape_2 (TensorFl [(None, 28, 28, 3, 2 0 activation_2[0][0] 75 __________________________________________________________________________________________________ 76 tf_op_layer_Transpose_1 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_2[0][0] 77 __________________________________________________________________________________________________ 78 tf_op_layer_Reshape_3 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_1[0][0] 79 __________________________________________________________________________________________________ 80 depthwise_conv2d_1 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_3[0][0] 81 __________________________________________________________________________________________________ 82 batch_normalization_4 (BatchNor (None, 28, 28, 60) 240 depthwise_conv2d_1[0][0] 83 __________________________________________________________________________________________________ 84 tf_op_layer_split_3 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_4[0][0] 85 __________________________________________________________________________________________________ 86 conv2d_10 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][0] 87 __________________________________________________________________________________________________ 88 conv2d_11 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][1] 89 __________________________________________________________________________________________________ 90 conv2d_12 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][2] 91 __________________________________________________________________________________________________ 92 concatenate_4 (Concatenate) (None, 28, 28, 240) 0 conv2d_10[0][0] 93 conv2d_11[0][0] 94 conv2d_12[0][0] 95 __________________________________________________________________________________________________ 96 batch_normalization_5 (BatchNor (None, 28, 28, 240) 960 concatenate_4[0][0] 97 __________________________________________________________________________________________________ 98 add (Add) (None, 28, 28, 240) 0 activation_1[0][0] 99 batch_normalization_5[0][0] 100 __________________________________________________________________________________________________ 101 activation_3 (Activation) (None, 28, 28, 240) 0 add[0][0] 102 __________________________________________________________________________________________________ 103 tf_op_layer_split_4 (TensorFlow [(None, 28, 28, 80), 0 activation_3[0][0] 104 __________________________________________________________________________________________________ 105 conv2d_13 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][0] 106 __________________________________________________________________________________________________ 107 conv2d_14 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][1] 108 __________________________________________________________________________________________________ 109 conv2d_15 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][2] 110 __________________________________________________________________________________________________ 111 concatenate_5 (Concatenate) (None, 28, 28, 60) 0 conv2d_13[0][0] 112 conv2d_14[0][0] 113 conv2d_15[0][0] 114 __________________________________________________________________________________________________ 115 batch_normalization_6 (BatchNor (None, 28, 28, 60) 240 concatenate_5[0][0] 116 __________________________________________________________________________________________________ 117 activation_4 (Activation) (None, 28, 28, 60) 0 batch_normalization_6[0][0] 118 __________________________________________________________________________________________________ 119 tf_op_layer_Reshape_4 (TensorFl [(None, 28, 28, 3, 2 0 activation_4[0][0] 120 __________________________________________________________________________________________________ 121 tf_op_layer_Transpose_2 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_4[0][0] 122 __________________________________________________________________________________________________ 123 tf_op_layer_Reshape_5 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_2[0][0] 124 __________________________________________________________________________________________________ 125 depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_5[0][0] 126 __________________________________________________________________________________________________ 127 batch_normalization_7 (BatchNor (None, 28, 28, 60) 240 depthwise_conv2d_2[0][0] 128 __________________________________________________________________________________________________ 129 tf_op_layer_split_5 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_7[0][0] 130 __________________________________________________________________________________________________ 131 conv2d_16 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][0] 132 __________________________________________________________________________________________________ 133 conv2d_17 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][1] 134 __________________________________________________________________________________________________ 135 conv2d_18 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][2] 136 __________________________________________________________________________________________________ 137 concatenate_6 (Concatenate) (None, 28, 28, 240) 0 conv2d_16[0][0] 138 conv2d_17[0][0] 139 conv2d_18[0][0] 140 __________________________________________________________________________________________________ 141 batch_normalization_8 (BatchNor (None, 28, 28, 240) 960 concatenate_6[0][0] 142 __________________________________________________________________________________________________ 143 add_1 (Add) (None, 28, 28, 240) 0 activation_3[0][0] 144 batch_normalization_8[0][0] 145 __________________________________________________________________________________________________ 146 activation_5 (Activation) (None, 28, 28, 240) 0 add_1[0][0] 147 __________________________________________________________________________________________________ 148 tf_op_layer_split_6 (TensorFlow [(None, 28, 28, 80), 0 activation_5[0][0] 149 __________________________________________________________________________________________________ 150 conv2d_19 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][0] 151 __________________________________________________________________________________________________ 152 conv2d_20 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][1] 153 __________________________________________________________________________________________________ 154 conv2d_21 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][2] 155 __________________________________________________________________________________________________ 156 concatenate_7 (Concatenate) (None, 28, 28, 60) 0 conv2d_19[0][0] 157 conv2d_20[0][0] 158 conv2d_21[0][0] 159 __________________________________________________________________________________________________ 160 batch_normalization_9 (BatchNor (None, 28, 28, 60) 240 concatenate_7[0][0] 161 __________________________________________________________________________________________________ 162 activation_6 (Activation) (None, 28, 28, 60) 0 batch_normalization_9[0][0] 163 __________________________________________________________________________________________________ 164 tf_op_layer_Reshape_6 (TensorFl [(None, 28, 28, 3, 2 0 activation_6[0][0] 165 __________________________________________________________________________________________________ 166 tf_op_layer_Transpose_3 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_6[0][0] 167 __________________________________________________________________________________________________ 168 tf_op_layer_Reshape_7 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_3[0][0] 169 __________________________________________________________________________________________________ 170 depthwise_conv2d_3 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_7[0][0] 171 __________________________________________________________________________________________________ 172 batch_normalization_10 (BatchNo (None, 28, 28, 60) 240 depthwise_conv2d_3[0][0] 173 __________________________________________________________________________________________________ 174 tf_op_layer_split_7 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_10[0][0] 175 __________________________________________________________________________________________________ 176 conv2d_22 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][0] 177 __________________________________________________________________________________________________ 178 conv2d_23 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][1] 179 __________________________________________________________________________________________________ 180 conv2d_24 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][2] 181 __________________________________________________________________________________________________ 182 concatenate_8 (Concatenate) (None, 28, 28, 240) 0 conv2d_22[0][0] 183 conv2d_23[0][0] 184 conv2d_24[0][0] 185 __________________________________________________________________________________________________ 186 batch_normalization_11 (BatchNo (None, 28, 28, 240) 960 concatenate_8[0][0] 187 __________________________________________________________________________________________________ 188 add_2 (Add) (None, 28, 28, 240) 0 activation_5[0][0] 189 batch_normalization_11[0][0] 190 __________________________________________________________________________________________________ 191 activation_7 (Activation) (None, 28, 28, 240) 0 add_2[0][0] 192 __________________________________________________________________________________________________ 193 tf_op_layer_split_8 (TensorFlow [(None, 28, 28, 80), 0 activation_7[0][0] 194 __________________________________________________________________________________________________ 195 conv2d_25 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][0] 196 __________________________________________________________________________________________________ 197 conv2d_26 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][1] 198 __________________________________________________________________________________________________ 199 conv2d_27 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][2] 200 __________________________________________________________________________________________________ 201 concatenate_9 (Concatenate) (None, 28, 28, 60) 0 conv2d_25[0][0] 202 conv2d_26[0][0] 203 conv2d_27[0][0] 204 __________________________________________________________________________________________________ 205 batch_normalization_12 (BatchNo (None, 28, 28, 60) 240 concatenate_9[0][0] 206 __________________________________________________________________________________________________ 207 activation_8 (Activation) (None, 28, 28, 60) 0 batch_normalization_12[0][0] 208 __________________________________________________________________________________________________ 209 tf_op_layer_Reshape_8 (TensorFl [(None, 28, 28, 3, 2 0 activation_8[0][0] 210 __________________________________________________________________________________________________ 211 tf_op_layer_Transpose_4 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_8[0][0] 212 __________________________________________________________________________________________________ 213 tf_op_layer_Reshape_9 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_4[0][0] 214 __________________________________________________________________________________________________ 215 depthwise_conv2d_4 (DepthwiseCo (None, 14, 14, 60) 540 tf_op_layer_Reshape_9[0][0] 216 __________________________________________________________________________________________________ 217 batch_normalization_13 (BatchNo (None, 14, 14, 60) 240 depthwise_conv2d_4[0][0] 218 __________________________________________________________________________________________________ 219 tf_op_layer_split_9 (TensorFlow [(None, 14, 14, 20), 0 batch_normalization_13[0][0] 220 __________________________________________________________________________________________________ 221 conv2d_28 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][0] 222 __________________________________________________________________________________________________ 223 conv2d_29 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][1] 224 __________________________________________________________________________________________________ 225 conv2d_30 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][2] 226 __________________________________________________________________________________________________ 227 concatenate_10 (Concatenate) (None, 14, 14, 240) 0 conv2d_28[0][0] 228 conv2d_29[0][0] 229 conv2d_30[0][0] 230 __________________________________________________________________________________________________ 231 average_pooling2d_1 (AveragePoo (None, 14, 14, 240) 0 activation_7[0][0] 232 __________________________________________________________________________________________________ 233 batch_normalization_14 (BatchNo (None, 14, 14, 240) 960 concatenate_10[0][0] 234 __________________________________________________________________________________________________ 235 concatenate_11 (Concatenate) (None, 14, 14, 480) 0 average_pooling2d_1[0][0] 236 batch_normalization_14[0][0] 237 __________________________________________________________________________________________________ 238 activation_9 (Activation) (None, 14, 14, 480) 0 concatenate_11[0][0] 239 __________________________________________________________________________________________________ 240 tf_op_layer_split_10 (TensorFlo [(None, 14, 14, 160) 0 activation_9[0][0] 241 __________________________________________________________________________________________________ 242 conv2d_31 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][0] 243 __________________________________________________________________________________________________ 244 conv2d_32 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][1] 245 __________________________________________________________________________________________________ 246 conv2d_33 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][2] 247 __________________________________________________________________________________________________ 248 concatenate_12 (Concatenate) (None, 14, 14, 120) 0 conv2d_31[0][0] 249 conv2d_32[0][0] 250 conv2d_33[0][0] 251 __________________________________________________________________________________________________ 252 batch_normalization_15 (BatchNo (None, 14, 14, 120) 480 concatenate_12[0][0] 253 __________________________________________________________________________________________________ 254 activation_10 (Activation) (None, 14, 14, 120) 0 batch_normalization_15[0][0] 255 __________________________________________________________________________________________________ 256 tf_op_layer_Reshape_10 (TensorF [(None, 14, 14, 3, 4 0 activation_10[0][0] 257 __________________________________________________________________________________________________ 258 tf_op_layer_Transpose_5 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_10[0][0] 259 __________________________________________________________________________________________________ 260 tf_op_layer_Reshape_11 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_5[0][0] 261 __________________________________________________________________________________________________ 262 depthwise_conv2d_5 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_11[0][0] 263 __________________________________________________________________________________________________ 264 batch_normalization_16 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_5[0][0] 265 __________________________________________________________________________________________________ 266 tf_op_layer_split_11 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_16[0][0] 267 __________________________________________________________________________________________________ 268 conv2d_34 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][0] 269 __________________________________________________________________________________________________ 270 conv2d_35 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][1] 271 __________________________________________________________________________________________________ 272 conv2d_36 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][2] 273 __________________________________________________________________________________________________ 274 concatenate_13 (Concatenate) (None, 14, 14, 480) 0 conv2d_34[0][0] 275 conv2d_35[0][0] 276 conv2d_36[0][0] 277 __________________________________________________________________________________________________ 278 batch_normalization_17 (BatchNo (None, 14, 14, 480) 1920 concatenate_13[0][0] 279 __________________________________________________________________________________________________ 280 add_3 (Add) (None, 14, 14, 480) 0 activation_9[0][0] 281 batch_normalization_17[0][0] 282 __________________________________________________________________________________________________ 283 activation_11 (Activation) (None, 14, 14, 480) 0 add_3[0][0] 284 __________________________________________________________________________________________________ 285 tf_op_layer_split_12 (TensorFlo [(None, 14, 14, 160) 0 activation_11[0][0] 286 __________________________________________________________________________________________________ 287 conv2d_37 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][0] 288 __________________________________________________________________________________________________ 289 conv2d_38 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][1] 290 __________________________________________________________________________________________________ 291 conv2d_39 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][2] 292 __________________________________________________________________________________________________ 293 concatenate_14 (Concatenate) (None, 14, 14, 120) 0 conv2d_37[0][0] 294 conv2d_38[0][0] 295 conv2d_39[0][0] 296 __________________________________________________________________________________________________ 297 batch_normalization_18 (BatchNo (None, 14, 14, 120) 480 concatenate_14[0][0] 298 __________________________________________________________________________________________________ 299 activation_12 (Activation) (None, 14, 14, 120) 0 batch_normalization_18[0][0] 300 __________________________________________________________________________________________________ 301 tf_op_layer_Reshape_12 (TensorF [(None, 14, 14, 3, 4 0 activation_12[0][0] 302 __________________________________________________________________________________________________ 303 tf_op_layer_Transpose_6 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_12[0][0] 304 __________________________________________________________________________________________________ 305 tf_op_layer_Reshape_13 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_6[0][0] 306 __________________________________________________________________________________________________ 307 depthwise_conv2d_6 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_13[0][0] 308 __________________________________________________________________________________________________ 309 batch_normalization_19 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_6[0][0] 310 __________________________________________________________________________________________________ 311 tf_op_layer_split_13 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_19[0][0] 312 __________________________________________________________________________________________________ 313 conv2d_40 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][0] 314 __________________________________________________________________________________________________ 315 conv2d_41 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][1] 316 __________________________________________________________________________________________________ 317 conv2d_42 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][2] 318 __________________________________________________________________________________________________ 319 concatenate_15 (Concatenate) (None, 14, 14, 480) 0 conv2d_40[0][0] 320 conv2d_41[0][0] 321 conv2d_42[0][0] 322 __________________________________________________________________________________________________ 323 batch_normalization_20 (BatchNo (None, 14, 14, 480) 1920 concatenate_15[0][0] 324 __________________________________________________________________________________________________ 325 add_4 (Add) (None, 14, 14, 480) 0 activation_11[0][0] 326 batch_normalization_20[0][0] 327 __________________________________________________________________________________________________ 328 activation_13 (Activation) (None, 14, 14, 480) 0 add_4[0][0] 329 __________________________________________________________________________________________________ 330 tf_op_layer_split_14 (TensorFlo [(None, 14, 14, 160) 0 activation_13[0][0] 331 __________________________________________________________________________________________________ 332 conv2d_43 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][0] 333 __________________________________________________________________________________________________ 334 conv2d_44 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][1] 335 __________________________________________________________________________________________________ 336 conv2d_45 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][2] 337 __________________________________________________________________________________________________ 338 concatenate_16 (Concatenate) (None, 14, 14, 120) 0 conv2d_43[0][0] 339 conv2d_44[0][0] 340 conv2d_45[0][0] 341 __________________________________________________________________________________________________ 342 batch_normalization_21 (BatchNo (None, 14, 14, 120) 480 concatenate_16[0][0] 343 __________________________________________________________________________________________________ 344 activation_14 (Activation) (None, 14, 14, 120) 0 batch_normalization_21[0][0] 345 __________________________________________________________________________________________________ 346 tf_op_layer_Reshape_14 (TensorF [(None, 14, 14, 3, 4 0 activation_14[0][0] 347 __________________________________________________________________________________________________ 348 tf_op_layer_Transpose_7 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_14[0][0] 349 __________________________________________________________________________________________________ 350 tf_op_layer_Reshape_15 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_7[0][0] 351 __________________________________________________________________________________________________ 352 depthwise_conv2d_7 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_15[0][0] 353 __________________________________________________________________________________________________ 354 batch_normalization_22 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_7[0][0] 355 __________________________________________________________________________________________________ 356 tf_op_layer_split_15 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_22[0][0] 357 __________________________________________________________________________________________________ 358 conv2d_46 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][0] 359 __________________________________________________________________________________________________ 360 conv2d_47 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][1] 361 __________________________________________________________________________________________________ 362 conv2d_48 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][2] 363 __________________________________________________________________________________________________ 364 concatenate_17 (Concatenate) (None, 14, 14, 480) 0 conv2d_46[0][0] 365 conv2d_47[0][0] 366 conv2d_48[0][0] 367 __________________________________________________________________________________________________ 368 batch_normalization_23 (BatchNo (None, 14, 14, 480) 1920 concatenate_17[0][0] 369 __________________________________________________________________________________________________ 370 add_5 (Add) (None, 14, 14, 480) 0 activation_13[0][0] 371 batch_normalization_23[0][0] 372 __________________________________________________________________________________________________ 373 activation_15 (Activation) (None, 14, 14, 480) 0 add_5[0][0] 374 __________________________________________________________________________________________________ 375 tf_op_layer_split_16 (TensorFlo [(None, 14, 14, 160) 0 activation_15[0][0] 376 __________________________________________________________________________________________________ 377 conv2d_49 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][0] 378 __________________________________________________________________________________________________ 379 conv2d_50 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][1] 380 __________________________________________________________________________________________________ 381 conv2d_51 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][2] 382 __________________________________________________________________________________________________ 383 concatenate_18 (Concatenate) (None, 14, 14, 120) 0 conv2d_49[0][0] 384 conv2d_50[0][0] 385 conv2d_51[0][0] 386 __________________________________________________________________________________________________ 387 batch_normalization_24 (BatchNo (None, 14, 14, 120) 480 concatenate_18[0][0] 388 __________________________________________________________________________________________________ 389 activation_16 (Activation) (None, 14, 14, 120) 0 batch_normalization_24[0][0] 390 __________________________________________________________________________________________________ 391 tf_op_layer_Reshape_16 (TensorF [(None, 14, 14, 3, 4 0 activation_16[0][0] 392 __________________________________________________________________________________________________ 393 tf_op_layer_Transpose_8 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_16[0][0] 394 __________________________________________________________________________________________________ 395 tf_op_layer_Reshape_17 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_8[0][0] 396 __________________________________________________________________________________________________ 397 depthwise_conv2d_8 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_17[0][0] 398 __________________________________________________________________________________________________ 399 batch_normalization_25 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_8[0][0] 400 __________________________________________________________________________________________________ 401 tf_op_layer_split_17 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_25[0][0] 402 __________________________________________________________________________________________________ 403 conv2d_52 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][0] 404 __________________________________________________________________________________________________ 405 conv2d_53 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][1] 406 __________________________________________________________________________________________________ 407 conv2d_54 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][2] 408 __________________________________________________________________________________________________ 409 concatenate_19 (Concatenate) (None, 14, 14, 480) 0 conv2d_52[0][0] 410 conv2d_53[0][0] 411 conv2d_54[0][0] 412 __________________________________________________________________________________________________ 413 batch_normalization_26 (BatchNo (None, 14, 14, 480) 1920 concatenate_19[0][0] 414 __________________________________________________________________________________________________ 415 add_6 (Add) (None, 14, 14, 480) 0 activation_15[0][0] 416 batch_normalization_26[0][0] 417 __________________________________________________________________________________________________ 418 activation_17 (Activation) (None, 14, 14, 480) 0 add_6[0][0] 419 __________________________________________________________________________________________________ 420 tf_op_layer_split_18 (TensorFlo [(None, 14, 14, 160) 0 activation_17[0][0] 421 __________________________________________________________________________________________________ 422 conv2d_55 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][0] 423 __________________________________________________________________________________________________ 424 conv2d_56 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][1] 425 __________________________________________________________________________________________________ 426 conv2d_57 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][2] 427 __________________________________________________________________________________________________ 428 concatenate_20 (Concatenate) (None, 14, 14, 120) 0 conv2d_55[0][0] 429 conv2d_56[0][0] 430 conv2d_57[0][0] 431 __________________________________________________________________________________________________ 432 batch_normalization_27 (BatchNo (None, 14, 14, 120) 480 concatenate_20[0][0] 433 __________________________________________________________________________________________________ 434 activation_18 (Activation) (None, 14, 14, 120) 0 batch_normalization_27[0][0] 435 __________________________________________________________________________________________________ 436 tf_op_layer_Reshape_18 (TensorF [(None, 14, 14, 3, 4 0 activation_18[0][0] 437 __________________________________________________________________________________________________ 438 tf_op_layer_Transpose_9 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_18[0][0] 439 __________________________________________________________________________________________________ 440 tf_op_layer_Reshape_19 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_9[0][0] 441 __________________________________________________________________________________________________ 442 depthwise_conv2d_9 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_19[0][0] 443 __________________________________________________________________________________________________ 444 batch_normalization_28 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_9[0][0] 445 __________________________________________________________________________________________________ 446 tf_op_layer_split_19 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_28[0][0] 447 __________________________________________________________________________________________________ 448 conv2d_58 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][0] 449 __________________________________________________________________________________________________ 450 conv2d_59 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][1] 451 __________________________________________________________________________________________________ 452 conv2d_60 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][2] 453 __________________________________________________________________________________________________ 454 concatenate_21 (Concatenate) (None, 14, 14, 480) 0 conv2d_58[0][0] 455 conv2d_59[0][0] 456 conv2d_60[0][0] 457 __________________________________________________________________________________________________ 458 batch_normalization_29 (BatchNo (None, 14, 14, 480) 1920 concatenate_21[0][0] 459 __________________________________________________________________________________________________ 460 add_7 (Add) (None, 14, 14, 480) 0 activation_17[0][0] 461 batch_normalization_29[0][0] 462 __________________________________________________________________________________________________ 463 activation_19 (Activation) (None, 14, 14, 480) 0 add_7[0][0] 464 __________________________________________________________________________________________________ 465 tf_op_layer_split_20 (TensorFlo [(None, 14, 14, 160) 0 activation_19[0][0] 466 __________________________________________________________________________________________________ 467 conv2d_61 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][0] 468 __________________________________________________________________________________________________ 469 conv2d_62 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][1] 470 __________________________________________________________________________________________________ 471 conv2d_63 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][2] 472 __________________________________________________________________________________________________ 473 concatenate_22 (Concatenate) (None, 14, 14, 120) 0 conv2d_61[0][0] 474 conv2d_62[0][0] 475 conv2d_63[0][0] 476 __________________________________________________________________________________________________ 477 batch_normalization_30 (BatchNo (None, 14, 14, 120) 480 concatenate_22[0][0] 478 __________________________________________________________________________________________________ 479 activation_20 (Activation) (None, 14, 14, 120) 0 batch_normalization_30[0][0] 480 __________________________________________________________________________________________________ 481 tf_op_layer_Reshape_20 (TensorF [(None, 14, 14, 3, 4 0 activation_20[0][0] 482 __________________________________________________________________________________________________ 483 tf_op_layer_Transpose_10 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_20[0][0] 484 __________________________________________________________________________________________________ 485 tf_op_layer_Reshape_21 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_10[0][0] 486 __________________________________________________________________________________________________ 487 depthwise_conv2d_10 (DepthwiseC (None, 14, 14, 120) 1080 tf_op_layer_Reshape_21[0][0] 488 __________________________________________________________________________________________________ 489 batch_normalization_31 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_10[0][0] 490 __________________________________________________________________________________________________ 491 tf_op_layer_split_21 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_31[0][0] 492 __________________________________________________________________________________________________ 493 conv2d_64 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][0] 494 __________________________________________________________________________________________________ 495 conv2d_65 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][1] 496 __________________________________________________________________________________________________ 497 conv2d_66 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][2] 498 __________________________________________________________________________________________________ 499 concatenate_23 (Concatenate) (None, 14, 14, 480) 0 conv2d_64[0][0] 500 conv2d_65[0][0] 501 conv2d_66[0][0] 502 __________________________________________________________________________________________________ 503 batch_normalization_32 (BatchNo (None, 14, 14, 480) 1920 concatenate_23[0][0] 504 __________________________________________________________________________________________________ 505 add_8 (Add) (None, 14, 14, 480) 0 activation_19[0][0] 506 batch_normalization_32[0][0] 507 __________________________________________________________________________________________________ 508 activation_21 (Activation) (None, 14, 14, 480) 0 add_8[0][0] 509 __________________________________________________________________________________________________ 510 tf_op_layer_split_22 (TensorFlo [(None, 14, 14, 160) 0 activation_21[0][0] 511 __________________________________________________________________________________________________ 512 conv2d_67 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][0] 513 __________________________________________________________________________________________________ 514 conv2d_68 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][1] 515 __________________________________________________________________________________________________ 516 conv2d_69 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][2] 517 __________________________________________________________________________________________________ 518 concatenate_24 (Concatenate) (None, 14, 14, 120) 0 conv2d_67[0][0] 519 conv2d_68[0][0] 520 conv2d_69[0][0] 521 __________________________________________________________________________________________________ 522 batch_normalization_33 (BatchNo (None, 14, 14, 120) 480 concatenate_24[0][0] 523 __________________________________________________________________________________________________ 524 activation_22 (Activation) (None, 14, 14, 120) 0 batch_normalization_33[0][0] 525 __________________________________________________________________________________________________ 526 tf_op_layer_Reshape_22 (TensorF [(None, 14, 14, 3, 4 0 activation_22[0][0] 527 __________________________________________________________________________________________________ 528 tf_op_layer_Transpose_11 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_22[0][0] 529 __________________________________________________________________________________________________ 530 tf_op_layer_Reshape_23 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_11[0][0] 531 __________________________________________________________________________________________________ 532 depthwise_conv2d_11 (DepthwiseC (None, 14, 14, 120) 1080 tf_op_layer_Reshape_23[0][0] 533 __________________________________________________________________________________________________ 534 batch_normalization_34 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_11[0][0] 535 __________________________________________________________________________________________________ 536 tf_op_layer_split_23 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_34[0][0] 537 __________________________________________________________________________________________________ 538 conv2d_70 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][0] 539 __________________________________________________________________________________________________ 540 conv2d_71 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][1] 541 __________________________________________________________________________________________________ 542 conv2d_72 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][2] 543 __________________________________________________________________________________________________ 544 concatenate_25 (Concatenate) (None, 14, 14, 480) 0 conv2d_70[0][0] 545 conv2d_71[0][0] 546 conv2d_72[0][0] 547 __________________________________________________________________________________________________ 548 batch_normalization_35 (BatchNo (None, 14, 14, 480) 1920 concatenate_25[0][0] 549 __________________________________________________________________________________________________ 550 add_9 (Add) (None, 14, 14, 480) 0 activation_21[0][0] 551 batch_normalization_35[0][0] 552 __________________________________________________________________________________________________ 553 activation_23 (Activation) (None, 14, 14, 480) 0 add_9[0][0] 554 __________________________________________________________________________________________________ 555 tf_op_layer_split_24 (TensorFlo [(None, 14, 14, 160) 0 activation_23[0][0] 556 __________________________________________________________________________________________________ 557 conv2d_73 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][0] 558 __________________________________________________________________________________________________ 559 conv2d_74 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][1] 560 __________________________________________________________________________________________________ 561 conv2d_75 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][2] 562 __________________________________________________________________________________________________ 563 concatenate_26 (Concatenate) (None, 14, 14, 120) 0 conv2d_73[0][0] 564 conv2d_74[0][0] 565 conv2d_75[0][0] 566 __________________________________________________________________________________________________ 567 batch_normalization_36 (BatchNo (None, 14, 14, 120) 480 concatenate_26[0][0] 568 __________________________________________________________________________________________________ 569 activation_24 (Activation) (None, 14, 14, 120) 0 batch_normalization_36[0][0] 570 __________________________________________________________________________________________________ 571 tf_op_layer_Reshape_24 (TensorF [(None, 14, 14, 3, 4 0 activation_24[0][0] 572 __________________________________________________________________________________________________ 573 tf_op_layer_Transpose_12 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_24[0][0] 574 __________________________________________________________________________________________________ 575 tf_op_layer_Reshape_25 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_12[0][0] 576 __________________________________________________________________________________________________ 577 depthwise_conv2d_12 (DepthwiseC (None, 7, 7, 120) 1080 tf_op_layer_Reshape_25[0][0] 578 __________________________________________________________________________________________________ 579 batch_normalization_37 (BatchNo (None, 7, 7, 120) 480 depthwise_conv2d_12[0][0] 580 __________________________________________________________________________________________________ 581 tf_op_layer_split_25 (TensorFlo [(None, 7, 7, 40), ( 0 batch_normalization_37[0][0] 582 __________________________________________________________________________________________________ 583 conv2d_76 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][0] 584 __________________________________________________________________________________________________ 585 conv2d_77 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][1] 586 __________________________________________________________________________________________________ 587 conv2d_78 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][2] 588 __________________________________________________________________________________________________ 589 concatenate_27 (Concatenate) (None, 7, 7, 480) 0 conv2d_76[0][0] 590 conv2d_77[0][0] 591 conv2d_78[0][0] 592 __________________________________________________________________________________________________ 593 average_pooling2d_2 (AveragePoo (None, 7, 7, 480) 0 activation_23[0][0] 594 __________________________________________________________________________________________________ 595 batch_normalization_38 (BatchNo (None, 7, 7, 480) 1920 concatenate_27[0][0] 596 __________________________________________________________________________________________________ 597 concatenate_28 (Concatenate) (None, 7, 7, 960) 0 average_pooling2d_2[0][0] 598 batch_normalization_38[0][0] 599 __________________________________________________________________________________________________ 600 activation_25 (Activation) (None, 7, 7, 960) 0 concatenate_28[0][0] 601 __________________________________________________________________________________________________ 602 tf_op_layer_split_26 (TensorFlo [(None, 7, 7, 320), 0 activation_25[0][0] 603 __________________________________________________________________________________________________ 604 conv2d_79 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][0] 605 __________________________________________________________________________________________________ 606 conv2d_80 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][1] 607 __________________________________________________________________________________________________ 608 conv2d_81 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][2] 609 __________________________________________________________________________________________________ 610 concatenate_29 (Concatenate) (None, 7, 7, 240) 0 conv2d_79[0][0] 611 conv2d_80[0][0] 612 conv2d_81[0][0] 613 __________________________________________________________________________________________________ 614 batch_normalization_39 (BatchNo (None, 7, 7, 240) 960 concatenate_29[0][0] 615 __________________________________________________________________________________________________ 616 activation_26 (Activation) (None, 7, 7, 240) 0 batch_normalization_39[0][0] 617 __________________________________________________________________________________________________ 618 tf_op_layer_Reshape_26 (TensorF [(None, 7, 7, 3, 80) 0 activation_26[0][0] 619 __________________________________________________________________________________________________ 620 tf_op_layer_Transpose_13 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_26[0][0] 621 __________________________________________________________________________________________________ 622 tf_op_layer_Reshape_27 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_13[0][0] 623 __________________________________________________________________________________________________ 624 depthwise_conv2d_13 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_27[0][0] 625 __________________________________________________________________________________________________ 626 batch_normalization_40 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_13[0][0] 627 __________________________________________________________________________________________________ 628 tf_op_layer_split_27 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_40[0][0] 629 __________________________________________________________________________________________________ 630 conv2d_82 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][0] 631 __________________________________________________________________________________________________ 632 conv2d_83 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][1] 633 __________________________________________________________________________________________________ 634 conv2d_84 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][2] 635 __________________________________________________________________________________________________ 636 concatenate_30 (Concatenate) (None, 7, 7, 960) 0 conv2d_82[0][0] 637 conv2d_83[0][0] 638 conv2d_84[0][0] 639 __________________________________________________________________________________________________ 640 batch_normalization_41 (BatchNo (None, 7, 7, 960) 3840 concatenate_30[0][0] 641 __________________________________________________________________________________________________ 642 add_10 (Add) (None, 7, 7, 960) 0 activation_25[0][0] 643 batch_normalization_41[0][0] 644 __________________________________________________________________________________________________ 645 activation_27 (Activation) (None, 7, 7, 960) 0 add_10[0][0] 646 __________________________________________________________________________________________________ 647 tf_op_layer_split_28 (TensorFlo [(None, 7, 7, 320), 0 activation_27[0][0] 648 __________________________________________________________________________________________________ 649 conv2d_85 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][0] 650 __________________________________________________________________________________________________ 651 conv2d_86 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][1] 652 __________________________________________________________________________________________________ 653 conv2d_87 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][2] 654 __________________________________________________________________________________________________ 655 concatenate_31 (Concatenate) (None, 7, 7, 240) 0 conv2d_85[0][0] 656 conv2d_86[0][0] 657 conv2d_87[0][0] 658 __________________________________________________________________________________________________ 659 batch_normalization_42 (BatchNo (None, 7, 7, 240) 960 concatenate_31[0][0] 660 __________________________________________________________________________________________________ 661 activation_28 (Activation) (None, 7, 7, 240) 0 batch_normalization_42[0][0] 662 __________________________________________________________________________________________________ 663 tf_op_layer_Reshape_28 (TensorF [(None, 7, 7, 3, 80) 0 activation_28[0][0] 664 __________________________________________________________________________________________________ 665 tf_op_layer_Transpose_14 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_28[0][0] 666 __________________________________________________________________________________________________ 667 tf_op_layer_Reshape_29 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_14[0][0] 668 __________________________________________________________________________________________________ 669 depthwise_conv2d_14 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_29[0][0] 670 __________________________________________________________________________________________________ 671 batch_normalization_43 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_14[0][0] 672 __________________________________________________________________________________________________ 673 tf_op_layer_split_29 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_43[0][0] 674 __________________________________________________________________________________________________ 675 conv2d_88 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][0] 676 __________________________________________________________________________________________________ 677 conv2d_89 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][1] 678 __________________________________________________________________________________________________ 679 conv2d_90 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][2] 680 __________________________________________________________________________________________________ 681 concatenate_32 (Concatenate) (None, 7, 7, 960) 0 conv2d_88[0][0] 682 conv2d_89[0][0] 683 conv2d_90[0][0] 684 __________________________________________________________________________________________________ 685 batch_normalization_44 (BatchNo (None, 7, 7, 960) 3840 concatenate_32[0][0] 686 __________________________________________________________________________________________________ 687 add_11 (Add) (None, 7, 7, 960) 0 activation_27[0][0] 688 batch_normalization_44[0][0] 689 __________________________________________________________________________________________________ 690 activation_29 (Activation) (None, 7, 7, 960) 0 add_11[0][0] 691 __________________________________________________________________________________________________ 692 tf_op_layer_split_30 (TensorFlo [(None, 7, 7, 320), 0 activation_29[0][0] 693 __________________________________________________________________________________________________ 694 conv2d_91 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][0] 695 __________________________________________________________________________________________________ 696 conv2d_92 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][1] 697 __________________________________________________________________________________________________ 698 conv2d_93 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][2] 699 __________________________________________________________________________________________________ 700 concatenate_33 (Concatenate) (None, 7, 7, 240) 0 conv2d_91[0][0] 701 conv2d_92[0][0] 702 conv2d_93[0][0] 703 __________________________________________________________________________________________________ 704 batch_normalization_45 (BatchNo (None, 7, 7, 240) 960 concatenate_33[0][0] 705 __________________________________________________________________________________________________ 706 activation_30 (Activation) (None, 7, 7, 240) 0 batch_normalization_45[0][0] 707 __________________________________________________________________________________________________ 708 tf_op_layer_Reshape_30 (TensorF [(None, 7, 7, 3, 80) 0 activation_30[0][0] 709 __________________________________________________________________________________________________ 710 tf_op_layer_Transpose_15 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_30[0][0] 711 __________________________________________________________________________________________________ 712 tf_op_layer_Reshape_31 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_15[0][0] 713 __________________________________________________________________________________________________ 714 depthwise_conv2d_15 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_31[0][0] 715 __________________________________________________________________________________________________ 716 batch_normalization_46 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_15[0][0] 717 __________________________________________________________________________________________________ 718 tf_op_layer_split_31 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_46[0][0] 719 __________________________________________________________________________________________________ 720 conv2d_94 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][0] 721 __________________________________________________________________________________________________ 722 conv2d_95 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][1] 723 __________________________________________________________________________________________________ 724 conv2d_96 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][2] 725 __________________________________________________________________________________________________ 726 concatenate_34 (Concatenate) (None, 7, 7, 960) 0 conv2d_94[0][0] 727 conv2d_95[0][0] 728 conv2d_96[0][0] 729 __________________________________________________________________________________________________ 730 batch_normalization_47 (BatchNo (None, 7, 7, 960) 3840 concatenate_34[0][0] 731 __________________________________________________________________________________________________ 732 add_12 (Add) (None, 7, 7, 960) 0 activation_29[0][0] 733 batch_normalization_47[0][0] 734 __________________________________________________________________________________________________ 735 activation_31 (Activation) (None, 7, 7, 960) 0 add_12[0][0] 736 __________________________________________________________________________________________________ 737 global_average_pooling2d (Globa (None, 960) 0 activation_31[0][0] 738 __________________________________________________________________________________________________ 739 dense (Dense) (None, 17) 16337 global_average_pooling2d[0][0] 740 ================================================================================================== 741 Total params: 902,741 742 Trainable params: 879,053 743 Non-trainable params: 23,688
数据增强
1 # 训练集数据进行数据增强 2 train_datagen = ImageDataGenerator( 3 rotation_range=20, # 随机旋转度数 4 width_shift_range=0.1, # 随机水平平移 5 height_shift_range=0.1, # 随机竖直平移 6 rescale=1 / 255, # 数据归一化 7 shear_range=10, # 随机错切变换 8 zoom_range=0.1, # 随机放大 9 horizontal_flip=True, # 水平翻转 10 brightness_range=(0.7, 1.3), # 亮度变化 11 fill_mode='nearest', # 填充方式 12 ) 13 # 测试集数据只需要归一化就可以 14 test_datagen = ImageDataGenerator( 15 rescale=1 / 255, # 数据归一化 16 )
数据生成器
1 # 训练集数据生成器，可以在训练时自动产生数据进行训练 2 # 从'data/train'获得训练集数据 3 # 获得数据后会把图片resize为image_size×image_size的大小 4 # generator每次会产生batch_size个数据 5 train_generator = train_datagen.flow_from_directory( 6 '../data/train', 7 target_size=(image_size, image_size), 8 batch_size=batch_size, 9 ) 10 11 # 测试集数据生成器 12 test_generator = test_datagen.flow_from_directory( 13 '../data/test', 14 target_size=(image_size, image_size), 15 batch_size=batch_size, 16 ) 17 # 字典的键为17个文件夹的名字，值为对应的分类编号 18 print(train_generator.class_indices)
回调设置
1 # 学习率调节函数，逐渐减小学习率
2 def adjust_learning_rate(epoch):
3 # 前40周期
4 if epoch<=40:
5 lr = 1e-4
6 # 前40到80周期
7 elif epoch>40 and epoch<=80:
8 lr = 1e-5
9 # 80到100周期
10 else:
11 lr = 1e-6
12 return lr
13
14 # 定义优化器
15 adam = Adam(lr=1e-4)
16
17 # 读取模型
18 checkpoint_save_path = "./checkpoint/ShuffleNetV1.ckpt"
19 if os.path.exists(checkpoint_save_path + '.index'):
20 print('-------------load the model-----------------')
21 model.load_weights(checkpoint_save_path)
22 # 保存模型
23 cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path,
24 save_weights_only=True,
25 save_best_only=True)
26
27 # 定义学习率衰减策略
28 callbacks = []
29 callbacks.append(LearningRateScheduler(adjust_learning_rate))
30 callbacks.append(cp_callback)
1 # 定义优化器，loss function，训练过程中计算准确率 2 model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy']) 3 4 # Tensorflow2.1版本(包括2.1)之后可以直接使用fit训练模型 5 history = model.fit(x=train_generator,epochs=epochs,validation_data=test_generator,callbacks=callbacks)
1 Epoch 1/100 2 34/34 [==============================] - 18s 526ms/step - loss: 2.9157 - accuracy: 0.1544 - val_loss: 2.8353 - val_accuracy: 0.0588 3 Epoch 2/100 4 34/34 [==============================] - 16s 478ms/step - loss: 2.2678 - accuracy: 0.2822 - val_loss: 2.8387 - val_accuracy: 0.0588 5 Epoch 3/100 6 34/34 [==============================] - 16s 483ms/step - loss: 1.9987 - accuracy: 0.3778 - val_loss: 2.8482 - val_accuracy: 0.0588 7 Epoch 4/100 8 34/34 [==============================] - 16s 481ms/step - loss: 1.8269 - accuracy: 0.4200 - val_loss: 2.8607 - val_accuracy: 0.0625 9 Epoch 5/100 10 34/34 [==============================] - 17s 487ms/step - loss: 1.7022 - accuracy: 0.4651 - val_loss: 2.8800 - val_accuracy: 0.0588 11 Epoch 6/100 12 34/34 [==============================] - 16s 479ms/step - loss: 1.5436 - accuracy: 0.5138 - val_loss: 2.9101 - val_accuracy: 0.0588 13 Epoch 7/100 14 34/34 [==============================] - 16s 484ms/step - loss: 1.4806 - accuracy: 0.5221 - val_loss: 2.9455 - val_accuracy: 0.0588 15 Epoch 8/100 16 34/34 [==============================] - 17s 486ms/step - loss: 1.3658 - accuracy: 0.5699 - val_loss: 3.0113 - val_accuracy: 0.0588 17 Epoch 9/100 18 34/34 [==============================] - 16s 478ms/step - loss: 1.3244 - accuracy: 0.5772 - val_loss: 3.0792 - val_accuracy: 0.0588 19 Epoch 10/100 20 34/34 [==============================] - 16s 480ms/step - loss: 1.2401 - accuracy: 0.6029 - val_loss: 3.2077 - val_accuracy: 0.0588 21 Epoch 11/100 22 34/34 [==============================] - 16s 480ms/step - loss: 1.1940 - accuracy: 0.6048 - val_loss: 3.3061 - val_accuracy: 0.0588 23 Epoch 12/100 24 34/34 [==============================] - 16s 480ms/step - loss: 1.1689 - accuracy: 0.6213 - val_loss: 3.4508 - val_accuracy: 0.0772 25 Epoch 13/100 26 34/34 [==============================] - 16s 481ms/step - loss: 1.1282 - accuracy: 0.6241 - val_loss: 3.5842 - val_accuracy: 0.0919 27 Epoch 14/100 28 34/34 [==============================] - 16s 479ms/step - loss: 1.0759 - accuracy: 0.6572 - val_loss: 3.5697 - val_accuracy: 0.1140 29 Epoch 15/100 30 34/34 [==============================] - 17s 488ms/step - loss: 1.0264 - accuracy: 0.6618 - val_loss: 3.6231 - val_accuracy: 0.1176 31 Epoch 16/100 32 34/34 [==============================] - 16s 475ms/step - loss: 1.0210 - accuracy: 0.6719 - val_loss: 3.3065 - val_accuracy: 0.1471 33 Epoch 17/100 34 34/34 [==============================] - 16s 477ms/step - loss: 0.9543 - accuracy: 0.6939 - val_loss: 2.9929 - val_accuracy: 0.2169 35 Epoch 18/100 36 34/34 [==============================] - 17s 493ms/step - loss: 0.9405 - accuracy: 0.6939 - val_loss: 2.3133 - val_accuracy: 0.2941 37 Epoch 19/100 38 34/34 [==============================] - 17s 488ms/step - loss: 0.9069 - accuracy: 0.7031 - val_loss: 1.8180 - val_accuracy: 0.4081 39 Epoch 20/100 40 34/34 [==============================] - 17s 491ms/step - loss: 0.8797 - accuracy: 0.7050 - val_loss: 1.7820 - val_accuracy: 0.4926 41 Epoch 21/100 42 34/34 [==============================] - 17s 493ms/step - loss: 0.8911 - accuracy: 0.7215 - val_loss: 1.4993 - val_accuracy: 0.5625 43 Epoch 22/100 44 34/34 [==============================] - 17s 496ms/step - loss: 0.8480 - accuracy: 0.7215 - val_loss: 1.4438 - val_accuracy: 0.5699 45 Epoch 23/100 46 34/34 [==============================] - 17s 496ms/step - loss: 0.7861 - accuracy: 0.7472 - val_loss: 1.2822 - val_accuracy: 0.6103 47 Epoch 24/100 48 34/34 [==============================] - 16s 477ms/step - loss: 0.8111 - accuracy: 0.7307 - val_loss: 1.7404 - val_accuracy: 0.5404 49 Epoch 25/100 50 34/34 [==============================] - 16s 477ms/step - loss: 0.7855 - accuracy: 0.7316 - val_loss: 1.4184 - val_accuracy: 0.6287 51 Epoch 26/100 52 34/34 [==============================] - 17s 491ms/step - loss: 0.7679 - accuracy: 0.7555 - val_loss: 1.2750 - val_accuracy: 0.6213 53 Epoch 27/100 54 34/34 [==============================] - 16s 481ms/step - loss: 0.7451 - accuracy: 0.7454 - val_loss: 1.4109 - val_accuracy: 0.5882 55 Epoch 28/100 56 34/34 [==============================] - 16s 482ms/step - loss: 0.7201 - accuracy: 0.7408 - val_loss: 1.3238 - val_accuracy: 0.6507 57 Epoch 29/100 58 34/34 [==============================] - 16s 475ms/step - loss: 0.7238 - accuracy: 0.7629 - val_loss: 1.2997 - val_accuracy: 0.6360 59 Epoch 30/100 60 34/34 [==============================] - 16s 482ms/step - loss: 0.6587 - accuracy: 0.7849 - val_loss: 1.2941 - val_accuracy: 0.6471 61 Epoch 31/100 62 34/34 [==============================] - 17s 488ms/step - loss: 0.6508 - accuracy: 0.7858 - val_loss: 1.4063 - val_accuracy: 0.6397 63 Epoch 32/100 64 34/34 [==============================] - 17s 495ms/step - loss: 0.6294 - accuracy: 0.8006 - val_loss: 1.1997 - val_accuracy: 0.6691 65 Epoch 33/100 66 34/34 [==============================] - 16s 482ms/step - loss: 0.6579 - accuracy: 0.7665 - val_loss: 1.2146 - val_accuracy: 0.6434 67 Epoch 34/100 68 34/34 [==============================] - 16s 481ms/step - loss: 0.6207 - accuracy: 0.7941 - val_loss: 1.3010 - val_accuracy: 0.6360 69 Epoch 35/100 70 34/34 [==============================] - 16s 480ms/step - loss: 0.5873 - accuracy: 0.8033 - val_loss: 1.2448 - val_accuracy: 0.6618 71 Epoch 36/100 72 34/34 [==============================] - 16s 477ms/step - loss: 0.5755 - accuracy: 0.7941 - val_loss: 1.2584 - val_accuracy: 0.6434 73 Epoch 37/100 74 34/34 [==============================] - 17s 491ms/step - loss: 0.5668 - accuracy: 0.8189 - val_loss: 1.1914 - val_accuracy: 0.6728 75 Epoch 38/100 76 34/34 [==============================] - 16s 485ms/step - loss: 0.5527 - accuracy: 0.8107 - val_loss: 1.2917 - val_accuracy: 0.6434 77 Epoch 39/100 78 34/34 [==============================] - 16s 475ms/step - loss: 0.5715 - accuracy: 0.8088 - val_loss: 1.2613 - val_accuracy: 0.6581 79 Epoch 40/100 80 34/34 [==============================] - 16s 477ms/step - loss: 0.5286 - accuracy: 0.8272 - val_loss: 1.2632 - val_accuracy: 0.6581 81 Epoch 41/100 82 34/34 [==============================] - 17s 495ms/step - loss: 0.5098 - accuracy: 0.8327 - val_loss: 1.1116 - val_accuracy: 0.6985 83 Epoch 42/100 84 34/34 [==============================] - 17s 493ms/step - loss: 0.4543 - accuracy: 0.8428 - val_loss: 1.0116 - val_accuracy: 0.7206 85 Epoch 43/100 86 34/34 [==============================] - 17s 492ms/step - loss: 0.4145 - accuracy: 0.8695 - val_loss: 0.9978 - val_accuracy: 0.7206 87 Epoch 44/100 88 34/34 [==============================] - 17s 489ms/step - loss: 0.4421 - accuracy: 0.8612 - val_loss: 0.9956 - val_accuracy: 0.7169 89 Epoch 45/100 90 34/34 [==============================] - 17s 486ms/step - loss: 0.4015 - accuracy: 0.8750 - val_loss: 0.9964 - val_accuracy: 0.7206 91 Epoch 46/100 92 34/34 [==============================] - 16s 476ms/step - loss: 0.3937 - accuracy: 0.8805 - val_loss: 1.0131 - val_accuracy: 0.7279 93 Epoch 47/100 94 34/34 [==============================] - 16s 477ms/step - loss: 0.4492 - accuracy: 0.8539 - val_loss: 1.0172 - val_accuracy: 0.7243 95 Epoch 48/100 96 34/34 [==============================] - 16s 483ms/step - loss: 0.4238 - accuracy: 0.8649 - val_loss: 1.0143 - val_accuracy: 0.7279 97 Epoch 49/100 98 34/34 [==============================] - 16s 480ms/step - loss: 0.4192 - accuracy: 0.8658 - val_loss: 1.0148 - val_accuracy: 0.7206 99 Epoch 50/100 100 34/34 [==============================] - 16s 479ms/step - loss: 0.4071 - accuracy: 0.8695 - val_loss: 1.0148 - val_accuracy: 0.7316 101 Epoch 51/100 102 34/34 [==============================] - 16s 483ms/step - loss: 0.4115 - accuracy: 0.8557 - val_loss: 1.0180 - val_accuracy: 0.7463 103 Epoch 52/100 104 34/34 [==============================] - 16s 480ms/step - loss: 0.4106 - accuracy: 0.8704 - val_loss: 1.0214 - val_accuracy: 0.7353 105 Epoch 53/100 106 34/34 [==============================] - 16s 477ms/step - loss: 0.4026 - accuracy: 0.8713 - val_loss: 1.0171 - val_accuracy: 0.7390 107 Epoch 54/100 108 34/34 [==============================] - 16s 476ms/step - loss: 0.4073 - accuracy: 0.8750 - val_loss: 1.0067 - val_accuracy: 0.7426 109 Epoch 55/100 110 34/34 [==============================] - 16s 485ms/step - loss: 0.3608 - accuracy: 0.8842 - val_loss: 1.0039 - val_accuracy: 0.7500 111 Epoch 56/100 112 34/34 [==============================] - 16s 478ms/step - loss: 0.4101 - accuracy: 0.8658 - val_loss: 1.0056 - val_accuracy: 0.7426 113 Epoch 57/100 114 34/34 [==============================] - 16s 478ms/step - loss: 0.3797 - accuracy: 0.8787 - val_loss: 0.9983 - val_accuracy: 0.7426 115 Epoch 58/100 116 34/34 [==============================] - 16s 484ms/step - loss: 0.4077 - accuracy: 0.8695 - val_loss: 1.0055 - val_accuracy: 0.7390 117 Epoch 59/100 118 34/34 [==============================] - 16s 474ms/step - loss: 0.4265 - accuracy: 0.8649 - val_loss: 1.0111 - val_accuracy: 0.7500 119 Epoch 60/100 120 34/34 [==============================] - 17s 492ms/step - loss: 0.3475 - accuracy: 0.8961 - val_loss: 0.9925 - val_accuracy: 0.7500 121 Epoch 61/100 122 34/34 [==============================] - 17s 497ms/step - loss: 0.3883 - accuracy: 0.8796 - val_loss: 0.9693 - val_accuracy: 0.7390 123 Epoch 62/100 124 34/34 [==============================] - 17s 494ms/step - loss: 0.3615 - accuracy: 0.8934 - val_loss: 0.9608 - val_accuracy: 0.7463 125 Epoch 63/100 126 34/34 [==============================] - 16s 480ms/step - loss: 0.4017 - accuracy: 0.8695 - val_loss: 0.9706 - val_accuracy: 0.7610 127 Epoch 64/100 128 34/34 [==============================] - 17s 495ms/step - loss: 0.3727 - accuracy: 0.8851 - val_loss: 0.9587 - val_accuracy: 0.7574 129 Epoch 65/100 130 34/34 [==============================] - 16s 480ms/step - loss: 0.4053 - accuracy: 0.8750 - val_loss: 0.9707 - val_accuracy: 0.7390 131 Epoch 66/100 132 34/34 [==============================] - 16s 481ms/step - loss: 0.3722 - accuracy: 0.8860 - val_loss: 0.9836 - val_accuracy: 0.7426 133 Epoch 67/100 134 34/34 [==============================] - 16s 477ms/step - loss: 0.3841 - accuracy: 0.8824 - val_loss: 0.9597 - val_accuracy: 0.7426 135 Epoch 68/100 136 34/34 [==============================] - 16s 480ms/step - loss: 0.4085 - accuracy: 0.8621 - val_loss: 0.9813 - val_accuracy: 0.7390 137 Epoch 69/100 138 34/34 [==============================] - 16s 479ms/step - loss: 0.3578 - accuracy: 0.8943 - val_loss: 0.9673 - val_accuracy: 0.7463 139 Epoch 70/100 140 34/34 [==============================] - 16s 478ms/step - loss: 0.3628 - accuracy: 0.8915 - val_loss: 0.9789 - val_accuracy: 0.7353 141 Epoch 71/100 142 34/34 [==============================] - 17s 486ms/step - loss: 0.3904 - accuracy: 0.8686 - val_loss: 0.9738 - val_accuracy: 0.7537 143 Epoch 72/100 144 34/34 [==============================] - 16s 484ms/step - loss: 0.3585 - accuracy: 0.8943 - val_loss: 0.9827 - val_accuracy: 0.7574 145 Epoch 73/100 146 34/34 [==============================] - 16s 479ms/step - loss: 0.3593 - accuracy: 0.8879 - val_loss: 0.9722 - val_accuracy: 0.7574 147 Epoch 74/100 148 34/34 [==============================] - 16s 481ms/step - loss: 0.3636 - accuracy: 0.8869 - val_loss: 0.9719 - val_accuracy: 0.7500 149 Epoch 75/100 150 34/34 [==============================] - 16s 479ms/step - loss: 0.3776 - accuracy: 0.8787 - val_loss: 0.9690 - val_accuracy: 0.7500 151 Epoch 76/100 152 34/34 [==============================] - 17s 490ms/step - loss: 0.3815 - accuracy: 0.8768 - val_loss: 0.9569 - val_accuracy: 0.7574 153 Epoch 77/100 154 34/34 [==============================] - 16s 480ms/step - loss: 0.3608 - accuracy: 0.8833 - val_loss: 0.9737 - val_accuracy: 0.7537 155 Epoch 78/100 156 34/34 [==============================] - 17s 496ms/step - loss: 0.3801 - accuracy: 0.8722 - val_loss: 0.9393 - val_accuracy: 0.7610 157 Epoch 79/100 158 34/34 [==============================] - 16s 478ms/step - loss: 0.3385 - accuracy: 0.8943 - val_loss: 0.9577 - val_accuracy: 0.7500 159 Epoch 80/100 160 34/34 [==============================] - 16s 477ms/step - loss: 0.3565 - accuracy: 0.8869 - val_loss: 0.9693 - val_accuracy: 0.7537 161 Epoch 81/100 162 34/34 [==============================] - 16s 482ms/step - loss: 0.3614 - accuracy: 0.8961 - val_loss: 0.9911 - val_accuracy: 0.7574 163 Epoch 82/100 164 34/34 [==============================] - 16s 478ms/step - loss: 0.3581 - accuracy: 0.8814 - val_loss: 0.9848 - val_accuracy: 0.7574 165 Epoch 83/100 166 34/34 [==============================] - 16s 479ms/step - loss: 0.3880 - accuracy: 0.8833 - val_loss: 0.9779 - val_accuracy: 0.7574 167 Epoch 84/100 168 34/34 [==============================] - 16s 478ms/step - loss: 0.3549 - accuracy: 0.8851 - val_loss: 0.9744 - val_accuracy: 0.7537 169 Epoch 85/100 170 34/34 [==============================] - 17s 487ms/step - loss: 0.3552 - accuracy: 0.8934 - val_loss: 0.9736 - val_accuracy: 0.7537 171 Epoch 86/100 172 34/34 [==============================] - 16s 483ms/step - loss: 0.3681 - accuracy: 0.8842 - val_loss: 0.9715 - val_accuracy: 0.7537 173 Epoch 87/100 174 34/34 [==============================] - 16s 479ms/step - loss: 0.3416 - accuracy: 0.8934 - val_loss: 0.9662 - val_accuracy: 0.7537 175 Epoch 88/100 176 34/34 [==============================] - 17s 486ms/step - loss: 0.3562 - accuracy: 0.8934 - val_loss: 0.9662 - val_accuracy: 0.7537 177 Epoch 89/100 178 34/34 [==============================] - 17s 485ms/step - loss: 0.3682 - accuracy: 0.8851 - val_loss: 0.9636 - val_accuracy: 0.7500 179 Epoch 90/100 180 34/34 [==============================] - 16s 480ms/step - loss: 0.3739 - accuracy: 0.8787 - val_loss: 0.9614 - val_accuracy: 0.7500 181 Epoch 91/100 182 34/34 [==============================] - 16s 483ms/step - loss: 0.3442 - accuracy: 0.8943 - val_loss: 0.9631 - val_accuracy: 0.7500 183 Epoch 92/100 184 34/34 [==============================] - 16s 478ms/step - loss: 0.3598 - accuracy: 0.8915 - val_loss: 0.9624 - val_accuracy: 0.7500 185 Epoch 93/100 186 34/34 [==============================] - 16s 474ms/step - loss: 0.3845 - accuracy: 0.8778 - val_loss: 0.9641 - val_accuracy: 0.7500 187 Epoch 94/100 188 34/34 [==============================] - 16s 478ms/step - loss: 0.3526 - accuracy: 0.8915 - val_loss: 0.9655 - val_accuracy: 0.7574 189 Epoch 95/100 190 34/34 [==============================] - 16s 477ms/step - loss: 0.3581 - accuracy: 0.8869 - val_loss: 0.9652 - val_accuracy: 0.7574 191 Epoch 96/100 192 34/34 [==============================] - 16s 477ms/step - loss: 0.3392 - accuracy: 0.8897 - val_loss: 0.9608 - val_accuracy: 0.7537 193 Epoch 97/100 194 34/34 [==============================] - 16s 478ms/step - loss: 0.3525 - accuracy: 0.8943 - val_loss: 0.9628 - val_accuracy: 0.7537 195 Epoch 98/100 196 34/34 [==============================] - 16s 480ms/step - loss: 0.3532 - accuracy: 0.8943 - val_loss: 0.9632 - val_accuracy: 0.7537 197 Epoch 99/100 198 34/34 [==============================] - 16s 479ms/step - loss: 0.3759 - accuracy: 0.8851 - val_loss: 0.9571 - val_accuracy: 0.7574 199 Epoch 100/100 200 34/34 [==============================] - 16s 476ms/step - loss: 0.3292 - accuracy: 0.8971 - val_loss: 0.9545 - val_accuracy: 0.7537
没有预训练的权重，训练出来的结果肯定不是特别好，但是还算能接受吧。
可视化
1 # 画出训练集准确率曲线图
2 plt.plot(np.arange(epochs),history.history['accuracy'],c='b',label='train_accuracy')
3 # 画出验证集准确率曲线图
4 plt.plot(np.arange(epochs),history.history['val_accuracy'],c='y',label='val_accuracy')
5 # 图例
6 plt.legend()
7 # x坐标描述
8 plt.xlabel('epochs')
9 # y坐标描述
10 plt.ylabel('accuracy')
11 # 显示图像
12 plt.show()
1 # 画出训练集loss曲线图
2 plt.plot(np.arange(epochs),history.history['loss'],c='b',label='train_loss')
3 # 画出验证集loss曲线图
4 plt.plot(np.arange(epochs),history.history['val_loss'],c='y',label='val_loss')
5 # 图例
6 plt.legend()
7 # x坐标描述
8 plt.xlabel('epochs')
9 # y坐标描述
10 plt.ylabel('loss')
11 # 显示图像
12 plt.show()
7、模型结构大图
这里给出1000分类的模型结构图，图太大，翻起来不好翻，需要pdf的留下邮箱我发给你。
声明：本文参照CSDN博主「别团等shy哥发育」的文章
原文链接：https://blog.csdn.net/qq_43753724/article/details/126415101
标签：________________________________________________________________________________ From： https://www.cnblogs.com/wsfbb/p/16783645.html