博客

CNN作为深度学习架构被提出的最初诉求，是降低对图像数据预处理的要求，以及避免复杂的特征工程。CNN训练的模型同样对缩放、平移、旋转等畸变具有不变性，有着很强的泛化性。CNN最大的特点在于卷积的权值共享结构，可以大幅减少神经网络的参数量，防止过拟合的同时又降低了神经网络模型的复杂度。

卷积神经网络可以利用空间结构关系减少需要学习的参数量，从而提高反向传播算法的训练效率。

每个卷积层会进行如下操作：

（1）图像通过多个不同的卷积核的滤波，并加偏置，提取局部特征，每一个卷积核会映射出一个新的2D图像。

（2）将前面卷积核的滤波输出结果，进行非线性的激活函数处理。目前最常见的是使用ReLU函数。

（3）对激活函数的结果再进行池化操作，目前一般是使用最大池化，保留最显著的特征，并提升模型的畸变容忍能力。

一个卷积层中可以有多个不同的卷积核，而每一个卷积核都对应一个滤波后映射出的新图像，同一个新图像中每一个像素都来自完全相同的卷积核，这就是卷积核的权值共享，可以降低模型复杂度，减轻过拟合并降低计算量。

图像中的基本特征无非就是点和边，无论多么复杂的图像都是点和边组合而成的，然后将点和边的信号传递给后面一层的神经元，再接着组合成高阶特征，比如三角形、正方形、直线、拐角等。再继续抽象组合，得到眼睛、鼻子和嘴等五官，最后再将五官组合而成一张脸，完成匹配识别。

卷积的好处是，不管图片尺寸如何，我们需要训练的权值数量只跟卷积核大小、卷积核数量有关，我们可以使用非常少的参数量处理任意大小的图片。还有就是隐含节点数量只和卷积的步长有关，数量为1/（步长x步长）。

卷积神经网络的要点就是局部连接、权值共享和池化操作中的下采样，其中局部连接和权值共享降低了参数量，使训练复杂度大大下降，并减轻了过拟合。同时权值共享还赋予了卷积网络对平移的容忍性，而池化操作下采样则进一步降低了输出参数量，并赋予模型对轻度形变的容忍性，提高了模型的泛化能力。

from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf
mnist=input_data.read_data_sets("MNIST_data/",one_hot=True)
sess=tf.InteractiveSession()
#给权值制造一些随机高斯噪声来打破完全对称，同时因为激活函数使用ReLU,也给偏置增加一些小的正值（0.1）来避免死亡节点。
def weight_variable(shape):
    initial = tf.truncated_normal(shape,stddev=0.1)
    return tf.Variable(initial)
def bias_variable(shape):
    initial=tf.constant(0.1,shape=shape)
    return tf.Variable(initial)
#接下来定义卷积层，下面x代表输入，W是卷积的参数如[5,5,1,32]代表卷积核大小5x5，1通道，数目为32，strides都为1代表划过图片每一个点
#padding='SAME'代表加入的padding使得卷积输入输出保持同样的尺寸。
def conv2d(x,W):
    return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding='SAME')
#接下来定义池化层，这里使用将2x2的最大池化降为1x1的像素
def max_pool_2x2(x):
    return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
#定义输入的placeholder，x是特征，y_是真实标签，因为卷积会利用空间信息，需要将1D输入向量转为2D图片结构，1x784->28x28，下面的-1代表样本数量不固定，最后的1代表颜色通道数
x=tf.placeholder(tf.float32,[None,784])
y_=tf.placeholder(tf.float32,[None,10])
x_image=tf.reshape(x,[-1,28,28,1])
#定义第一个卷积层，先卷积再加上偏置，在使用ReLU激活函数进行非线性处理，最后进行池化操作
W_conv1=weight_variable([5,5,1,32])
b_conv1=bias_variable([32])
h_conv1=tf.nn.relu(conv2d(x_image,W_conv1)+b_conv1)
h_pool1=max_pool_2x2(h_conv1)
#定义第二层卷积层，卷积核数量变为64
W_conv2=weight_variable([5,5,32,64])
b_conv2=bias_variable([64])
h_conv2=tf.nn.relu(conv2d(h_pool1,W_conv2)+b_conv2)
h_pool2=max_pool_2x2(h_conv2)
#将第二个卷积层的输出转成1D的向量，然后连接一个全连接层，隐含节点为1024，并使用ReLU激活函数
W_fc1=weight_variable([7 * 7 * 64,1024])
b_fc1=bias_variable([1024])
h_pool2_flat=tf.reshape(h_pool2,[-1,7 * 7 * 64])
h_fc1=tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1)
#为了减轻过拟合，下面使用一个Dropout层，通过一个placeholder传入keep_prob比率来控制的。
keep_prob=tf.placeholder(tf.float32)
h_fc1_drop=tf.nn.dropout(h_fc1,keep_prob)
#最后将Dropout层的输出连接一个Softmax层，得到最后的概率输出
W_fc2=weight_variable([1024,10])
b_fc2=bias_variable([10])
y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop,W_fc2) + b_fc2)
#定义损失函数为交叉熵，优化器使用Adam，并给予一个比较小的学习速率1e-4
cross_entropy=tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv),reduction_indices=[1]))
train_step=tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
#再继续定义评测准确率的操作
correct_prediction = tf.equal(tf.argmax(y_conv,1),tf.argmax(y_,1))
accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
#下面开始训练过程
tf.global_variables_initializer().run()
for i in range(20000):
    batch = mnist.train.next_batch(50)
    if i%100 ==0:
        train_accuracy=accuracy.eval(feed_dict={x:batch[0],y_:batch[1],keep_prob:1.0})
        print("step %d,training accuracy %g"%(i,train_accuracy))
    train_step.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5})
#在最终的测试集上进行全面的测试，得到整体的分类准确率
print("test accuracy %g"%accuracy.eval(feed_dict={x:mnist.test.images,y_:mnist.test.labels,keep_prob:1.0}))

结果：

/usr/local/Cellar/anaconda/bin/python /Users/new/Documents/JLIFE/procedure/python_tr/py_train/train1.py
Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
2017-08-03 00:14:51.401937: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-03 00:14:51.401962: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-08-03 00:14:51.401968: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-03 00:14:51.401973: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
step 0,training accuracy 0.02
step 100,training accuracy 0.78
step 200,training accuracy 0.82
step 300,training accuracy 0.9
step 400,training accuracy 0.94
step 500,training accuracy 0.98
step 600,training accuracy 0.96
step 700,training accuracy 0.96
step 800,training accuracy 0.9
step 900,training accuracy 0.98
step 1000,training accuracy 0.96
step 1100,training accuracy 1
step 1200,training accuracy 0.98
step 1300,training accuracy 0.92
step 1400,training accuracy 0.98
step 1500,training accuracy 0.98
step 1600,training accuracy 0.98
step 1700,training accuracy 0.96
step 1800,training accuracy 0.98
step 1900,training accuracy 0.98
step 2000,training accuracy 0.96
step 2100,training accuracy 0.98
step 2200,training accuracy 1
step 2300,training accuracy 0.98
step 2400,training accuracy 0.96
step 2500,training accuracy 0.94
step 2600,training accuracy 0.98
step 2700,training accuracy 0.98
step 2800,training accuracy 1
step 2900,training accuracy 0.98
step 3000,training accuracy 1
step 3100,training accuracy 0.94
step 3200,training accuracy 0.98
step 3300,training accuracy 1
step 3400,training accuracy 0.96
step 3500,training accuracy 1
step 3600,training accuracy 1
step 3700,training accuracy 0.96
step 3800,training accuracy 0.98
step 3900,training accuracy 0.98
step 4000,training accuracy 0.92
step 4100,training accuracy 1
step 4200,training accuracy 0.98
step 4300,training accuracy 1
step 4400,training accuracy 1
step 4500,training accuracy 0.98
step 4600,training accuracy 1
step 4700,training accuracy 0.98
step 4800,training accuracy 0.96
step 4900,training accuracy 1
step 5000,training accuracy 1
step 5100,training accuracy 0.98
step 5200,training accuracy 1
step 5300,training accuracy 0.94
step 5400,training accuracy 0.98
step 5500,training accuracy 0.98
step 5600,training accuracy 0.98
step 5700,training accuracy 1
step 5800,training accuracy 1
step 5900,training accuracy 0.96
step 6000,training accuracy 1
step 6100,training accuracy 1
step 6200,training accuracy 1
step 6300,training accuracy 0.98
step 6400,training accuracy 0.98
step 6500,training accuracy 1
step 6600,training accuracy 1
step 6700,training accuracy 0.94
step 6800,training accuracy 1
step 6900,training accuracy 1
step 7000,training accuracy 0.98
step 7100,training accuracy 1
step 7200,training accuracy 1
step 7300,training accuracy 0.96
step 7400,training accuracy 1
step 7500,training accuracy 0.96
step 7600,training accuracy 0.98
step 7700,training accuracy 1
step 7800,training accuracy 1
step 7900,training accuracy 0.98
step 8000,training accuracy 1
step 8100,training accuracy 1
step 8200,training accuracy 0.98
step 8300,training accuracy 1
step 8400,training accuracy 1
step 8500,training accuracy 1
step 8600,training accuracy 1
step 8700,training accuracy 0.98
step 8800,training accuracy 1
step 8900,training accuracy 1
step 9000,training accuracy 0.98
step 9100,training accuracy 0.98
step 9200,training accuracy 1
step 9300,training accuracy 1
step 9400,training accuracy 1
step 9500,training accuracy 1
step 9600,training accuracy 1
step 9700,training accuracy 1
step 9800,training accuracy 1
step 9900,training accuracy 1
step 10000,training accuracy 1
step 10100,training accuracy 1
step 10200,training accuracy 1
step 10300,training accuracy 1
step 10400,training accuracy 1
step 10500,training accuracy 1
step 10600,training accuracy 1
step 10700,training accuracy 1
step 10800,training accuracy 0.98
step 10900,training accuracy 1
step 11000,training accuracy 1
step 11100,training accuracy 1
step 11200,training accuracy 1
step 11300,training accuracy 1
step 11400,training accuracy 1
step 11500,training accuracy 0.98
step 11600,training accuracy 1
step 11700,training accuracy 1
step 11800,training accuracy 1
step 11900,training accuracy 1
step 12000,training accuracy 1
step 12100,training accuracy 0.98
step 12200,training accuracy 1
step 12300,training accuracy 1
step 12400,training accuracy 1
step 12500,training accuracy 1
step 12600,training accuracy 1
step 12700,training accuracy 0.96
step 12800,training accuracy 1
step 12900,training accuracy 1
step 13000,training accuracy 0.96
step 13100,training accuracy 1
step 13200,training accuracy 1
step 13300,training accuracy 1
step 13400,training accuracy 1
step 13500,training accuracy 1
step 13600,training accuracy 1
step 13700,training accuracy 1
step 13800,training accuracy 1
step 13900,training accuracy 0.98
step 14000,training accuracy 1
step 14100,training accuracy 1
step 14200,training accuracy 1
step 14300,training accuracy 1
step 14400,training accuracy 1
step 14500,training accuracy 1
step 14600,training accuracy 1
step 14700,training accuracy 1
step 14800,training accuracy 1
step 14900,training accuracy 1
step 15000,training accuracy 1
step 15100,training accuracy 0.98
step 15200,training accuracy 1
step 15300,training accuracy 1
step 15400,training accuracy 1
step 15500,training accuracy 1
step 15600,training accuracy 1
step 15700,training accuracy 1
step 15800,training accuracy 1
step 15900,training accuracy 1
step 16000,training accuracy 1
step 16100,training accuracy 1
step 16200,training accuracy 0.98
step 16300,training accuracy 1
step 16400,training accuracy 1
step 16500,training accuracy 1
step 16600,training accuracy 1
step 16700,training accuracy 1
step 16800,training accuracy 1
step 16900,training accuracy 1
step 17000,training accuracy 1
step 17100,training accuracy 1
step 17200,training accuracy 1
step 17300,training accuracy 1
step 17400,training accuracy 1
step 17500,training accuracy 1
step 17600,training accuracy 1
step 17700,training accuracy 1
step 17800,training accuracy 1
step 17900,training accuracy 1
step 18000,training accuracy 1
step 18100,training accuracy 1
step 18200,training accuracy 1
step 18300,training accuracy 1
step 18400,training accuracy 1
step 18500,training accuracy 1
step 18600,training accuracy 1
step 18700,training accuracy 1
step 18800,training accuracy 1
step 18900,training accuracy 0.98
step 19000,training accuracy 1
step 19100,training accuracy 1
step 19200,training accuracy 1
step 19300,training accuracy 1
step 19400,training accuracy 1
step 19500,training accuracy 1
step 19600,training accuracy 1
step 19700,training accuracy 1
step 19800,training accuracy 1
step 19900,training accuracy 1
test accuracy 0.9923

Process finished with exit code 0