论文部分内容阅读
在语音识别领域,基于深度神经网络(Deep Neural Network,DNN)的声学模型与传统的基于高斯混合模型的声学模型相比具有更出色的识别效果.DNN模型主要使用误差反向传播(Back Propagation,BP)算法进行训练.由于DNN参数规模非常庞大,使用BP算法对其训练是一个异常耗时的过程.目前主要通过高性能的GPU设备对DNN模型的训练训练进行加速.本文探究了BP训练算法在多GPU设备上并行化时所存在的收敛性和带宽问题,并结合NVIDIA Kepler架构的硬件特性提出一种并行优化策略.实验结果表明优化后的算法可以有效增加小mini-batch下的GPU利用率,同时减少数据传输的开销.在相同的mini-batch尺寸下,优化后的算法在4块NVIDIA Tesla K20m设备上与单个NVIDIA Tesla K20m设备相比可以取得高达3.89倍的加速比.
In the field of speech recognition, the acoustic model based on Deep Neural Network (DNN) has better recognition performance than the traditional acoustic model based on Gaussian mixture model.DNN model mainly uses Back Propagation BP) algorithm.Due to the huge size of DNN parameters, it is an abnormal and time-consuming process to train BPN.Nowadays, the training of DNN model is accelerated mainly by high-performance GPU devices.This paper explores the advantages of BP training algorithm This paper presents a parallel optimization strategy based on the hardware features of NVIDIA Kepler architecture.The experimental results show that the optimized algorithm can effectively increase the GPU utilization under the mini-batch Rate while reducing the data transfer overhead.The optimized algorithm achieves up to 3.89x faster than a single NVIDIA Tesla K20m device on 4 NVIDIA Tesla K20m devices in the same mini-batch size.