Adjusting precision and topology parameters for neural network training based on a performance metric
user-5f8cf9244c775ec6fa691c99(2020)
摘要
Apparatus and methods for training neural networks based on a performance metric, including adjusting numerical precision and topology as training progresses are disclosed. In some examples, block floating-point formats having relatively lower accuracy are used during early stages of training. Accuracy of the floating-point format can be increased as training progresses based on a determined performance metric. In some examples, values for the neural network are transformed to normal precision floating-point formats. The performance metric can be determined based on entropy of values for the neural network, accuracy of the neural network, or by other suitable techniques. Accelerator hardware can be used to implement certain implementations, including hardware having direct support for block floating-point formats.
更多查看译文
关键词
Performance metric,Artificial neural network,Entropy (information theory),Computer science,Topology,Implementation,Numerical precision
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络