27th International Conference on International Conference on Machine Learning 2010 

『Rectified Linear Units Improve Restricted Boltzmann Machines』 

Authors

Vinod Nair, Geoffrey E. Hinton 

Abstract

Restricted Boltzmann machines were developed using binary stochastic hidden units. These can be generalized by replacing each binary unit by an infinite number of copies that all have the same weights but have progressively more negative biases. The learning and inference rules for these "Stepped Sigmoid Units" are unchanged. They can be approximated efficiently by noisy, rectified linear units. Compared with binary units, these units learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset. Unlike binary units, rectified linear units preserve information about relative intensities as information travels through multiple layers of feature detectors. 

Review

Restricted Boltzmann machines은 이진 유닛을 사용하여 개발되었지만, 이들을 무한한 수의 동일한 가중치와 점점 더 부정적인 편향을 가진 복사본으로 일반화

이러한 "단계별 시그모이드 유닛"은 이진 유닛과 비교하여 객체 인식 및 얼굴 검증에 더 적합한 특성을 학습하며, 정보를 다층 특징 검출기를 통과하면서 상대적 강도에 대한 정보를 보존