Conference item icon

Conference item

On the impact of the activation function on deep neural networks training

Abstract:

The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propagation. Understanding the theoretical properties of untrained random networks is key to identifying which deep networks may be trained successfully as recently demonstrated by Sa...

Expand abstract
Publication status:
Published
Peer review status:
Peer reviewed

Actions


Access Document


Files:

Authors


More by this author
Institution:
University of Oxford
Oxford college:
Hertford College
Role:
Author
More by this author
Institution:
University of Oxford
Division:
MPLS
Department:
Statistics
Oxford college:
Hertford College
Role:
Author
ORCID:
0000-0002-7662-419X
More by this author
Institution:
University of Oxford
Division:
MPLS Division
Department:
Statistics
Oxford college:
Jesus College
Role:
Author
Publisher:
Journal of Machine Learning Research Publisher's website
Journal:
Proceedings of Machine Learning Research Journal website
Host title:
Proceedings of Machine Learning Research
Publication date:
2019-06-12
Acceptance date:
2019-04-30
ISSN:
2640-3498
Source identifiers:
1043248
Pubs id:
pubs:1043248
UUID:
uuid:1e9a7c39-7519-4796-9784-537b14ba1941
Local pid:
pubs:1043248
Deposit date:
2019-08-13

Terms of use


Views and Downloads






If you are the owner of this record, you can report an update to it here: Report update to this record

TO TOP