Advantages of ReLU Posted on 2017-12-27 Edited on 2019-03-21 In study Valine: speed up trainning: gradient computation -> 0 or 1 computation of ReLU is also easy, 0 or original vanishing gradient problem sparse output, reduce overfitting, (like L1 reg)