Paper

• Gradient-Based Learning Applied to Document Recognition
• Authors: Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner

LeNet:

• LeNet-5 is a pioneering convolutional neural network (CNN) that played a foundational role in the field of deep learning, particularly in the application of using deep learning to image recognition tasks.
• Key Contributions:
  • Conv Layer: Demonstrated the effectiveness of using learnable filters for feature extraction from images.
  • Subsampling/Pooling Layer: Introduced the concept of spatial pooling (also known down-sampling) to reduce the spatial size of the representation, control overfitting, and reduce computational requirements.
  • One more aspect about LeNet-5 regarding activations is that,
    • for intermediate layers it used tanh
    • for final output layer radial basis function (RBF) network to classify images.
• Architecture: LeNet-5’s architecture is relatively simple by today’s standards but was revolutionary at the time.