Deep Learning on Bdim

Understanding RNNs and LSTMs: What's the Deal?

Fri, 20 Sep 2024 00:01:11 +0800

What Are They?

RNNs are like learners with short-term memory – they remember recent information but get fuzzy as time passes.

Simply put, a Recurrent Neural Network (RNN) is like a “learner” with memory. Imagine this learner reading an article, remembering the meaning of each word temporarily before moving on to the next. While traditional neural networks treat each input independently (like starting from scratch each time), RNNs “remember” previous information (earlier words) and pass this memory to the next step, creating context for the entire sentence.

Understanding Cross-Entropy

Wed, 18 Sep 2024 00:01:11 +0800

What is it?

In deep learning, “cross-entropy” is a function used to measure the difference between two probability distributions, typically used in classification tasks.

Analogy: Guessing the Color of Candies

Imagine you have a bag of candies containing red, green, and blue colors. You guess the color distribution of the candies is 50% red, 30% green, and 20% blue. However, the actual distribution is 60% red, 20% green, and 20% blue.

Understanding Embedding Layers

Fri, 13 Sep 2024 00:01:11 +0800

Imagine you’re in a classroom with many students, each with their own name. The teacher wants to assign tasks based on students’ names, but the names themselves are meaningless and can’t help the teacher make decisions directly.

So, the teacher assigns each student a number, like Xiaoming is No. 1 and Xiaohong is No. 2. These numbers act as “labels” for the students, helping the teacher organize them better. However, the numbers alone aren’t enough, as they don’t carry much information.

OMR Related Paper Review: Practical End-to-End Optical Music Recognition for Pianoform Music

Tue, 27 Aug 2024 00:01:11 +0800

Core Ideas

1. Background of the Study:

This paper explores how to implement an end-to-end Optical Music Recognition (OMR) system using deep learning methods, with a focus on recognizing pianoform music. While recent progress has been made in monophonic music recognition, existing OMR models struggle to handle the multi-voice and multi-staff nature of piano music.
The complexity of piano music stems from its independent parallel voices, which can freely appear and disappear within a composition. This complexity introduces additional challenges for the output of OMR models.

2. Key Contributions:

How to Understand Batch Normalization?

Sun, 11 Aug 2024 00:01:11 +0800

Batch Normalization (BN)

When I first tried to understand Batch Normalization (BN), I went through a lot of resources, but I still found the specific implementation and purpose somewhat unclear. A few days ago, I asked my teacher for an explanation, and he gave me an example that made it much easier to grasp. I found it very helpful, and after going back to review other materials, everything started to make sense.

My note on learning LeNet 5

Sat, 13 Jul 2024 00:01:11 +0800

Some words

“The more I get, the less I know.”
——bdim

Recently, I am trying to learn more about image recognition. One of the topics that I am interested in is LeNet, which is a classic convolutional neural network. In this article, I will try to re-implement a LeNet using PyTorch and then abstractly understand some parts that I do not quite understand. The contents of this article may be inaccurate and will continue to update the details. Welcome to discuss with me about this article.