The title refers to the song by Bernice Petker and Irving Caesar.
Late at night in a music shop, the mice come out to play. They turn on the gramophone and dance to the titular tune. They enjoy playing with the various musical instruments in the shop. A hungry cat outside in the rain spies them through the window. He climbs to the roof and enters through the chimney. He chases and corners one, but before he can eat it, it calls for help. The other mice use the instruments as weapons and force the cat out of the store.
- It was nominated for the Academy Award for Best Short Subject (Cartoon), making this the first Warner Bros. short to get an Academy Award nomination.
- This cartoon entered the Public Domain in 1960.