The iPod does’t have a stop button. You merely pause the song.
The iPod is never completely off, it just falls asleep. Because of this “instant-on” feature the iPod responds as quickly as it does when used again.
The song starts playing again where you left off. You can manually put your iPod to sleep by pressing and holding the play/pause button.