A punk is a baby elephant, according to a rather antiquated definition (it has largely been replaced by the more generic calf in this context). Given this information, you might consider Baby Elephant Walk by Henry Mancini (the guy who wrote the theme to the Pink Panther) to be the first "punk" song. Several punk bands have picked up on this bit of trivia and done very incongruous covers of Mancini's tune.