As far as I'm aware this phrase originated as one of many used on propaganda posters during World War II. It was basically a reminder to civilians involved in the war effort to be wary about what they spoke about, or to whom, for fear of information getting into the hands of enemy spies. I remember seeing a poster (in a museum, I'm not that old) with this slogan imprinted across it, and and a picture of two young women chatting about something they'd overheard, whilst behind a nearby wall a stereotypical-looking German was hastily scribbling in a notebook. In other words, even if you think you're alone and can't be overheard, you probably can.

Modern surveillance equipment makes this saying even more apt, and it was taken to its literal extreme during the Cold War. The Russians very generously offered to build the United States a lovely shiny new embassy building in Moscow in the 1980s. Obviously the Americans were suspicious, but their engineers oversaw the entire construction and after completion the entire place was carefully swept to ensure there were no bugs or other listening devices. Everything was clean, until by chance someone rested a bug detector against a wall. The KGB had embedded highly sensitive microphones-cum-transmitters into the very concrete used to build the walls, which were found to literally have ears...