Doesn't this story sound good? They found an actual bug in the machine, and thus we call electronic glitches bugs? Actually, this is a bit of hacker folklore. Grace Hopper liked to tell this story, and it was published in "Annals of the History of Computing", Vol. 3, No. 3 (July 1981), pp. 285-286. The text of the log entry (from September 9, 1947), reads "1545 Relay #70 Panel F (moth) in relay. First actual case of bug being found".
Alas, as Rear Admiral Hopper herself would gladly have told you, use of the word "bug" to mean a glitch or error was in common use during WWII, as the text of the log entry indeed implies with the words "First actual case...". Some reports claim that the use of the word bug go back to telegraphist slang of the 19th century. For more on this issue, see the Jargon File under "bug"
Update: the entire Jargon File has been autonoded, I see... look under bug for the whole story on this matter.