"JHC" is an monogram for "Jesus," being the first three letters of "Jesus" in Greek: iota, eta, sigma. The same monogram is sometimes seen as "IHS" or "JHS." As I child I was told that "IHS" was a monogram for Jesus, meaning "I Have Suffered": this, obviously from someone not knowing the Greek alphabet. "Jesus H. Christ" is not so far from the same mark.
In the same vein, is the tizzy some get in over the abbreviation "Xmas." A third grade teacher of mine went ballistic: "X is an unknown quantity; Jesus is not an unknown quantity." The truth is that in this case "X" is not an "Ex" but the Greek letter chi, the first letter in the Greek word Christ and thus a symbol for Jesus. Worrying about Jesus' middle name begs the question, since Jesus doesn't have a last name. "Christ" is a title, not a name. It means "The Anointed." Thus we often see "Jesus the Christ." If Jesus were to have a last name it would, I suppose be "Josephson" or "Godson".