This phrase is commonly used in at least two specific situations.
1. You don't want to use
profanity. This would be used to replace "
Holy Shit"! Or "Holy
Fuck"! In this case, it would be appropriate to use "Holy Bajeezus" in the company of small children.
2. You don't want to
piss off any religious-types by actually using "
Jesus". Adding the "ba" to the beginning of the name keeps you from getting in any
trouble. In this case, it would be appropriate to use "Holy Bajeezus" in the company of your
Catholic Grandmother.