Bayesian Network

A mathematical model for modelling and calculation on the causal influence between thing. Also called causal network or influence diagram.

Basicly, a Bayesesian Network is an oriented graph where the nodes are variables (representing some phenomena, e.g. I got drunk last night or "I have a hangover") and the edges represent causal influence (e.g. if i got drunk last knight, I'm more likely to have a hangover than if i didn't get drunk).

Each variable has a finite number of states. Variables which are not influenced by other variables has an a priori probability distribution between its states. Influenced variables have argumented probabilities, so that the probability that it is in each of its states is known, if the states of the variables that has causal influence on it are known.

Bayesian Networks are used in decision support systems and troubleshooters.

Nebulon gives a good description of Bayesian networks, but I think that Bayes nets are best understood with a diagram.

Consider the following example:

Let R be the random variable indicating that it will rain tomorrow.
Let S indicate whether of not I go to sleep late tonight.
Let L indicate whether or not I am late to class tomorrow.

Tonight, I don't know if it will rain tomorrow, so the fact that I will go to sleep late is independent of rain tomorrow. However, rain tomorrow and a late bedtime tonight both increase the chance that I will be late to class in the morning.

If I was to represent this example as a Bayes net, it would look like this:

 _________                  _________
|         |                |         |
|  Rain   |                |  Sleep  |
|_________|                |_________|
       \                     /
        \                   /
         \                 /
          \               /
           \             /
           \/ __________\/
             |          |
             |   Late   |
             |__________|

To represent all of the relevant probabilities in this example, I need to provide the probabilities of R and S, P(R) and P(S), as well as all of the conditional probabilities of L, P(L | R & S), P(L | R & ~S), P(L | ~R & S), P(L | ~R & ~S). However, I gain by not having to provide conditional probabilities for R and S; I know that they are independent of each other.

Since there are three Boolean variables in this example, to provide the full joint probability distribution, I would have to give 8 pieces of information. By using the conditional information represented in my Bayes net, I can give only 6 pieces of information, and still provide the user with enough information to calculate any joint distribution that he desires.

Now, suppose I want to add another variable to my example.

Let U be the variable that indicates whether or not I understand tomorrow's lecture.
I am more likely to understand the lecture if I arrive on time, so U is dependent on L.

My new Bayesian network looks like this:

 _________                  _________
|         |                |         |
|  Rain   |                |  Sleep  |
|_________|                |_________|
       \                     /
        \                   /
         \                 /
          \               /
           \             /
           \/ __________\/
             |          |
             |   Late   |
             |__________|
                  |
                  |
                 \ /
             ____________
            |            |
            | Understand |
            |____________|

Looking at this network, it is clear that U is not truly independent of R or S, since if it rains, I am more likely to be late, and therefore misunderstand the lecture. However, if I am given the value for L, I can gain no more information about U by knowing either R or S.

"If I know that I was late to class (L = 1), then knowing if it rained or if I went to bed late does not help me to predict if I will understand lecture."

What this means is that although U is not fully independent of R or S, it is conditionally independent of R and S.

P(U | L & R) = P(U | L)
P(U | L & S) = P(U | L)