Simple mathematical model for a neuron.
Invented by Rosenblatt in the 1960s.
A perceptron is a function f: Rn -> R defined by 2 functions g: Rn -> R and t: R -> R per f(x) = t(g(x)). R is the set of real numbers.
g is called the summation function and t is called the activation/transfer function.
Usually g(x) is the inner product of x with a weight vector w minus a real threshold value theta; g(x) = < x,w> - threshold
t can be any monotonic function, but usually boundedness is demanded. Very often the sign function is used.

Perceptrons were believed to be very powerful in the 1960s, until Minsky and Papert proved in their book "Perceptrons" that a single perceptron (with the sign function) can't solve the XOR problem.

Note that often authors use the above example for g, the sign function for t and regard other functions for t as non-perceptrons.

A perceptron is a simple kind of neural network. A perceptron may have m weighted inputs and a bias, which are summed. This sum is passed through a hard limit transfer function, which forces the output to one if the input is greater than or equal to zero, or to zero if the input is less than zero. This hard limit function is what makes a perceptron a perceptron. More general neural networks can use many different transfer functions.

A neural network consisting of a single node can classify input vectors into one of two categories. In general, a perceptron network with n nodes can classify input vectors into n^2 categories.

A single layer perceptron network can be trained using the perceptron learning rule. More complex networks use backpropagation training.

An example perceptron might have two inputs:

              +------+
p1 --> w1 --> | ---- |
              | \    | --> hardlim(p1w1 + p2w2 + b) --> a
p2 --> w2 --> | /    |
              | ---- |
              +------+
                  |
                  b

In this example, the input vector P is (p1, p2), and the weight vector is (w1, w2). The input is weighted and summed together with the bias, and then passed to the hard limiter, so we have:

a = hardlim(PWT + b)

If p1 and p2 are either 1 or 0, and the weight matrix is (0.5, 0.5), and the bias b is -1, then this perceptron performs the boolean AND function. Example:

hardlim((1 0)(0.5 0.5)T + (-0.9)) = hardlim(-0.4) = 0

hardlim((0 1)(0.5 0.5)T + (-0.9)) = hardlim(-0.4) = 0

hardlim((0 0)(0.5 0.5)T + (-0.9)) = hardlim(-0.9) = 0

hardlim((1 1)(0.5 0.5)T + (-0.9)) = hardlim(0.1) = 1

There are many combinations of weights and bias's that can produce the same output.

If the perceptron has two inputs, then one can graph those inputs by placing the first input on the x axis, and the second input on the y axis. A decision boundary defined by the weight vector can be made to intersect this plane, so the all of the inputs on one side of the line correspond to an output of zero and all of the inputs on the other side of the line correspond to a one.

If the weight vector is (1 0), then we can see that the output doesn't depend on the second input at all; the decision boundary is a vertical line at x=0 (assuming a bias of zero).

The bias input serves to move the decision boundary off of the origin. If we again have a weight vector of (1 0), and a bias of 1, then the decision boundary has moved to x = -1 (a = hardlim((-1 1) (1 0) + 1) = hardlim(-1 + 1) = hardlim(0) = 1). Any value for p1 greater than or equal to -1 results in an output of 1, while a p1 < -1 give a result of 0.

If the perceptron has three inputs, then the input space becomes a volume, and the decision boundary becomes a surface. Higher dimensions are hard to visualize.


Computational Intellegence class notes, from memory as practice.

Log in or register to write something here or to contact authors.