In quantum mechanics, define the commutator of two operators A and B to be [ A,B ] = AB-BA. It's essentially a way of stating how two operators fail to
commute.
If they do commute, then [ A,B ] = 0 ie AB = BA and the operators are said to be simultaneously
diagonalizable - that is, their
eigenspaces coincide and
kets can be labeled by eigenvalues of both operators.
The relationship between this definition and the one given by ariels comes from Lie algebras: suppose we approximate a as 1+δA and b as 1+εB. Then to first order in δ and ε,
aba-1b-1 goes to (1+δA)(1+εB)(1-δA)(1+εB) = 1+(AB-BA)δε = 1+[ A,B ]δε
so as a->A and b->B, [ a,b ]->[ A,B ].