This is actually a fairly serious question in philosphy: how do we know what "blue" is? How can i be certain that the way this wavelength of light appears to me as the exact same shade of grey that you see?

Picture if you will: a person whose eyes function differently than ours. S/he is shown a specific color on a card. If we saw through his/her eyes, we would call it "red", but looking at the card through our own eyes we would see it as green. S/he goes through life seeing "green" things as "red" ones, but nobody ever knows because s/he learned that that specific color was "green".

If my green is not your green, is it still green?

As for the problem with the Matrix, allow me this hypothesis:

The Matrix interfaces directly with the brain. Whatever it tells the brain is what is accepted as "true". When one is freed from the Matrix, the brain has only that information to rely on. The Matrix probably managed to download the pertinent information from the brains of those first trapped within. Those who managed to break free saw the world through eys tainted by exposure the Matrix. Those born free would still have been taught that "this is what tuna fish tastes like by those who were grown in the batteries. Thusly, the (mis?)information propagates itself and the Matrix shows a "realistic" view of the world.

Ye gods, i hope this made sense...