Someone called me a 'toff'. What's that?
'Toff' is British slang for a rich, fancy, upper-class person. Synonyms include dandy and swell (as nouns, not adjective). It means you're posh.
So should I be offended?
'Toff' is considered a derogatory term. The lower-classes look at those of us with an education, a wealthy family, and fancy pants with scorn, and they shouldn't, because we're better than they are. If someone calls you a toff, call them a 'chav' (unless they're fully wretched, in which case you can just give them a good kick off to the side).
Where'd this word come from, anyhoo?
The word came about in the eighteen hundreds (1851 says etymonline.com). The standard claim is that it's a modification of tuft, that being the tassel worn by students at Cambridge and Oxford Universities. Those rich snobs and their 1337 education. Pshaw! They're not so great.
Who uses this word?
Well, the British, obviously, and the lower-classes at that. Of course, it's a perfectly fine word, and anyone is welcome to use it. But if you're a toff, why would you call someone else one? Leave it for the lower- and middle-class slobs. Yeah, that's right.