During the early 1800s the gentleman's pocket watch was a very important element of his appearance, a chance for him to show off his wealth and taste. The watch itself was obviously a very important part of the ensemble, but it was generally hidden in the fob pocket. As time went on the other end of the watch chain, which could hang out visibly across the gentleman's chest, became more and more important.
One of the first ornaments to appear was a silver or enamel pendant bearing the arms of some club or society. This soon became a larger object, and one of the more popular decorative objects was a seal. Just as the woman's practical chatelaine evolved into a gaudy exhibition of overdone jewelry, the gentleman's watch seal soon became fancier and less practical. Watch seals qua seals remained popular, but it was not uncommon for a gentleman's chest to be adorned with 3 to 5 'seals' - including a variety of pendants, mounted gems, watch keys (these later evolved into class keys), and other doodads.
The watch chain might connect to any number of practical objects as well, particularly once the Double Albert, a double chain with all the more room for attaching trinkets, came into fashion. Vesta cases, miniature compasses, sovereign cases, cigar cutters, lorgnette, and pen knifes for sharpening quills were common objects, but they generally remained hidden in the vest pocket, as opposed to hanging down across the chest.
As time went on, it became more and more common to refer to these items as watch fobs, although technically the watch fob is the connector between the watch and the chain or the fob pocket. These days even the fanciest three-piece suit is unlikely to sport anything much more ostentatious than a nicely draped watch chain, and so terminology is not particularly important.