Throughout Buchanan's presidency there were rumors that he was a homosexual. While there is no concrete proof, the facts are pretty striking. Buchanan remained a bachelor his whole life, and for many years in Washington he lived with William Rufus King, a Senator from Alabama who later became vice-president under Franklin Pierce. King was also never married.

They spent so much time together many Washington circles referred to them as "the Siamese twins," Andrew Jackson called King "Miss Nancy," and James K. Polk named him "Mrs. James Buchanan."

King himself penned long and intimate letters to Buchanan. When appointed minister to France, King wrote to Buchanan, "I am selfish enough to hope you will not be able to procure an associate who will cause you to feel no regret at our separation." Buchanan later referred to King as "among the best, purest, and most consistent public men I have ever known, and is also a sound judging and discreet fellow," as well as a "very gay, elegant looking fellow."