I was always led to believe (although it's probably an urban myth) that the "we" to whom Victoria referred was herself and the late Prince Albert of Saxe-Coburg-Gotha.

Victoria never got over Albert's death and continued to mourn for him, wearing black for the rest of her life. (Does this make Victoria the first geek chick?... perhaps I shouldn't go there.) Nonetheless she continued to refer to him in conversation as if he was still by her side.