Bishop's most famous exploit is also his most controversial. On June 2, 1917, he claimed that he attacked a German airfield which was more than 25 km inside enemy territory and shot down three enemy aircraft. His plane was in tatters when he landed almost an hour later. Bishop had already won several medals, including the Military Cross, but the account of this daring, one-man raid won him the highest of all British honours for bravery in battle, the Victoria Cross.

Bishop's victories were an inspiration to the troops during World War I, but some critics have suggested that his famous raid never took place. There were no eyewitnesses and no German record of the event. Those who support Bishop charge that these accusations are an insult to a national hero. A film that questioned Bishop's claims caused a furor in Parliament and led to a Senate investigation, which was not conclusive. Bishop was also the subject of the popular playBilly Bishop Goes to War, by John Gray. A hero or not, Bishop seems destined to remain part of the Canadian imagination.