Extremely
common in the
Northeast US. Probably installed in
houses from the early 1900s to the 1950s or 1960s when
natural gas pipelines were installed throughout the US. You don't see it much in the rest of the country since most
development happened after
WWII, and before that people probably used
coal or
wood to
heat.
Oil heat is powered by (get this) heating oil. Heating oil is somewhere between gasoline and kerosene in consistency. If your house is heated with oil heat you have a huge oil tank either buried near your house, in your basement, or just sitting outside. An oil heat furnace or boiler requires electricity to pump the oil into the burner to keep running. In addition, heating oil will foul the burner over time, and such heaters need annual maintenance to keep operating at peak efficiency.
Recent fluctuations in oil prices have meant heating oil is very expensive this winter which may cause hardship for poorer people in the Northeast. This is why heating oil subsidies are important.
If you have a buried oil tank, it's very likely it has rusted and is leaking oil into the soil and possibly the groundwater. In New Jersey it is the law that oil tanks have to be fully disclosed when a house is sold, and any cleanup of a leaking tank is the responsibility of the seller. Cleanup usually involves removing the tank and any contaminated soil nearby, and filling with clean soil.