Two substances are miscible if they are capable of mixing. It's not clear why we don't just say 'mixable', which is easily understood, doesn't sound like 'missable', and has been a word since at least the seventeenth century, but there you go. English is silly. Probably someone thought the Latinate version sounded fancier. My money is on 'mixable' displacing it in the next hundred years or so.
The word is usually applied to liquids, since all gases are miscible with each other, while no solids are capable of mixing in the same way. If a solid or gas is miscible with a liquid, we usually say it's soluble, instead, but apart from the change of state it amounts to much the same thing. Sometimes we talk about liquids dissolving in each other, too. When liquids aren't miscible they are said to be immiscible (or unmixable), which happens when the molecules of each prefer their own company, usually because one of the liquids is polar while the other is not. For example, water molecules are quite strongly attracted to each other by hydrogen bonding; oil molecules are quite strongly attracted to each other by London dispersion forces. Neither one is much attracted to the other because oil molecules, being non-polar, are immune to hydrogen bonding, while water molecules are too small to be much moved by London dispersion forces.
If both liquids are polar, all their molecules are attracted to whichever poles they get near; if they're non-polar, they're attracted to each other just as indiscriminately. Some liquids have molecules that are only partially polar, like acetone and alcohol, and these are attracted to both polar and non-polar molecules, so they tend to mix at least a bit with liquids of either kind. This can get a bit complicated when you have three or more liquids all together: one way to test for oil in food is by soaking the food in alcohol to dissolve the oil, then adding water to the mixture. If it turns cloudy, you know there's oil, or at least some kind of lipid: whereas pure alcohol dissolves nicely in water, when it's mixed with oil it loses its solubility. That means it forms an emulsion with water, tiny droplets separating out and scattering light. Ouzo does something similar when you dilute it, thanks to the essential oils that give it its flavour.
An emulsion is the closest you can get to a mixture of two immiscible liquids. To form a stable emulsion, you need some kind of emulsifier - something that's attracted to both liquids. This is the basis of soap: you can't just rinse oily stuff off your hands, pans and clothing, but if you can turn the oil from a layer into an emulsion, the droplets rinse away easily. Emulsions are important in the kitchen, too. A salad dressing of oil and vinegar alone will quickly separate into layers, but a bit of mustard added to the mix helps it stay emulsified. Similarly, the lecithin in egg yolk allows mayonnaise to be a stable emulsion. Milk is a natural emulsion, and stays as such when you turn it into cream or butter.