Modal verbs are a set of verbs in English that follow some specific and unusual rules. They can be seen as a subset of auxiliary verbs.
The current modal verbs are:
"Ought" is the only modal verb that is always followed by the preposition "to". "Dare" and "need" are not commonly used as modal verbs in the contemporary United States, but they might still be found in older texts.
Modal verbs follow a few special grammatical rules. First, do not change forms. We do not say "I musted done it" or "He cans run fast". They are always used in one form, and they are always followed by the bare infinitive form of the verb. Modal verbs, in standard English, are also used singly. You can't mix modal verbs with other modal verbs, or with the auxiliary verbs to be or to do. "I can must run" and "I do will go" are not sensical statements in English. A modal verb also can't exist in isolation, because they require a standard verb to modify.
Conceptually, modal verbs communicate two concepts: possibility (or certainty) and obligation (or permission). In this role, they have a combination of meanings that can be quite confusing for non-native speakers, although native speakers naturally know all the rules. For example, "must" can be used to communicate both deduction, and obligation. "It must be raining" when we see someone enter a room carrying an umbrella, is quite different than the must in "It is raining, you must bring your umbrella". The first communicates deduction, the second obligation. "It might rain" and "It may rain" are interchangeable when discussing possibility, but "You may sit down" is a statement of permission, while "You might sit down" is a statement of possibility. "Should" is used to communicate both possibility and obligation, so the statement "He should be here by 3 PM" can mean either "He is expected to arrive at 3 PM" or "He has an obligation to be here by 3 PM", and possibly both. It would take a complicated venn diagram to explain all the shades of meanings of different modal verbs.
There are a few exceptions to the rules about modal verbs, further confusing things: at one time, modal verbs did change form, so you can read Shakespeare write things like "Canst thou not minister to a mind diseased" in MacBeth. Also, there are some times when modal verbs are used as full verbs. I thought this was merely a modern humorous form, when I read "I am unable to can" on tumblr, but again, Shakespeare himself would use this form with such phrases as "I must away!".
In general, though, modal verbs follow specific grammatical rules, and have ambiguous and overlapping meanings. For native speakers, this is an intuitive part of the language. For non-native speakers, this is one of the most difficult things to learn, both in form and content.