This is mainly for solo improv.
As shaynetonio says, "learn your scales." The pentatonic scale (just those black keys if you're using a piano or keyboard) is pretty handy to get started with, since it's fairly easy to get almost anything to "sound good" with these notes.
Choose a few notes (maybe 2 to 4) to be the ones you play the most. Then randomly throw others in there to taste. If there isn't enough variation, then what you play starts to sound boring. If there is too much variation, then it may become unpredictable and lose what most people would consider a sense of melody.
Learn some commonly used chords and possibly some often used chord progressions after mastering the basics. You don't have to slavishly follow them though. Depending on how much predictability or variation you want, vary how often you change chords and how many notes in the new chord match the notes in the previous one. If you find yourself getting bored with what you're currently playing, it's probably a good time to start switching things up a bit.
Don't limit your changes to just chords - you can also throw in changes to tempo, volume, and rhythm. Changes to tempo and volume can be especially helpful in creating a dramatic effect (assuming you don't mind being called bombastic =)
More good advice from shaynetonio: "Make your mistakes big and loud." Actually, by the time you've realized you made a mistake, it's usually too late to go back and make it bigger and louder. However, you can change the next notes you play to match your mistake, turning what at first sounded like mismatched notes into something that is now a predictable part of the pattern you're playing.