Friendly AI is a futurist term for an artificial general intelligence that has a positive effect on humanity. It usually assumes that the AI under discussion is very powerful, usually a superintelligence, and probably at or past the point of singularity.
A friendly AI is not just one that has a 'good' goal; a powerful AI that had the goal of increasing human per capita wealth and did this by drastically reducing the human population would probably be an unfriendly AI. Likewise, an AI with the goal of maximizing human per capita wealth and human population, but achieved this through the means of 500 billion hungry people each with an extensive collection of Fabergé eggs would also be an unfriendly AI.
All proposed sets of rules that you could put on a superintelligent AI seem to go one of two ways: they are either insufficient to close all loopholes that a being much smarter than you can come up with, or they are so restrictive in requiring that the AI check everything with the controller that nothing gets done.... and then you still find that there are unclosed loopholes.
The general goal is what Eliezer Yudkowsky calls Coherent Extrapolated Volition -- that an AI would do what we would do "if we knew more, thought faster, were more the people we wished we were, and had grown up closer together." (Which sounds great until you consider what sort of people some people want to be more like.) Obviously, this is not itself a solution to the problem, but it is a framework in which a solution might be developed.
It is a matter of some debate as to how important it is to define and perhaps design a friendly AI. However, it appears that we currently have all the foundational technologies for a superintelligent agent, and we honestly do not know what might trigger a computer system to go from "trying to solve a problem" to "improving its own resources until it does solve the problem". We already have complex computer programs designed to evolve in order to tackle problems more effectively, and they have the resources to self-improve in unexpected ways, making it reasonable to suspect that this is a problem that we will have to face in the foreseeable future.