I am explaining this in the context of research/academic job hunting, but similar principles can be applied to other situations.
The effectiveness E of a particular individual as your reference can be expressed by the following equation:
E = wcQ
, where w is the weight of the individual, c the strength of the connection, and Q the quality of the recommendation.
. w is the weight of that individual in the particular field you are applying your job for. A nobody has a weight with absolute value close to 0, while a bigwig has weight with absolutely value close to 1. Note the mention of absolute value; the weight can be positive as well as negative. (Think about having Charles Darwin as reference for a scientific versus theological post.)
. c is the strength of connection between you and that individual, a real number between 0 (e.g. you do not really know each other) and 1 (e.g. the two of you fought in the same foxhole during WWII).
. Q is the quality of recommendation, a real number between -1 (e.g. you did not realize you actually pissed off that guy) and 1 (e.g. you wrote a 1000+ citation paper with your PhD adviser).
Your job is to identify individuals who can give you sufficiently large E values. Thus, do not ask people who are not very successful (near 0 w) or (worse) are archenemies with the institution you are applying for (negative w), who barely know you (weak c), or who are too busy (near 0 Q) or too pissed (negative Q) to write good recommendation.
You want a reasonable number of large positive E values.
The definition of *reasonable number* is application dependent (pun intended), but I would say 5 is about right for a research/academic job, 3 will be a bit too weak, and you probably do not need more than 7 unless you are applying for a senior position.
Also, make sure your E values are all sufficiently large and positive; having a bunch of weak E will not help, and can give the impression that you have difficulty finding good references.
Last but not least, ask and obtain consensus from people before listing them as your references. It can be very impolite, and downright dangerous, if you do not do so.
If you feel these are all common senses, you will be surprised by how many people commit the very basic mistakes.