whether the fan comes on or not also depends on your current CPU load. If that is high, and the temperature low, the fan will still go on.
If you experience this behaviour, open Activity Monitor and look for the culprit program which is responsible for the CPU load. Once you kill it, your fan will turn off in no time