Heater problem? Anyone know?

Before I say anything, I've already done searches on this specific topic, in which I couldn't find anyone w/ the same problem..

My heater doesn't seem to be working properly?? I am not sure what's the cause. I have a 97 integra GSR, below 70K on mileage. Everytime I start up the car and let it idle and warm up for over 20 min, the temperature meter/gauge (not sure what it's called) on the left of the RPM Gauge displays with the needle almost in the center of "Cold" and "Hot," slightly off (about 1-2cm) to the Cold side (NOTE: The needle has NEVER reached the middle, even after hard driving, long driving, etc.. temp gauge not accurate? not sure...) This seems fairly normal? Normal? I guess..? At this time, even if I push the temp control to max heat, only warm air would come through my vents (my vents isn't clogged also, if I put it to full blast, it blast pretty well). As I begin driving for about 5 min or so, the temp gauge drops back to the "Cold" side. There is like 2cm in length that separates the "Cold" mark and the needle..The once warm air that comes through the vent is now cool and cold (IT'S WINTER HERE ALSO!). It goes on like this everytime I drive =/

Anyone know the cause?? Problem?? SOLUTIoN??? Help??