The U's have now lost just one of their last nine games to continue their progress up the League One table, but they were given a fright by the Robins.
All the main action occurred in the second half.
The visitors grabbed the lead in the 57th minute thanks to a goal from debutant Chris Westwood, who climbed high to head home a corner by Ian Westlake.
But that nearly prompted a Colchester revival, sparked off by Steven Gillespie who came on just seconds after Westwood's opener.
Gillespie should really have scored the equaliser himself on 58 minutes only for his goalbound header to be saved at point-blank range by his old team-mate Scott Brown.
Yet Brown was powerless to prevent Colchester skipper Dean Hammond from levelling the game with a sweet header from a free-kick by Mark Yeates just after the hour mark.
Cheltenham's cause was not helped by substitute John Finnigan, who was on the pitch for only seven minutes before he was sent off for two bookable offences - the second of them a late challenge on Yeates.
And Gillespie made the Robins pay by heading the U's into a 2-1 lead in the 78th minute.
Gillespie had spent two successful years at Cheltenham before his move to Essex last summer so this goal meant a lot to him.
Martin Allen's men had given their all during the first hour, though it's not surprising that they tired late on with Colchester always looking to add to their tally.
Man of the match Yeates had already been denied by a couple of extra saves from the agile Brown, before he finally sealed the win for the U's in the 89th minute.
Yeates latched on to a lay-off from substitute Lewis Gobern, who was making his debut following his loan move from Wolves, and the winger fired home in some style.
This result leaves Cheltenham rooted to the bottom four, while the U's are inching closer to the top ten.