Former Bristol Rovers striker Barry Hayles struck twice in the first half to put the Robins in control, but they had to defend strongly in the second half in order to collect three much needed points.
Cheltenham launched themselves into the task from the start and spent the first 45 minutes playing at a tempo that Rovers were unable to match.
Hayles ran onto a pass from Alex Russell before shooting wide and although Rovers went close with a Rickie Lambert header from a Chris Lines corner, the home side took the lead with 19 minutes played.
The goal resulted from a slick move, Russell playing a one-two with Lloyd Owusu before threading a pass through for Hayles to beat goalkeeper Steve Phillips from a tight angle.
The lead was doubled 12 minutes later when a corner was cleared only as far as Russell wide on the right. He showed presence of mind to switch the play right to the other side of the field, where defender Shane Duff headed down for Hayles to flip the ball beyond Phillips.
Rovers almost struck back from a goalmouth scramble just before half-time and they went on to dominate long periods of the second half.
They pulled a goal back five minutes after the re-start when Cheltenham 's new loan signing, Scott Murray, gave the ball away in midfield. Jeff Hughes got away down the left and crossed for Lambert to score with a well-struck half-volley from the edge of the penalty area.
Duff was forced to throw himself in front of a shot from Jo Kuffour and Allen made a bold decision to switch to five at the back with half an hour left to stem the tide.
The tactics worked and Rovers were restricted to only a couple more good chances, with Kuffour blasting over the bar and Lambert seeing an injury-time shot held by goalkeeper Higgs.