The home side, meanwhile, look condemned to relegation after their sixth consecutive defeat and 13th match without a win.
It all looked so different at half-time with Cheltenham having played some confident football against a sluggish Millwall side that took a full 45 minutes to get going.
Cheltenham midfielder Ian Westlake saw a good early chance saved by David Forde and it was generally the bottom-of-the-table side that made the running in the early stages.
They were rewarded after half-an-hour when teenage winger Morley Watkins, making his first start for the first team, delivered a cross from the right and Millwall defender Richard Duffy turned the ball into his own net.
Watkins almost made it 2-0 at the start of the second half when he headed just wide from a Nicholas Bignall cross but the game turned on a decision made by an assistant referee five minutes into the second period.
Impressive winger James Henry darted down the right and fired over a low cross that hit defender Lee Ridley. Referee Scott Mathieson waved away the penalty claims but his assistant had a better view and signalled for a penalty.
Henry took it himself, beating goalkeeper Scott Brown with a low shot, and from that point on the confidence grew in Millwall and drained from Cheltenham.
Ten minutes after the equaliser came the decisive second goal, substitute Ashley Grimes scoring at close range from another inviting Henry cross, and after that it was merely a question of whether Millwall would get any more.
That they duly did in the final minute of normal time when Grimes turned provider with a low cross from the left after a quick breakaway. Fellow substitute Marc Laird had made good ground through the middle and beat Brown with a low finish.