Millwall's win lifts them above Cheltenham in the table, but more importantly it means that a number of lower mid-table sides are now anxiously looking over their shoulders.
Alexander's strike - his seventh league goal of the season - came in the 55th minute when he timed his jump to perfection at the far post and met a Dave Brammer cross from far out on the right-hand side.
Cheltenham goalkeeper Shane Higgs had no chance with it and Millwall went on to defend their lead tenaciously for the rest of the game.
In fact, the home side, who had won their previous four games without conceding a goal, were unable to muster a shot on target until second-half injury time.
Both Steven Gillespie and Damian Spencer saw headers finger-tipped away by Millwall goalkeeper Rhys Evans as the Robins finally came to life in front of goal.
Cheltenham had played the better football in a low-key first half, although they almost fell a goal behind in the early stages when Alexander slipped between two defenders to meet a speculative ball from Scott Barron, but Higgs was out quickly to block.
Cheltenham's Alex Russell supplied Steve Brooker for a long-range shot that cleared the crossbar, but aside from that, the first half was lacking in goalmouth action.
The second half offered little more, although the driving midfield runs of Adrian Forbes and Ali Fuseini and the knowhow of the veteran Brammer proved too much for a Cheltenham side unable to recapture the form of recent weeks.
Meanwhile, Millwall worked hard and improved as the game wore on to claim their first win of 2008 and only their second away from home this season.