Orient wasted an early chance after Gary Alexander found himself one on one with Higgs, but the former Exeter striker dragged his shot wide. Taylor was then unlucky to see his powerful free-kick cannon back off the cross bar.
Odejayi scored his first Cheltenham goal after 23 minutes. Latching on to a long ball from Jamie Victory, the former Forest Green striker headed over Morris to give his side a deserved lead.
Orient’s Wayne Purser then fired in a free-kick almost identical to Taylor’s effort and once again the ball rebounded to safety after crashing down off the bar.
Cheltenham extended their lead on the half-hour mark, with Greg Heald deflecting Darren Jones’ long-range effort past Morris and into his own net.
Two minutes later, Taylor made it three, receiving the ball from Bertrand Cozic before supplying a neat finish.
Good work from Amankwaah on the right-wing led to Taylor’s second and Cheltenham’s fourth. The on-loan defender beat two defenders and produced a powerful ball into the box that Taylor calmly finished from close-range.
The first half ended with Orient reduced to 10 men. Goalkeeper Morris was sent off for impeding Odejayi when the quick forward was through on goal. Morris departed and midfielder Justin Miller was substituted for replacement keeper Lee Harrison.
This resulted in unrest in sections of the home support, many of whom were already calling for the sacking of boss Paul Brush.
Graham Fyfe fired narrowly wide as Cheltenham attempted to make if five in the second half. Antony Griffin replaced Mike Duff at the centre of defence as the Robins played lengthy periods of keep-ball, much to the frustration of the home side.
Richard Forsyth and Martin Devaney were also given a run-out as Cheltenham played out the remainder of the half to make it two wins in a row.