An injury to striker Steven Gillespie ten minutes before half-time saw left-winger Spencer switched up front and the move paid dividends.
The first half was goalless but the home side took the lead three minutes after the restart when Malvin Kamara slotted home.
Huddersfield should have capitalised but wayward finishing from Andy Booth, Chris Brandon and Luke Beckett proved costly.
Booth's downward header failed to test Shane Higgs and Brandon couldn't get enough curl on a shot which drifted wide.
The Terriers players had extra shooting practice this week after firing blanks in the 2-0 defeat at Crewe but they still missed chances.
Spencer pulled the Robins level just past the hour with a back-post header from Andy Lindegaard's free-kick.
Poor Huddersfield defending allowed Spencer to put Cheltenham ahead when he powered into the box and slammed a low shot into the bottom corner.
Seven minutes from time skipper John Finnigan sealed the points when he finished from close range after keeper Matt Glennon had saved from Spencer.
Centre-back Richard Keogh, making his home debut on loan from Bristol City, set up a frantic four minutes of stoppage time when he lashed home from 25 yards to give Huddersfield hope.
Beckett, who had an effort ruled out for offside in the first 20 seconds, was denied again at the end.
Beckett's header was disallowed for a foul and Huddersfield boss Andy Ritchie was left seething at what he later claimed was a poor performance from referee Rob Shoebridge.