The Astros lost their sixth in a row.
Jed Lowrie started the comeback with a leadoff home run against Chad Qualls (0-1).
Josh Donaldson walked and Yoenis Cespedes then singled. After Qualls struck out pinch-hitter John Jaso, Alberto Callaspo hit a tying single and Reddick followed with a one-out liner to right-center.
Dan Otero (3-0) pitched the ninth for the win after starter Scott Kazmir had gone eight strong innings.
Jonathan Villar homered and scored twice for Houston.
A day after scoring seven runs in the first inning and hitting four home runs overall, the A's were kept mostly quiet until their outburst in the ninth.
Oakland had stranded 12 runners through the first eight innings and left the bases loaded in the sixth.
Reddick's single off Qualls was the fourth game-ending hit of his career and continued a recent offensive surge for Oakland's outfielder. Reddick had two hits and a home run in Friday's 11-3 win after starting the season in a 4-for-41 rut.
The Athletics' rally in the ninth spoiled a solid effort from Houston starter Brett Oberholtzer, who allowed one run and eight hits over 5 2-3 innings but remains winless in four starts.
Oberholtzer pitched with runners on in each of the first five innings but limited the damage each time. He got Derek Norris to fly out with runners on the corners in the third, then speared Norris' comebacker and threw to first to get out of another two-out, two-on jam in the fifth.
Kazmir allowed two earned runs and six hits. He struck out five and walked none.
The Astros got to Kazmir early when Villar homered on an 0-2 pitch in the third. Villar also doubled off the wall in left-center in the fifth and scored on a throwing error by Norris, the catcher.
Oakland tied it at 1 when Craig Gentry, who crashed into the fence while backpedaling chasing after Villar's home run, hit an RBI double that scored Daric Barton.
Former A's first baseman Chris Carter doubled and scored on Matt Dominguez's RBI single in the fourth to put the Astros back in front.
The game was halted briefly with two outs in the top of the eighth when umpires lost track of the pitch count. A meeting was held near the mound before crew chief Mike Winters went to replay for clarification.
NOTES: Cespedes reached on an infield single in the fifth to extend his hitting streak against Houston to 14 games. ... Astros rookie outfielder George Springer doubled in the ninth and has a hit in all four games since being called up from the minors on Wednesday. ... A's reliever Eric O'Flaherty, on the disabled list since undergoing Tommy John surgery in the offseason, threw another bullpen session and could be ready to face hitters by the time Oakland returns from its next road trip in early May. ... RHP Jesse Chavez, winless in three starts despite a 1.35 ERA, pitches the series finale for Oakland. RHP Brad Peacock (0-1) makes his first start for Houston after opening the season in the bullpen.