His first, a belter from outside the box on 70 minutes, levelled for the hosts after Simon Church put the O's ahead, before the south Londoners' top scorer headed home the winner four minutes from the end.
Orient were handed a perfect opportunity to go ahead inside ten minutes when referee Graham Horwood pointed to the spot after Tony Craig caught Charlie Daniels inside the Lions' box.
But Church's on-loan strike partner, Scott McGleish, fired tamely at goal as Lions keeper David Forde guessed the right way to keep out his third penalty of the season.
The Lions dominated in every department from start to finish but could not find a way through the resilient backline of the Eastenders.
Centre-back Zak Whitbread tried his luck with a free-kick from inside his own half on 16 minutes that only missed the upright by a whisker, having spotted Jamie Jones off his line for what would have been an extraordinary goal.
Jason Price then forced a top save from the busy O's keeper with his volley on the turn, having found space inside the opposition box. Jones' one-hand parry turned the stinging effort over the bar.
James Henry wasted a brilliant opportunity to put the Lions ahead having raced into the clear, but he shot straight at keeper Jones who gratefully took it into his chest.
Immediately after the break, the O's took the lead after a clinical breakaway. Church got goal-side of Whitbread from McGleish's flick and the young Reading starlet slotted past Forde as the ball crept into the far corner.
David Martin had a chance to equalise on the hour mark but couldn't react quickly enough to turn home. Sub Neil Harris sent an inswinging cross into the danger area that Jordan Spence missed as the ball trickled into the winger's path, but the chance was gone.
The equaliser, that appeared to be coming, arrived in spectacular style. Sub Lewis Grabban picked out the frontman with a brilliant ball that Alexander chested past the last defender.
He then leathered it into the top corner leaving keeper Jones no chance as the Den rocked.
The game appeared to be petering out in a similar fashion to the previous encounter at Brisbane Road, but Alexander snatched the winner late on.
He rose highest at the far post to nod home Henry's hanging cross with four minutes to go as the Lions move up into third position ahead of MK Dons, while Orient remain clear of the drop zone.