The south London club are through to the last eight after Shaun Cummings scored a 90th-minute winner to heap more pressure on Leicester manager Claudio Ranieri and his beleaguered side.
Millwall were reduced to 10 men for much of the second half after Jake Cooper was dismissed for his second bookable offence following a rash tackle on Ahmed Musa.
But the deficit only sought to galvanise the League One club, with Cummings cooly scoring the winner in the closing stages of the game.
"I think people thought we had a chance, and we did," said Harris
"We assumed Claudio would make changes but the quality of player you are playing against is outstanding.
"We were the underdog
There is no doubting that
We knew we could cause Leicester problems by playing high-tempo, aggressive football.
"Am I surprised we won? I wouldn't say surprised
We knew we would have to catch Leicester cold, which we did for the last half hour."
Ranieri, with one eye on his side's trip to Sevilla in the Champions League on Wednesday night and their precarious position just above the relegation zone in the Premier League, made 10 changes for the visit to Millwall.
Demarai Gray was the only survivor from the team which lost at Swansea last Sunday, with Jamie Vardy and Riyad Mahrez relegated to the bench.
It was a decision which backfired on the Italian and courted fierce criticism from former Leicester striker Gary Lineker.
He tweeted: "Congratulations to Millwall
This reserve team stuff though in the 5th round of the FA Cup is utter b*******
4 days until Champions League.
"Great to be in the @ChampionsLeague but the obsession with a competition that there's zero chance of winning has hugely damaged @LCFC season."
Ranieri, however, defended his team selection.
"We made 10 changes also against Derby in the last round and we won the match," he said.
"In this period when you play three matches in nine days you need to give also the opportunity to other players.
"If you play well and you win, okay, don't change too much and continue with the same team
But when you don't play well, you lose a match, you are to give the opportunity to the other players
"They showed a good performance 11 v 11 but we lost the desire and the concentration because they found a goal and nobody marked the man."
Source : PA