The decision came after a string of poor results, culminating in a 1-0 home defeat by Preston North End last night. Club captain Dennis Wise has taken over on a temporary basis.
The Lions currently sit in eigth place in Division One and are just six points behind leaders Sheffield United but the South London club have played more games than their competitors and recent performances have left a lot to be desired. McGhee constantly played about with the formation of the side and recently told popular players Robbie Ryan and Matt Lawrence that their services were no longer required.
Speaking to the club's official website, chairman Theo Paphitis said: "Last season was a frustrating one for the club and whilst expectations were high at the start of the current campaign, we have struggled to live up to them."
McGhee joined Millwall three years ago and led the club to promotion from the Second Division during his first season in charge. The following season The Lions narrowly missed out on promotion, losing out in the play-offs to Birmingham City but the former Wolves and Reading boss endured a difficult campaign last season.
Paphitis added: "We met with Mark after the Preston game on Tuesday night and he agreed that it is in the best interests of both parties that there is a parting of the ways. "
Graham Rix has already thrown his hat into the ring and over candidates are thought to include former Arsenal and England captain Tony Adams.