There was an ancient city standing on the place of Mahdia today during the Phoenicians and roman times and was destroyed during the Arab conquest of North Africa. In Fatimid caliphs time, Obeid Allah, the first Fatimid caliph known as El-Mahdi, choose the place of this narrow peninsula as a port and base in 916 AD, as it was considered a suitable and easy to defend place of refuge for his Shiite followers. He also saw it as an advantageous costal base from where he can launch his attacks on his ultimate goal, Egypt. The Fatimid Calipsh Abdallah El Fatimi after that made Mahdia the capital city of Ifriqiya.

Thick defensive walls were surrounding the city of Mahdia during the Fatimid period that reached up to 10m in thickness and ran across the peninsula at its narrowest point at the place of Skifa el-Kahla today, with smaller walls encircling the remaining area of the peninsula. The areas within the walls were reserved for the Mahdi and his entourage while inhabitants of the city were living outside these walls. Fatimids finally abandoned Mahdia as the capital of their kingdom in AD 947.

Mahdia reached its present state from the 14th century when the city became the wealthiest city on the Barbary Coast. Now, the city of Mahdia is inhabited by some 30,000 Tunisians, however the areas of living was reversed from their previous locations as now most of the inhabitants live in the modern suburbs in the west from Skifa El Kahla or the Black Passage. Some structures in the city dates from the 10th and 11th centuries, such as the Great Mosque and the Casbah, helped make the city an important tourist attraction as well.