The original part of Palazzo Vescovile was built on foundations provided by part of the Roman amphitheatre, to the left of the Duomo, soon after the diocese was re-establised in 1218.

As part of the restoration of the Duomo in ca. 1650, Bishop Cardinal Francesco Angelo Rapaccioli commissioned:

1. 

   ✴the extension to Palazzo Vescovile, to the left of the Duomo; and
   

1. 

   ✴Palazzo del Seminario, to the right, which now houses the Museo Diocesano. 
   

These palaces were not completed until 1740.