The antecedent of the Royal Palace of Visegrád was the house built by the king Károly Róbert after his seat was placed there in 1323. Zách Felicián attacked the royal family in this house in 1330. However, the association was not fulfilled as the tutors of the young princes hindered it. After the death of Károly in 1242 Lajos I. began expanding the house of his father in town. He began building a chapel; however, the works were halted due to wars in Venice. Still, the king moved to Buda. The construction works of the palace of Visegrád restarted only in the second half of the 1350'. The palace of Lajos I. was really made up of a lot of smaller buildings on a large area. He only began to build a more regular palace towards the end of his life. That his predecessors, the queen Mária and Zsigmond of Luxemburg finished at the end of the 14th century and the first years of the 15th century.

The new palace was located on 123 x 123 m. The gate tower stood in the middle of the facade facing the street. On one side the kitchen, on the other side stores and wards attached to the tower. The first yard of the palace was a large inn yard. Flower gardens of the king and the queen belonged to the building. A large orchard on the north and a Franciscan monastery to the right were built to it. Water from sources were led in the palace which also supplied the fountains and baths.

The suits with colorful tile stoves were all furnished with luxury. King Zsigmond moved his court to Buda from 1405 until 1408 and began the building of a new palace. This However did not mean that he neglected his palace in Visegrád. However, his predecessors used it less by the middle of the 15th Century and by 1470 it was in a bad shape. After king Mátyás married Beatrix he began to rebuild the palace in late gothic style but renaissance designs already appeared on it. The white marble for the palace probably came from Venice, from the workshop of Verrocchio. The red marble probably came from Gregorio di Niccoló of Florence.

Ulászló II, the follower of Mátyás on the throne continued the reconstruction works with the renovation of the Franciscan monastery and the living quarters in the palace of Visegrád. The palace turned ruins in the Turkish times, later in the 18th century even the ruins were moved. The excavations began in 1834 and are still going on.