The museum building is the larges renaissance mansion of Hungary. This makes it a highly valued monument. The history of the building stretches back long before. In the 14th century it was part of the Tokaj castle. After the Ottoman occupation it was transmitted to the Serédi, then the Alaghy family. In 1571 the Mágóchy family obtained ownership over it. The walls of the mansion still senn today were erected between 1560 and 1581.