Herbert House, formerly known as The Friars or White Friars, survived until 1967 as the incomplete remains of a large house of the late 16th century, standing on the site of Grey Friars, a Franciscan friary. After the friary had been dissolved in 1538, the site was sold to Sir George Herbert of Swansea who subsequently became Earl of Pembroke. His grandson, Sir William Herbert has been credited with having built the house in 1582.

Full descriptive account may be found in 'An Inventory of the Ancient Monuments in Glamorgan: Volume IV: Domestic Architecture from the Reformation to the Industrial Revolution: Part I. The Greater Houses', RCAHMW 1981

Demolished June, 1967. Datestone 1582.