Gwydir Uchaf was built as a summer house in 1604, the site, on top of the crag above the main house, being chosen for its views. Later in the seventeenth century it was found more convenient than the castle as the main residence and at some stage it was extended to the west, but in the eighteenth century the castle again became the principal house and eventually Gwydir Uchaf became derelict through neglect, and was partly demolished - Thomas Pennant, writing in the 1780s, says that this had happened 'lately'. Pennant also mentions an inscription which had been over the door, as follows:

Bryn Gwedir gwelir goleu adeilad
Uwch dolydd a chaurau
Bryn gwiech adail yn ail ne;
Bron wen Henllys bren hinlle.

This he translated as 'A conspicuous edifice on Gwedir hill, towering over the adjacent land; a well-chosen situation, a second paradise, a fair bank, a palace of royalty', which may not be strictly accurate but is faithful to the spirit of the original.

In 1808 the ruins of Gwydir Uchaf were inhabited by 'poor persons', but later in the nineteenth century the house was restored, and has since become the headquarters of the Llanrwst Forest District of Forest Enterprise. It is a two-storey stone building with a central porch on the main block, and a slate roof.

Gwydir Uchaf chapel was built on the crag near Gwydir Uchaf as a private chapel for the Wynnes in 1673 (at which time the family were probably resident at Gwydir Uchaf). The chapel seems never to have been licensed. The family employed a chaplain, and in Pennant's time four services a year were still held there. It is a small, simple building of dressed sandstone with a slate roof, and it retains its original painted ceiling, crudely but colourfully executed on a boarded barrel vault.