Overlooking Sennybridge are the scanty remains of what is thought to be a thirteenth century castle. The remains consist of a ruinous wall, about 26m east-north-east to west-south-west which could form part of the southern side of the castle enclosure. A D-shaped tower projects from the centre of the wall and is about 8.5m across with 1.8m thick walls. It stood up to 7.6m high in 1973.

The castle may have been constructed by Llywelyn ab Gruffudd from 1262 and was possibly the 'new castle beyond Brecon' destroyed by the then Lord Edward in 1265. Einion Sais, a local Welsh lord or prince, dated letters from Rhyd-y-briw in 1271.
The apparent form of the castle can be compared to those of Dolforwyn (NPRN 300392) and Dinas Bran (NPRN 307064), both castles built by Welsh princes which feature projecting D-plan towers.

Two south-facing WWII pill-boxes have been built into and camouflaged as, the castle remains.

John Wiles, RCAHMW, 24 July 2007