Probably built c1870 and unaltered externally. It was built by Lady Llangattock as a Working Men's Gymnasium and may have been designed by Benjamin Lawrence of Newport who designed the contemporary Working Men's Free Institute for the same client (see Monk Street). Previous to 1970 the Nelson Museum was housed here, hence the name of the building.

A building constructed of coursed, squared, rock-faced, red sandstone rubble with Bath stone ashlar quoins and dressings and a tiled roof. The plan is a large hall above a basement, the building being two storeys, possibly partly with an attic, with the upper floor much taller than the ground floor. There is a rectangular corner block with four bays to Glendower Street and the gable end to Agincourt Street.
The Glendower Street elevation has three cross-framed windows on the left of the ground floor and a door flanked by single light ones on the right. The upper floor has windows, all with two transoms and all lights with leaded glazing. The 3-light window is in a slightly projecting oriel with an apron and a castellated parapet. There are three string courses, above and below the apron and below the castellated parapet, which go round the building. There is a coped gable with a single light window above the oriel and a blank plaque on parapet above the door.
The Agincourt Street elevation has a canted 2-storey bay with 1 + 3 + 1 lights, a single light window below, and with two transoms above, all with leaded lights. The first floor window has a stone apron and a castellated parapet. There is a plain 3-light window above, coped gable with ball finials.
The rear elevation is similar, but more plainly finished.

The building has a single room on each floor with the lower one having a much lower ceiling. The upper hall is in five bays with four false hammerbeam trusses, in fact being heavy tiebeams with brackets and arched braced collars above. There are vertical iron restraints through the centre of each truss. The eastern bay has a balcony.
(Source; Cadw listing database) S Fielding RCAHMW 05/06/2006