St Mary's Church was sited to the immediate east of St Mary's passage (which seperates the site from the Guildhall (now Carmarthen crown and migistrates courts). Its south side bordered St Mary's Street. The church is known to have dated to before 1282, in which year Henry III directed that places to the west of the church be marked out for shops with fixed rents payable to the king. In 1328 leave was given to alienate certain lands to the priory of St John and to find two chaplians to celebrate divine service daily in the chapel of St Mary's, for the souls of the King's progenitors, Kings of England, John Gogh, the King's clerk, Walter Winter, Griffin de Cauntyton and the faithful departed. In chantry certificates of Edward VI (1547-1553) the church is refered to as 'one ffree chappell of Our Ladie comonly callyd the Rood Churche in the town of Karmarden'. At this time it posessed one silver chalice, one set of vestments of blue satin , two alter cloths, and two small bells (valued at a total of 15 shillings). Nicholas Byford was chantry priest, and lands and tenements brought in 9 pounds and 10 shillings per annum. In 1655 the town Corporation granted the lease of a house'lying and being over agaist the east ent of the Towne Hall, the wall of the Rood Church on the south side, and the street called St Mary on the north side'. In 1701 the Corporation granted 'the half roof on the south side of the dissolved church...'. A length of carved stone arcading, thought to have been part of the chancel decoration, was excavated around 1830. Its total length is given as 8ft 6in, with a depth of 1ft 3in. The piece is noted to have consisted of three arched compartments, each with a pier dividing it, and each with a figure in the centre. It was subsequently broken into three sections (each consisting of an arch). In 1917 one section had reportedly been inserted into the wall at the corner of St Mary's Lane (Passage). The second section had been built into the exterior wall of St Peter's Vicarge, with the third section having been placed in the baptistry of St David's Church (NPRN 100103).

The remains of the church were identified during building work around 1957, and recorded by the curator of Carmarthenshire Museum. The east wall was reveled, with an arch at its north-east end and a tomb recess at its north-west end (the outer arch of which was preserved in the wall of the adjacent building). The position of the chancel's south wall was recorded, and elements of a thirteenth century piscina were removed to Carmarthenshire Museum. Various other architectural fragments are noted to have been collected, including thirteenth century hollow chamfered arch voisoirs (similar to those at Caernarfon Castle) and window fragments (some of contemporary style and some with perpendicular moulding). In 1957 these were located in the open yard at the rear of Carmarthenshire Museum (then located in nearby Quay Street).

N Vousden, RCAHMW, 26 March 2013