Building work began in 1295 on the castle on the 'fair marsh', 'Beau Mareys' in Norman French. It was the last of the great Welsh castles built for Edward I by his chief military architect, Master James of St George.

Construction of the castle began immediately following the quelling of a Welsh uprising under Madog Ap Llewelyn. At its peak the enterprise employed around 2600 men, but by 1298 the substantial funds required had dried up and work came to a halt. Building resumed between 1306 and 1330, but at a much reduced scale, and the great plans for the castle were never completed. The lavish accommodation planned for the north gatehouse never acquired its second storey, while the block planned for the south gate never rose above its footings. None of the towers gained their turrets, and it is uncertain whether any of the buildings such as the hall, kitchens and stables were ever built.

The castle has an almost perfect symmetrical concentric layout, with a high inner ring of defences surrounded by a lower outer circuit of walls, surrounded by a moat. At the southern end was a tidal dock for shipping, protected by the shooting deck on Gunners' Walk. The castle would have been virtually impregnable, but its defences were never truly put to the test.