Yarmouth Castle was the last coastal fort built by Henry VIII. When Henry broke away from the Catholic Church, he ordered a series of artillery forts to be built all along the south coast to protect the country from the threat of invasion from Catholic France and Spain. However it was not seen necessary to build a fort at Yarmouth. This opinion changed in 1545 when a French force sailed into the Solent and landed on the Isle of Wight, safe from the guns and forts on the mainland. The local militia defeated the French, but a weakness in the island’s defences had been revealed and a castle was soon being constructed at Yarmouth.

Yarmouth Castle does not share the same design as Henry’s other coastal forts, with their semicircular bastions fronting a higher, central tower; instead Yarmouth is a simple square with no central tower. The north and west sides are washed by the sea and the east and south sides were protected by a moat. There is a single ‘arrow-head’ bastion on the south-east corner, the first example of its kind in Britain. The castle was completed by 1547, but did not remain unchanged for long. Within a few decades the northern half of the courtyard was filled with earth to form a large gun battery and domestic buildings were constructed in the southern half leaving just a narrow passage between.

Towards the end of the 17th century the moat was filled in and a house, now the George Hotel, was built on the east side. The main entrance to the castle, which had been on this side, was moved to the south. The fort never saw action but was garrisoned until 1885.