Etal Castle started out as a three-storey tower house, but its location near the border with Scotland made it vulnerable to attack. In 1341, the owner, Robert Manners, was granted a licence to fortify his home. He created a roughly square courtyard enclosed by curtain walls, with the tower house in one corner and a large gatehouse diagonally opposite and a tower at each of the other corners. The tower house was improved with the addition of another storey and crenellations.

By the start of the 16th century the Manners were living elsewhere and the castle was in the care of a constable. In 1513 the castle fell to the army of James IV of Scotland during his failed invasion of England. James was killed nearby during the Battle of Flodden, when a hastily recruited army of 20,000 Northerners decisively beat his army of 30,000 Scots.

In 1549 the castle was ceded to the Crown, possibly in an attempt to reduce the neglect of this strategic border castle. With the union of the English and Scottish crowns in 1603 Etal ceased to have any military purpose and the decay, which had already set in was allowed to continue unabated.