Although St Andrews castle is in a ruined state it has two features that are very popular with visitors. There is a mine dug by attackers during a siege of 1546-7, and a countermine dug by the defenders in an attempt to halt the attack. The countermine successfully connected with the mine, and although both mines were filled in after the siege, they were excavated in the 19th century and can now be explored by visitors. The other popular feature is a bottle dungeon cut into solid rock beneath the Sea Tower.

The castle was the main residence of Scotland's leading churchmen, the bishops, and later the archbishops, of St Andrews. It is located on a headland protected by the sea on two sides and by rock-cut ditches on the landward side. There was a castle on the site from the late 12th century, but most of the existing remains date from the 14th century and later.

In 1290, the castle was captured by the English forces of King Edward I, but in 1314 it was retaken by the Scots. It was back in English hands by the 1330's but was once again recaptured by the Scots, led by Sir Andrew Moray. The castle was then heavily slighted to prevent any further use by the English. In the late 14th century the castle was rebuilt by Bishop Walter Trail, and it was his castle that became the basis for all later development.

The castle was greatly strengthened by Archbishop James Beaton during the first half of the 16th century. Heavy artillery was now a serious threat and the castle was strengthened to withstand such an attack and at the same time it was modified to make use of the new weapons with the addition of two circular gun towers at each end of the main front. These new defences would soon be put to the test. The successor to James Beaton was his nephew David Beaton, who became Archbishop of St Andrews and Cardinal of the Church in 1538. He was an ardent Catholic who became very unpopular in an increasingly Protestant Scotland. In March 1546, he had the Protestant preacher, George Wishart, burnt at the stake in front of the castle. In May of that year a group of Protestant Fife lairds gained access to the castle, killed the Cardinal and hung his naked body from the castle walls. A long siege followed, with the castle bombarded from land and sea, reducing much of it to rubble. Many of the defeated garrison, including John Knox, the future leader of the Protestant Reformers, were sent to act as galley slaves in the French fleet.

The castle was rebuilt after the siege by Archbishop John Hamilton, but following his death in 1571 it was mainly occupied by a succession of constables. In 1606 Parliament separated the castle from the archbishopric and it was granted to the Earl of Dunbar, constable since 1603. In 1612 it was returned to Archbishop Gordon Gledstanes, but further attempts to re-establish the former estates of the Archbishop failed. The castle had lost its importance, and by 1654 stone was being removed from the castle to repair the harbour.