Threave Castle stands on an island in the middle of the River Dee. During the medieval period the water level was higher and the island was only about a third of the size that it is today. Legend tells that Threave Island was the home of the ancient rulers of Galloway, but there is no physical proof of this. Archaeological evidence does seem to suggest that there was an earlier castle on the site, which by tradition was the residence of Fergus, Lord of Galloway, and his descendants from the mid 11th century onwards, and was probably destroyed by Robert the Bruce in 1308.

The current remains date back to the late 14th century when Archibald 'The Grim', Third Earl of Douglas, built a castle on the island. The fortress was centred on one of the earliest tower houses to be built in Scotland. The Douglas family were the most powerful family in Scotland and dominated Scottish politics. By the mid 15th century the king, James II, was determined to break their power by systematically destroying all the major Douglas strongholds across the country. By June 1455, Threave was the last castle still being defended in the name of the now exiled Earl of Douglas. During a two month siege the king used the latest cannons and bombards (giant siege guns) in an attempt to take the castle by force, but its final surrender was more likely due to bribery than the effectiveness of these weapons. The castle's ability to stand up to such an attack was probably due to the artillery fortification that was built around the tower house shortly before 1455. The artillery wall had vertical slits through which defenders could fire long-bows and cross-bows, and at three of the corners was a round tower that was designed to accommodate small guns. It was one of the first artillery defences to be built in Britain.

Threave Castle then became a Crown property. A succession of custodians was put in charge, and in 1513 the Maxwells were made keepers, a position that became hereditary in 1526. The last Maxwell keeper finally abandoned the castle in 1640 following a 13 week siege by an army of Covenanters. The castle was slighted and partly dismantled, but was still substantial enough in the early 19th century to act as a prison for French troops captured during the Napoleonic wars.