Crichton Castle stands in a secluded rural setting overlooking the River Tyne. It was constructed in several distinct phases over a 200 year period. The earliest part is a tower house built by John de Crichton at the end of the 14th century. It was three storeys high with a vaulted basement that contained a kitchen and pit-prison with a hall and private chambers above. The top storey was removed in a later remodelling of the castle.

In the mid 15th century, William Crichton, a powerful man who held the office of Chancellor, developed the castle into a residence more suitable for a man of his high standing. He incorporated the tower house into a range of buildings around a square courtyard. William spent most of his life in conflict with the Black Douglases and played his part in the infamous 'Black Dinner' in Edinburgh Castle at which the Sixth Earl of Douglas and his brother were lured to their death. In 1444, Sir John Forrester of Corstorphine, one of the Earl of Douglas's aides, marched on Crichton and destroyed the castle.

In 1484 the estates and offices of the Crichton family were forfeited after the third Lord Crichton was implicated in a conspiracy against the king. The estate of Crichton was bestowed upon Sir John Ramsey of Bothwell, later made Lord Bothwell and Treasurer of Scotland. He fled to England after the defeat of his king at the battle of Sauchieburn in 1488. The now vacant Lordships of Crichton and Bothwell were combined to create the Earldom of Bothwell that was first granted to Sir Patrick Hepburn.

In 1559 James, fourth Earl of Bothwell, and third husband of Mary Queen of Scots, seized a sum of 4000 crowns sent by Elizabeth I of England to assist the Protestant Lords. In retaliation the Earl of Arran besieged and captured Crichton Castle. Following Mary's defeat in 1567 James fled to Elgin and Crichton was left without an owner until 1581 when Francis Stewart, grandson of the third Earl, became Earl of Bothwell. He was a well travelled and educated man who had spent a lot of time in Italy and Spain, but he was also known as a bit of a ruffian. He set about transforming Crichton in the renaissance style he had seen on his travels in Europe. He remodelled much of the castle, adding an arcade to the north range at ground-level, facing into the courtyard, and dressed the wall above with a diamond-faceted facade. In 1595 the Earl's unruly behaviour got the better of him and he was forced to flee abroad, never to return.