Farnham Castle was built by Henry of Blois, Bishop of Winchester, during the first half of the 12th century. Little is known about this early castle other than it had a tower, possibly made of stone, on top of a large motte. This was demolished on the orders of Henry II in 1155, but recent archaeological excavations have revealed the large basement and foundations of the original tower at the centre of the motte. The castle was rebuilt in the late 12th and early 13th centuries, with the earlier motte entirely surrounded by a shell keep, and an outer curtain wall constructed. The space between the wall of the shell keep and the slopes of the motte were filled in sometime in the 13th century.

The castle remained a residence of the Bishops of Winchester through to the early 20th century, and many new buildings were added within the castle grounds. The most impressive of these date from the mid 17th century and were built by Bishop George Morley. These courtyard buildings and grounds are now privately owned, although occasional tours can be arranged. The keep is managed by English Heritage and is open to the public.