Castle Stalker is a small tower house that is sited on an islet, known as the Rock of the Cormorants, at the mouth of Loch Laich. It was built in the mid 15th century by Sir John Stewart, Lord of Lorn. The castle was only ever reachable by boat, which made it highly defendable.

In around 1620 the Castle passed into the hands of the Campbells of Airds as a result of a drunken wager by the 7th Stewart Chief, Duncan, in exchange for an eight-oared wherry. The castle did come back into the hands of the Stewarts for a short while before being lost to the Campbells once more in 1690. The castle was finally abandoned around 1840 when it lost its roof. It was bought, in 1965, by Lt. Col. Stewart Allward who spent the next ten years restoring it.