Swainby is a village to the south of Stokesley surrounded by drystone walls, hills, sheep and moorland.The parish has always had a sound economy owing to the abundance of clean water for both drinking and industry. In the 19 century ironstone and jet were mined and the population rose to its highest level
Here are the remains of a castle supposed to have been erected about the time of Richard II. little now remains of the ancient fortress, except the lofty gateway tower, on which may yet be seen the arms of D'Arcy, Meynell, and Gray; who seem to have been successively lords of this mansion, an Augustinian cell to Guisbrough priory was founded in the time of Henry I; and both have left some vestiges.
Following the death of Sir James Strangeways in 1541 his Whorlton Estate was granted to Matthew, Earl of Lennox, whose wife was niece to Henry VIII. A letter was said to have been written at Whorlton Castle by the Countess which successfully proposed a marriage between her son Henry Lord Damley and Mary Queen of Scots.
The Estate was granted in 1604 to Thomas Bruce by their son, James I, remaining in the same family until 1887 when it was sold to James Emerson of Easby Manor. The Bruces felled the woodland and divided Whorlton Great Park into five farms.