Politician and mineral collector. Ferguson lived on the family estates at Raith, near Kirkcaldy. He became a member of the Royal Company of Archers in 1792, and along with his brother, Lieutenant-General Sir Ronald Ferguson (1773 - 1841), features in the Raeburn portrait The Archers. He travelled across Europe between 1795 and 1805 collecting minerals and fossils. The mineral Fergusonite is named after him.
Ferguson was Member of Parliament for Kirkcaldy (1831-2), then the first County Member for East Lothian (1835-7) and Lord Lieutenant of Fifeshire (1837). He was also a fellow of the Royal Societies of London and Edinburgh.
He married Lady Elgin, a native of Dirleton (East Lothian), following one of many scandalous affairs. Her husband, Lord Elgin (of the Elgin Marbles fame), sued Ferguson for the breakdown of his marriage and won £10,000.
He died in London, but is remembered by a remarkable memorial at Haddington.