The rectory, also known as Staveley Hall, was built in c 1604 at the north east corner of St John the Baptist Church on the site of one of more preceeding houses. It was acquired by Sir Peter Frecheville in 1681, the Frecheville Family graves can be seen the church. The rectory was partially demolished in the early 19th century, and remodelled in a jacobean style by Scott in 1867. The building is now used as District Council Offices.