It is difficult in summer to see the handsome Old Rectory because of the trees which have grown up on the boundary of the garden, but it is still possible to catch a glimpse. The Rectory was built for Revd Dr William Wotton between 1696 and 1711, replacing an earlier stone built house round a courtyard on the same site. It is the largest house in the village, and would be far too big for the present clergy to maintain. It was purchased from the Oxford Diocese by a private individual in the early 1960s, and was eventually acquired by the Milton Keynes Development Corporation and split into four separate apartments which were later sold on. It is a Grade 2 listed building.
The gardens were remodelled in the 1960s by the new owner, but other than that, the exterior is little changed. The garden wall facing Willen Road is built of stone and thus probably dates from the time of the earlier stone built Rectory. It is separately listed (Grade 2).
The Rectory was the home of all the Rectors of Milton Keynes from the time of William Wotton. The last Rector to reside there was John Franklin Cheyne. Only those Rectors having private means found it possible to maintain and even an attempt by the Diocese to increase the stipend of the Rector by joining the parishes of Broughton and Milton Keynes was only partially successful, and sale of the Rectory became inevitable.