Swerford is older than the Domesday Book, in which it is called Surford. At that time it was owned by one of William the Conqueror's henchmen, Robert D'Oily. His rent to the king for the land was a tablecloth, and according to a local guidebook, this could be the origin of the term doily for a small tablecloth.
More information is provided on the West Oxfordshire Conservation Area Character Appraisal document, which, from pages 2-4, gives an excellent description of the character of the village and further information on its history. Click here to see this document.
Victoria County History has collected information on many of the villages around. The link below will take you to some very interesting documents on Swerford written by Stephen Mileson which are well worth reading. Scroll down to find Swerford & Showell.
There is more history of the village on the villages own website swerford.org , and there is the Geoffrey Castle pamphlet created in 1997 "Swerford a Village Guide", now sadly out of print,which covers other buildings of interest.