Castle Doyle is a Rural locality located in New South Wales and part of the Rest of NSW , with a population of 164. Castle Doyle Castle Doyle covers an area of 90.983 square km. Its elevation is 963 meters. It is governed by Armidale Regional (Area). .