A castle was built on this Monastic site in 1228 by the Anglo-Norman House of Burke.
After more than three-and-a-half centuries under the de Burgos, whose surname became Burke or Bourke, Ashford passed into the hands of a new master, following a fierce battle between the forces of the de Burgos and those of the English official Sir Richard Bingham, Lord President of Connaught, when a truce was agreed. In 1589, the castle fell to Bingham, who added a fortified enclave within its precincts.
Dominick Browne, of the Browne Family (Baron Oranmore) received the estate in a Royal Grant in either 1670 or 1678. In 1715, the estate of Ashford was established by the Browne family and a hunting lodge in the style of a 17th-century French chateau was constructed. The double-headed eagles still visible on the roof represent the coat of arms of the Brownes.
In the late 18th-century a branch of the family inhabited the castle. In the early 19th-century, one Thomas Elwood was agent for the Brownes at Ashford and was recorded as living there in 1814.