Colonel James McKay, a British Army officer and Scotsman, began construction on his home at Prospect Hill, a site in Buffalo that overlooked the Niagara River and is now the eastern terminal of the Peace Bridge. He modeled his home on a Scottish castle design; it was one story with numerous rooms and a central passage that ran from front to back.
McKay never finished his home and sold it in 1847 to the U.S. government which incorporated it into the new Fort Porter. The building was used by a military engineering corps. In 1926, Fort Porter was eliminated as a military post and the land it occupied was divided between the City of Buffalo and the new Fort Erie Public Bridge Company. The 'castle' was on bridge land and the City of Buffalo paid to have it dismantled and moved onto the city's portion of the tract. It served as an office for the Buffalo Parks Department; a portion was used by the Girl Scouts.
Read about Fort Porter Training here.