The castle reconstruction project explores new ways for measuring, modeling, and estimating ancient structures using the ruins of a Crusader fortress, Apollonia-Arsuf, as a case example. Novel measurement devices have been developed that empower archaeologists to create 3D surface models by integrating a dense collection of (x,y,z) surface measurements from in-situ architectural remains with digital photographs. Another aspect of the project develops a specialized computer language, which allows users to specify shape grammar, enabling researchers to compactly represent complex architectural structures in terms of simpler sub-structures, e.g., the wall of a building consists of floors, each floor consists of brick and mortar with facade features inserted at specific locations such as windows. By changing the variables in the computerized shape grammar, different variations of an architectural structure can be automatically generated. The final part of the project seeks to automatically or semi-automatically reconstruct damaged and collapsed architecture from 3D scan data and a user-specified shape grammar. Here, the measured 3D scan data is used to constrain some or all of the variables of the shape grammar program in an effort to automatically or semi-automatically reconstruct the most likely structure of the damaged architecture. Work on this project is supported through NSF Grant NSF-IIS 0808718.