The armory and arsenal were destroyed during the Civil War. The lower town was little more than a ghost town for many years until the National Park Service began restoration work in the 1950s.