Children, homeless people and even city officials entered the Berger building unaware they were inhaling toxic dust.