This paper proposes the use of a physical structure’s Transmissibility functions as input to a novel composite architecture consisting of Deep CNN followed by multivariate linear regressors to detect, localize, and quantify the damage extent in a system.