Structural health monitoring (SHM) is currently applied to control regularly the health of high-rise buildings which have deteriorated after being subjected to a sudden loading. Damage detection at element levels of a structure consisting of an enormous number of elements becomes the main objective. In this study, the complicated problem is simplified by a two-step solution. Damaged storeys are preliminarily detected before a full damage scenario at an element level is achieved. In Step 1, to overcome the issues related to the huge number of degrees of freedom (DOFs), the full building is simplified to a beam-like system using the Guyan condensation technique. As the natural characteristics of the two lowest modes at the intact and a damaged stage are obtained, the eigenvalue problem based inverse solution is applied to approximately detect damaged storeys. Furthermore, an updating procedure that is proposed in this study effectively enhances the first prediction. In Step 2, an artificial neural network (ANN) model is designed to indicate damaged members on detected storeys using only the first three modal modes. Compared to other approaches applied to detect damages on high-rise buildings, the robustness of the proposed method is that the required number of lowest modal modes is two and three in Step 1 and Step 2 respectively. Furthermore, regardless of the extension of the building in the horizontal direction, only one lateral displacement of each storey is measured to detect damaged storeys in Step 1 and generally detect damaged elements in Step 2. For light and asymmetrical damage scenarios, two more vertical displacements should be considered to obtain accurate element-level detection. However, for all cases, the required number of DOFs is significantly lower than the full system.