In this study, the effects of recycled glass (RG) and recycled concrete (RC) fine aggregates on acid and sulfate resistance as well as alkali-silica reaction (ASR) expansion of mortar mixture were investigated comparatively. In addition, durability performance of mortar mixtures was supported by microstructural analysis. For this purpose, nine different mortar mixtures were prepared by replacing 25, 50, 75, and 100 wt% of crushed-limestone fine aggregate with recycled aggregates. Except for ASR and acid resistance, the sulfate resistance of mortar mixture improved by using RG aggregate. The performance of mortar in terms of ASR and acid sulfuric resistance increased by increasing RC aggregate replacement level. The ASR expansions in RG mixtures increased by increasing the RG aggregate content up to 100%. The RC mixtures containing more than 50% recycled aggregate, showed better performance than the control mixture in terms of sulfate resistance.