Magic Reduction reduces the damage taken from the Spells Attack Type. Most spells and abilities deal Spell damage, especially nuker heroes such as Zeus, Lina, and Rylai Crestfall. Spell damage is not to be confused with the Magical Damage Type, which simply determines whether or not a skill's damage is reduced by Armor.
All heroes have an innate 25% magic reduction, and there are items which can be bought to further reduce the damage taken from this attack type. These items are, as of version 6.53, the Hood of Defiance(30%) and Planeswalker's Cloak(15%). Armor does not reduce Spell damage, unless the damage type is Normal, in which case the damage is commonly referred to as Mixed.
Stacking Spell Resistance
Damage Reduction stacks with diminishing returns, as described by the formula 1-((1-A)*(1-B)*(1-C)...) where A, B and C are the amounts of spell resistance converted into decimals. Several versions ago, one had to acquire the item before the skill in order to stack spell resistance properly, otherwise the item spell resistance would completely override the spell resistance that the skill gives. However, in a recent stealth change, the system was modified so that spell resistance stacks regardless of order of acquisition.