Why not allow the option to refund an ability point, assuming you have not used any abilites yet?

Would it be that bad to allow such a thing? I just hate putting points into the wrong ability, really hurts.