Yes, move the stupid button.
It automatically places your mouse pointer there. As I was in a hurry today to finish a quest, I just clicked and Turbine successfully stole my shards. I would never use this stupid function and would absolutely turn it off if I had the choice.
It's quite clearly where it is to get you to spend your shards via misclicks - an utterly dishonest business model.