Get Greasemonkey and you can limit the video speed controller to certain websites or turn it off when you're not using it. (And also disable auto update of the script).