Not that that's my intention, but yes—in a certain sense, fun has to be justified, for example, by competence. Having no competence and having a lot of fun isn't a good mix.
Furthermore, your comparisons are flawed because they are innocently chosen. But shouldn't nefarious capitalists, ruthless politicians, and uncritical managers also be allowed to have fun - at the expense of others?