I'm not much of an athlete, and I don't have a smartwatch, but the idea of leaving my phone at home sometimes without being totally disconnected from comms does appeal to me somewhat.
Would that not just be replacing a phone with another smaller phone strapped to your wrist?
Doesn't the fact that you are connected and communicable make whatever device you choose to use essentially a phone?
I will say, if it is possible, going out without any form of internet/comms enabled device can be very liberating. We all used to do it, and I think many of us have gotten used to the idea that we need to be on call or have some sort of utility in case of emergencies that are very unlikely to happen.