My wife took my young sons to Pismo Beach often. She let them play in the surf with no worries, because the surfers were vigilant of little kids.
As for afforable housing, we have friends that live inexpensively in rural AG, back in the hills. Very beautiful.
SLO is a great college town. Lots of engineering students and faculty for potential friends. Lots of fun things to do in downtown SLO.