It depends what time of the day you log in too. I'm in the GMT time zone, I can literally see a comment go from +20 upvotes in the morning to negative numbers when Americans start waking up. It really shifts your perspective of the site too, because comments move down or even disappear based on the number of votes.
I would strongly encourage everyone to read HN with `showdead` enabled (it's in your profile page). There aren't actually all that many downvoted comments, and while mosts are low-level trolling, even with `showdead` you see them at the end of the parent thread and they are greyed out, so it's not all that distracting. But being able to see some of the things that get downvoted / killed unjustly (and then vouch & upvote them) is how you get a better HN.
You can "vouch" for them, which makes them non-dead (and upvotable again). But, yes, it does have some karma limit - I'm not sure if the specifics are documented anywhere, the FAQ just says "small karma threshold".