I'm having a hard time grasping the concept so bear with me.
By this logic does this mean that if a baby drowns in a bucket 10 feet from you and you sit in a chair and do nothing, since it's not your baby and you did nothing, then you have no responsibility? and in fact, are entitled to complain about the parents bad decisions?
This is where you are misunderstanding.
If one has the power, the ability, the way/knowledge to actually help the baby, and he does not, then he should have known better. He should have been responsible and saved the baby. The parents are equally (more so, actually) responsible.
The difference in the baby and the car/kid example, and the voting discussion, is that in the case of the baby, you have the ability to effect/change something, where as the kid/voting you have 0 ability to effect/change something.
I'm not having the ability to change the country, my vote is useless, it doesn't do anything (neither does yours, which is why I said I don't complain anyways).
The better example of what you're saying would be if there is a baby drowning on the other side of a glass wall that I cannot break, where I cannot save him, can I complain about the parents who are on the right side of the wall that can actually help the baby but choose not to?