Try not to put on clothes that are uncomfortable, but the most important thing is that you are protected.
CAUSE
It is true that children lose more heat through the head, and that hats (which also cover their ears) are highly recommended in cold weather. But the baby is not aware of it. He is used to walking home with nothing on his head, and when you put it on to go outside, he notices that something strange bothers him and most likely he tries to take it off.
SOLUTION
If you do not tolerate it, you can take the child in the stroller protected by the plastic. When it is not possible, another option is the caps that are closed with a velcro strip or snaps, because they cannot be removed. Be careful not to tie the models too tightly with ribbons, as they can be pulled and hurt.