Does a stuffy nose, itchy eyes, and trouble sleeping at night affect your ability to go about your daily life with a little discomfort?
You can still eat, talk, and breathe with a stuffy nose, so primitive man could still eat, communicate about important and unimportant matters, and run and do other physical tasks requiring hard breathing.
You can still drive with itchy eye, still read with itchy eyes, so the aim of primitive man might not be affected enough to prevent them from killing their food or gathering it.
You can still drive with trouble sleep, you can still go about daily tasks with poor sleep, so primitive man could still be a functioning member of his or her community, even if they don’t feel like it.
The fact is, if our ancestors couldn’t hunt/survive with these symptoms, none of us would be here. So this did not affect them the way you might think it would.
Also, evolution goes for “good enough”. Sickle cell disease will greatly shorten your lifespan, especially short without treatment. However, it also protects equatorial populations from malaria (large mosquito populations pass malaria to humans), as those with the disease are immune. Some random genetic mutation kept someone alive long enough to reproduce and pass the gene on to their kids, and their kids lived long enough to reproduce, etc. Those who got malaria were pretty much dead, so a lot of babies and children died along with adults. But people with sickle cell didn’t, so they passed on their genes with enough gusto that it still exists today.
Even though primitive man with sickle cell wasn’t going to live as long as other adults in his community who never had that disease or contracted malaria, evolution never selected against it because it worked to pass on the genes. If sickle cell mutates to the point where it kills people before puberty, it will be selected against since there will be too few people to pass it on.
These primitive people with allergies? Whatever the effects of those allergies, they stayed alive long enough to have kids, who stayed alive long enough to have more kids, etc. That’s the reason we are not immune to pollen.