One of the main reasons why some men are unable to grow a beard or have a patchy beard is their genes, which decide how thick the facial hair grows, according to dermatologist Dr John Anthony. He added that patchiness may be due to alopecia areata, which can resolve by itself. Other reasons may be stress and diet, he said.