If the belly button goes unclean for too long, sweat, dirt and dead skin cells can get trapped inside, causing a foul odour. It can also cause candida, a fungus that grows naturally on the skin. Candida can overgrow when it comes into contact with moist, unclean environment like the belly button. Not cleaning it may also cause burning rash.