Latex is basically natural rubber. The minor differences between pure natural rubber and latex have to do with chemical compounds added to the liquid rubber sap as it is processed. These chemicals enhance the ability of natural rubber to maintain "solid surface tension" while being stretched (so holes don't appear in the stretched glove) and also enhance the natural rubber's resistance to ultraviolet radiation (sunlight). In addition, of course, the latex is almost always given a tint with special dyes while it is still in liquid state.
These chemical additives are generally non-allergenic. The common allergy that some people develop to latex products is an allergy to the natural rubber proteins. Those who develop an allergy to latex most often switch to nitrile or vinyl gloves.