No shame necessary
The elemental attributions of the Major Arcana come from their astrological correspondances, which, in the Thoth, are at the bottom of the card next to its name. So, for example, the Hermit is Virgo, which is an Earth sign. The Tower is Mars, which is Fire. Three of the Majors--the Fool, the Hanged Man, and the Aeon--have only elemental correspondances (Air, Water, and Fire respectively) without an astrological aspect. But mainly it's a matter of knowing your astrology.