It is well-known that axions, if they exist, are produced in hot, dense stellar cores, e.g. those of white dwarfs and neutron stars. Once produced, they free-stream out of the star due to their weak interactions with matter, like neutrinos, but unlike neutrinos, they may convert into photons in the magnetosphere of the object. These photons are in the X-ray energy range due to the ~keV temperatures of the stellar cores. I discuss past and ongoing searches for axion-induced X-ray emission at both white dwarfs and neutron stars using data from the XMM-Newton and Chandra telescopes. In these searches, we discovered an X-ray excess at several neutron stars that is well-explained by the axion interpretation; I discuss ongoing follow-up searches with the NuSTAR telescope that can further probe the origin of this excess.