We have used three independent methods to determine an accurate and precise geometric center of SNR 0509-67.5, at RA=05:09:31.208, DEC=−67:31:17.48 (J2000). This supernova, which occurred approximately 400 years ago in the Large Magellanic Cloud, was confirmed to be a Type Ia by Rest et al. (2005), Rest et al. (2008) based on spectra of a light echo from the eruption. If this supernova had a single-degenerate progenitor system, we would see the “leftover” companion star within a certain distance of the remnant's center. Accounting for an offset due to enhanced ISM in the west-southwest quadrant of the remnant, we find the eruption position to be at RA=05:09:30.976, DEC=−67:31:17.90; the error circle which should contain any possible ex-companion star has a radius of 1.60″ for 99.73% (3-sigma) containment. This accounts for the proper motion of the stars, the possibility of kicks from the supernova, and asymmetries in the explosion and remnant expansion. We find no possible ex-companion stars within this ellipse, to a limiting magnitude of V=26.9: there are no red giants, which precludes symbiotic progenitors, no subgiants, which when combined with the lack of red giants precludes recurrent nova progenitors, and no main sequence stars with mass greater than 1.16 solar masses (V brighter than 22.7 mag), which precludes persistent supersoft X-ray source progenitors. Indeed, all published SD models are eliminated, so we conclude that this particular Type Ia supernova had a double-degenerate progenitor.