Abstract: A thorough spectral study of the intrinsic single-photon detection efficiency in superconducting TaN and NbN nanowires with different widths has been performed. The experiment shows that the cut-off of the intrinsic detection efficiency at near-infrared wavelengths is most likely controlled by the local suppression of the barrier for vortex nucleation around the absorption site. Beyond the cut-off quasi-particle diffusion in combination with spontaneous, thermally activated vortex crossing explains the detection process. For both materials, the reciprocal cut-off wavelength scales linearly with the wire width where the scaling factor agrees with the hot-spot detection model.