||We present a detailed experimental study of the direct detection effect in a small volume (0.15pm x lpm) NbN hot electron bolometer mixer. It is a quasioptical mixer with a twin slot antenna designed for 700 GHz and the measurement was done at a LO frequency of 670 GHz. The direct detection effect is characterized by a change in the mixer bias current when switching broadband radiation from a 300 K hot load to a 77 K cold load in a standard Y factor measurement. The result is, depending on the receiver under study, an increase or decrease in the receiver noise temperature. We find that the small signal noise temperature, which is the noise temperature that would be observed without the presence of the direct detection effect, and thus the one that is relevant for an astronomical observation, is 20% lower than the noise temperature obtained using 300 K and 77 K calibration loads. Thus, in our case the direct detection effect reduces the mixer sensitivity. These results are in good agreement with previous measurement at THz frequencies . Other experiments report an increase in mixer sensitivity . To analyze this discrepancy we have designed a separate set of experiments to find out the physical origin of the direct detection effect. Possible candidates are the bias current dependence of the mixer gain and the bias current dependence of the IF match. We measured directly the change in mixer IF match and receiver gain due to the direct detection effect. From these measurements we conclude that the direct detection effect is caused by a combination of bias current reduction when switching form the 77 K to the 300 K load in combination with the bias current dependence of the receiver gain. The bias current dependence of the receiver gain is shown to be mainly caused by the current dependence of the mixer gain. We also find that an increase in receiver sensitivity due to the direct detection effect is only possible if the noise temperature change due to the direct detection is dominated by the mixer-amplifier IF match.  J.J.A. Baselmans, A. Baryshev, S.F. Reker, M. Hajenius, J.R. Gao, T.M. Klapwijk, Yu.Vachtomin, S. Maslennikov, S. Antipov, B. Voronov, and G. Gol'tsman., Appl. Phys. Lett. 86, 163503 (2005).  S. Svechnokov, A. Verevkin, B. Voronov, E. Menschikov. E. Gershenzon, G. Gol'tsman, 9th Int. Symp. On Space THz. Techn., 45, (1999).