Direct tunneling gate leakage current in metal-oxide-semiconductor (MOS) structures with ultrathin gate oxides is studied. The effects of inelastic scattering of inversion carriers in the gate-oxide region is taken into account in the current calculation. Open boundary conditions, incorporating the effects of wave function penetration into the gate oxide, are used to solve Schrödinger's equation. The proposed technique, based on the Green's function formalism, is numerically efficient and does not require determination of complex eigenenergies of a non-Hermitian matrix. Self-consistent calculations for n-type MOS devices are compared with experimental results. Excellent agreement between simulated and measured data is obtained when appropriate spatial and gate bias dependence of the inelastic scattering rate is taken into account. It is shown that due to inelastic scattering, at low gate voltages, the gate current increases significantly in devices with oxide thickness equal to 2 nm or higher. However, when the oxide thickness is reduced below 2 nm, inelastic scattering has no significant effect on gate current. The existing mismatch at lower gate voltages between experimental and modeled direct tunneling currents in devices with gate-oxide width equal to or greater than 2 nm is explained in terms of inelastic scattering effects.