Recently, transformerless (TL) inverters are being extensively used in small-scale photovoltaic (PV) systems due to their compact-size, lighter-weight, lower-cost and higher-efficiency compared to their counterpart with transformer. However, the growing penetration of these small-scale based PV systems and low inertia induction motor loads in low-voltage distribution networks (DNs) make the networks more vulnerable to short-term voltage stability (STVS). Hence, this paper thoroughly examines the STVS of DN with high-penetration of TL-PV units, and provides countermeasures by TL-PV systems to improve the STVS. The detailed dynamic modelling of the TL inverter is developed first. Next, three control strategies: (1) constant peak current (CPC), (2) constant active current (CAC), and (3) constant active power with low voltage ride through (LVRT) and dynamic voltage support (DVS) capability are proposed to improve STVS. The impacts of different level of penetration and LVRT capability of TL-PV inverter on the STVS are explored. Moreover, countermeasures such as LVRT with CAC and CAP control and DVS capability by the TL-PV systems are designed and implemented. Several case studies are carried out on an IEEE-4 bus system first, and later extended to IEEE-13 node test feeder. The results show that DVS can further improve STVS in low-voltage DNs.