Nulling interferometry enables astronomers to advance beyond the resolving power of ground-based telescopes with the goal of directly detecting exo-planets. By diminishing the overwhelming emission of the host star through destructive interference, radiation from young companions can be observed. The atmospheric transmission window centered around 4 µm wavelength is of particular interest because it has a favorable contrast between star and planet as well as a reduced atmospheric disturbance. For robustness and high stability, it is desirable to employ integrated devices based on optical waveguide technology. Their development is hindered at this wavelength range due to the lack of suitable host materials and compatible fabrication techniques to create low-loss photonic devices. This paper details our work on femtosecond laser direct-written optical waveguides and key components for an on-chip nulling interferometer inside gallium lanthanum sulphur glass. By combining cumulative heating fabrication with the multiscan technique, single-mode optical waveguides with propagation losses as low as 0.22 ± 0.02 dB/cm at 4 µm and polarization-dependent losses of < 0.1 dB/cm were realized. Furthermore, S-bends with negligible bending loss and broadband Y-splitters with 50/50 power division across a 600 nm wavelength window (3.6 - 4.2 µm) and low losses of < 0.5 dB are demonstrated. Directional couplers with an equal splitting ratio complement these main building blocks to create a future compact nulling interferometer with a total projected intrinsic loss of < 1 dB, a value that is sufficient to perform future on-sky experiments in relatively short observation runs on ground-based telescopes.