In this paper, we study a bivariate shot noise self-exciting process. This process includes both externally excited joint jumps, which are distributed according to a shot noise Cox process, and two separate self-excited jumps, which are distributed according to the branching structure of a Hawkes process with an exponential fertility rate, respectively. A constant rate of exponential decay is included in this process as it can play a role as the time value of money in economics, finance and insurance applications. We analyse this process systematically for its theoretical distributional properties, based on the piecewise deterministic Markov process theory developed by Davis (1984), and the martingale methodology used by Dassios and Jang (2003). The analytic expressions of the Laplace transforms of this process and the moments are presented, which have the potential to be applicable to a variety of problems in economics, finance and insurance. In this paper, as an application of this process, we provide insurance premium calculations based on its moments. Numerical examples show that this point process can be used for the modelling of discounted aggregate losses from catastrophic events.