Quantifying mangrove soil organic carbon (SOC) is key to better understanding the global carbon cycle, a critical phenomenon in reducing greenhouse gas emissions. However, it is challenging to have a large sample size in soil carbon measurements and analysis due to the high costs associated with them. In the current research, we propose a novel hybridized artificial intelligence model based on the categorical boosting regression (CBR) and the particle swarm optimization (PSO) algorithm for feature selection, namely, the CBR-PSO model for estimating mangrove SOC. We integrated multisensor optical (Sentinel-2) and synthetic aperture radar (Sentinel-1 and ALOS-2 PALSAR-2) remote sensing data to construct and verify the proposed model, drawing upon a survey in 85 soil cores at 100 cm depth in the Red River Delta, Vietnam. The CBR-PSO model estimated the mangrove SOC ranging from 44.74 to 91.92 Mg ha−1 (average = 68.76 Mg ha−1) with satisfactory accuracy (coefficient of determination (R 2) = 0.809 and root-mean-square error (RMSE) = 9.30 Mg ha−1). We also compared the proposed model’s capability with four machine learning techniques, i.e. support vector regression (SVR), random forest regression (RFR), extreme gradient boosting regression (XGBR), and XGBR-PSO models. We show that multimodal and multisensor earth observation dataset combined with the CBR-PSO model can significantly improve the estimates of mangrove SOC. Our findings contribute novel and advanced machine learning approaches for robustness of SOC estimation using open-source software. Our novel framework, which is automated, fast, and reliable, developed in this study can be easily applicable to other mangrove ecosystems across the world, thus providing insights for a voluntary blue carbon offset marketplace for sustainable mangrove conservation.