My research focuses on problems in Statistics, Data Science, Monte Carlo Simulation, and Computer Science. In particular, I consider hard problems in these areas and develop efficient computer algorithms to tackle those problems.


Some of the keywords associated with my research are Model Selection, Best Subset Selection, Deep Learning, MCMC Methods, Spatial Point Processes, Bayesian Inference, Perfect Sampling, Importance Sampling, Unbiased Estimation, Large Deviations Theory, Variance Reduction Techniques, and Queueing Theory.

 

Book Writing

Currently writing a book on The Mathematical Engineering of Deep Learning jointly with Prof. Benoit Liquet (MQ) and A/Prof. Yoni Nazarathy (UQ). Completed chapters are freely available at https://deeplearningmath.org/.

 

Ongoing Projects

The following are several research projects that are currently taking most of my time. If you are interested in discussing them, feel free to drop an email.

COMBSS: Continuous Optimization towards Best Subset Selection: Recent rapid developments in information technology have enabled the collection of high-dimensional complex data, including in engineering, economics, finance, biology, and health sciences. High-dimensional means that the number of features is large and often far higher than the number of collected data samples. In many of these applications, it is desirable to find a small best subset of predictors so that the resulting model has desirable prediction accuracy. This is a hard problem (NP-hard). In this project, we recast the challenge of best subset selection in linear regression as well as non-linear regression as a continuous optimization problem. We show that this reframing has enormous potential and substantially advances research into larger dimensional and exhaustive feature selection in regression, making available technology that can reliably select significant variables when the number of features is well in excess of 1000s. 

The first paper (preprint) is available at https://arxiv.org/abs/2205.02617.

Partial Rejection Sampling for Markov Random Fields: With the rapid acceleration of computational power over the last half-century, sampling techniques have become ubiquitous in engineering and scientific disciplines and financial and industrial applications. When attempting to sample from a probability distribution, these Monte Carlo techniques fall into two main categories: approximate methods and exact or perfect sampling techniques. A typical approximate approach is to construct a Markov chain whose distribution asymptotically converges to the target distribution. Two key drawbacks of Markov chain methods are that they are sequentially and it is hard to bound the approximation error. In this project, we develop partial rejection sampling methods for Markov random fields. These methods are exact and parallelizable. Some of the applications we focus on are Gibbs point processes, graph colouring, and sampling of solutions of stochastic differential equations.

Primary work from this project is published in Bernoulli (click here).

 

Large Deviations in Spatial Point Processes:   Various phenomena in statistical physics, wireless communication networks, forestry, chemistry, biology, and material science, to model phenomena as diverse as gas atoms in a chamber, tree locations in a forest, base stations of a cellular network over a city, to name a few. Estimating the rare-event probabilities of spatial point processes and establishing large deviation results of these rare events are important for developing efficient sampling and estimation methods.

 

Some of the published work in this direction:

Variance Reduction for Matrix Simulation:   This project focuses on variance reduction for matrix computations via matrix factorization, with a particular focus on the estimation of the log-likelihood of Gaussian processes.

 

The primary work is published in VALUETOOLS 2021 (Click here).