Qifan Song, Ph.D., Department of Statistics, Purdue University

"Bayesian sharp minimaxity via FDR penalization"

We consider Bayesian inference for the high dimensional regression problem, where y = Xβ + ε and β is the unknown sparse coefficient vector. In literature, various Bayesian approaches are proposed and shown to be consistent for model selection. These approaches have been viewed as gold standard, and commonly used in practice. However, in this work, we show that any model-selection consistent estimator can never be rate minimax over the parameter space of all sparse β, in terms of L2 error. Especially when the true β is relatively dense and contains many weak signals, model-selection consistent estimators are rate suboptimal. This also implies that in order to yield minimax estimation, it is necessary to sacrifice selection consistency and be tolerant to false discoveries. Inspired by the FDR-type penalization and its minimaxity under normal means model, we propose a Bayesian modeling that corresponds to L0 penalization ci=1k log(p/i) for general regression models, where k equals to the number of selected variables. We show that its corresponding posterior contraction rate is rate-minimax, and the number of false discoveries selected by posterior is bounded. More importantly, we find that for certain particular cases, i.e. X = √n I or all entries of X are i.i.d standard normal, the posterior is asymptotically sharply minimax, as

E[π(||β - β*||) ≥ (1 + ε) √ (2slog(p/s)/n) | X,y] → 0, 

and the number of false discoveries is bounded by delta s, for arbitrary small constants ε and δ, where s denotes the true sparsity.

Stay connected TwitterFacebookLinkedInYouTubeInstagram