arXiv Statistics @arxiv_stats@qoto.org

Sparsified Learning is ubiquitous in many machine learning tasks. It aims to regularize the objective function by adding a penalization term that considers the constraints made on the learned parameters. This paper considers the problem of learning heavy-tailed LSP. We develop a flexible and robust sparse learning framework capable of handling heavy-tailed data with locally stationary behavior and propose concentration inequalities. We further provide non-asymptotic oracle inequalities for different types of sparsity, including $\ell_1$-norm and total variation penalization for the least square loss.

Research on chemotherapy, heart surgery, and vaccines has indicated that the risks and benefits of a treatment could vary depending on the time of day it is administered. A challenge with performing studies on timing treatment administration is that the optimal treatment time is different for each patient, as it would be based on a patient's internal clock time (ICT) rather than the 24-hour day-night cycle time. Prediction methods have been developed to determine a patient's ICT based on biomarker measurements, which can be leveraged to personalize treatment time. However, these methods face two limitations. First, these methods are designed to output predictions given biomarker measurements from a single tissue sample, when multiple tissue samples can be collected over time. Second, these methods are based on linear modelling frameworks, which would not capture the potentially complex relationships between biomarkers and a patient's ICT. To address these two limitations, this paper introduces a recurrent neural network framework, which we refer to as LassoRNet, for predicting the ICT at which a patient's biomarkers are measured as well as the underlying offset between a patient's ICT and the 24-hour day-night cycle time, or that patient's dim-light melatonin onset (DLMO) time. A novel feature of LassoRNet is a proposed variable selection scheme that minimizes the number of biomarkers needed to predict ICT. We evaluate LassoRNet on three longitudinal circadian transcriptome study data sets where DLMO time was determined for each study participant, and find that it consistently outperforms state-of-the art in both ICT and DLMO time prediction. Notably, LassoRNet obtains a median absolute error of approximately one hour in ICT prediction and 30 to 40 minutes in DLMO time prediction, where DLMO time prediction is performed using three samples collected at sequential time points.

This paper presents a student-led activity designed to explore the use of statistical software in academic research across economics, political science, and statistics. Students reviewed replication files from major journals and repositories, gaining hands-on experience with reproducible workflows while contributing to cross-disciplinary datasets. Web-scraped metadata and student data collection, together covering more than 10,000 papers, reveal clear disciplinary patterns: Stata remains dominant in economics, while R is increasingly popular in political science and is the standard in statistics. Within the social sciences, a growing number of articles also use multiple software platforms within a single manuscript. Students reported increased understanding of academic workflows and greater awareness of software diversity in quantitative research. The activity is easy to adapt across course levels and disciplines, and we offer suggestions for follow-up assignments that reinforce key concepts in reproducibility and data fluency. The resulting insights into current software practices are also valuable for instructors seeking to align their teaching with evolving trends in research.

This paper aims to develop practical applications of the model for the highly technical measure-valued populations developed by the authors in \cite{FanEtal20}. We consider the problem of estimation of parameters in the general age and population-dependent model, in which the individual birth and death rates depend not only on the age of the individual but also on the whole population composition. We derive new estimators of the rates based on the use of test functions in the functional Law of Large Numbers and Central Limit Theorem for populations with a large carrying capacity. We consider the rates to be simple functions, that take finitely many values both in age $x$ and measure $A$, which leads to systems of linear equations. The proposed method of using test functions for estimation is a radically new approach which can be applied to a wide range of models of dynamical systems.

This paper investigates sequential change-point detection in reconfigurable sensor networks. In this problem, data from multiple sensors are observed sequentially. Each sensor can have a unique change point, and the data distribution differs before and after the change. We aim to detect these changes as quickly as possible once they have occurred while controlling the false discovery rate at all times. Our setting is more realistic than traditional settings in that (1) the set of active sensors - i.e., those from which data can be collected - can change over time through the deactivation of existing sensors and the addition of new sensors, and (2) dependencies can occur both between sensors and across time points. We propose powerful e-value-based detection procedures that control the false discovery rate uniformly over time. Numerical experiments demonstrate that, with the same false discovery rate target, our procedures achieve superior performance compared to existing methods, exhibiting lower false non-discovery rates and reduced detection delays.

Network pruning is used to reduce inference latency and power consumption in large neural networks. However, most existing methods struggle to accurately assess the importance of individual weights due to their inherent interrelatedness, leading to poor performance, especially at extreme sparsity levels. We introduce Hyperflows, a dynamic pruning approach that estimates each weight's importance by observing the network's gradient response to the weight's removal. A global pressure term continuously drives all weights toward pruning, with those critical for accuracy being automatically regrown based on their flow, the aggregated gradient signal when they are absent. We explore the relationship between final sparsity and pressure, deriving power-law equations similar to those found in neural scaling laws. Empirically, we demonstrate state-of-the-art results with ResNet-50 and VGG-19 on CIFAR-10 and CIFAR-100.

We review the literature on algorithms for estimating the index space in a multi-index model. The primary focus is on computationally efficient (polynomial-time) algorithms in Gaussian space, the assumptions under which consistency is guaranteed by these methods, and their sample complexity. In many cases, a gap is observed between the sample complexity of the best known computationally efficient methods and the information-theoretical minimum. We also review algorithms based on estimating the span of gradients using nonparametric methods, and algorithms based on fitting neural networks using gradient descent

Tangent approximation form a popular class of variational inference (VI) techniques for Bayesian analysis in intractable non-conjugate models. It is based on the principle of convex duality to construct a minorant of the marginal likelihood, making the problem tractable. Despite its extensive applications, a general methodology for tangent approximation encompassing a large class of likelihoods beyond logit models with provable optimality guarantees is still elusive. In this article, we propose a general Tangent Approximation based Variational InferencE (TAVIE) framework for strongly super-Gaussian (SSG) likelihood functions which includes a broad class of flexible probability models. Specifically, TAVIE obtains a quadratic lower bound of the corresponding log-likelihood, thus inducing conjugacy with Gaussian priors over the model parameters. Under mild assumptions on the data-generating process, we demonstrate the optimality of our proposed methodology in the fractional likelihood setup. Furthermore, we illustrate the empirical performance of TAVIE through extensive simulations and an application on the U.S. 2000 Census real data.

The gut microbiome plays a crucial role in human health, yet the mechanisms underlying host-microbiome interactions remain unclear, limiting its translational potential. Recent microbiome multiomics studies, particularly paired microbiome-metabolome studies (PM2S), provide valuable insights into gut metabolism as a key mediator of these interactions. Our preliminary data reveal strong correlations among certain gut metabolites, suggesting shared metabolic pathways and microbial co-metabolism. However, these findings are confounded by various factors, underscoring the need for a more rigorous statistical approach. Thus, we introduce microbial correlation, a novel metric that quantifies how two metabolites are co-regulated by the same gut microbes while accounting for confounders. Statistically, it is based on a partially linear model that isolates microbial-driven associations, and a consistent estimator is established based on semi-parametric theory. To improve efficiency, we develop a calibrated estimator with a parametric rate, maximizing the use of large external metagenomic datasets without paired metabolomic profiles. This calibrated estimator also enables efficient p-value calculation for identifying significant microbial co-metabolism signals. Through extensive numerical analysis, our method identifies important microbial co-metabolism patterns for healthy individuals, serving as a benchmark for future studies in diseased populations.

Degradation data are essential for determining the reliability of high-end products and systems, especially when covering multiple degradation characteristics (DCs). Modern degradation studies not only measure these characteristics but also record dynamic system usage and environmental factors, such as temperature, humidity, and ultraviolet exposures, referred to as the dynamic covariates. Most current research either focuses on a single DC with dynamic covariates or multiple DCs with fixed covariates. This paper presents a Bayesian framework to analyze data with multiple DCs, which incorporates dynamic covariates. We develop a Bayesian framework for mixed effect nonlinear general path models to describe the degradation path and use Bayesian shape-constrained P-splines to model the effects of dynamic covariates. We also detail algorithms for estimating the failure time distribution induced by our degradation model, validate the developed methods through simulation, and illustrate their use in predicting the lifespan of organic coatings in dynamic environments.

High-dimensional vector autoregressive (VAR) models offer a versatile framework for multivariate time series analysis, yet face critical challenges from over-parameterization and uncertain lag order. In this paper, we systematically compare three Bayesian shrinkage priors (horseshoe, lasso, and normal) and two frequentist regularization approaches (ridge and nonparametric shrinkage) under three carefully crafted simulation scenarios. These scenarios encompass (i) overfitting in a low-dimensional setting, (ii) sparse high-dimensional processes, and (iii) a combined scenario where both large dimension and overfitting complicate inference. We evaluate each method in quality of parameter estimation (root mean squared error, coverage, and interval length) and out-of-sample forecasting (one-step-ahead forecast RMSE). Our findings show that local-global Bayesian methods, particularly the horseshoe, dominate in maintaining accurate coverage and minimizing parameter error, even when the model is heavily over-parameterized. Frequentist ridge often yields competitive point forecasts but underestimates uncertainty, leading to sub-nominal coverage. A real-data application using macroeconomic variables from Canada illustrates how these methods perform in practice, reinforcing the advantages of local-global priors in stabilizing inference when dimension or lag order is inflated.

Computer experiments with quantitative and qualitative inputs are widely used to study many scientific and engineering processes. Much of the existing work has focused on design and modeling or process optimization for such experiments. This paper proposes an adaptive design approach for estimating a contour from computer experiments with quantitative and qualitative inputs. A new criterion is introduced to search for the follow-up inputs. The key features of the proposed criterion are (a) the criterion yields adaptive search regions; and (b) it is region-based cooperative in that for each stage of the sequential procedure, the candidate points in the design space is divided into two disjoint groups using confidence bounds, and within each group, an acquisition function is used to select a candidate point. Among the two selected points, a point that is closer to the contour level with the higher uncertainty or that has higher uncertainty when the distance between its prediction and the contour level is within a threshold is chosen. The proposed approach provides empirically more accurate contour estimation than existing approaches as illustrated in numerical examples and a real application. Theoretical justification of the proposed adaptive search region is given.

Non-linear survival models are flexible models in which the proportional hazard assumption is not required. This poses difficulties in their evaluation. We introduce a new discrimination measure, time-dependent Uno's C-index, to assess the discrimination performance of non-linear survival models. This is an unbiased version of Antolini's time-dependent concordance. We prove convergence of both measures employing Nolan and Pollard's results on U-statistics. We explore the relationship between these measures and, in particular, the bias of Antolini's concordance in the presence of censoring using simulated data. We demonstrate the value of time-dependent Uno's C-index for the evaluation of models trained on censored real data and for model tuning.

Restricted Boltzmann machines (RBMs) are energy-based models analogous to the Ising model and are widely applied in statistical machine learning. The standard inverse Ising problem with a complete dataset requires computing both data and model expectations and is computationally challenging because model expectations have a combinatorial explosion. Furthermore, in many applications, the available datasets are partially incomplete, making it difficult to compute even data expectations. In this study, we propose a approximation framework for these expectations in the practical inverse Ising problems that integrates mean-field approximation or persistent contrastive divergence to generate refined initial points and spatial Monte Carlo integration to enhance estimator accuracy. We demonstrate that the proposed method effectively and accurately tunes the model parameters in comparison to the conventional method.

In social science researches, causal inference regarding peer effects often faces significant challenges due to homophily bias and contextual confounding. For example, unmeasured health conditions (e.g., influenza) and psychological states (e.g., happiness, loneliness) can spread among closely connected individuals, such as couples or siblings. To address these issues, we define four effect estimands for dyadic data to characterize direct effects and spillover effects. We employ dual instrumental variables to achieve nonparametric identification of these causal estimands in the presence of unobserved confounding. We then derive the efficient influence functions for these estimands under the nonparametric model. Additionally, we develop a triply robust and locally efficient estimator that remains consistent even under partial misspecification of the observed data model. The proposed robust estimators can be easily adapted to flexible approaches such as machine learning estimation methods, provided that certain rate conditions are satisfied. Finally, we illustrate our approach through simulations and an empirical application evaluating the peer effects of retirement on fluid cognitive perception among couples.

Reinforcement learning from human feedback (RLHF) has emerged as a key technique for aligning the output of large language models (LLMs) with human preferences. To learn the reward function, most existing RLHF algorithms use the Bradley-Terry model, which relies on assumptions about human preferences that may not reflect the complexity and variability of real-world judgments. In this paper, we propose a robust algorithm to enhance the performance of existing approaches under such reward model misspecifications. Theoretically, our algorithm reduces the variance of reward and policy estimators, leading to improved regret bounds. Empirical evaluations on LLM benchmark datasets demonstrate that the proposed algorithm consistently outperforms existing methods, with 77-81% of responses being favored over baselines on the Anthropic Helpful and Harmless dataset.

Forecast combination methods have traditionally emphasized symmetric loss functions, particularly squared error loss, with equally weighted combinations often justified as a robust approach under such criteria. However, these justifications do not extend to asymmetric loss functions, where optimally weighted combinations may provide superior predictive performance. This study introduces a novel contribution by incorporating modal regression into forecast combinations, offering a Bayesian hierarchical framework that models the conditional mode of the response through combinations of time-varying parameters and exponential discounting. The proposed approach utilizes error distributions characterized by asymmetry and heavy tails, specifically the asymmetric Laplace, asymmetric normal, and reverse Gumbel distributions. Simulated data validate the parameter estimation for the modal regression models, confirming the robustness of the proposed methodology. Application of these methodologies to a real-world analyst forecast dataset shows that modal regression with asymmetric Laplace errors outperforms mean regression based on two key performance metrics: the hit rate, which measures the accuracy of classifying the sign of revenue surprises, and the win rate, which assesses the proportion of forecasts surpassing the equally weighted consensus. These results underscore the presence of skewness and fat-tailed behavior in forecast combination errors for revenue forecasting, highlighting the advantages of modal regression in financial applications.

Persistent homology is an area within topological data analysis (TDA) that can uncover different dimensional holes (connected components, loops, voids, etc.) in data. The holes are characterized, in part, by how long they persist across different scales. Noisy data can result in many additional holes that are not true topological signal. Various robust TDA techniques have been proposed to reduce the number of noisy holes, however, these robust methods have a tendency to also reduce the topological signal. This work introduces Maximal TDA (MaxTDA), a statistical framework addressing a limitation in TDA wherein robust inference techniques systematically underestimate the persistence of significant homological features. MaxTDA combines kernel density estimation with level-set thresholding via rejection sampling to generate consistent estimators for the maximal persistence features that minimizes bias while maintaining robustness to noise and outliers. We establish the consistency of the sampling procedure and the stability of the maximal persistence estimator. The framework also enables statistical inference on topological features through rejection bands, constructed from quantiles that bound the estimator's deviation probability. MaxTDA is particularly valuable in applications where precise quantification of statistically significant topological features is essential for revealing underlying structural properties in complex datasets. Numerical simulations across varied datasets, including an example from exoplanet astronomy, highlight the effectiveness of MaxTDA in recovering true topological signals.

Selecting important features in high-dimensional survival analysis is critical for identifying confirmatory biomarkers while maintaining rigorous error control. In this paper, we propose a derandomized knockoffs procedure for Cox regression that enhances stability in feature selection while maintaining rigorous control over the k-familywise error rate (k-FWER). By aggregating across multiple randomized knockoff realizations, our approach mitigates the instability commonly observed with conventional knockoffs. Through extensive simulations, we demonstrate that our method consistently outperforms standard knockoffs in both selection power and error control. Moreover, we apply our procedure to a clinical dataset on primary biliary cirrhosis (PBC) to identify key prognostic biomarkers associated with patient survival. The results confirm the superior stability of the derandomized knockoffs method, allowing for a more reliable identification of important clinical variables. Additionally, our approach is applicable to datasets containing both continuous and categorical covariates, broadening its utility in real-world biomedical studies. This framework provides a robust and interpretable solution for high-dimensional survival analysis, making it particularly suitable for applications requiring precise and stable variable selection.