arXiv Statistics @arxiv_stats@qoto.org

Classic supervised learning involves algorithms trained on $n$ labeled examples to produce a hypothesis $h \in \mathcal{H}$ aimed at performing well on unseen examples. Meta-learning extends this by training across $n$ tasks, with $m$ examples per task, producing a hypothesis class $\mathcal{H}$ within some meta-class $\mathbb{H}$. This setting applies to many modern problems such as in-context learning, hypernetworks, and learning-to-learn. A common method for evaluating the performance of supervised learning algorithms is through their learning curve, which depicts the expected error as a function of the number of training examples. In meta-learning, the learning curve becomes a two-dimensional learning surface, which evaluates the expected error on unseen domains for varying values of $n$ (number of tasks) and $m$ (number of training examples). Our findings characterize the distribution-free learning surfaces of meta-Empirical Risk Minimizers when either $m$ or $n$ tend to infinity: we show that the number of tasks must increase inversely with the desired error. In contrast, we show that the number of examples exhibits very different behavior: it satisfies a dichotomy where every meta-class conforms to one of the following conditions: (i) either $m$ must grow inversely with the error, or (ii) a \emph{finite} number of examples per task suffices for the error to vanish as $n$ goes to infinity. This finding illustrates and characterizes cases in which a small number of examples per task is sufficient for successful learning. We further refine this for positive values of $\varepsilon$ and identify for each $\varepsilon$ how many examples per task are needed to achieve an error of $\varepsilon$ in the limit as the number of tasks $n$ goes to infinity. We achieve this by developing a necessary and sufficient condition for meta-learnability using a bounded number of examples per domain.

We describe the DISC (Different Individuals, Same Clusters) design, a sampling scheme that can improve the precision of difference-in-differences (DID) estimators in settings involving repeated sampling of a population at multiple time points. Although cohort designs typically lead to more efficient DID estimators relative to repeated cross-sectional (RCS) designs, they are often impractical in practice due to high rates of loss-to-follow-up, individuals leaving the risk set, or other reasons. The DISC design represents a hybrid between a cohort sampling design and a RCS sampling design, an alternative strategy in which the researcher takes a single sample of clusters, but then takes different cross-sectional samples of individuals within each cluster at two or more time points. We show that the DISC design can yield DID estimators with much higher precision relative to a RCS design, particularly if random cluster effects are present in the data-generating mechanism. For example, for a design in which 40 clusters and 25 individuals per cluster are sampled (for a total sample size of n=1,000), the variance of a commonly-used DID treatment effect estimator is 2.3 times higher in the RCS design for an intraclass correlation coefficient (ICC) of 0.05, 3.8 times higher for an ICC of 0.1, and 7.3 times higher for an ICC of 0.2.

In epidemiological research, causal models incorporating potential mediators along a pathway are crucial for understanding how exposures influence health outcomes. This work is motivated by integrated epidemiological and blood biomarker studies, investigating the relationship between long-term adherence to a Mediterranean diet and cardiometabolic health, with plasma metabolomes as potential mediators. Analyzing causal mediation in such high-dimensional omics data presents substantial challenges, including complex dependencies among mediators and the need for advanced regularization or Bayesian techniques to ensure stable and interpretable estimation and selection of indirect effects. To this end, we propose a novel Bayesian framework for identifying active pathways and estimating indirect effects in the presence of high-dimensional multivariate mediators. Our approach adopts a multivariate stochastic search variable selection method, tailored for such complex mediation scenarios. Central to our method is the introduction of a set of priors for the selection: a Markov random field prior and sequential subsetting Bernoulli priors. The first prior's Markov property leverages the inherent correlations among mediators, thereby increasing power to detect mediated effects. The sequential subsetting aspect of the second prior encourages the simultaneous selection of relevant mediators and their corresponding indirect effects from the two model parts, providing a more coherent and efficient variable selection framework, specific to mediation analysis. Comprehensive simulation studies demonstrate that the proposed method provides superior power in detecting active mediating pathways. We further illustrate the practical utility of the method through its application to metabolome data from two cohort studies, highlighting its effectiveness in real data setting.

Model selection/optimization in conformal inference is challenging, since it may break the exchangeability between labeled and unlabeled data. We study this problem in the context of conformal selection, which uses conformal p-values to select ``interesting'' instances with large unobserved labels from a pool of unlabeled data, while controlling the FDR in finite sample. For validity, existing solutions require the model choice to be independent of the data used to construct the p-values and calibrate the selection set. However, when presented with many model choices and limited labeled data, it is desirable to (i) select the best model in a data-driven manner, and (ii) mitigate power loss due to sample splitting. This paper presents OptCS, a general framework that allows valid statistical testing (selection) after flexible data-driven model optimization. We introduce general conditions under which OptCS constructs valid conformal p-values despite substantial data reuse and handles complex p-value dependencies to maintain finite-sample FDR control via a novel multiple testing procedure. We instantiate this general recipe to propose three FDR-controlling procedures, each optimizing the models differently: (i) selecting the most powerful one among multiple pre-trained candidate models, (ii) using all data for model fitting without sample splitting, and (iii) combining full-sample model fitting and selection. We demonstrate the efficacy of our methods via simulation studies and real applications in drug discovery and alignment of large language models in radiology report generation.

A central question in multimodal neuroimaging analysis is to understand the association between two imaging modalities and to identify brain regions where such an association is statistically significant. In this article, we propose a Bayesian nonparametric spatially varying correlation model to make inference of such regions. We build our model based on the thresholded correlation Gaussian process (TCGP). It ensures piecewise smoothness, sparsity, and jump discontinuity of spatially varying correlations, and is well applicable even when the number of subjects is limited or the signal-to-noise ratio is low. We study the identifiability of our model, establish the large support property, and derive the posterior consistency and selection consistency. We also develop a highly efficient Gibbs sampler and its variant to compute the posterior distribution. We illustrate the method with both simulations and an analysis of functional magnetic resonance imaging data from the Human Connectome Project.

In a multiple regression model where features (predictors) form an orthonormal basis, we prove that there exists a uniformly most powerful unbiased (UMPU) test for testing that the coefficient of a single feature is negative (or zero) versus positive. The test statistic used is the same as the coefficient t-test commonly reported by standard statistical software, and has the same null distribution. This result suggests that orthogonalizing features around the predictor of interest, prior to fitting the multiple regression, might be a way to increase power in single coefficient testing, compared to regressing on the raw, correlated features. This paper determines the conditions under which this is true, and argues that those conditions are fulfilled in a majority of applications.

This short text tried to establish a big picture of what evidential statistics is about and how an ideal inference method should behave. Moreover, by examining shortcomings of some of the currently used methods for measuring evidence and utilizing some intuitive principles, we motivated the Relative Belief Ratio as the primary method of characterizing statistical evidence. Number of topics has been omitted for the interest of this text and the reader is strongly advised to refer to (Evans, 2015) as the primary source for further readings of the subject.

Information Theory provides a fundamental basis for analysis, and for a variety of subsequent methodological approaches, in relation to uncertainty quantifi- cation. The transversal character of concepts and derived results justifies its omnipresence in scientific research, in almost every area of knowledge, particu- larly in Physics, Communications, Geosciences, Life Sciences, etc. Information- theoretic aspects underlie modern developments on complexity and risk. A proper use and exploitation of structural characteristics inherent to spatial data motivates, according to the purpose, special considerations in this context. In this paper, some of the most relevant approaches introduced, in particular recent contributions and directions, regarding the informational analysis of spa- tial data and related aspects concerning complexity analysis, are reviewed under a conceptually connective evolutionary perspective. The discussion involves the cases of spatial data from magnitude measurements and spatial point patterns, with the latter possibly being of a multifractal nature.

In experimental and observational data settings, researchers often have limited knowledge of the reasons for censored or missing outcomes. To address this uncertainty, we propose bounds on causal effects for censored outcomes, accommodating the scenario where censoring is an unobserved mixture of informative and non-informative components. Within this mixed censoring framework, we explore several assumptions to derive bounds on causal effects, including bounds expressed as a function of user-specified sensitivity parameters. We develop influence-function based estimators of these bounds to enable flexible, non-parametric, and machine learning based estimation, achieving root-n convergence rates and asymptotic normality under relatively mild conditions. We further consider the identification and estimation of bounds for other causal quantities that remain meaningful when informative censoring reflects a competing risk, such as death. We conduct extensive simulation studies and illustrate our methodology with a study on the causal effect of antipsychotic drugs on diabetes risk using a health insurance dataset.

This paper addresses the critical and challenging task of developing emulators for simulating human operational motions in industrial workplaces. We conceptualize human motion as a sequence of human body shapes. Leveraging statistical shape theory, we develop statistical generative models for sequences of human (body) shapes of human workers in workplaces. Our goal is to create emulators that generate random, realistic human motions statistically consistent with past work performances, essential for simulating industrial operations involving human labor. We derive the space of shapes as a Riemannian shape manifold, modeling human motion as a continuous-time stochastic process on this manifold. Representing such processes is challenging due to the nonlinearity of the shape manifold, variability in execution rates across observations, infinite dimensionality of stochastic processes, and population variability within and across action classes. This paper studies multiple solutions to these problems. This paper proposes multiple solutions to these challenges, presenting a comprehensive framework that incorporates (1) time warping for temporal alignment of training data, (2) Riemannian geometry for tackling manifold nonlinearity, and (3) Shape- and Functional-PCA for dimension reduction leading to traditional finite-dimensional Euclidean representations. In particular, it develops the transported velocity field representation for motion sequences and imposes a Gaussian model on the resulting Euclidean spaces. It then applies these models to emulate human shape sequences from an industrial operation dataset and evaluates these simulations in multiple ways.

The reliability of clinical trial outcomes is crucial, especially in guiding medical decisions. In this paper, we introduce the Fragility Index (FI) for time-to-event endpoints in single-arm clinical trials - a novel metric designed to quantify the robustness of study conclusions. The FI represents the smallest number of censored observations that, when reclassified as uncensored events, causes the posterior probability of the median survival time exceeding a specified threshold to fall below a predefined confidence level. While drug effectiveness is typically assessed by determining whether the posterior probability exceeds a specified confidence level, the FI offers a complementary measure, indicating how robust these conclusions are to potential shifts in the data. Using a Bayesian approach, we develop a practical framework for computing the FI based on the exponential survival model. To facilitate the application of our method, we developed an R package fi, which provides a tool to compute the Fragility Index. Through real world case studies involving time to event data from single arms clinical trials, we demonstrate the utility of this index. Our findings highlight how the FI can be a valuable tool for assessing the robustness of survival analyses in single-arm studies, aiding researchers and clinicians in making more informed decisions.

Multivariate extreme value analysis quantifies the probability and magnitude of joint extreme events. River discharges from the upper Danube River basin provide a challenging dataset for such analysis because the data, which is measured on a spatial network, exhibits both asymptotic dependence and asymptotic independence. To account for both features, we extend the conditional multivariate extreme value model (CMEVM) with a new approach for the residual distribution. This allows sparse (graphical) dependence structures and fully parametric prediction. Our approach fills a current gap in statistical methodology for graphical extremes, where existing models require asymptotic independence. Further, the model can be used to learn the graphical dependence structure when it is unknown a priori. To support inference in high dimensions, we propose a stepwise inference procedure that is computationally efficient and loses no information or predictive power. We show our method is flexible and accurately captures the extremal dependence for the upper Danube River basin discharges.

Bayesian statistics emphasizes the importance of prior distributions, yet finding an appropriate one is practically challenging. When multiple sample results are taken regarding the frequency of the same event, these samples may be influenced by different selection effects. In the absence of suitable prior distributions to correct for these selection effects, it is necessary to exclude outlier sample results to avoid compromising the final result. However, defining outliers based on different thresholds may change the result, which makes the result less persuasive. This work proposes a definition of outliers without the need to set thresholds.

Graphical structure learning is an effective way to assess and visualize cross-biomarker dependencies in biomedical settings. Standard approaches to estimating graphs rely on conditional independence tests that may not be sensitive to associations that manifest at the tails of joint distributions, i.e., they may miss connections among variables that exhibit associations mainly at lower or upper quantiles. In this work, we propose a novel measure of quantile-specific conditional association called QuACC: Quantile Association via Conditional Concordance. For a pair of variables and a conditioning set, QuACC quantifies agreement between the residuals from two quantile regression models, which may be linear or more complex, e.g., quantile forests. Using this measure as the basis for a test of null (quantile) association, we introduce a new class of quantile-specific graphical models. Through simulation we show our method is powerful for detecting dependencies under dependencies that manifest at the tails of distributions. We apply our method to biobank data from All of Us and identify quantile-specific patterns of conditional association in a multivariate setting.

Utility-preserving data privatization is of utmost importance for data-producing agencies. The popular noise-addition privacy mechanism distorts autocorrelation patterns in time series data, thereby marring utility; in response, McElroy et al. (2023) introduced all-pass filtering (FLIP) as a utility-preserving time series data privatization method. Adapting this concept to multivariate data is more complex, and in this paper we propose a multivariate all-pass (MAP) filtering method, employing an optimization algorithm to achieve the best balance between data utility and privacy protection. To test the effectiveness of our approach, we apply MAP filtering to both simulated and real data, sourced from the U.S. Census Bureau's Quarterly Workforce Indicator (QWI) dataset.

Conformal Prediction offers a powerful framework for quantifying uncertainty in machine learning models, enabling the construction of prediction sets with finite-sample validity guarantees. While easily adaptable to non-probabilistic models, applying conformal prediction to probabilistic generative models, such as Normalising Flows is not straightforward. This work proposes a novel method to conformalise conditional normalising flows, specifically addressing the problem of obtaining prediction regions for multi-step time series forecasting. Our approach leverages the flexibility of normalising flows to generate potentially disjoint prediction regions, leading to improved predictive efficiency in the presence of potential multimodal predictive distributions.

Marine accidents highlight the crucial need for human safety. They result in loss of life, environmental harm, and significant economic costs, emphasizing the importance of being proactive and taking precautionary steps. This study aims to identify the root causes of accidents, to develop effective strategies for preventing them. Due to the lack of accurate quantitative data or reliable probability information, we employ qualitative approaches to assess the reliability of complex systems. We collect expert judgments regarding the failure likelihood of each basic event and aggregate those opinions using the Similarity-based Aggregation Method (SAM) to form a collective assessment. In SAM, we convert expert opinions into failure probability using interval-valued triangular fuzzy numbers. Since each expert possesses different knowledge and various levels of experience, we need to assign weights to their opinions to reflect their relative expertise. We employ the Best-Worst Method (BWM) to calculate the weights of each criterion, and then use the weighting scores to determine the weights of each expert. Ranking of basic events according to their criticality is a crucial step, and in this study, we use the FVI measure to prioritize and rank these events according to their criticality level. To demonstrate the effectiveness and validity of our proposed methodology, we apply our method to two case studies: (1) chemical cargo contamination, and (2) the loss of ship steering ability. These case studies serve as examples to illustrate the practicality and utility of our approach in evaluating criticality and assessing risk in complex systems.

We study the distributional properties of linear neural networks with random parameters in the context of large networks, where the number of layers diverges in proportion to the number of neurons per layer. Prior works have shown that in the infinite-width regime, where the number of neurons per layer grows to infinity while the depth remains fixed, neural networks converge to a Gaussian process, known as the Neural Network Gaussian Process. However, this Gaussian limit sacrifices descriptive power, as it lacks the ability to learn dependent features and produce output correlations that reflect observed labels. Motivated by these limitations, we explore the joint proportional limit in which both depth and width diverge but maintain a constant ratio, yielding a non-Gaussian distribution that retains correlations between outputs. Our contribution extends previous works by rigorously characterizing, for linear activation functions, the limiting distribution as a nontrivial mixture of Gaussians.

A large body of work in the statistics and computer science communities dating back to Huber (Huber, 1960) has led to statistically and computationally efficient outlier-robust estimators. Two particular outlier models have received significant attention: the adversarial and heavy-tailed models. While the former models outliers as the result of a malicious adversary manipulating the data, the latter relaxes distributional assumptions on the data allowing outliers to naturally occur as part of the data generating process. In the first setting, the goal is to develop estimators robust to the largest fraction of outliers while in the second, one seeks estimators to combat the loss of statistical efficiency, where the dependence on the failure probability is paramount. Despite these distinct motivations, the algorithmic approaches to both these settings have converged, prompting questions on the relationship between the models. In this paper, we investigate and provide a principled explanation for this phenomenon. First, we prove that any adversarially robust estimator is also resilient to heavy-tailed outliers for any statistical estimation problem with i.i.d data. As a corollary, optimal adversarially robust estimators for mean estimation, linear regression, and covariance estimation are also optimal heavy-tailed estimators. Conversely, for arguably the simplest high-dimensional estimation task of mean estimation, we construct heavy-tailed estimators whose application to the adversarial setting requires any black-box reduction to remove almost all the outliers in the data. Taken together, our results imply that heavy-tailed estimation is likely easier than adversarially robust estimation opening the door to novel algorithmic approaches for the heavy-tailed setting. Additionally, confidence intervals obtained for adversarially robust estimation also hold with high-probability.

Functional data contains two components: shape (or amplitude) and phase. This paper focuses on a branch of functional data analysis (FDA), namely Shape-Based FDA, that isolates and focuses on shapes of functions. Specifically, this paper focuses on Scalar-on-Shape (ScoSh) regression models that incorporate the shapes of predictor functions and discard their phases. This aspect sets ScoSh models apart from the traditional Scalar-on-Function (ScoF) regression models that incorporate full predictor functions. ScoSh is motivated by object data analysis, {\it, e.g.}, for neuro-anatomical objects, where object morphologies are relevant and their parameterizations are arbitrary. ScoSh also differs from methods that arbitrarily pre-register data and uses it in subsequent analysis. In contrast, ScoSh models perform registration during regression, using the (non-parametric) Fisher-Rao inner product and nonlinear index functions to capture complex predictor-response relationships. This formulation results in novel concepts of {\it regression phase} and {\it regression mean} of functions. Regression phases are time-warpings of predictor functions that optimize prediction errors, and regression means are optimal regression coefficients. We demonstrate practical applications of the ScoSh model using extensive simulated and real-data examples, including predicting COVID outcomes when daily rate curves are predictors.