Statistical Foundations

Causal inference in statistics: An overview

Judea Pearl · Statistics Surveys · 2009 · 2,309 citations

This review presents empirical researchers with recent advances in causal inference, and stresses the paradigmatic shifts that must be undertaken in moving from traditional statistical analysis to causal analysis of multivariate data. Special emphasis is placed on the assumptions that underly all causal inferences, the languages used in formulating those assumptions, the conditional nature of all causal and counterfactual claims, and the methods that have been developed for the assessment of such claims. These advances are illustrated using a general theory of causation based on the Structural Causal Model (SCM) described in Pearl (2000a), which subsumes and unifies other approaches to causation, and provides a coherent mathematical foundation for the analysis of causes and counterfactuals. In particular, the paper surveys the development of mathematical tools for inferring (from a combination of data and assumptions) answers to three types of causal queries: (1) queries about the effects of potential interventions, (also called “causal effects” or “policy evaluation”) (2) queries about probabilities of counterfactuals, (including assessment of “regret,” “attribution” or “causes of effects”) and (3) queries about direct and indirect effects (also known as “mediation”). Finally, the paper defines the formal and conceptual relationships between the structural and potential-outcome frameworks and presents tools for a symbiotic analysis that uses the strong features of both.

The Design of Experiments

Ronald A. Fisher · Oliver and Boyd · 1935

The foundational text on randomized experiments, experimental control, and the statistical logic of designed experiments.

Elements of Causal Inference: Foundations and Learning Algorithms

Jonas Peters, Dominik Janzing, Bernhard Scholkopf · MIT Press · 2017

A machine-learning-facing treatment of structural causal models, causal discovery, invariance, and causal learning algorithms.

Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

Nicholas Larsen, Alex Deng, Jiheng Zhang +2 more · The American Statistician · 2024 · 70 citations

A modern review of statistical methodology for online controlled experiments, including sensitivity, heterogeneity, long-term effects, optional stopping, and interference.

On Pearl's Hierarchy and the Foundations of Causal Inference

Elias Bareinboim, Juan D. Correa, Duligur Ibeling +1 more · Probabilistic and Causal Inference: The Works of Judea Pearl · 2022

An overview of the Pearl Causal Hierarchy and the formal distinctions between associational, interventional, and counterfactual queries.

Double/Debiased Machine Learning for Treatment and Causal Parameters

Victor Chernozhukov, Denis Chetverikov, Mert Demirer +4 more · 2016

Most modern supervised statistical/machine learning (ML) methods are explicitly designed to solve prediction problems very well. Achieving this goal does not imply that these methods automatically deliver good estimators of causal parameters. Examples of such parameters include individual regression coefficients, average treatment effects, average lifts, and demand or supply elasticities. In fact, estimates of such causal parameters obtained via naively plugging ML estimators into estimating equations for such parameters can behave very poorly due to the regularization bias. Fortunately, this regularization bias can be removed by solving auxiliary prediction problems via ML tools. Specifically, we can form an orthogonal score for the target low-dimensional parameter by combining auxiliary and main ML predictions. The score is then used to build a de-biased estimator of the target parameter which typically will converge at the fastest possible 1/root(n) rate and be approximately unbiased and normal, and from which valid confidence intervals for these parameters of interest may be constructed. The resulting method thus could be called a "double ML" method because it relies on estimating primary and auxiliary predictive models. In order to avoid overfitting, our construction also makes use of the K-fold sample splitting, which we call cross-fitting. This allows us to use a very broad set of ML predictive methods in solving the auxiliary and main prediction problems, such as random forest, lasso, ridge, deep neural nets, boosted trees, as well as various hybrids and aggregators of these methods.

<b>brms</b> : An <i>R</i> Package for Bayesian Multilevel Models Using <i>Stan</i>

Paul‐Christian Bürkner · Journal of Statistical Software · 2017 · 9,191 citations

The brms package implements Bayesian multilevel models in R using the probabilistic programming language Stan. A wide range of distributions and link functions are supported, allowing users to fit - among others - linear, robust linear, binomial, Poisson, survival, ordinal, zero-inflated, hurdle, and even non-linear models all in a multilevel context. Further modeling options include autocorrelation of the response variable, user defined covariance structures, censored data, as well as meta-analytic standard errors. Prior specifications are flexible and explicitly encourage users to apply prior distributions that actually reflect their beliefs. In addition, model fit can easily be assessed and compared with the Watanabe-Akaike information criterion and leave-one-out cross-validation.

Activation likelihood estimation meta-analysis revisited

Simon B. Eickhoff, Danilo Bzdok, Angela R. Laird +2 more · NeuroImage · 2011 · 1,493 citations

Fractional Brownian Motions, Fractional Noises and Applications

Benoît B. Mandelbrot, John W. Van Ness · SIAM Review · 1968 · 7,657 citations

This foundational theoretical paper introduced and formalized the mathematical concepts of Fractional Brownian Motion and Fractional Gaussian Noise, providing a framework for understanding and modeling phenomena with "long-range dependence" or "long memory" where past events have a persistent influence on future events, which is crucial for analyzing complex systems in various fields.

The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective

John K. Kruschke, Torrin M. Liddell · Psychonomic Bulletin & Review · 2017 · 1,326 citations

Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach

Giovanni Di Leo, Francesco Sardanelli · European Radiology Experimental · 2020 · 436 citations

Here, we summarise the unresolved debate about p value and its dichotomisation. We present the statement of the American Statistical Association against the misuse of statistical significance as well as the proposals to abandon the use of p value and to reduce the significance threshold from 0.05 to 0.005. We highlight reasons for a conservative approach, as clinical research needs dichotomic answers to guide decision-making, in particular in the case of diagnostic imaging and interventional radiology. With a reduced p value threshold, the cost of research could increase while spontaneous research could be reduced. Secondary evidence from systematic reviews/meta-analyses, data sharing, and cost-effective analyses are better ways to mitigate the false discovery rate and lack of reproducibility associated with the use of the 0.05 threshold. Importantly, when reporting p values, authors should always provide the actual value, not only statements of "p < 0.05" or "p ≥ 0.05", because p values give a measure of the degree of data compatibility with the null hypothesis. Notably, radiomics and big data, fuelled by the application of artificial intelligence, involve hundreds/thousands of tested features similarly to other "omics" such as genomics, where a reduction in the significance threshold, based on well-known corrections for multiple testing, has been already adopted.

Improving the estimation of parameter uncertainty distributions in nonlinear mixed effects models using sampling importance resampling

Anne‐Gaëlle Dosne, Martin Bergstrand, Kajsa Harling +1 more · Journal of Pharmacokinetics and Pharmacodynamics · 2016 · 248 citations

Taking parameter uncertainty into account is key to make drug development decisions such as testing whether trial endpoints meet defined criteria. Currently used methods for assessing parameter uncertainty in NLMEM have limitations, and there is a lack of diagnostics for when these limitations occur. In this work, a method based on sampling importance resampling (SIR) is proposed, which has the advantage of being free of distributional assumptions and does not require repeated parameter estimation. To perform SIR, a high number of parameter vectors are simulated from a given proposal uncertainty distribution. Their likelihood given the true uncertainty is then approximated by the ratio between the likelihood of the data given each vector and the likelihood of each vector given the proposal distribution, called the importance ratio. Non-parametric uncertainty distributions are obtained by resampling parameter vectors according to probabilities proportional to their importance ratios. Two simulation examples and three real data examples were used to define how SIR should be performed with NLMEM and to investigate the performance of the method. The simulation examples showed that SIR was able to recover the true parameter uncertainty. The real data examples showed that parameter 95 % confidence intervals (CI) obtained with SIR, the covariance matrix, bootstrap and log-likelihood profiling were generally in agreement when 95 % CI were symmetric. For parameters showing asymmetric 95 % CI, SIR 95 % CI provided a close agreement with log-likelihood profiling but often differed from bootstrap 95 % CI which had been shown to be suboptimal for the chosen examples. This work also provides guidance towards the SIR workflow, i.e.,which proposal distribution to choose and how many parameter vectors to sample when performing SIR, using diagnostics developed for this purpose. SIR is a promising approach for assessing parameter uncertainty as it is applicable in many situations where other methods for assessing parameter uncertainty fail, such as in the presence of small datasets, highly nonlinear models or meta-analysis.

The Molecular Signatures Database Hallmark Gene Set Collection

Arthur Liberzon, Chet Birger, Helga Thorvaldsdóttir +3 more · Cell Systems · 2015 · 14,282 citations

SwissADME: a free web tool to evaluate pharmacokinetics, drug-likeness and medicinal chemistry friendliness of small molecules

Antoine Daina, Olivier Michielin, Vincent Zoete · Scientific Reports · 2017 · 16,793 citations

To be effective as a drug, a potent molecule must reach its target in the body in sufficient concentration, and stay there in a bioactive form long enough for the expected biologic events to occur. Drug development involves assessment of absorption, distribution, metabolism and excretion (ADME) increasingly earlier in the discovery process, at a stage when considered compounds are numerous but access to the physical samples is limited. In that context, computer models constitute valid alternatives to experiments. Here, we present the new SwissADME web tool that gives free access to a pool of fast yet robust predictive models for physicochemical properties, pharmacokinetics, drug-likeness and medicinal chemistry friendliness, among which in-house proficient methods such as the BOILED-Egg, iLOGP and Bioavailability Radar. Easy efficient input and interpretation are ensured thanks to a user-friendly interface through the login-free website http://www.swissadme.ch. Specialists, but also nonexpert in cheminformatics or computational chemistry can predict rapidly key parameters for a collection of molecules to support their drug discovery endeavours.

Fitting Linear Mixed-Effects Models Using <b>lme4</b>

Douglas M. Bates, Martin Mächler, Benjamin M. Bolker +1 more · Journal of Statistical Software · 2015 · 84,511 citations

Maximum likelihood or restricted maximum likelihood (REML) estimates of the parameters in linear mixed-effects models can be determined using the lmer function in the lme4 package for R. As for most model-fitting functions in R, the model is described in an lmer call by a formula, in this case including both fixed- and random-effects terms. The formula and data together determine a numerical representation of the model from which the profiled deviance or the profiled REML criterion can be evaluated as a function of some of the model parameters. The appropriate criterion is optimized, using one of the constrained optimization functions in R, to provide the parameter estimates. We describe the structure of the model, the steps in evaluating the profiled deviance or REML criterion, and the structure of classes or types that represents such a model. Sufficient detail is included to allow specialization of these structures by users who wish to write functions to fit specialized linear mixed models, such as models incorporating pedigrees or smoothing splines, that are not easily expressible in the formula language used by lmer.

Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates

Anders Eklund, Thomas E. Nichols, Hans Knutsson · Proceedings of the National Academy of Sciences · 2016 · 3,632 citations

The most widely used task functional magnetic resonance imaging (fMRI) analyses use parametric statistical methods that depend on a variety of assumptions. In this work, we use real resting-state data and a total of 3 million random task group analyses to compute empirical familywise error rates for the fMRI software packages SPM, FSL, and AFNI, as well as a nonparametric permutation method. For a nominal familywise error rate of 5%, the parametric statistical methods are shown to be conservative for voxelwise inference and invalid for clusterwise inference. Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian shape. By comparison, the nonparametric permutation test is found to produce nominal results for voxelwise as well as clusterwise inference. These findings speak to the need of validating the statistical methods being used in the field of neuroimaging.

pROC: an open-source package for R and S+ to analyze and compare ROC curves

Xavier Robin, Natacha Turck, Alexandre Hainard +4 more · BMC Bioinformatics · 2011 · 13,864 citations

BACKGROUND: Receiver operating characteristic (ROC) curves are useful tools to evaluate classifiers in biomedical and bioinformatics applications. However, conclusions are often reached through inconsistent use or insufficient statistical analysis. To support researchers in their ROC curves analysis we developed pROC, a package for R and S+ that contains a set of tools displaying, analyzing, smoothing and comparing ROC curves in a user-friendly, object-oriented and flexible interface. RESULTS: With data previously imported into the R or S+ environment, the pROC package builds ROC curves and includes functions for computing confidence intervals, statistical tests for comparing total or partial area under the curve or the operating points of different classifiers, and methods for smoothing ROC curves. Intermediary and final results are visualised in user-friendly interfaces. A case study based on published clinical and biomarker data shows how to perform a typical ROC analysis with pROC. CONCLUSIONS: pROC is a package for R and S+ specifically dedicated to ROC analysis. It proposes multiple statistical tests to compare ROC curves, and in particular partial areas under the curve, allowing proper ROC interpretation. pROC is available in two versions: in the R programming language or with a graphical user interface in the S+ statistical software. It is accessible at http://expasy.org/tools/pROC/ under the GNU General Public License. It is also distributed through the CRAN and CSAN public repositories, facilitating its installation.

Bayesian statistics and modelling

Rens van de Schoot, Sarah Depaoli, Ruth King +8 more · Nature Reviews Methods Primers · 2021 · 1,152 citations

Analysis of compositions of microbiomes with bias correction

Lin Huang, Shyamal Peddada · Nature Communications · 2020 · 2,208 citations

Differential abundance (DA) analysis of microbiome data continues to be a challenging problem due to the complexity of the data. In this article we define the notion of "sampling fraction" and demonstrate a major hurdle in performing DA analysis of microbiome data is the bias introduced by differences in the sampling fractions across samples. We introduce a methodology called Analysis of Compositions of Microbiomes with Bias Correction (ANCOM-BC), which estimates the unknown sampling fractions and corrects the bias induced by their differences among samples. The absolute abundance data are modeled using a linear regression framework. This formulation makes a fundamental advancement in the field because, unlike the existing methods, it (a) provides statistically valid test with appropriate p-values, (b) provides confidence intervals for differential abundance of each taxon, (c) controls the False Discovery Rate (FDR), (d) maintains adequate power, and (e) is computationally simple to implement.

Genome sequence-based species delimitation with confidence intervals and improved distance functions

Jan P. Meier‐Kolthoff, Alexander F. Auch, Hans-Peter Klenk +1 more · BMC Bioinformatics · 2013 · 6,654 citations

BACKGROUND: For the last 25 years species delimitation in prokaryotes (Archaea and Bacteria) was to a large extent based on DNA-DNA hybridization (DDH), a tedious lab procedure designed in the early 1970s that served its purpose astonishingly well in the absence of deciphered genome sequences. With the rapid progress in genome sequencing time has come to directly use the now available and easy to generate genome sequences for delimitation of species. GBDP (Genome Blast Distance Phylogeny) infers genome-to-genome distances between pairs of entirely or partially sequenced genomes, a digital, highly reliable estimator for the relatedness of genomes. Its application as an in-silico replacement for DDH was recently introduced. The main challenge in the implementation of such an application is to produce digital DDH values that must mimic the wet-lab DDH values as close as possible to ensure consistency in the Prokaryotic species concept. RESULTS: Correlation and regression analyses were used to determine the best-performing methods and the most influential parameters. GBDP was further enriched with a set of new features such as confidence intervals for intergenomic distances obtained via resampling or via the statistical models for DDH prediction and an additional family of distance functions. As in previous analyses, GBDP obtained the highest agreement with wet-lab DDH among all tested methods, but improved models led to a further increase in the accuracy of DDH prediction. Confidence intervals yielded stable results when inferred from the statistical models, whereas those obtained via resampling showed marked differences between the underlying distance functions. CONCLUSIONS: Despite the high accuracy of GBDP-based DDH prediction, inferences from limited empirical data are always associated with a certain degree of uncertainty. It is thus crucial to enrich in-silico DDH replacements with confidence-interval estimation, enabling the user to statistically evaluate the outcomes. Such methodological advancements, easily accessible through the web service at http://ggdc.dsmz.de, are crucial steps towards a consistent and truly genome sequence-based classification of microorganisms.

Testing measurement invariance of composites using partial least squares

Jörg Henseler, Christian M. Ringle, Marko Sarstedt · International Marketing Review · 2016 · 2,729 citations

Purpose – Research on international marketing usually involves comparing different groups of respondents. When using structural equation modeling (SEM), group comparisons can be misleading unless researchers establish the invariance of their measures. While methods have been proposed to analyze measurement invariance in common factor models, research lacks an approach in respect of composite models. The purpose of this paper is to present a novel three-step procedure to analyze the measurement invariance of composite models (MICOM) when using variance-based SEM, such as partial least squares (PLS) path modeling. Design/methodology/approach – A simulation study allows us to assess the suitability of the MICOM procedure to analyze the measurement invariance in PLS applications. Findings – The MICOM procedure appropriately identifies no, partial, and full measurement invariance. Research limitations/implications – The statistical power of the proposed tests requires further research, and researchers using the MICOM procedure should take potential type-II errors into account. Originality/value – The research presents a novel procedure to assess the measurement invariance in the context of composite models. Researchers in international marketing and other disciplines need to conduct this kind of assessment before undertaking multigroup analyses. They can use MICOM procedure as a standard means to assess the measurement invariance.

Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications

Eric‐Jan Wagenmakers, Maarten Marsman, Tahira Jamil +10 more · Psychonomic Bulletin & Review · 2017 · 1,697 citations

Bayesian parameter estimation and Bayesian hypothesis testing present attractive alternatives to classical inference using confidence intervals and p values. In part I of this series we outline ten prominent advantages of the Bayesian approach. Many of these advantages translate to concrete opportunities for pragmatic researchers. For instance, Bayesian hypothesis testing allows researchers to quantify evidence and monitor its progression as data come in, without needing to know the intention with which the data were collected. We end by countering several objections to Bayesian hypothesis testing. Part II of this series discusses JASP, a free and open source software program that makes it easy to conduct Bayesian estimation and testing for a range of popular statistical scenarios (Wagenmakers et al. this issue).

Sparse and Compositionally Robust Inference of Microbial Ecological Networks

Zachary Kurtz, Christian L. Müller, Emily R. Miraldi +3 more · PLoS Computational Biology · 2015 · 1,863 citations

16S ribosomal RNA (rRNA) gene and other environmental sequencing techniques provide snapshots of microbial communities, revealing phylogeny and the abundances of microbial populations across diverse ecosystems. While changes in microbial community structure are demonstrably associated with certain environmental conditions (from metabolic and immunological health in mammals to ecological stability in soils and oceans), identification of underlying mechanisms requires new statistical tools, as these datasets present several technical challenges. First, the abundances of microbial operational taxonomic units (OTUs) from amplicon-based datasets are compositional. Counts are normalized to the total number of counts in the sample. Thus, microbial abundances are not independent, and traditional statistical metrics (e.g., correlation) for the detection of OTU-OTU relationships can lead to spurious results. Secondly, microbial sequencing-based studies typically measure hundreds of OTUs on only tens to hundreds of samples; thus, inference of OTU-OTU association networks is severely under-powered, and additional information (or assumptions) are required for accurate inference. Here, we present SPIEC-EASI (SParse InversE Covariance Estimation for Ecological Association Inference), a statistical method for the inference of microbial ecological networks from amplicon sequencing datasets that addresses both of these issues. SPIEC-EASI combines data transformations developed for compositional data analysis with a graphical model inference framework that assumes the underlying ecological association network is sparse. To reconstruct the network, SPIEC-EASI relies on algorithms for sparse neighborhood and inverse covariance selection. To provide a synthetic benchmark in the absence of an experimentally validated gold-standard network, SPIEC-EASI is accompanied by a set of computational tools to generate OTU count data from a set of diverse underlying network topologies. SPIEC-EASI outperforms state-of-the-art methods to recover edges and network properties on synthetic data under a variety of scenarios. SPIEC-EASI also reproducibly predicts previously unknown microbial associations using data from the American Gut project.

phyloseq: An R Package for Reproducible Interactive Analysis and Graphics of Microbiome Census Data

Paul J. McMurdie, Susan Holmes · PLoS ONE · 2013 · 22,235 citations

BACKGROUND: the analysis of microbial communities through dna sequencing brings many challenges: the integration of different types of data with methods from ecology, genetics, phylogenetics, multivariate statistics, visualization and testing. With the increased breadth of experimental designs now being pursued, project-specific statistical analyses are often needed, and these analyses are often difficult (or impossible) for peer researchers to independently reproduce. The vast majority of the requisite tools for performing these analyses reproducibly are already implemented in R and its extensions (packages), but with limited support for high throughput microbiome census data. RESULTS: Here we describe a software project, phyloseq, dedicated to the object-oriented representation and analysis of microbiome census data in R. It supports importing data from a variety of common formats, as well as many analysis techniques. These include calibration, filtering, subsetting, agglomeration, multi-table comparisons, diversity analysis, parallelized Fast UniFrac, ordination methods, and production of publication-quality graphics; all in a manner that is easy to document, share, and modify. We show how to apply functions from other R packages to phyloseq-represented data, illustrating the availability of a large number of open source analysis techniques. We discuss the use of phyloseq with tools for reproducible research, a practice common in other fields but still rare in the analysis of highly parallel microbiome census data. We have made available all of the materials necessary to completely reproduce the analysis and figures included in this article, an example of best practices for reproducible research. CONCLUSIONS: The phyloseq project for R is a new open-source software package, freely available on the web from both GitHub and Bioconductor.

Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors

Jennifer A. Hoeting, David Madigan, Adrian E. Raftery +1 more · Statistical Science · 1999 · 4,164 citations

Standard statistical practice ignores model uncertainty. Data analysts typically select a model from some class of models and then proceed as if the selected model had generated the data. This approach ignores the uncertainty in model selection, leading to over-confident inferences and decisions that are more risky than one thinks they are. Bayesian model averaging (BMA)provides a coherent mechanism for accounting for this model uncertainty. Several methods for implementing BMA have recently emerged. We discuss these methods and present a number of examples.In these examples, BMA provides improved out-of-sample predictive performance. We also provide a catalogue of currently available BMA software.

A kernel two-sample test

Arthur Gretton, Karsten Borgwardt, Malte J. Rasch +2 more · MPG.PuRe (Max Planck Society) · 2012 · 2,230 citations

We propose a framework for analyzing and comparing distributions, which we use to construct statistical tests to determine if two samples are drawn from different distributions. Our test statistic is the largest difference in expectations over functions in the unit ball of a reproducing kernel Hilbert space (RKHS), and is called the maximum mean discrepancy (MMD).We present two distributionfree tests based on large deviation bounds for the MMD, and a third test based on the asymptotic distribution of this statistic. The MMD can be computed in quadratic time, although efficient linear time approximations are available. Our statistic is an instance of an integral probability metric, and various classical metrics on distributions are obtained when alternative function classes are used in place of an RKHS. We apply our two-sample tests to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where they perform strongly. Excellent performance is also obtained when comparing distributions over graphs, for which these are the first such tests.

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

Liu Yang, Xuhui Meng, George Em Karniadakis · Journal of Computational Physics · 2020 · 1,025 citations

Inference Under Covariate-Adaptive Randomization

Federico A. Bugni, Ivan A. Canay, Azeem M. Shaikh · Journal of the American Statistical Association · 2017 · 127 citations

This article studies inference for the average treatment effect in randomized controlled trials with covariate-adaptive randomization. Here, by covariate-adaptive randomization, we mean randomization schemes that first stratify according to baseline covariates and then assign treatment status so as to achieve “balance” within each stratum. Our main requirement is that the randomization scheme assigns treatment status within each stratum so that the fraction of units being assigned to treatment within each stratum has a well behaved distribution centered around a proportion π as the sample size tends to infinity. Such schemes include, for example, Efron’s biased-coin design and stratified block randomization. When testing the null hypothesis that the average treatment effect equals a prespecified value in such settings, we first show the usual two-sample t-test is conservative in the sense that it has limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level. We show, however, that a simple adjustment to the usual standard error of the two-sample t-test leads to a test that is exact in the sense that its limiting rejection probability under the null hypothesis equals the nominal level. Next, we consider the usual t-test (on the coefficient on treatment assignment) in a linear regression of outcomes on treatment assignment and indicators for each of the strata. We show that this test is exact for the important special case of randomization schemes with π=12, but is otherwise conservative. We again provide a simple adjustment to the standard errors that yields an exact test more generally. Finally, we study the behavior of a modified version of a permutation test, which we refer to as the covariate-adaptive permutation test, that only permutes treatment status for units within the same stratum. When applied to the usual two-sample t-statistic, we show that this test is exact for randomization schemes with π=12 and that additionally achieve what we refer to as “strong balance.” For randomization schemes with π≠12, this test may have limiting rejection probability under the null hypothesis strictly greater than the nominal level. When applied to a suitably adjusted version of the two-sample t-statistic, however, we show that this test is exact for all randomization schemes that achieve “strong balance,” including those with π≠12. A simulation study confirms the practical relevance of our theoretical results. We conclude with recommendations for empirical practice and an empirical illustration. Supplementary materials for this article are available online.

Extending the Limits of Quantitative Proteome Profiling with Data-Independent Acquisition and Application to Acetaminophen-Treated Three-Dimensional Liver Microtissues

Roland Bruderer, Oliver M. Bernhardt, Tejas Gandhi +10 more · Molecular & Cellular Proteomics · 2015 · 1,280 citations

The data-independent acquisition (DIA) approach has recently been introduced as a novel mass spectrometric method that promises to combine the high content aspect of shotgun proteomics with the reproducibility and precision of selected reaction monitoring. Here, we evaluate, whether SWATH-MS type DIA effectively translates into a better protein profiling as compared with the established shotgun proteomics.We implemented a novel DIA method on the widely used Orbitrap platform and used retention-time-normalized (iRT) spectral libraries for targeted data extraction using Spectronaut. We call this combination hyper reaction monitoring (HRM). Using a controlled sample set, we show that HRM outperformed shotgun proteomics both in the number of consistently identified peptides across multiple measurements and quantification of differentially abundant proteins. The reproducibility of HRM in peptide detection was above 98%, resulting in quasi complete data sets compared with 49% of shotgun proteomics.Utilizing HRM, we profiled acetaminophen (APAP) 1The abbreviations used are:APAPacetaminophenATPadenosine triphosphateCVcoefficient of variationDIAdata-independent acquisitionDDAdata-dependent acquisitionFDRfalse discovery rateHRMhyper reaction monitoringiRTindexed retention timeNAPQIN-acetyl-p-benzoquinone imineSRMselected reaction monitoring; DIA with 32 sequential windows of 25 Dalton width. treated three-dimensional human liver microtissues. An early onset of relevant proteome changes was revealed at subtoxic doses of APAP. Further, we detected and quantified for the first time human NAPQI-protein adducts that might be relevant for the toxicity of APAP. The adducts were identified on four mitochondrial oxidative stress related proteins (GATM, PARK7, PRDX6, and VDAC2) and two other proteins (ANXA2 and FTCD).Our findings imply that DIA should be the preferred method for quantitative protein profiling. The data-independent acquisition (DIA) approach has recently been introduced as a novel mass spectrometric method that promises to combine the high content aspect of shotgun proteomics with the reproducibility and precision of selected reaction monitoring. Here, we evaluate, whether SWATH-MS type DIA effectively translates into a better protein profiling as compared with the established shotgun proteomics. We implemented a novel DIA method on the widely used Orbitrap platform and used retention-time-normalized (iRT) spectral libraries for targeted data extraction using Spectronaut. We call this combination hyper reaction monitoring (HRM). Using a controlled sample set, we show that HRM outperformed shotgun proteomics both in the number of consistently identified peptides across multiple measurements and quantification of differentially abundant proteins. The reproducibility of HRM in peptide detection was above 98%, resulting in quasi complete data sets compared with 49% of shotgun proteomics. Utilizing HRM, we profiled acetaminophen (APAP) 1The abbreviations used are:APAPacetaminophenATPadenosine triphosphateCVcoefficient of variationDIAdata-independent acquisitionDDAdata-dependent acquisitionFDRfalse discovery rateHRMhyper reaction monitoringiRTindexed retention timeNAPQIN-acetyl-p-benzoquinone imineSRMselected reaction monitoring; DIA with 32 sequential windows of 25 Dalton width. treated three-dimensional human liver microtissues. An early onset of relevant proteome changes was revealed at subtoxic doses of APAP. Further, we detected and quantified for the first time human NAPQI-protein adducts that might be relevant for the toxicity of APAP. The adducts were identified on four mitochondrial oxidative stress related proteins (GATM, PARK7, PRDX6, and VDAC2) and two other proteins (ANXA2 and FTCD). acetaminophen adenosine triphosphate coefficient of variation data-independent acquisition data-dependent acquisition false discovery rate hyper reaction monitoring indexed retention time N-acetyl-p-benzoquinone imine selected reaction monitoring; DIA with 32 sequential windows of 25 Dalton width. Our findings imply that DIA should be the preferred method for quantitative protein profiling. Quantitative mass spectrometry is a powerful and widely used approach to identify differentially abundant proteins, e.g. for proteome profiling and biomarker discovery (1Liu Y. Hittenhain R. Collins B. Aebersold R. Mass spectrometric protein maps for biomarker discovery and clinical research.Expert Rev. Mol. Diagn. 2013; 13: 811-825Crossref PubMed Scopus (99) Google Scholar). Several tens of thousands of peptides and thousands of proteins can be routinely identified from a single sample injection in shotgun proteomics (2Mann M. Kulak N.A. Nagaraj N. Cox J. The coming age of complete, accurate, and ubiquitous proteomes.Mol. Cell. 2013; 49: 583-590Abstract Full Text Full Text PDF PubMed Scopus (285) Google Scholar). Shotgun proteomics, however, is limited by low analytical reproducibility. This is due to the complexity of the samples that results in under sampling (supplemental Fig. 1) and to the fact that the acquisition of MS2 spectra is often triggered outside of the elution peak apex. As a result, only 17% of the detectable peptides are typically fragmented, and less than 60% of those are identified. This translates in reliable identification of only 10% of the detectable peptides (3Michalski A. Cox J. Mann M. More than 100,000 detectable peptide species elute in single shotgun proteomics runs but the majority is inaccessible to data-dependent LC-MS/MS.J. Proteome Res. 2011; 10: 1785-1793Crossref PubMed Scopus (476) Google Scholar). The overlap of peptide identification across technical replicates is typically 35–60% (4Tabb D. Vega-Montoto L. Rudnick P.A. Variyath A.M. Ham A.J. Bunk D.M. Kilpatrick L.E. Billheimer D.D. Blackman R.K. Cardasis H.L. Carr S.A. Clauser K.R. Jaffe J.D. Kowalski K.A. Neubert T.A. Regnier F.E. Schilling B. Tegeler T.J. Wang M. Wang P. Whiteaker J.R. Zimmerman L.J. Fisher S.J. Gibson B.W. Kinsinger C.R. Mesri M. Rodriguez H Stein S.E. Tempst P. Paulovich A.G. Liebler D.C. Spiegelman C. Repeatability and reproducibility in proteomic identifications by liquid chromatography-tandem mass spectrometry.J. Proteome Res. 2009; 9: 761-776Crossref Scopus (389) Google Scholar), which results in inconsistent peptide quantification. Alternatively to shotgun proteomics, selected reaction monitoring (SRM) enables quantification of up to 200–300 peptides at very high reproducibility, accuracy, and precision (5Barnidge D.R. Dratz E.A Martin T. Bonilla L.E. Moran L.B. Lindall A. Absolute quantification of the G protein-coupled receptor rhodopsin by LC/MS/MS using proteolysis product peptides and synthetic peptide standards.Anal. Chem. 2003; 75: 445-451Crossref PubMed Scopus (196) Google Scholar, 6Gerber S.A. Rush J. Stemman O. Kirschner M.W. Gygi S.P. Absolute quantification of proteins and phosphoproteins from cell lysates by tandem MS.Proc. Natl. Acad. Sci. U.S.A. 2003; 100: 6940-6945Crossref PubMed Scopus (1542) Google Scholar, 7Keshishian H. Addona T. Burgess M. Kuhn E. Carr S.A. Quantitative, multiplexed assays for low abundance proteins in plasma by targeted mass spectrometry and stable isotope dilution.Mol. Cell. Proteomics. 2007; 6: 2212-2229Abstract Full Text Full Text PDF PubMed Scopus (576) Google Scholar, 8Gillette M.A. Carr S.A. Quantitative analysis of peptides and proteins in biomedicine by targeted mass spectrometry.Nat. Methods. 2013; 10: 28-34Crossref PubMed Scopus (359) Google Scholar). Data-independent acquisition (DIA), a novel acquisition type, overcomes the semistochastic nature of shotgun proteomics (9Venable J. Dong M. Wohlschlegel J. Automated approach for quantitative analysis of complex peptide mixtures from tandem mass spectra.Nat. Methods. 2004; 1: 39-45Crossref PubMed Scopus (509) Google Scholar, 10Plumb R.S. Johnson K.A. Rainville P. Smith B.W. Wilson I.D. Castro-Perez J.M. Nicholson J.K. UPLC/MSE; a new approach for generating molecular fragment information for biomarker structure elucidation.Rapid Commun. Mass Spectrom. 2006; 20: 1989-1994Crossref PubMed Scopus (389) Google Scholar, 11Distler U. Kuharev J. Navarro P. Levin Y. Drift time-specific collision energies enable deep-coverage data-independent acquisition proteomics.Nat. Methods. 2015; 11Google Scholar, 12Moran D. Cross T. Brown L.M. Colligan R.M. Dunbar D. Data-independent acquisition (MSE) with ion mobility provides a systematic method for analysis of a bacteriophage structural proteome.J. Virol. Methods. 2014; 195: 9-17Crossref PubMed Scopus (16) Google Scholar, 13Geiger T. Cox J. Mann M. Proteomics on an Orbitrap benchtop mass spectrometer using all-ion fragmentation.Mol. Cell. Proteomics. 2010; 9: 2252-2261Abstract Full Text Full Text PDF PubMed Scopus (189) Google Scholar, 14Panchaud A. Jung S. Shaffer S.A Aitchison J.D. Goodlett D.R. Faster, quantitative, and accurate precursor acquisition independent from ion count.Anal. Chem. 2011; 83: 2250-2257Crossref PubMed Scopus (65) Google Scholar, 15Pak H. Nikitin F. Gluck F. Lisacek F. Scherl A. Muller M. Clustering and filtering tandem mass spectra acquired in data-independent mode.J. Am. Soc. Mass Spectrom. 2013; 24: 1862-1871Crossref PubMed Scopus (13) Google Scholar, 16Weisbrod C.R. Eng J.K. Hoopmann M.R. Baker T. Bruce J.E. Accurate peptide fragment mass analysis: Multiplexed peptide identification and quantification.J. Proteome Res. 2012; 11: 1621-1632Crossref PubMed Scopus (72) Google Scholar, 17Carvalho P.C. Han X. Xu T. Cociorva D. Carvalho Mda. G. Barbosa V.C. Yates 3rd., J.R. XDIA: Improving on the label-free data-independent analysis.Bioinformatics. 2010; 26: 847-848Crossref PubMed Scopus (70) Google Scholar, 18Egertson J.D. Kuehn A. Merrihew G.E. Bateman N.W. MacLean B.X. Ting Y.S. Canterbury J.D. Marsh D.M. Kellmann M. Zabrouskov V. Wu C.C. MacCoss M.J. Multiplexed MS/MS for improved data-independent acquisition.Nat. Methods. 2013; 10: 744-746Crossref PubMed Scopus (207) Google Scholar). Spectra are acquired according to a predefined schema instead of dependent on the analysis of DIA data was introduced with SWATH-MS Navarro P. S. H. N. L. R. Aebersold R. data extraction of the MS/MS spectra by data-independent a new for and accurate proteome Proteomics. 2012; Full Text Full Text PDF PubMed Scopus Google Scholar). the the mass spectrometer 32 25 precursor and fragment ion spectra Navarro P. S. H. N. L. R. Aebersold R. data extraction of the MS/MS spectra by data-independent a new for and accurate proteome Proteomics. 2012; Full Text Full Text PDF PubMed Scopus Google Scholar). This results in a of detectable of the selected mass The of SWATH-MS was in the analysis of the DIA fragment are using which results in targeted are by in O. L. and L. and for of data independent acquisition of the on Mass and B. D.M. N. M. B. R. Liebler D.C. MacCoss M.J. An for and targeted proteomics 2010; 26: PubMed Scopus Google Scholar), and H.L. G. Navarro P. L. Collins J. L. Aebersold R. enables targeted analysis of data-independent acquisition 2014; PubMed Scopus Google Scholar). The of peptide identification is on the method L. O. P. Hittenhain R. M. Aebersold R. Automated data and for Methods. 2011; PubMed Scopus Google Scholar). We a novel DIA hyper reaction monitoring in in mass independent analysis and hyper reaction Rev. Proteomics. 2013; 10: PubMed Scopus Google implemented on a of DIA acquisition and targeted data analysis with retention-time-normalized spectral libraries C. L. MacLean B. R. F. J. MacCoss M.J. O. Using a retention time for targeted of 2012; PubMed Scopus Google Scholar). high of peptide identification and quantification is due to we a improved DIA we the L. O. P. Hittenhain R. M. Aebersold R. Automated data and for Methods. 2011; PubMed Scopus Google approach in the we and retention-time-normalized (iRT) spectral We compared HRM and shotgun proteomics in of to differentially abundant proteins. this we used a sample with proteins at into a stable human cell protein This in quasi complete data sets for HRM and the detection of a number of differentially abundant proteins as compared with shotgun proteomics. We HRM to identify changes in the proteome in three-dimensional human liver A. J. D. for in using and a as an 2013; PubMed Scopus Google Scholar, H. M.R. A. of proteomics of in Sci. 2011; PubMed Scopus Google Scholar, S. J.M. type human liver for 2013; PubMed Scopus Google Scholar). a of only the abundance of proteins was quantified an novel proteins adducts that might be relevant for the toxicity of were and quantified on proteins. was from and were from was from The were by and were from was from was from peptides were from of was by in and The was with for at the was with 25 for at The was to and with at a to at for The samples were at at for The peptides were using from The according to the peptides were in and of and and were as for the cell The HRM was to of the samples according to for the DIA analysis using of human and human were in with S. J.M. type human liver for 2013; PubMed Scopus Google Scholar). The liver were treated at with and in the for of were with single from were in an the were at for at and with with as The were in of and for and at for at the samples were as for the cell of the samples was on a analytical with at using an to a mass spectrometer The peptides were by a of from to with at by a to in and for the method from was used with the C. R. and acquisition for shotgun proteomics on a Orbitrap mass Proteome Res. 2012; 11: PubMed Scopus Google Scholar). The was The for the MS/MS was to collision was 10% at The HRM DIA method of a at from to of injection DIA windows were acquired at and for injection (supplemental collision was 10% at The spectra were in The MS/MS spectra were from to was on a to of on a using a as with time and width. sample was with the stable isotope The mass spectrometric data were at the the is and the is The DIA data were with a mass spectrometer from The were used for the time type was to for was to on MS2 was The false discovery rate was to at peptide The spectra were with the analysis using with the J. Mann M. enables high peptide identification mass and protein 26: PubMed Scopus Google Scholar). The peptide was to of as a of and as The mass for the precursor was and for the fragment was The were the human the in proteins and the peptide The identifications were to of on peptide and protein of the spectral measurements of the sample measurements of the were spectra were as and a spectral was using spectral in to H. Eng J.K. N. Stein S.E. Aebersold R. and of a spectral method for peptide identification from 2007; PubMed Scopus Google Scholar). from of and fragment the retention the of the assays of the a was the The was to the profiling sample set, peptides the proteins, and from were that were identified multiple using the identification was Full peptide were as peptide precursor species and The were with of and to of than was using and analysis of using 2009; PubMed Scopus Google Scholar). both and the of the peptides were across the samples and and for in a implemented in M. T. D. T. MacLean B. O. An for analysis of quantitative mass proteomic 2014; PubMed Scopus Google Scholar). The is sample precursor and the of the and the of the that are by the systematic of in the number of runs the of the of the of the the of the of The is for the of a with The was used to the of for The were for multiple using the method Y. Y. the false discovery and powerful approach to multiple R. Soc. Scholar). by the was to to The was used to the and the under the The under the with two acquisition were compared using the D.M. the under two a PubMed Scopus Google Scholar). novel acquisition method was to enable high DIA on a of and DIA that to the ion complexity of mass and This method is the new DIA of which enables of DIA windows with This the time by a of as compared with the used method for DIA R. S. O. and L. multiplexed protein profiling across sets of of acquisition with shotgun on Mass and the time of the SWATH-MS acquisition method Navarro P. S. H. N. L. R. Aebersold R. data extraction of the MS/MS spectra by data-independent a new for and accurate proteome Proteomics. 2012; Full Text Full Text PDF PubMed Scopus Google Scholar). The DIA method is to the used peak resulting on in to data of peptides from spectra of the DIA we a mass spectrometer Spectronaut. and is to L. O. P. Hittenhain R. M. Aebersold R. Automated data and for Methods. 2011; PubMed Scopus Google Scholar), with fragment can be the nature of DIA the of and for peak and time information can be used to the elution of the peptides by of This can be of as a which the of ion extraction windows with the of a of the to of peptides peptide quantification is improved by of an detection The method of is for the of the resulting J.E. Gygi S.P. for mass Mol. 2010; PubMed Scopus Google Scholar). The implemented in was by peptides in from a human cell The of the was compared with the of the spectra of the peptides the profiling of HRM and to with the established shotgun proteomics, we a of controlled mixtures the profiling sample proteins (supplemental were into a The profiling sample two for mass spectrometric detection of a number of differentially abundant proteins a of proteins and quantification. The proteins were into were to in and 60% at of detection and at and The was in a at the of detection Spectra from the profiling sample were acquired on a mass spectrometer in and DIA in technical The of runs was to and O. of quantitative mass proteomic Proteome Res. 2009; PubMed Scopus Google Scholar). the targeted analysis of the HRM we a spectral using shotgun proteomics of peptide assays for protein with an of the HRM approach can only identify and peptides and proteins for which the assays are in the spectral the content of the spectral the to the of shotgun proteomics runs at the we a of spectral libraries from an number of The was controlled at protein L. M. S.P. M. A. J.M. Aebersold R. identification false discovery for very proteomics data sets by tandem mass Cell. Proteomics. 2009; Full Text Full Text PDF PubMed Scopus Google Scholar). The number of proteins in the to to the peptides to runs of shotgun proteomics the peptide protein in the the into the the peptide identifications were for the of at fragment and for on in retention The DIA spectra were in targeted with using the spectral was The retention time extraction for the targeted HRM was by The implemented in in a of the and (supplemental Fig. peptides were identified in The spectra of the profiling sample were with the J. Mann M. enables high peptide identification mass and protein 26: PubMed Scopus Google Scholar). was peptides were identified in shotgun proteomics. the HRM approach identified on 60% peptides in a single than shotgun proteomics. This is due to the nature of precursor with DIA and to the spectral the two data was on the peptide to for and S.J. Johnson Smith for systematic with mass spectrometry and label-free proteomics Proteome Res. 2006; PubMed Scopus Google (supplemental Fig. in of identification was for HRM and shotgun proteomics. the HRM data of peptides of runs of peptides of proteins and shotgun proteomics of HRM a quasi complete data set, with only of those are compared with for shotgun proteomics This the of the targeted HRM to identify peptides and the quantitative precision of shotgun proteomics and HRM, we the of variation for the peptides that were quantified in both and detected in the The of the HRM were than of the of shotgun proteomics of for HRM, for shotgun The results for shotgun proteomics are with the which for technical replicates on the mass spectrometer (4Tabb D. Vega-Montoto L. Rudnick P.A. Variyath A.M. Ham A.J. Bunk D.M. Kilpatrick L.E. Billheimer D.D. Blackman R.K. Cardasis H.L. Carr S.A. Clauser K.R. Jaffe J.D. Kowalski K.A. Neubert T.A. Regnier F.E. Schilling B. Tegeler T.J. Wang M. Wang P. Whiteaker J.R. Zimmerman L.J. Fisher S.J. Gibson B.W. Kinsinger C.R. Mesri M. Rodriguez H Stein S.E. Tempst P. Paulovich A.G. Liebler D.C. Spiegelman C. Repeatability and reproducibility in proteomic identifications by liquid chromatography-tandem mass spectrometry.J. Proteome Res. 2009; 9: 761-776Crossref Scopus (389) Google Scholar). the for HRM were to be than those for shotgun proteomics the (supplemental Fig. analysis of the protein that HRM and peptide for and and Fig. We quantitative of the identified peptides to the of the two to differentially abundant proteins in the profiling sample The were for shotgun proteomics and for HRM, using a of implemented in M. T. D. T. MacLean B. O. An for analysis of quantitative mass proteomic 2014; PubMed Scopus Google Scholar). protein and of the of the the was used to the The of of of mixtures were used to of proteins, by the in the for HRM and shotgun proteomics. The of proteins was for of protein and the of the of abundance was on the on the of changes in the and the proteins. Shotgun proteomics identified the number of proteins. The at the of that the HRM approach identified than as in proteins The revealed the of HRM and HRM shotgun proteomics was to changes of with the HRM a better to changes of 60% and (supplemental Fig. The of three-dimensional human liver from was used to identify relevant proteomic changes using HRM to the of on the the liver were treated with a of and the The was by an of on the was with a of to subtoxic and and were for HRM profiling. spectral was from the samples peptides of protein with on The samples were acquired in DIA using a (supplemental The spectra were with Spectronaut. peptides were identified The high reproducibility of peptide detection in a quasi complete data set, the of and technical of were from measurements and The of peptides identified in technical replicates were on The was using the of of in of proteins compared with proteins and in M.R. H. and of in to and Res. 2013; PubMed Scopus Google novel and as and were of was at the as S. E. M. to proteins and toxicity Rev. PubMed Scopus Google were which are of to M.R. H. and of in to and Res. 2013; PubMed Scopus Google Scholar). proteins for were identified at was from the and were up at The for was and the for was The of the and protein was using stable isotope (supplemental Fig. proteins, as of cell PubMed Scopus Google and protein were into which is to and to proteins. the measurements of the of was identified at the on four related proteins (GATM, PARK7, PRDX6, and VDAC2) and on and HRM analysis of the peptides that are detectable at of APAP. The peptides for the and Fig. We a of quantification of HRM and shotgun proteomics using a profiling sample The HRM outperformed shotgun proteomics in number of consistently identified in precision of and in detection of differentially abundant proteins. The and of the HRM approach for profiling is the quasi complete data that can be data is the of shotgun proteomics for quantification Smith and for label-free 2012; 13: PubMed Scopus Google Scholar). The HRM approach this The data can be using e.g. as for HRM O. R. T. S. and L. novel for protein profiling on in data implemented in Spectronaut. on Mass and for shotgun proteomics N.W. S.P. MacCoss M.J. Wu C.C. peptide identification in proteomic data-dependent Cell. Proteomics. 2013; 13: Full Text Full Text PDF PubMed Scopus Google Scholar, J. Nagaraj N. Mann M. Accurate label-free quantification by and peptide Cell. Proteomics. 2014; 13: Full Text Full Text PDF PubMed Scopus Google Scholar). We that the of HRM to the for to better and data HRM and shotgun proteomics is the of the quantified shotgun proteomics, the quantification is on the precursor in HRM on the of the on the fragment ion is less to as of for fragment is shotgun proteomics method of the acquisition time for spectra that are used for quantification. The DIA method introduced of the acquisition time for MS2 spectra that are used for quantification. The of that quantification on the MS2 is The findings are with compared the of and of MS2 J.D. Kuehn A. Merrihew G.E. Bateman N.W. MacLean B.X. Ting Y.S. Canterbury J.D. Marsh D.M. Kellmann M. Zabrouskov V. Wu C.C. MacCoss M.J. Multiplexed MS/MS for improved data-independent acquisition.Nat. Methods. 2013; 10: 744-746Crossref PubMed Scopus (207) Google Scholar). The ion of can be and in a peptide from be quantified to on (supplemental Fig. Using HRM profiling with proteins from of were identified as the HRM protein profiling for discovery novel is that the of cell by is the of the to proteins, M.R. H. and of in to and Res. 2013; PubMed Scopus Google Scholar). than of on human proteins were We profiled on proteins. an in mitochondrial oxidative stress G. S. N. B. J. H. D. O. T. R. and mitochondrial due to of protein 2010; PubMed Scopus Google the detected at R.M. Wilson M.A. R. C. S. M.J. D. M.R. The protein is due to mitochondrial Natl. Acad. Sci. U.S.A. 2004; PubMed Scopus Google Scholar). is in the of mitochondrial in liver T. N. T. S. E. M. M.J. Fisher mitochondrial and liver in J. 2009; PubMed Scopus Google Scholar). The was to be in to oxidative stress by of into the M. T. T. A. S. T. H. of a of mitochondrial is by and by with Res. S. 2014; Scholar). has been in in oxidative stress and J.K. G. the of in Commun. 2014; PubMed Scopus Google Scholar). findings that relevant of protein were which might to cell The reproducibility and high of HRM the to mass proteomics in the shotgun proteomics but is used for the has the of spectral libraries that as for HRM targeted data The of DIA acquisition for high content discovery is a novel approach with the for that has been We of M. and M.A. for of the We M. and R. for mass spectrometric sample and with

Determinants of accelerated metabolomic and epigenetic aging in a UK cohort

Oliver Robinson, Marc Chadeau‐Hyam, İbrahim Karaman +14 more · Aging Cell · 2020 · 192 citations

Markers of biological aging have potential utility in primary care and public health. We developed a model of age based on untargeted metabolic profiling across multiple platforms, including nuclear magnetic resonance spectroscopy and liquid chromatography-mass spectrometry in urine and serum, within a large sample (N = 2,239) from the UK Airwave cohort. We validated a subset of model predictors in a Finnish cohort including repeat measurements from 2,144 individuals. We investigated the determinants of accelerated aging, including lifestyle and psychological risk factors for premature mortality. The metabolomic age model was well correlated with chronological age (mean r = .86 across independent test sets). Increased metabolomic age acceleration (mAA) was associated after false discovery rate (FDR) correction with overweight/obesity, diabetes, heavy alcohol use and depression. DNA methylation age acceleration measures were uncorrelated with mAA. Increased DNA methylation phenotypic age acceleration (N = 1,110) was associated after FDR correction with heavy alcohol use, hypertension and low income. In conclusion, metabolomics is a promising approach for the assessment of biological age and appears complementary to established epigenetic clocks.

Bioinformatics enrichment tools: paths toward the comprehensive functional analysis of large gene lists

Da Wei Huang, Brad T. Sherman, Richard A. Lempicki · Nucleic Acids Research · 2008 · 14,721 citations

Functional analysis of large gene lists, derived in most cases from emerging high-throughput genomic, proteomic and bioinformatics scanning approaches, is still a challenging and daunting task. The gene-annotation enrichment analysis is a promising high-throughput strategy that increases the likelihood for investigators to identify biological processes most pertinent to their study. Approximately 68 bioinformatics enrichment tools that are currently available in the community are collected in this survey. Tools are uniquely categorized into three major classes, according to their underlying enrichment algorithms. The comprehensive collections, unique tool classifications and associated questions/issues will provide a more comprehensive and up-to-date view regarding the advantages, pitfalls and recent trends in a simpler tool-class level rather than by a tool-by-tool approach. Thus, the survey will help tool designers/developers and experienced end users understand the underlying algorithms and pertinent details of particular tool categories/tools, enabling them to make the best choices for their particular research interests.

An Introduction to Statistical Methods and Data Analysis

· Technometrics · 1994 · 5,383 citations

PART I: INTRODUCTION 1. WHAT IS STATISTICS? Introduction / Why Study Statistics? / Some Current Applications of Statistics / What Do Statisticians Do? / Quality and Process Improvement / A Note to the Student / Summary / Supplementary Exercises PART II: COLLECTING THE DATA 2. USING SURVEYS AND SCIENTIFIC STUDIES TO COLLECT DATA Introduction / Surveys / Scientific Studies / Observational Studies / Data Management: Preparing Data for Summarization and Analysis / Summary PART III: SUMMARIZING DATA 3. DATA DESCRIPTION Introduction / Describing Data on a Single Variable: Graphical Methods / Describing Data on a Single Variable: Measures of Central Tendency / Describing Data on a Single Variable: Measures of Variability / The Box Plot / Summarizing Data from More Than One Variable / Calculators, Computers, and Software Systems / Summary / Key Formulas / Supplementary Exercises PART IV: TOOLS AND CONCEPTS 4. PROBABILITY AND PROBABILITY DISTRIBUTIONS How Probability Can Be Used in Making Inferences / Finding the Probability of an Event / Basic Event Relations and Probability Laws / Conditional Probability and Independence / Bayes's Formula / Variables: Discrete and Continuous / Probability Distributions for Discrete Random Variables / A Useful Discrete Random Variable: The Binomial / Probability Distributions for Continuous Random Variables / A Useful Continuous Random Variable: The Normal Distribution / Random Sampling / Sampling Distributions / Normal Approximation to the Binomial / Summary / Key Formulas / Supplementary Exercises PART V: ANALYZING DATA: CENTRAL VALUES, VARIANCES, AND PROPORTIONS 5. INFERENCES ON A POPULATION CENTRAL VALUE Introduction and Case Study / Estimation of / Choosing the Sample Size for Estimating / A Statistical Test for / Choosing the Sample Size for Testing / The Level of Significance of a Statistical Test / Inferences about for Normal Population, s Unknown / Inferences about the Population Median / Summary / Key Formulas / Supplementary Exercises 6. COMPARING TWO POPULATION CENTRAL VALUES Introduction and Case Study / Inferences about 1 - 2: Independent Samples / A Nonparametric Alternative: The Wilcoxon Rank Sum Test / Inferences about 1 - 2: Paired Data / A Nonparametric Alternative: The Wilcoxon Signed-Rank Test / Choosing Sample Sizes for Inferences about 1 - 2 / Summary / Key Formulas / Supplementary Exercises 7. INFERENCES ABOUT POPULATION VARIANCES Introduction and Case Study / Estimation and Tests for a Population Variance / Estimation and Tests for Comparing Two Population Variances / Tests for Comparing k > 2 Population Variances / Summary / Key Formulas / Supplementary Exercises 8. INFERENCES ABOUT POPULATION CENTRAL VALUES Introduction and Case Study / A Statistical Test About More Than Two Population Variances / Checking on the Assumptions / Alternative When Assumptions are Violated: Transformations / A Nonparametric Alternative: The Kruskal-Wallis Test / Summary / Key Formulas / Supplementary Exercises 9. MULTIPLE COMPARISONS Introduction and Case Study / Planned Comparisons Among Treatments: Linear Contrasts / Which Error Rate Is Controlled / Multiple Comparisons with the Best Treatment / Comparison of Treatments to a Control / Pairwise Comparison on All Treatments / Summary / Key Formulas / Supplementary Exercises 10. CATEGORICAL DATA Introduction and Case Study / Inferences about a Population Proportion p / Comparing Two Population Proportions p1 - p2 / Probability Distributions for Discrete Random Variables / The Multinomial Experiment and Chi-Square Goodness-of-Fit Test / The Chi-Square Test of Homogeneity of Proportions / The Chi-Square Test of Independence of Two Nominal Level Variables / Fisher's Exact Test, a Permutation Test / Measures of Association / Combining Sets of Contingency Tables / Summary / Key Formulas / Supplementary Exercises PART VI: ANALYZING DATA: REGRESSION METHODS, MODEL BUILDING 11. SIMPLE LINEAR REGRESSION AND CORRELATION Linear Regression and the Method of Least Squares / Transformations to Linearize Data / Correlation / A Look Ahead: Multiple Regression / Summary of Key Formulas. Supplementary Exercises. 12. INFERENCES RELATED TO LINEAR REGRESSION AND CORRELATION Introduction and Case Study / Diagnostics for Detecting Violations of Model Conditions / Inferences about the Intercept and Slope of the Regression Line / Inferences about the Population Mean for a Specified Value of the Explanatory Variable / Predictions and Prediction Intervals / Examining Lack of Fit in the Model / The Inverse Regression Problem (Calibration): Predicting Values for x for a Specified Value of y / Summary / Key Formulas / Supplementary Exercises 13. MULTIPLE REGRESSION AND THE GENERAL LINEAR MODEL Introduction and Case Study / The General Linear Model / Least Squares Estimates of Parameters in the General Linear Model / Inferences about the Parameters in the General Linear Model / Inferences about the Population Mean and Predictions from the General Linear Model / Comparing the Slope of Several Regression Lines / Logistic Regression / Matrix Formulation of the General Linear Model / Summary / Key Formulas / Supplementary Exercises 14. BUILDING REGRESSION MODELS WITH DIAGNOSTICS Introduction and Case Study / Selecting the Variables (Step 1) / Model Formulation (Step 2) / Checking Model Conditions (Step 3) / Summary / Key Formulas / Supplementary Exercises PART VII: ANALYZING DATA: DESIGN OF EXPERIMENTS AND ANOVA 15. DESIGN CONCEPTS FOR EXPERIMENTS AND STUDIES Experiments, Treatments, Experimental Units, Blocking, Randomization, and Measurement Units / How Many Replications? / Studies for Comparing Means versus Studies for Comparing Variances / Summary / Key Formulas / Supplementary Exercises 16. ANALYSIS OF VARIANCE FOR STANDARD DESIGNS Introduction and Case Study / Completely Randomized Design with Single Factor / Randomized Block Design / Latin Square Design / Factorial Experiments in a Completely Randomized Design / The Estimation of Treatment Differences and Planned Comparisons in the Treatment Means / Checking Model Conditions / Alternative Analyses: Transformation and Friedman's Rank-Based Test / Summary / Key Formulas / Supplementary Exercises 17. ANALYSIS OF COVARIANCE Introduction and Case Study / A Completely Randomized Design with One Covariate / The Extrapolation Problem / Multiple Covariates and More Complicated Designs / Summary / Key Formulas / Supplementary Exercises 18. ANALYSIS OF VARIANCE FOR SOME UNBALANCED DESIGNS Introduction and Case Study / A Randomized Block Design with One or More Missing Observations / A Latin Square Design with Missing Data / Incomplete Block Designs / Summary / Key Formulas / Supplementary Exercises 19. ANALYSIS OF VARIANCE FOR SOME FIXED EFFECTS, RANDOM EFFECTS, AND MIXED EFFECTS MODELS Introduction and Case Study / A One-Factor Experiment with Random Treatment Effects / Extensions of Random-Effects Models / A Mixed Model: Experiments with Both Fixed and Random Treatment Effects / Models with Nested Factors / Rules for Obtaining Expected Mean Squares / Summary / Key Formulas / Supplementary Exercises 20. SPLIT-PLOT DESIGNS AND EXPERIMENTS WITH REPEATED MEASURES Introduction and Case Study / Split-Plot Designs / Single-Factor Experiments with Repeated Measures / Two-Factor Experiments with Repeated Measures on One of the Factors / Crossover Design / Summary / Key Formulas / Supplementary Exercises PART VIII: COMMUNICATING AND DOCUMENTING THE RESULTS OF A STUDY OR EXPERIMENT 21. COMMUNICATING AND DOCUMENTING THE RESULTS OF A STUDY OR EXPERIMENT Introduction / The Difficulty of Good Communication / Communication Hurdles: Graphical Distortions / Communication Hurdles: Biased Samples / Communication Hurdles: Sample Size / The Statistical Report / Documentation and Storage of Results / Summary / Supplementary Exercises

Analysis of molecular variance inferred from metric distances among DNA haplotypes: application to human mitochondrial DNA restriction data.

Laurent Excoffier, Peter E. Smouse, Joseph M. Quattro · Genetics · 1992 · 13,984 citations

We present here a framework for the study of molecular variation within a single species. Information on DNA haplotype divergence is incorporated into an analysis of variance format, derived from a matrix of squared-distances among all pairs of haplotypes. This analysis of molecular variance (AMOVA) produces estimates of variance components and F-statistic analogs, designated here as phi-statistics, reflecting the correlation of haplotypic diversity at different levels of hierarchical subdivision. The method is flexible enough to accommodate several alternative input matrices, corresponding to different types of molecular data, as well as different types of evolutionary assumptions, without modifying the basic structure of the analysis. The significance of the variance components and phi-statistics is tested using a permutational approach, eliminating the normality assumption that is conventional for analysis of variance but inappropriate for molecular data. Application of AMOVA to human mitochondrial DNA haplotype data shows that population subdivisions are better resolved when some measure of molecular differences among haplotypes is introduced into the analysis. At the intraspecific level, however, the additional information provided by knowing the exact phylogenetic relations among haplotypes or by a nonlinear translation of restriction-site change into nucleotide diversity does not significantly modify the inferred population genetic structure. Monte Carlo studies show that site sampling does not fundamentally affect the significance of the molecular variance components. The AMOVA treatment is easily extended in several different directions and it constitutes a coherent and flexible framework for the statistical analysis of molecular data.

Extremely randomized trees

Pierre Geurts, Damien Ernst, Louis Wehenkel · Machine Learning · 2006 · 8,537 citations

Measures of location and scale for velocities in clusters of galaxies - A robust approach

Timothy C. Beers, Kevin Flynn, Karl Gebhardt · The Astronomical Journal · 1990 · 1,419 citations

Recent observational evidence suggests that few clusters and groups of galaxies have achieved dynamical equilibrium, where a Gaussian distribution of radial velocities might be expected. The canonical estimation techniques, which either assume Gaussian parent populations or clip observed velocity distributions until the Gaussian assumption is satisfied, are not, in general, minimum variance estimators of the kinematic properties of such clusters. In addition, a detailed examination of the local kinematical properties of clusters requires the use of efficient statistical estimators which are insensitive to localized misbehavior in small datasets. For these reasons we suggest that the traditional methods of assigning cluster mean velocities, dispersions, and confidence intervals on these quantities are no longer adequate. In this paper we discuss alternative estimators of the kinematical properties of clusters of galaxies-estimators that are resistant in the presence of outliers, and robust for a broad range of non-Gaussian underlying populations. Because a number of different estimators may be used for any given quantity, we urge a change in the nomenclature to one that does not imply an underlying probabilistic model: we suggest C_u_ for the central location ( "mean"), S_v_ for the scale ( "dispersion"), and IC_u,v_ and IS_v_ for the set of confidence intervals about C_u_ and S_v_, respectively. The subscripts u and v indicate the methods used to obtain the sample estimate. Extensive simulations for a number of common situations realizable in small to large samples of cluster radial velocities allow us to identify minimum variance estimators. We also explore the estimation of confidence intervals using the jackknife and bootstrap resampling techniques, and compare these methods to simple formulas based on sample estimates of central location and scale. Our tests reveal that the family of location and scale estimators based on Tukey's biweight prove consistently superior for most applications. Confidence intervals on location based on the biweight also prove superior. Estimators of confidence intervals on scale require resampling- although bootstrapping is preferred, less computationally demanding estimators based on the jackknife of the biweight scale are shown to be adequate for most situations.

Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods

David P. MacKinnon, Chondra M. Lockwood, Jason Williams · Multivariate Behavioral Research · 2004 · 7,576 citations

The most commonly used method to test an indirect effect is to divide the estimate of the indirect effect by its standard error and compare the resulting z statistic with a critical value from the standard normal distribution. Confidence limits for the indirect effect are also typically based on critical values from the standard normal distribution. This article uses a simulation study to demonstrate that confidence limits are imbalanced because the distribution of the indirect effect is normal only in special cases. Two alternatives for improving the performance of confidence limits for the indirect effect are evaluated: (a) a method based on the distribution of the product of two normal random variables, and (b) resampling methods. In Study 1, confidence limits based on the distribution of the product are more accurate than methods based on an assumed normal distribution but confidence limits are still imbalanced. Study 2 demonstrates that more accurate confidence limits are obtained using resampling methods, with the bias-corrected bootstrap the best method overall.

Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate

Christopher R. Genovese, Nicole A. Lazar, Thomas E. Nichols · NeuroImage · 2002 · 5,015 citations

Bayesian Network Classifiers

Nir Friedman, Dan Geiger, Moisés Goldszmidt · Machine Learning · 1997 · 4,729 citations

Nonparametric statistical testing of EEG- and MEG-data

Eric Maris, Robert Oostenveld · Journal of Neuroscience Methods · 2007 · 9,151 citations

Learning Bayesian Networks: The Combination of Knowledge and Statistical Data

David Heckerman, Dan Geiger, David M. Chickering · Machine Learning · 1995 · 3,239 citations

Valid Inference in Random Effects Meta‐Analysis

Dean Follmann, Michael A. Proschan · Biometrics · 1999 · 129 citations

The standard approach to inference for random effects meta-analysis relies on approximating the null distribution of a test statistic by a standard normal distribution. This approximation is asymptotic on k, the number of studies, and can be substantially in error in medical meta-analyses, which often have only a few studies. This paper proposes permutation and ad hoc methods for testing with the random effects model. Under the group permutation method, we randomly switch the treatment and control group labels in each trial. This idea is similar to using a permutation distribution for a community intervention trial where communities are randomized in pairs. The permutation method theoretically controls the type I error rate for typical meta-analyses scenarios. We also suggest two ad hoc procedures. Our first suggestion is to use a t-reference distribution with k-1 degrees of freedom rather than a standard normal distribution for the usual random effects test statistic. We also investigate the use of a simple t-statistic on the reported treatment effects.

R: A Language and Environment for Statistical Computing

R Core Team · 2000 · 352,998 citations

Most R novices will start with Appendix A [A sample session], page 80.This should give some familiarity with the style of R sessions and more importantly some instant feedback on what actually happens.Many users will come to R mainly for its graphical facilities.

Local Indicators of Spatial Association—LISA

Luc Anselin · Geographical Analysis · 1995 · 12,413 citations

The capabilities for visualization, rapid data retrieval, and manipulation in geographic information systems (GIS) have created the need for new techniques of exploratory data analysis that focus on the “spatial” aspects of the data. The identification of local patterns of spatial association is an important concern in this respect. In this paper, I outline a new general class of local indicators of spatial association (LISA) and show how they allow for the decomposition of global indicators, such as Moran's I, into the contribution of each observation. The LISA statistics serve two purposes. On one hand, they may be interpreted as indicators of local pockets of nonstationarity, or hot spots, similar to the G i and G* i statistics of Getis and Ord (1992). On the other hand, they may be used to assess the influence of individual locations on the magnitude of the global statistic and to identify “outliers,” as in Anselin's Moran scatterplot (1993a). An initial evaluation of the properties of a LISA statistic is carried out for the local Moran, which is applied in a study of the spatial pattern of conflict for African countries and in a number of Monte Carlo simulations.

CONFIDENCE LIMITS ON PHYLOGENIES: AN APPROACH USING THE BOOTSTRAP

Joseph Felsenstein · Evolution · 1985 · 41,353 citations

The recently-developed statistical method known as the "bootstrap" can be used to place confidence intervals on phylogenies. It involves resampling points from one's own data, with replacement, to create a series of bootstrap samples of the same size as the original data. Each of these is analyzed, and the variation among the resulting estimates taken to indicate the size of the error involved in making estimates from the original data. In the case of phylogenies, it is argued that the proper method of resampling is to keep all of the original species while sampling characters with replacement, under the assumption that the characters have been independently drawn by the systematist and have evolved independently. Majority-rule consensus trees can be used to construct a phylogeny showing all of the inferred monophyletic groups that occurred in a majority of the bootstrap samples. If a group shows up 95% of the time or more, the evidence for it is taken to be statistically significant. Existing computer programs can be used to analyze different bootstrap samples by using weights on the characters, the weight of a character being how many times it was drawn in bootstrap sampling. When all characters are perfectly compatible, as envisioned by Hennig, bootstrap sampling becomes unnecessary; the bootstrap method would show significant evidence for a group if it is defined by three or more characters.

CLUMPP: a cluster matching and permutation program for dealing with label switching and multimodality in analysis of population structure

Mattias Jakobsson, Noah A. Rosenberg · Bioinformatics · 2007 · 6,470 citations

MOTIVATION: Clustering of individuals into populations on the basis of multilocus genotypes is informative in a variety of settings. In population-genetic clustering algorithms, such as BAPS, STRUCTURE and TESS, individual multilocus genotypes are partitioned over a set of clusters, often using unsupervised approaches that involve stochastic simulation. As a result, replicate cluster analyses of the same data may produce several distinct solutions for estimated cluster membership coefficients, even though the same initial conditions were used. Major differences among clustering solutions have two main sources: (1) 'label switching' of clusters across replicates, caused by the arbitrary way in which clusters in an unsupervised analysis are labeled, and (2) 'genuine multimodality,' truly distinct solutions across replicates. RESULTS: To facilitate the interpretation of population-genetic clustering results, we describe three algorithms for aligning multiple replicate analyses of the same data set. We have implemented these algorithms in the computer program CLUMPP (CLUster Matching and Permutation Program). We illustrate the use of CLUMPP by aligning the cluster membership coefficients from 100 replicate cluster analyses of 600 chickens from 20 different breeds. AVAILABILITY: CLUMPP is freely available at http://rosenberglab.bioinformatics.med.umich.edu/clumpp.html.

A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion

Tan Bui–Thanh, Omar Ghattas, James L. Martin +1 more · SIAM Journal on Scientific Computing · 2013 · 369 citations

We present a computational framework for estimating the uncertainty in the numerical solution of linearized infinite-dimensional statistical inverse problems. We adopt the Bayesian inference formulation: given observational data and their uncertainty, the governing forward problem and its uncertainty, and a prior probability distribution describing uncertainty in the parameter field, find the posterior probability distribution over the parameter field. The prior must be chosen appropriately in order to guarantee well-posedness of the infinite-dimensional inverse problem and facilitate computation of the posterior. Furthermore, straightforward discretizations may not lead to convergent approximations of the infinite-dimensional problem. And finally, solution of the discretized inverse problem via explicit construction of the covariance matrix is prohibitive due to the need to solve the forward problem as many times as there are parameters. Our computational framework builds on the infinite-dimensional formulation proposed by Stuart [Acta Numer., 19 (2010), pp. 451--559] and incorporates a number of components aimed at ensuring a convergent discretization of the underlying infinite-dimensional inverse problem. The framework additionally incorporates algorithms for manipulating the prior, constructing a low rank approximation of the data-informed component of the posterior covariance operator, and exploring the posterior that together ensure scalability of the entire framework to very high parameter dimensions. We demonstrate this computational framework on the Bayesian solution of an inverse problem in three-dimensional global seismic wave propagation with hundreds of thousands of parameters.

Properties of the urn randomization in clinical trials

Lee‐Jen Wei, John M. Lachin · Controlled Clinical Trials · 1988 · 308 citations

Nonparametric permutation tests for functional neuroimaging: A primer with examples

Thomas E. Nichols, Andrew P. Holmes · Human Brain Mapping · 2001 · 6,444 citations

Requiring only minimal assumptions for validity, nonparametric permutation testing provides a flexible and intuitive methodology for the statistical analysis of data from functional neuroimaging experiments, at some computational expense. Introduced into the functional neuroimaging literature by Holmes et al. ([1996]: J Cereb Blood Flow Metab 16:7-22), the permutation approach readily accounts for the multiple comparisons problem implicit in the standard voxel-by-voxel hypothesis testing framework. When the appropriate assumptions hold, the nonparametric permutation approach gives results similar to those obtained from a comparable Statistical Parametric Mapping approach using a general linear model with multiple comparisons corrections derived from random field theory. For analyses with low degrees of freedom, such as single subject PET/SPECT experiments or multi-subject PET/SPECT or fMRI designs assessed for population effects, the nonparametric approach employing a locally pooled (smoothed) variance estimate can outperform the comparable Statistical Parametric Mapping approach. Thus, these nonparametric techniques can be used to verify the validity of less computationally expensive parametric approaches. Although the theory and relative advantages of permutation approaches have been discussed by various authors, there has been no accessible explication of the method, and no freely distributed software implementing it. Consequently, there have been few practical applications of the technique. This article, and the accompanying MATLAB software, attempts to address these issues. The standard nonparametric randomization and permutation testing ideas are developed at an accessible level, using practical examples from functional neuroimaging, and the extensions for multiple comparisons described. Three worked examples from PET and fMRI are presented, with discussion, and comparisons with standard parametric approaches made where appropriate. Practical considerations are given throughout, and relevant statistical concepts are expounded in appendices.

Learning Bayesian networks: The combination of knowledge and statistical data

David Heckerman, Dan Geiger, David M. Chickering · Machine Learning · 1995 · 2,040 citations

INTERPOLATING, EXTRAPOLATING, AND COMPARING INCIDENCE-BASED SPECIES ACCUMULATION CURVES

Robert K. Colwell, Chang Xuan Mao, Jing Chang · Ecology · 2004 · 1,790 citations

A general binomial mixture model is proposed for the species accumulation function based on presence–absence (incidence) of species in a sample of quadrats or other sampling units. The model covers interpolation between zero and the observed number of samples, as well as extrapolation beyond the observed sample set. For interpolation (sample-based rarefaction), easily calculated, closed-form expressions for both expected richness and its confidence limits are developed (using the method of moments) that completely eliminate the need for resampling methods and permit direct statistical comparison of richness between sample sets. An incidence-based form of the Coleman (random-placement) model is developed and compared with the moment-based interpolation method. For extrapolation beyond the empirical sample set (and simultaneously, as an alternative method of interpolation), a likelihood-based estimator with a bootstrap confidence interval is described that relies on a sequential, AIC-guided algorithm to fit the mixture model parameters. Both the moment-based and likelihood-based estimators are illustrated with data sets for temperate birds and tropical seeds, ants, and trees. The moment-based estimator is confidently recommended for interpolation (sample-based rarefaction). For extrapolation, the likelihood-based estimator performs well for doubling or tripling the number of empirical samples, but it is not reliable for estimating the richness asymptote. The sensitivity of individual-based and sample-based rarefaction to spatial (or temporal) patchiness is discussed.