[Download PDF](/articles/s41586-021-03819-2.pdf)

### Subjects

* [Computational biophysics](/subjects/computational-biophysics)
* [Machine learning](/subjects/machine-learning)
* [Protein structure predictions](/subjects/protein-structure-predictions)
* [Structural biology](/subjects/structural-biology)

Abstract
--------

Proteins are essential to life, and understanding their structure can facilitate a mechanistic understanding of their function. Through an enormous experimental effort[1](#ref-CR1 "Thompson, M. C., Yeates, T. O. & Rodriguez, J. A. Advances in methods for atomic resolution macromolecular structure determination. F1000Res. 9, 667 (2020)."),[2](#ref-CR2 "Bai, X.-C., McMullan, G. & Scheres, S. H. W. How cryo-EM is revolutionizing structural biology. Trends Biochem. Sci. 40, 49–57 (2015)."),[3](#ref-CR3 "Jaskolski, M., Dauter, Z. & Wlodawer, A. A brief history of macromolecular crystallography, illustrated by a family tree and its Nobel fruits. FEBS J. 281, 3985–4009 (2014)."),[4](/articles/s41586-021-03819-2#ref-CR4 "Wüthrich, K. The way to NMR structures of proteins. Nat. Struct. Biol. 8, 923–925 (2001)."), the structures of around 100,000 unique proteins have been determined[5](/articles/s41586-021-03819-2#ref-CR5 "wwPDB Consortium. Protein Data Bank: the single global archive for 3D macromolecular structure data. Nucleic Acids Res. 47, D520–D528 (2018)."), but this represents a small fraction of the billions of known protein sequences[6](/articles/s41586-021-03819-2#ref-CR6 "Mitchell, A. L. et al. MGnify: the microbiome analysis resource in 2020. Nucleic Acids Res. 48, D570–D578 (2020)."),[7](/articles/s41586-021-03819-2#ref-CR7 "Steinegger, M., Mirdita, M. & Söding, J. Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold. Nat. Methods 16, 603–606 (2019)."). Structural coverage is bottlenecked by the months to years of painstaking effort required to determine a single protein structure. Accurate computational approaches are needed to address this gap and to enable large-scale structural bioinformatics. Predicting the three-dimensional structure that a protein will adopt based solely on its amino acid sequence—the structure prediction component of the ‘protein folding problem’[8](/articles/s41586-021-03819-2#ref-CR8 "Dill, K. A., Ozkan, S. B., Shell, M. S. & Weikl, T. R. The protein folding problem. Annu. Rev. Biophys. 37, 289–316 (2008).")—has been an important open research problem for more than 50 years[9](/articles/s41586-021-03819-2#ref-CR9 "Anfinsen, C. B. Principles that govern the folding of protein chains. Science 181, 223–230 (1973)."). Despite recent progress[10](#ref-CR10 "Senior, A. W. et al. Improved protein structure prediction using potentials from deep learning. Nature 577, 706–710 (2020)."),[11](#ref-CR11 "Wang, S., Sun, S., Li, Z., Zhang, R. & Xu, J. Accurate de novo prediction of protein contact map by ultra-deep learning model. PLOS Comput. Biol. 13, e1005324 (2017)."),[12](#ref-CR12 "Zheng, W. et al. Deep-learning contact-map guided protein structure prediction in CASP13. Proteins 87, 1149–1164 (2019)."),[13](#ref-CR13 "Abriata, L. A., Tamò, G. E. & Dal Peraro, M. A further leap of improvement in tertiary structure prediction in CASP13 prompts new routes for future assessments. Proteins 87, 1100–1112 (2019)."),[14](/articles/s41586-021-03819-2#ref-CR14 "Pearce, R. & Zhang, Y. Deep learning techniques have significantly impacted protein structure prediction and protein design. Curr. Opin. Struct. Biol. 68, 194–207 (2021)."), existing methods fall far short of atomic accuracy, especially when no homologous structure is available. Here we provide the first computational method that can regularly predict protein structures with atomic accuracy even in cases in which no similar structure is known. We validated an entirely redesigned version of our neural network-based model, AlphaFold, in the challenging 14th Critical Assessment of protein Structure Prediction (CASP14)[15](/articles/s41586-021-03819-2#ref-CR15 "Moult, J., Fidelis, K., Kryshtafovych, A., Schwede, T. & Topf, M. Critical assessment of techniques for protein structure prediction, fourteenth round. CASP 14 Abstract Book
                  https://www.predictioncenter.org/casp14/doc/CASP14_Abstracts.pdf

                 (2020)."), demonstrating accuracy competitive with experimental structures in a majority of cases and greatly outperforming other methods. Underpinning the latest version of AlphaFold is a novel machine learning approach that incorporates physical and biological knowledge about protein structure, leveraging multi-sequence alignments, into the design of the deep learning algorithm.

Main
----

The development of computational methods to predict three-dimensional (3D) protein structures from the protein sequence has proceeded along two complementary paths that focus on either the physical interactions or the evolutionary history. The physical interaction programme heavily integrates our understanding of molecular driving forces into either thermodynamic or kinetic simulation of protein physics[16](/articles/s41586-021-03819-2#ref-CR16 "Brini, E., Simmerling, C. & Dill, K. Protein storytelling through physics. Science 370, eaaz3041 (2020).") or statistical approximations thereof[17](/articles/s41586-021-03819-2#ref-CR17 "Sippl, M. J. Calculation of conformational ensembles from potentials of mean force. An approach to the knowledge-based prediction of local structures in globular proteins. J. Mol. Biol. 213, 859–883 (1990)."). Although theoretically very appealing, this approach has proved highly challenging for even moderate-sized proteins due to the computational intractability of molecular simulation, the context dependence of protein stability and the difficulty of producing sufficiently accurate models of protein physics. The evolutionary programme has provided an alternative in recent years, in which the constraints on protein structure are derived from bioinformatics analysis of the evolutionary history of proteins, homology to solved structures[18](/articles/s41586-021-03819-2#ref-CR18 "Šali, A. & Blundell, T. L. Comparative protein modelling by satisfaction of spatial restraints. J. Mol. Biol. 234, 779–815 (1993)."),[19](/articles/s41586-021-03819-2#ref-CR19 "Roy, A., Kucukural, A. & Zhang, Y. I-TASSER: a unified platform for automated protein structure and function prediction. Nat. Protocols 5, 725–738 (2010).") and pairwise evolutionary correlations[20](#ref-CR20 "Altschuh, D., Lesk, A. M., Bloomer, A. C. & Klug, A. Correlation of co-ordinated amino acid substitutions with function in viruses related to tobacco mosaic virus. J. Mol. Biol. 193, 693–707 (1987)."),[21](#ref-CR21 "Shindyalov, I. N., Kolchanov, N. A. & Sander, C. Can three-dimensional contacts in protein structures be predicted by analysis of correlated mutations? Protein Eng. 7, 349–358 (1994)."),[22](#ref-CR22 "Weigt, M., White, R. A., Szurmant, H., Hoch, J. A. & Hwa, T. Identification of direct residue contacts in protein–protein interaction by message passing. Proc. Natl Acad. Sci. USA 106, 67–72 (2009)."),[23](#ref-CR23 "Marks, D. S. et al. Protein 3D structure computed from evolutionary sequence variation. PLoS ONE 6, e28766 (2011)."),[24](/articles/s41586-021-03819-2#ref-CR24 "Jones, D. T., Buchan, D. W. A., Cozzetto, D. & Pontil, M. PSICOV: precise structural contact prediction using sparse inverse covariance estimation on large multiple sequence alignments. Bioinformatics 28, 184–190 (2012)."). This bioinformatics approach has benefited greatly from the steady growth of experimental protein structures deposited in the Protein Data Bank (PDB)[5](/articles/s41586-021-03819-2#ref-CR5 "wwPDB Consortium. Protein Data Bank: the single global archive for 3D macromolecular structure data. Nucleic Acids Res. 47, D520–D528 (2018)."), the explosion of genomic sequencing and the rapid development of deep learning techniques to interpret these correlations. Despite these advances, contemporary physical and evolutionary-history-based approaches produce predictions that are far short of experimental accuracy in the majority of cases in which a close homologue has not been solved experimentally and this has limited their utility for many biological applications.

In this study, we develop the first, to our knowledge, computational approach capable of predicting protein structures to near experimental accuracy in a majority of cases. The neural network AlphaFold that we developed was entered into the CASP14 assessment (May–July 2020; entered under the team name ‘AlphaFold2’ and a completely different model from our CASP13 AlphaFold system[10](/articles/s41586-021-03819-2#ref-CR10 "Senior, A. W. et al. Improved protein structure prediction using potentials from deep learning. Nature 577, 706–710 (2020).")). The CASP assessment is carried out biennially using recently solved structures that have not been deposited in the PDB or publicly disclosed so that it is a blind test for the participating methods, and has long served as the gold-standard assessment for the accuracy of structure prediction[25](/articles/s41586-021-03819-2#ref-CR25 "Moult, J., Pedersen, J. T., Judson, R. & Fidelis, K. A large-scale experiment to assess protein structure prediction methods. Proteins 23, ii–iv (1995)."),[26](/articles/s41586-021-03819-2#ref-CR26 "Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K. & Moult, J. Critical assessment of methods of protein structure prediction (CASP)-round XIII. Proteins 87, 1011–1020 (2019).").

In CASP14, AlphaFold structures were vastly more accurate than competing methods. AlphaFold structures had a median backbone accuracy of 0.96 Å r.m.s.d.95 (Cα root-mean-square deviation at 95% residue coverage) (95% confidence interval = 0.85–1.16 Å) whereas the next best performing method had a median backbone accuracy of 2.8 Å r.m.s.d.95 (95% confidence interval = 2.7–4.0 Å) (measured on CASP domains; see Fig. [1a](/articles/s41586-021-03819-2#Fig1) for backbone accuracy and Supplementary Fig. [14](/articles/s41586-021-03819-2#MOESM1) for all-atom accuracy). As a comparison point for this accuracy, the width of a carbon atom is approximately 1.4 Å. In addition to very accurate domain structures (Fig. [1b](/articles/s41586-021-03819-2#Fig1)), AlphaFold is able to produce highly accurate side chains (Fig. [1c](/articles/s41586-021-03819-2#Fig1)) when the backbone is highly accurate and considerably improves over template-based methods even when strong templates are available. The all-atom accuracy of AlphaFold was 1.5 Å r.m.s.d.95 (95% confidence interval = 1.2–1.6 Å) compared with the 3.5 Å r.m.s.d.95 (95% confidence interval = 3.1–4.2 Å) of the best alternative method. Our methods are scalable to very long proteins with accurate domains and domain-packing (see Fig. [1d](/articles/s41586-021-03819-2#Fig1) for the prediction of a 2,180-residue protein with no structural homologues). Finally, the model is able to provide precise, per-residue estimates of its reliability that should enable the confident use of these predictions.

**Fig. 1: AlphaFold produces highly accurate structures.**[![figure 1](//media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_Fig1_HTML.png)](/articles/s41586-021-03819-2/figures/1)**a**, The performance of AlphaFold on the CASP14 dataset (*n* = 87 protein domains) relative to the top-15 entries (out of 146 entries), group numbers correspond to the numbers assigned to entrants by CASP. Data are median and the 95% confidence interval of the median, estimated from 10,000 bootstrap samples. **b**, Our prediction of CASP14 target T1049 (PDB 6Y4F, blue) compared with the true (experimental) structure (green). Four residues in the C terminus of the crystal structure are *B*-factor outliers and are not depicted. **c**, CASP14 target T1056 (PDB 6YJ1). An example of a well-predicted zinc-binding site (AlphaFold has accurate side chains even though it does not explicitly predict the zinc ion). **d**, CASP target T1044 (PDB 6VR4)—a 2,180-residue single chain—was predicted with correct domain packing (the prediction was made after CASP using AlphaFold without intervention). **e**, Model architecture. Arrows show the information flow among the various components described in this paper. Array shapes are shown in parentheses with *s*, number of sequences (*N*seq in the main text); *r*, number of residues (*N*res in the main text); *c*, number of channels.

[Full size image](/articles/s41586-021-03819-2/figures/1)We demonstrate in Fig. [2a](/articles/s41586-021-03819-2#Fig2) that the high accuracy that AlphaFold demonstrated in CASP14 extends to a large sample of recently released PDB structures; in this dataset, all structures were deposited in the PDB after our training data cut-off and are analysed as full chains (see [Methods](/articles/s41586-021-03819-2#Sec10), Supplementary Fig. [15](/articles/s41586-021-03819-2#MOESM1) and Supplementary Table [6](/articles/s41586-021-03819-2#MOESM1) for more details). Furthermore, we observe high side-chain accuracy when the backbone prediction is accurate (Fig. [2b](/articles/s41586-021-03819-2#Fig2)) and we show that our confidence measure, the predicted local-distance difference test (pLDDT), reliably predicts the Cα local-distance difference test (lDDT-Cα) accuracy of the corresponding prediction (Fig. [2c](/articles/s41586-021-03819-2#Fig2)). We also find that the global superposition metric template modelling score (TM-score)[27](/articles/s41586-021-03819-2#ref-CR27 "Zhang, Y. & Skolnick, J. Scoring function for automated assessment of protein structure template quality. Proteins 57, 702–710 (2004).") can be accurately estimated (Fig. [2d](/articles/s41586-021-03819-2#Fig2)). Overall, these analyses validate that the high accuracy and reliability of AlphaFold on CASP14 proteins also transfers to an uncurated collection of recent PDB submissions, as would be expected (see [Supplementary Methods 1.15](/articles/s41586-021-03819-2#MOESM1) and Supplementary Fig. [11](/articles/s41586-021-03819-2#MOESM1) for confirmation that this high accuracy extends to new folds).

**Fig. 2: Accuracy of AlphaFold on recent PDB structures.**[![figure 2](//media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_Fig2_HTML.png)](/articles/s41586-021-03819-2/figures/2)The analysed structures are newer than any structure in the training set. Further filtering is applied to reduce redundancy (see [Methods](/articles/s41586-021-03819-2#Sec10)). **a**, Histogram of backbone r.m.s.d. for full chains (Cα r.m.s.d. at 95% coverage). Error bars are 95% confidence intervals (Poisson). This dataset excludes proteins with a template (identified by hmmsearch) from the training set with more than 40% sequence identity covering more than 1% of the chain (*n* = 3,144 protein chains). The overall median is 1.46 Å (95% confidence interval = 1.40–1.56 Å). Note that this measure will be highly sensitive to domain packing and domain accuracy; a high r.m.s.d. is expected for some chains with uncertain packing or packing errors. **b**, Correlation between backbone accuracy and side-chain accuracy. Filtered to structures with any observed side chains and resolution better than 2.5 Å (*n* = 5,317 protein chains); side chains were further filtered to *B*-factor <30 Å2. A rotamer is classified as correct if the predicted torsion angle is within 40°. Each point aggregates a range of lDDT-Cα, with a bin size of 2 units above 70 lDDT-Cα and 5 units otherwise. Points correspond to the mean accuracy; error bars are 95% confidence intervals (Student *t*-test) of the mean on a per-residue basis. **c**, Confidence score compared to the true accuracy on chains. Least-squares linear fit lDDT-Cα = 0.997 × pLDDT − 1.17 (Pearson’s *r* = 0.76). *n* = 10,795 protein chains. The shaded region of the linear fit represents a 95% confidence interval estimated from 10,000 bootstrap samples. In the companion paper[39](/articles/s41586-021-03819-2#ref-CR39 "Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. Nature
                  https://doi.org/10.1038/s41586-021-03828-1

                 (2021)."), additional quantification of the reliability of pLDDT as a confidence measure is provided. **d**, Correlation between pTM and full chain TM-score. Least-squares linear fit TM-score = 0.98 × pTM + 0.07 (Pearson’s *r* = 0.85). *n* = 10,795 protein chains. The shaded region of the linear fit represents a 95% confidence interval estimated from 10,000 bootstrap samples.

[Full size image](/articles/s41586-021-03819-2/figures/2)The AlphaFold network
---------------------

AlphaFold greatly improves the accuracy of structure prediction by incorporating novel neural network architectures and training procedures based on the evolutionary, physical and geometric constraints of protein structures. In particular, we demonstrate a new architecture to jointly embed multiple sequence alignments (MSAs) and pairwise features, a new output representation and associated loss that enable accurate end-to-end structure prediction, a new equivariant attention architecture, use of intermediate losses to achieve iterative refinement of predictions, masked MSA loss to jointly train with the structure, learning from unlabelled protein sequences using self-distillation and self-estimates of accuracy.

The AlphaFold network directly predicts the 3D coordinates of all heavy atoms for a given protein using the primary amino acid sequence and aligned sequences of homologues as inputs (Fig. [1e](/articles/s41586-021-03819-2#Fig1); see [Methods](/articles/s41586-021-03819-2#Sec10) for details of inputs including databases, MSA construction and use of templates). A description of the most important ideas and components is provided below. The full network architecture and training procedure are provided in the [Supplementary Methods](/articles/s41586-021-03819-2#MOESM1).

The network comprises two main stages. First, the trunk of the network processes the inputs through repeated layers of a novel neural network block that we term Evoformer to produce an *N*seq × *N*res array (*N*seq, number of sequences; *N*res, number of residues) that represents a processed MSA and an *N*res × *N*res array that represents residue pairs. The MSA representation is initialized with the raw MSA (although see [Supplementary Methods 1.2.7](/articles/s41586-021-03819-2#MOESM1) for details of handling very deep MSAs). The Evoformer blocks contain a number of attention-based and non-attention-based components. We show evidence in ‘Interpreting the neural network’ that a concrete structural hypothesis arises early within the Evoformer blocks and is continuously refined. The key innovations in the Evoformer block are new mechanisms to exchange information within the MSA and pair representations that enable direct reasoning about the spatial and evolutionary relationships.

The trunk of the network is followed by the structure module that introduces an explicit 3D structure in the form of a rotation and translation for each residue of the protein (global rigid body frames). These representations are initialized in a trivial state with all rotations set to the identity and all positions set to the origin, but rapidly develop and refine a highly accurate protein structure with precise atomic details. Key innovations in this section of the network include breaking the chain structure to allow simultaneous local refinement of all parts of the structure, a novel equivariant transformer to allow the network to implicitly reason about the unrepresented side-chain atoms and a loss term that places substantial weight on the orientational correctness of the residues. Both within the structure module and throughout the whole network, we reinforce the notion of iterative refinement by repeatedly applying the final loss to outputs and then feeding the outputs recursively into the same modules. The iterative refinement using the whole network (which we term ‘recycling’ and is related to approaches in computer vision[28](/articles/s41586-021-03819-2#ref-CR28 "Tu, Z. & Bai, X. Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1744–1757 (2010)."),[29](/articles/s41586-021-03819-2#ref-CR29 "Carreira, J., Agrawal, P., Fragkiadaki, K. & Malik, J. Human pose estimation with iterative error feedback. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 4733–4742 (2016).")) contributes markedly to accuracy with minor extra training time (see [Supplementary Methods 1.8](/articles/s41586-021-03819-2#MOESM1) for details).

Evoformer
---------

The key principle of the building block of the network—named Evoformer (Figs. [1](/articles/s41586-021-03819-2#Fig1)e, [3a](/articles/s41586-021-03819-2#Fig3))—is to view the prediction of protein structures as a graph inference problem in 3D space in which the edges of the graph are defined by residues in proximity. The elements of the pair representation encode information about the relation between the residues (Fig. [3b](/articles/s41586-021-03819-2#Fig3)). The columns of the MSA representation encode the individual residues of the input sequence while the rows represent the sequences in which those residues appear. Within this framework, we define a number of update operations that are applied in each block in which the different update operations are applied in series.

**Fig. 3: Architectural details.**[![figure 3](//media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_Fig3_HTML.png)](/articles/s41586-021-03819-2/figures/3)**a**, Evoformer block. Arrows show the information flow. The shape of the arrays is shown in parentheses. **b**, The pair representation interpreted as directed edges in a graph. **c**, Triangle multiplicative update and triangle self-attention. The circles represent residues. Entries in the pair representation are illustrated as directed edges and in each diagram, the edge being updated is *ij*. **d**, Structure module including Invariant point attention (IPA) module. The single representation is a copy of the first row of the MSA representation. **e**, Residue gas: a representation of each residue as one free-floating rigid body for the backbone (blue triangles) and *χ* angles for the side chains (green circles). The corresponding atomic structure is shown below. **f**, Frame aligned point error (FAPE). Green, predicted structure; grey, true structure; (*R**k*, **t***k*), frames; **x**i, atom positions.

[Full size image](/articles/s41586-021-03819-2/figures/3)The MSA representation updates the pair representation through an element-wise outer product that is summed over the MSA sequence dimension. In contrast to previous work[30](/articles/s41586-021-03819-2#ref-CR30 "Mirabello, C. & Wallner, B. rawMSA: end-to-end deep learning using raw multiple sequence alignments. PLoS ONE 14, e0220182 (2019)."), this operation is applied within every block rather than once in the network, which enables the continuous communication from the evolving MSA representation to the pair representation.

Within the pair representation, there are two different update patterns. Both are inspired by the necessity of consistency of the pair representation—for a pairwise description of amino acids to be representable as a single 3D structure, many constraints must be satisfied including the triangle inequality on distances. On the basis of this intuition, we arrange the update operations on the pair representation in terms of triangles of edges involving three different nodes (Fig. [3c](/articles/s41586-021-03819-2#Fig3)). In particular, we add an extra logit bias to axial attention[31](/articles/s41586-021-03819-2#ref-CR31 "Huang, Z. et al. CCNet: criss-cross attention for semantic segmentation. In Proc. IEEE/CVF International Conference on Computer Vision 603–612 (2019).") to include the ‘missing edge’ of the triangle and we define a non-attention update operation ‘triangle multiplicative update’ that uses two edges to update the missing third edge (see [Supplementary Methods 1.6.5](/articles/s41586-021-03819-2#MOESM1) for details). The triangle multiplicative update was developed originally as a more symmetric and cheaper replacement for the attention, and networks that use only the attention or multiplicative update are both able to produce high-accuracy structures. However, the combination of the two updates is more accurate.

We also use a variant of axial attention within the MSA representation. During the per-sequence attention in the MSA, we project additional logits from the pair stack to bias the MSA attention. This closes the loop by providing information flow from the pair representation back into the MSA representation, ensuring that the overall Evoformer block is able to fully mix information between the pair and MSA representations and prepare for structure generation within the structure module.

End-to-end structure prediction
-------------------------------

The structure module (Fig. [3d](/articles/s41586-021-03819-2#Fig3)) operates on a concrete 3D backbone structure using the pair representation and the original sequence row (single representation) of the MSA representation from the trunk. The 3D backbone structure is represented as *N*res independent rotations and translations, each with respect to the global frame (residue gas) (Fig. [3e](/articles/s41586-021-03819-2#Fig3)). These rotations and translations—representing the geometry of the N-Cα-C atoms—prioritize the orientation of the protein backbone so that the location of the side chain of each residue is highly constrained within that frame. Conversely, the peptide bond geometry is completely unconstrained and the network is observed to frequently violate the chain constraint during the application of the structure module as breaking this constraint enables the local refinement of all parts of the chain without solving complex loop closure problems. Satisfaction of the peptide bond geometry is encouraged during fine-tuning by a violation loss term. Exact enforcement of peptide bond geometry is only achieved in the post-prediction relaxation of the structure by gradient descent in the Amber[32](/articles/s41586-021-03819-2#ref-CR32 "Hornak, V. et al. Comparison of multiple Amber force fields and development of improved protein backbone parameters. Proteins 65, 712–725 (2006).") force field. Empirically, this final relaxation does not improve the accuracy of the model as measured by the global distance test (GDT)[33](/articles/s41586-021-03819-2#ref-CR33 "Zemla, A. LGA: a method for finding 3D similarities in protein structures. Nucleic Acids Res. 31, 3370–3374 (2003).") or lDDT-Cα[34](/articles/s41586-021-03819-2#ref-CR34 "Mariani, V., Biasini, M., Barbato, A. & Schwede, T. lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests. Bioinformatics 29, 2722–2728 (2013).") but does remove distracting stereochemical violations without the loss of accuracy.

The residue gas representation is updated iteratively in two stages (Fig. [3d](/articles/s41586-021-03819-2#Fig3)). First, a geometry-aware attention operation that we term ‘invariant point attention’ (IPA) is used to update an *N*res set of neural activations (single representation) without changing the 3D positions, then an equivariant update operation is performed on the residue gas using the updated activations. The IPA augments each of the usual attention queries, keys and values with 3D points that are produced in the local frame of each residue such that the final value is invariant to global rotations and translations (see [Methods](/articles/s41586-021-03819-2#Sec10) ‘IPA’ for details). The 3D queries and keys also impose a strong spatial/locality bias on the attention, which is well-suited to the iterative refinement of the protein structure. After each attention operation and element-wise transition block, the module computes an update to the rotation and translation of each backbone frame. The application of these updates within the local frame of each residue makes the overall attention and update block an equivariant operation on the residue gas.

Predictions of side-chain *χ* angles as well as the final, per-residue accuracy of the structure (pLDDT) are computed with small per-residue networks on the final activations at the end of the network. The estimate of the TM-score (pTM) is obtained from a pairwise error prediction that is computed as a linear projection from the final pair representation. The final loss (which we term the frame-aligned point error (FAPE) (Fig. [3f](/articles/s41586-021-03819-2#Fig3))) compares the predicted atom positions to the true positions under many different alignments. For each alignment, defined by aligning the predicted frame (*R**k*, **t***k*) to the corresponding true frame, we compute the distance of all predicted atom positions **x***i* from the true atom positions. The resulting *N*frames × *N*atoms distances are penalized with a clamped *L*1 loss. This creates a strong bias for atoms to be correct relative to the local frame of each residue and hence correct with respect to its side-chain interactions, as well as providing the main source of chirality for AlphaFold ([Supplementary Methods 1.9.3](/articles/s41586-021-03819-2#MOESM1) and Supplementary Fig. [9](/articles/s41586-021-03819-2#MOESM1)).

Training with labelled and unlabelled data
------------------------------------------

The AlphaFold architecture is able to train to high accuracy using only supervised learning on PDB data, but we are able to enhance accuracy (Fig. [4a](/articles/s41586-021-03819-2#Fig4)) using an approach similar to noisy student self-distillation[35](/articles/s41586-021-03819-2#ref-CR35 "Xie, Q., Luong, M.-T., Hovy, E. & Le, Q. V. Self-training with noisy student improves imagenet classification. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 10687–10698 (2020)."). In this procedure, we use a trained network to predict the structure of around 350,000 diverse sequences from Uniclust30[36](/articles/s41586-021-03819-2#ref-CR36 "Mirdita, M. et al. Uniclust databases of clustered and deeply annotated protein sequences and alignments. Nucleic Acids Res. 45, D170–D176 (2017).") and make a new dataset of predicted structures filtered to a high-confidence subset. We then train the same architecture again from scratch using a mixture of PDB data and this new dataset of predicted structures as the training data, in which the various training data augmentations such as cropping and MSA subsampling make it challenging for the network to recapitulate the previously predicted structures. This self-distillation procedure makes effective use of the unlabelled sequence data and considerably improves the accuracy of the resulting network.

**Fig. 4: Interpreting the neural network.**[![figure 4](//media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_Fig4_HTML.png)](/articles/s41586-021-03819-2/figures/4)**a**, Ablation results on two target sets: the CASP14 set of domains (*n* = 87 protein domains) and the PDB test set of chains with template coverage of ≤30% at 30% identity (*n* = 2,261 protein chains). Domains are scored with GDT and chains are scored with lDDT-Cα. The ablations are reported as a difference compared with the average of the three baseline seeds. Means (points) and 95% bootstrap percentile intervals (error bars) are computed using bootstrap estimates of 10,000 samples. **b**, Domain GDT trajectory over 4 recycling iterations and 48 Evoformer blocks on CASP14 targets LmrP (T1024) and Orf8 (T1064) where D1 and D2 refer to the individual domains as defined by the CASP assessment. Both T1024 domains obtain the correct structure early in the network, whereas the structure of T1064 changes multiple times and requires nearly the full depth of the network to reach the final structure. Note, 48 Evoformer blocks comprise one recycling iteration.

[Full size image](/articles/s41586-021-03819-2/figures/4)Additionally, we randomly mask out or mutate individual residues within the MSA and have a Bidirectional Encoder Representations from Transformers (BERT)-style[37](/articles/s41586-021-03819-2#ref-CR37 "Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1, 4171–4186 (2019).") objective to predict the masked elements of the MSA sequences. This objective encourages the network to learn to interpret phylogenetic and covariation relationships without hardcoding a particular correlation statistic into the features. The BERT objective is trained jointly with the normal PDB structure loss on the same training examples and is not pre-trained, in contrast to recent independent work[38](/articles/s41586-021-03819-2#ref-CR38 "Rao, R. et al. MSA transformer. In Proc. 38th International Conference on Machine Learning PMLR 139, 8844–8856 (2021).").

Interpreting the neural network
-------------------------------

To understand how AlphaFold predicts protein structure, we trained a separate structure module for each of the 48 Evoformer blocks in the network while keeping all parameters of the main network frozen ([Supplementary Methods 1.14](/articles/s41586-021-03819-2#MOESM1)). Including our recycling stages, this provides a trajectory of 192 intermediate structures—one per full Evoformer block—in which each intermediate represents the belief of the network of the most likely structure at that block. The resulting trajectories are surprisingly smooth after the first few blocks, showing that AlphaFold makes constant incremental improvements to the structure until it can no longer improve (see Fig. [4b](/articles/s41586-021-03819-2#Fig4) for a trajectory of accuracy). These trajectories also illustrate the role of network depth. For very challenging proteins such as ORF8 of SARS-CoV-2 (T1064), the network searches and rearranges secondary structure elements for many layers before settling on a good structure. For other proteins such as LmrP (T1024), the network finds the final structure within the first few layers. Structure trajectories of CASP14 targets T1024, T1044, T1064 and T1091 that demonstrate a clear iterative building process for a range of protein sizes and difficulties are shown in Supplementary Videos [1](/articles/s41586-021-03819-2#MOESM3)–[4](/articles/s41586-021-03819-2#MOESM6). In [Supplementary Methods 1.16](/articles/s41586-021-03819-2#MOESM1) and Supplementary Figs. [12](/articles/s41586-021-03819-2#MOESM1), [13](/articles/s41586-021-03819-2#MOESM1), we interpret the attention maps produced by AlphaFold layers.

Figure [4a](/articles/s41586-021-03819-2#Fig4) contains detailed ablations of the components of AlphaFold that demonstrate that a variety of different mechanisms contribute to AlphaFold accuracy. Detailed descriptions of each ablation model, their training details, extended discussion of ablation results and the effect of MSA depth on each ablation are provided in [Supplementary Methods 1.13](/articles/s41586-021-03819-2#MOESM1) and Supplementary Fig. [10](/articles/s41586-021-03819-2#MOESM1).

MSA depth and cross-chain contacts
----------------------------------

Although AlphaFold has a high accuracy across the vast majority of deposited PDB structures, we note that there are still factors that affect accuracy or limit the applicability of the model. The model uses MSAs and the accuracy decreases substantially when the median alignment depth is less than around 30 sequences (see Fig. [5a](/articles/s41586-021-03819-2#Fig5) for details). We observe a threshold effect where improvements in MSA depth over around 100 sequences lead to small gains. We hypothesize that the MSA information is needed to coarsely find the correct structure within the early stages of the network, but refinement of that prediction into a high-accuracy model does not depend crucially on the MSA information. The other substantial limitation that we have observed is that AlphaFold is much weaker for proteins that have few intra-chain or homotypic contacts compared to the number of heterotypic contacts (further details are provided in a companion paper[39](/articles/s41586-021-03819-2#ref-CR39 "Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. Nature
                  https://doi.org/10.1038/s41586-021-03828-1

                 (2021).")). This typically occurs for bridging domains within larger complexes in which the shape of the protein is created almost entirely by interactions with other chains in the complex. Conversely, AlphaFold is often able to give high-accuracy predictions for homomers, even when the chains are substantially intertwined (Fig. [5b](/articles/s41586-021-03819-2#Fig5)). We expect that the ideas of AlphaFold are readily applicable to predicting full hetero-complexes in a future system and that this will remove the difficulty with protein chains that have a large number of hetero-contacts.

**Fig. 5: Effect of MSA depth and cross-chain contacts.**[![figure 5](//media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_Fig5_HTML.png)](/articles/s41586-021-03819-2/figures/5)**a**, Backbone accuracy (lDDT-Cα) for the redundancy-reduced set of the PDB after our training data cut-off, restricting to proteins in which at most 25% of the long-range contacts are between different heteromer chains. We further consider two groups of proteins based on template coverage at 30% sequence identity: covering more than 60% of the chain (*n* = 6,743 protein chains) and covering less than 30% of the chain (*n* = 1,596 protein chains). MSA depth is computed by counting the number of non-gap residues for each position in the MSA (using the *N*eff weighting scheme; see [Methods](/articles/s41586-021-03819-2#Sec10) for details) and taking the median across residues. The curves are obtained through Gaussian kernel average smoothing (window size is 0.2 units in log10(*N*eff)); the shaded area is the 95% confidence interval estimated using bootstrap of 10,000 samples. **b**, An intertwined homotrimer (PDB 6SK0) is correctly predicted without input stoichiometry and only a weak template (blue is predicted and green is experimental).

[Full size image](/articles/s41586-021-03819-2/figures/5)Related work
------------

The prediction of protein structures has had a long and varied development, which is extensively covered in a number of reviews[14](/articles/s41586-021-03819-2#ref-CR14 "Pearce, R. & Zhang, Y. Deep learning techniques have significantly impacted protein structure prediction and protein design. Curr. Opin. Struct. Biol. 68, 194–207 (2021)."),[40](#ref-CR40 "Kuhlman, B. & Bradley, P. Advances in protein structure prediction and design. Nat. Rev. Mol. Cell Biol. 20, 681–697 (2019)."),[41](#ref-CR41 "Marks, D. S., Hopf, T. A. & Sander, C. Protein structure prediction from sequence variation. Nat. Biotechnol. 30, 1072–1080 (2012)."),[42](#ref-CR42 "Qian, N. & Sejnowski, T. J. Predicting the secondary structure of globular proteins using neural network models. J. Mol. Biol. 202, 865–884 (1988)."),[43](/articles/s41586-021-03819-2#ref-CR43 "Fariselli, P., Olmea, O., Valencia, A. & Casadio, R. Prediction of contact maps with neural networks and correlated mutations. Protein Eng. 14, 835–843 (2001)."). Despite the long history of applying neural networks to structure prediction[14](/articles/s41586-021-03819-2#ref-CR14 "Pearce, R. & Zhang, Y. Deep learning techniques have significantly impacted protein structure prediction and protein design. Curr. Opin. Struct. Biol. 68, 194–207 (2021)."),[42](/articles/s41586-021-03819-2#ref-CR42 "Qian, N. & Sejnowski, T. J. Predicting the secondary structure of globular proteins using neural network models. J. Mol. Biol. 202, 865–884 (1988)."),[43](/articles/s41586-021-03819-2#ref-CR43 "Fariselli, P., Olmea, O., Valencia, A. & Casadio, R. Prediction of contact maps with neural networks and correlated mutations. Protein Eng. 14, 835–843 (2001)."), they have only recently come to improve structure prediction[10](/articles/s41586-021-03819-2#ref-CR10 "Senior, A. W. et al. Improved protein structure prediction using potentials from deep learning. Nature 577, 706–710 (2020)."),[11](/articles/s41586-021-03819-2#ref-CR11 "Wang, S., Sun, S., Li, Z., Zhang, R. & Xu, J. Accurate de novo prediction of protein contact map by ultra-deep learning model. PLOS Comput. Biol. 13, e1005324 (2017)."),[44](/articles/s41586-021-03819-2#ref-CR44 "Yang, J. et al. Improved protein structure prediction using predicted interresidue orientations. Proc. Natl Acad. Sci. USA 117, 1496–1503 (2020)."),[45](/articles/s41586-021-03819-2#ref-CR45 "Li, Y. et al. Deducing high-accuracy protein contact-maps from a triplet of coevolutionary matrices through deep residual convolutional networks. PLOS Comput. Biol. 17, e1008865 (2021)."). These approaches effectively leverage the rapid improvement in computer vision systems[46](/articles/s41586-021-03819-2#ref-CR46 "He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (2016).") by treating the problem of protein structure prediction as converting an ‘image’ of evolutionary couplings[22](#ref-CR22 "Weigt, M., White, R. A., Szurmant, H., Hoch, J. A. & Hwa, T. Identification of direct residue contacts in protein–protein interaction by message passing. Proc. Natl Acad. Sci. USA 106, 67–72 (2009)."),[23](#ref-CR23 "Marks, D. S. et al. Protein 3D structure computed from evolutionary sequence variation. PLoS ONE 6, e28766 (2011)."),[24](/articles/s41586-021-03819-2#ref-CR24 "Jones, D. T., Buchan, D. W. A., Cozzetto, D. & Pontil, M. PSICOV: precise structural contact prediction using sparse inverse covariance estimation on large multiple sequence alignments. Bioinformatics 28, 184–190 (2012).") to an ‘image’ of the protein distance matrix and then integrating the distance predictions into a heuristic system that produces the final 3D coordinate prediction. A few recent studies have been developed to predict the 3D coordinates directly[47](#ref-CR47 "AlQuraishi, M. End-to-end differentiable learning of protein structure. Cell Syst. 8, 292–301 (2019)."),[48](#ref-CR48 "Senior, A. W. et al. Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13). Proteins 87, 1141–1148 (2019)."),[49](#ref-CR49 "Ingraham, J., Riesselman, A. J., Sander, C. & Marks, D. S. Learning protein structure with a differentiable simulator. in Proc. International Conference on Learning Representations (2019)."),[50](/articles/s41586-021-03819-2#ref-CR50 "Li, J. Universal transforming geometric network. Preprint at
                  https://arxiv.org/abs/1908.00723

                 (2019)."), but the accuracy of these approaches does not match traditional, hand-crafted structure prediction pipelines[51](/articles/s41586-021-03819-2#ref-CR51 "Xu, J., McPartlon, M. & Li, J. Improved protein structure prediction by deep learning irrespective of co-evolution information. Nat. Mach. Intell. 3, 601–609 (2021)."). In parallel, the success of attention-based networks for language processing[52](/articles/s41586-021-03819-2#ref-CR52 "Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems 5998–6008 (2017).") and, more recently, computer vision[31](/articles/s41586-021-03819-2#ref-CR31 "Huang, Z. et al. CCNet: criss-cross attention for semantic segmentation. In Proc. IEEE/CVF International Conference on Computer Vision 603–612 (2019)."),[53](/articles/s41586-021-03819-2#ref-CR53 "Wang, H. et al. Axial-deeplab: stand-alone axial-attention for panoptic segmentation. in European Conference on Computer Vision 108–126 (Springer, 2020).") has inspired the exploration of attention-based methods for interpreting protein sequences[54](#ref-CR54 "Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M. & Church, G. M. Unified rational protein engineering with sequence-based deep representation learning. Nat. Methods 16, 1315–1322 (2019)."),[55](#ref-CR55 "Heinzinger, M. et al. Modeling aspects of the language of life through transfer-learning protein sequences. BMC Bioinformatics 20, 723 (2019)."),[56](/articles/s41586-021-03819-2#ref-CR56 "Rives, A. et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proc. Natl Acad. Sci. USA 118, e2016239118 (2021).").

Discussion
----------

The methodology that we have taken in designing AlphaFold is a combination of the bioinformatics and physical approaches: we use a physical and geometric inductive bias to build components that learn from PDB data with minimal imposition of handcrafted features (for example, AlphaFold builds hydrogen bonds effectively without a hydrogen bond score function). This results in a network that learns far more efficiently from the limited data in the PDB but is able to cope with the complexity and variety of structural data.

In particular, AlphaFold is able to handle missing the physical context and produce accurate models in challenging cases such as intertwined homomers or proteins that only fold in the presence of an unknown haem group. The ability to handle underspecified structural conditions is essential to learning from PDB structures as the PDB represents the full range of conditions in which structures have been solved. In general, AlphaFold is trained to produce the protein structure most likely to appear as part of a PDB structure. For example, in cases in which a particular stochiometry, ligand or ion is predictable from the sequence alone, AlphaFold is likely to produce a structure that respects those constraints implicitly.

AlphaFold has already demonstrated its utility to the experimental community, both for molecular replacement[57](/articles/s41586-021-03819-2#ref-CR57 "Pereira, J. et al. High-accuracy protein structure prediction in CASP14. Proteins
                  https://doi.org/10.1002/prot.26171

                 (2021).") and for interpreting cryogenic electron microscopy maps[58](/articles/s41586-021-03819-2#ref-CR58 "Gupta, M. et al. CryoEM and AI reveal a structure of SARS-CoV-2 Nsp2, a multifunctional protein involved in key host processes. Preprint at
                  https://doi.org/10.1101/2021.05.10.443524

                 (2021)."). Moreover, because AlphaFold outputs protein coordinates directly, AlphaFold produces predictions in graphics processing unit (GPU) minutes to GPU hours depending on the length of the protein sequence (for example, around one GPU minute per model for 384 residues; see [Methods](/articles/s41586-021-03819-2#Sec10) for details). This opens up the exciting possibility of predicting structures at the proteome-scale and beyond—in a companion paper[39](/articles/s41586-021-03819-2#ref-CR39 "Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. Nature
                  https://doi.org/10.1038/s41586-021-03828-1

                 (2021)."), we demonstrate the application of AlphaFold to the entire human proteome[39](/articles/s41586-021-03819-2#ref-CR39 "Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. Nature
                  https://doi.org/10.1038/s41586-021-03828-1

                 (2021).").

The explosion in available genomic sequencing techniques and data has revolutionized bioinformatics but the intrinsic challenge of experimental structure determination has prevented a similar expansion in our structural knowledge. By developing an accurate protein structure prediction algorithm, coupled with existing large and well-curated structure and sequence databases assembled by the experimental community, we hope to accelerate the advancement of structural bioinformatics that can keep pace with the genomics revolution. We hope that AlphaFold—and computational approaches that apply its techniques for other biophysical problems—will become essential tools of modern biology.

Methods
-------

### Full algorithm details

Extensive explanations of the components and their motivations are available in [Supplementary Methods 1.1–1.10](/articles/s41586-021-03819-2#MOESM1), in addition, pseudocode is available in [Supplementary Information Algorithms 1–32](/articles/s41586-021-03819-2#MOESM1), network diagrams in Supplementary Figs. [1](/articles/s41586-021-03819-2#MOESM1)–[8](/articles/s41586-021-03819-2#MOESM1), input features in Supplementary Table [1](/articles/s41586-021-03819-2#MOESM1) and additional details are provided in Supplementary Tables [2](/articles/s41586-021-03819-2#MOESM1), [3](/articles/s41586-021-03819-2#MOESM1). Training and inference details are provided in [Supplementary Methods 1.11–1.12](/articles/s41586-021-03819-2#MOESM1) and Supplementary Tables [4](/articles/s41586-021-03819-2#MOESM1), [5](/articles/s41586-021-03819-2#MOESM1).

### IPA

The IPA module combines the pair representation, the single representation and the geometric representation to update the single representation (Supplementary Fig. [8](/articles/s41586-021-03819-2#MOESM1)). Each of these representations contributes affinities to the shared attention weights and then uses these weights to map its values to the output. The IPA operates in 3D space. Each residue produces query points, key points and value points in its local frame. These points are projected into the global frame using the backbone frame of the residue in which they interact with each other. The resulting points are then projected back into the local frame. The affinity computation in the 3D space uses squared distances and the coordinate transformations ensure the invariance of this module with respect to the global frame (see [Supplementary Methods 1.8.2](/articles/s41586-021-03819-2#MOESM1) ‘Invariant point attention (IPA)’ for the algorithm, proof of invariance and a description of the full multi-head version). A related construction that uses classic geometric invariants to construct pairwise features in place of the learned 3D points has been applied to protein design[59](/articles/s41586-021-03819-2#ref-CR59 "Ingraham, J., Garg, V. K., Barzilay, R. & Jaakkola, T. Generative models for graph-based protein design. in Proc. 33rd Conference on Neural Information Processing Systems (2019).").

In addition to the IPA, standard dot product attention is computed on the abstract single representation and a special attention on the pair representation. The pair representation augments both the logits and the values of the attention process, which is the primary way in which the pair representation controls the structure generation.

### Inputs and data sources

Inputs to the network are the primary sequence, sequences from evolutionarily related proteins in the form of a MSA created by standard tools including jackhmmer[60](/articles/s41586-021-03819-2#ref-CR60 "Johnson, L. S., Eddy, S. R. & Portugaly, E. Hidden Markov model speed heuristic and iterative HMM search procedure. BMC Bioinformatics 11, 431 (2010).") and HHBlits[61](/articles/s41586-021-03819-2#ref-CR61 "Remmert, M., Biegert, A., Hauser, A. & Söding, J. HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment. Nat. Methods 9, 173–175 (2012)."), and 3D atom coordinates of a small number of homologous structures (templates) where available. For both the MSA and templates, the search processes are tuned for high recall; spurious matches will probably appear in the raw MSA but this matches the training condition of the network.

One of the sequence databases used, Big Fantastic Database (BFD), was custom-made and released publicly (see ‘Data availability’) and was used by several CASP teams. BFD is one of the largest publicly available collections of protein families. It consists of 65,983,866 families represented as MSAs and hidden Markov models (HMMs) covering 2,204,359,010 protein sequences from reference databases, metagenomes and metatranscriptomes.

BFD was built in three steps. First, 2,423,213,294 protein sequences were collected from UniProt (Swiss-Prot&TrEMBL, 2017-11)[62](/articles/s41586-021-03819-2#ref-CR62 "The UniProt Consortium. UniProt: the universal protein knowledgebase in 2021. Nucleic Acids Res. 49, D480–D489 (2020)."), a soil reference protein catalogue and the marine eukaryotic reference catalogue[7](/articles/s41586-021-03819-2#ref-CR7 "Steinegger, M., Mirdita, M. & Söding, J. Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold. Nat. Methods 16, 603–606 (2019)."), and clustered to 30% sequence identity, while enforcing a 90% alignment coverage of the shorter sequences using MMseqs2/Linclust[63](/articles/s41586-021-03819-2#ref-CR63 "Steinegger, M. & Söding, J. Clustering huge protein sequence sets in linear time. Nat. Commun. 9, 2542 (2018)."). This resulted in 345,159,030 clusters. For computational efficiency, we removed all clusters with less than three members, resulting in 61,083,719 clusters. Second, we added 166,510,624 representative protein sequences from Metaclust NR (2017-05; discarding all sequences shorter than 150 residues)[63](/articles/s41586-021-03819-2#ref-CR63 "Steinegger, M. & Söding, J. Clustering huge protein sequence sets in linear time. Nat. Commun. 9, 2542 (2018).") by aligning them against the cluster representatives using MMseqs2[64](/articles/s41586-021-03819-2#ref-CR64 "Steinegger, M. & Söding, J. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nat. Biotechnol. 35, 1026–1028 (2017)."). Sequences that fulfilled the sequence identity and coverage criteria were assigned to the best scoring cluster. The remaining 25,347,429 sequences that could not be assigned were clustered separately and added as new clusters, resulting in the final clustering. Third, for each of the clusters, we computed an MSA using FAMSA[65](/articles/s41586-021-03819-2#ref-CR65 "Deorowicz, S., Debudaj-Grabysz, A. & Gudyś, A. FAMSA: fast and accurate multiple sequence alignment of huge protein families. Sci. Rep. 6, 33964 (2016).") and computed the HMMs following the Uniclust HH-suite database protocol[36](/articles/s41586-021-03819-2#ref-CR36 "Mirdita, M. et al. Uniclust databases of clustered and deeply annotated protein sequences and alignments. Nucleic Acids Res. 45, D170–D176 (2017).").

The following versions of public datasets were used in this study. Our models were trained on a copy of the PDB[5](/articles/s41586-021-03819-2#ref-CR5 "wwPDB Consortium. Protein Data Bank: the single global archive for 3D macromolecular structure data. Nucleic Acids Res. 47, D520–D528 (2018).") downloaded on 28 August 2019. For finding template structures at prediction time, we used a copy of the PDB downloaded on 14 May 2020, and the PDB70[66](/articles/s41586-021-03819-2#ref-CR66 "Steinegger, M. et al. HH-suite3 for fast remote homology detection and deep protein annotation. BMC Bioinformatics 20, 473 (2019).") clustering database downloaded on 13 May 2020. For MSA lookup at both training and prediction time, we used Uniref90[67](/articles/s41586-021-03819-2#ref-CR67 "Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B. & Wu, C. H. UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches. Bioinformatics 31, 926–932 (2015).") v.2020\_01, BFD, Uniclust30[36](/articles/s41586-021-03819-2#ref-CR36 "Mirdita, M. et al. Uniclust databases of clustered and deeply annotated protein sequences and alignments. Nucleic Acids Res. 45, D170–D176 (2017).") v.2018\_08 and MGnify[6](/articles/s41586-021-03819-2#ref-CR6 "Mitchell, A. L. et al. MGnify: the microbiome analysis resource in 2020. Nucleic Acids Res. 48, D570–D578 (2020).") v.2018\_12. For sequence distillation, we used Uniclust30[36](/articles/s41586-021-03819-2#ref-CR36 "Mirdita, M. et al. Uniclust databases of clustered and deeply annotated protein sequences and alignments. Nucleic Acids Res. 45, D170–D176 (2017).") v.2018\_08 to construct a distillation structure dataset. Full details are provided in [Supplementary Methods 1.2](/articles/s41586-021-03819-2#MOESM1).

For MSA search on BFD + Uniclust30, and template search against PDB70, we used HHBlits[61](/articles/s41586-021-03819-2#ref-CR61 "Remmert, M., Biegert, A., Hauser, A. & Söding, J. HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment. Nat. Methods 9, 173–175 (2012).") and HHSearch[66](/articles/s41586-021-03819-2#ref-CR66 "Steinegger, M. et al. HH-suite3 for fast remote homology detection and deep protein annotation. BMC Bioinformatics 20, 473 (2019).") from hh-suite v.3.0-beta.3 (version 14/07/2017). For MSA search on Uniref90 and clustered MGnify, we used jackhmmer from HMMER3[68](/articles/s41586-021-03819-2#ref-CR68 "Eddy, S. R. Accelerated profile HMM searches. PLOS Comput. Biol. 7, e1002195 (2011)."). For constrained relaxation of structures, we used OpenMM v.7.3.1[69](/articles/s41586-021-03819-2#ref-CR69 "Eastman, P. et al. OpenMM 7: rapid development of high performance algorithms for molecular dynamics. PLOS Comput. Biol. 13, e1005659 (2017).") with the Amber99sb force field[32](/articles/s41586-021-03819-2#ref-CR32 "Hornak, V. et al. Comparison of multiple Amber force fields and development of improved protein backbone parameters. Proteins 65, 712–725 (2006)."). For neural network construction, running and other analyses, we used TensorFlow[70](/articles/s41586-021-03819-2#ref-CR70 "Ashish, A. M. A. et al. TensorFlow: large-scale machine learning on heterogeneous systems. Preprint at
                  https://arxiv.org/abs/1603.04467

                 (2015)."), Sonnet[71](/articles/s41586-021-03819-2#ref-CR71 "Reynolds, M. et al. Open sourcing Sonnet – a new library for constructing neural networks. DeepMind
                  https://deepmind.com/blog/open-sourcing-sonnet/

                 (7 April 2017)."), NumPy[72](/articles/s41586-021-03819-2#ref-CR72 "Harris, C. R. et al. Array programming with NumPy. Nature 585, 357–362 (2020)."), Python[73](/articles/s41586-021-03819-2#ref-CR73 "Van Rossum, G. & Drake, F. L. Python 3 Reference Manual (CreateSpace, 2009).") and Colab[74](/articles/s41586-021-03819-2#ref-CR74 "Bisong, E. in Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners 59–64 (Apress, 2019).").

To quantify the effect of the different sequence data sources, we re-ran the CASP14 proteins using the same models but varying how the MSA was constructed. Removing BFD reduced the mean accuracy by 0.4 GDT, removing Mgnify reduced the mean accuracy by 0.7 GDT, and removing both reduced the mean accuracy by 6.1 GDT. In each case, we found that most targets had very small changes in accuracy but a few outliers had very large (20+ GDT) differences. This is consistent with the results in Fig. [5a](/articles/s41586-021-03819-2#Fig5) in which the depth of the MSA is relatively unimportant until it approaches a threshold value of around 30 sequences when the MSA size effects become quite large. We observe mostly overlapping effects between inclusion of BFD and Mgnify, but having at least one of these metagenomics databases is very important for target classes that are poorly represented in UniRef, and having both was necessary to achieve full CASP accuracy.

### Training regimen

To train, we use structures from the PDB with a maximum release date of 30 April 2018. Chains are sampled in inverse proportion to cluster size of a 40% sequence identity clustering. We then randomly crop them to 256 residues and assemble into batches of size 128. We train the model on Tensor Processing Unit (TPU) v3 with a batch size of 1 per TPU core, hence the model uses 128 TPU v3 cores. The model is trained until convergence (around 10 million samples) and further fine-tuned using longer crops of 384 residues, larger MSA stack and reduced learning rate (see [Supplementary Methods 1.11](/articles/s41586-021-03819-2#MOESM1) for the exact configuration). The initial training stage takes approximately 1 week, and the fine-tuning stage takes approximately 4 additional days.

The network is supervised by the FAPE loss and a number of auxiliary losses. First, the final pair representation is linearly projected to a binned distance distribution (distogram) prediction, scored with a cross-entropy loss. Second, we use random masking on the input MSAs and require the network to reconstruct the masked regions from the output MSA representation using a BERT-like loss[37](/articles/s41586-021-03819-2#ref-CR37 "Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1, 4171–4186 (2019)."). Third, the output single representations of the structure module are used to predict binned per-residue lDDT-Cα values. Finally, we use an auxiliary side-chain loss during training, and an auxiliary structure violation loss during fine-tuning. Detailed descriptions and weighting are provided in the [Supplementary Information](/articles/s41586-021-03819-2#MOESM1).

An initial model trained with the above objectives was used to make structure predictions for a Uniclust dataset of 355,993 sequences with the full MSAs. These predictions were then used to train a final model with identical hyperparameters, except for sampling examples 75% of the time from the Uniclust prediction set, with sub-sampled MSAs, and 25% of the time from the clustered PDB set.

We train five different models using different random seeds, some with templates and some without, to encourage diversity in the predictions (see Supplementary Table [5](/articles/s41586-021-03819-2#MOESM1) and [Supplementary Methods 1.12.1](/articles/s41586-021-03819-2#MOESM1) for details). We also fine-tuned these models after CASP14 to add a pTM prediction objective ([Supplementary Methods 1.9.7](/articles/s41586-021-03819-2#MOESM1)) and use the obtained models for Fig. [2d](/articles/s41586-021-03819-2#Fig2).

### Inference regimen

We inference the five trained models and use the predicted confidence score to select the best model per target.

Using our CASP14 configuration for AlphaFold, the trunk of the network is run multiple times with different random choices for the MSA cluster centres (see [Supplementary Methods 1.11.2](/articles/s41586-021-03819-2#MOESM1) for details of the ensembling procedure). The full time to make a structure prediction varies considerably depending on the length of the protein. Representative timings for the neural network using a single model on V100 GPU are 4.8 min with 256 residues, 9.2 min with 384 residues and 18 h at 2,500 residues. These timings are measured using our open-source code, and the open-source code is notably faster than the version we ran in CASP14 as we now use the XLA compiler[75](/articles/s41586-021-03819-2#ref-CR75 "TensorFlow. XLA: Optimizing Compiler for TensorFlow.
                  https://www.tensorflow.org/xla

                 (2018).").

Since CASP14, we have found that the accuracy of the network without ensembling is very close or equal to the accuracy with ensembling and we turn off ensembling for most inference. Without ensembling, the network is 8× faster and the representative timings for a single model are 0.6 min with 256 residues, 1.1 min with 384 residues and 2.1 h with 2,500 residues.

Inferencing large proteins can easily exceed the memory of a single GPU. For a V100 with 16 GB of memory, we can predict the structure of proteins up to around 1,300 residues without ensembling and the 256- and 384-residue inference times are using the memory of a single GPU. The memory usage is approximately quadratic in the number of residues, so a 2,500-residue protein involves using unified memory so that we can greatly exceed the memory of a single V100. In our cloud setup, a single V100 is used for computation on a 2,500-residue protein but we requested four GPUs to have sufficient memory.

Searching genetic sequence databases to prepare inputs and final relaxation of the structures take additional central processing unit (CPU) time but do not require a GPU or TPU.

### Metrics

The predicted structure is compared to the true structure from the PDB in terms of lDDT metric[34](/articles/s41586-021-03819-2#ref-CR34 "Mariani, V., Biasini, M., Barbato, A. & Schwede, T. lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests. Bioinformatics 29, 2722–2728 (2013)."), as this metric reports the domain accuracy without requiring a domain segmentation of chain structures. The distances are either computed between all heavy atoms (lDDT) or only the Cα atoms to measure the backbone accuracy (lDDT-Cα). As lDDT-Cα only focuses on the Cα atoms, it does not include the penalty for structural violations and clashes. Domain accuracies in CASP are reported as GDT[33](/articles/s41586-021-03819-2#ref-CR33 "Zemla, A. LGA: a method for finding 3D similarities in protein structures. Nucleic Acids Res. 31, 3370–3374 (2003).") and the TM-score[27](/articles/s41586-021-03819-2#ref-CR27 "Zhang, Y. & Skolnick, J. Scoring function for automated assessment of protein structure template quality. Proteins 57, 702–710 (2004).") is used as a full chain global superposition metric.

We also report accuracies using the r.m.s.d.95 (Cα r.m.s.d. at 95% coverage). We perform five iterations of (1) a least-squares alignment of the predicted structure and the PDB structure on the currently chosen Cα atoms (using all Cα atoms in the first iteration); (2) selecting the 95% of Cα atoms with the lowest alignment error. The r.m.s.d. of the atoms chosen for the final iterations is the r.m.s.d.95. This metric is more robust to apparent errors that can originate from crystal structure artefacts, although in some cases the removed 5% of residues will contain genuine modelling errors.

### Test set of recent PDB sequences

For evaluation on recent PDB sequences (Figs. [2](/articles/s41586-021-03819-2#Fig2)a–d, [4](/articles/s41586-021-03819-2#Fig4)a, [5a](/articles/s41586-021-03819-2#Fig5)), we used a copy of the PDB downloaded 15 February 2021. Structures were filtered to those with a release date after 30 April 2018 (the date limit for inclusion in the training set for AlphaFold). Chains were further filtered to remove sequences that consisted of a single amino acid as well as sequences with an ambiguous chemical component at any residue position. Exact duplicates were removed, with the chain with the most resolved Cα atoms used as the representative sequence. Subsequently, structures with less than 16 resolved residues, with unknown residues or solved by NMR methods were removed. As the PDB contains many near-duplicate sequences, the chain with the highest resolution was selected from each cluster in the PDB 40% sequence clustering of the data. Furthermore, we removed all sequences for which fewer than 80 amino acids had the alpha carbon resolved and removed chains with more than 1,400 residues. The final dataset contained 10,795 protein sequences.

The procedure for filtering the recent PDB dataset based on prior template identity was as follows. Hmmsearch was run with default parameters against a copy of the PDB SEQRES fasta downloaded 15 February 2021. Template hits were accepted if the associated structure had a release date earlier than 30 April 2018. Each residue position in a query sequence was assigned the maximum identity of any template hit covering that position. Filtering then proceeded as described in the individual figure legends, based on a combination of maximum identity and sequence coverage.

The MSA depth analysis was based on computing the normalized number of effective sequences (*N*eff) for each position of a query sequence. Per-residue *N*eff values were obtained by counting the number of non-gap residues in the MSA for this position and weighting the sequences using the *N*eff scheme[76](/articles/s41586-021-03819-2#ref-CR76 "Wu, T., Hou, J., Adhikari, B. & Cheng, J. Analysis of several key factors influencing deep learning-based inter-residue contact prediction. Bioinformatics 36, 1091–1098 (2020).") with a threshold of 80% sequence identity measured on the region that is non-gap in either sequence.

### Reporting summary

Further information on research design is available in the [Nature Research Reporting Summary](/articles/s41586-021-03819-2#MOESM2) linked to this paper.

Data availability
-----------------

All input data are freely available from public sources.

Structures from the PDB were used for training and as templates (<https://www.wwpdb.org/ftp/pdb-ftp-sites>; for the associated sequence data and 40% sequence clustering see also <https://ftp.wwpdb.org/pub/pdb/derived_data/> and <https://cdn.rcsb.org/resources/sequence/clusters/bc-40.out>). Training used a version of the PDB downloaded 28 August 2019, while the CASP14 template search used a version downloaded 14 May 2020. The template search also used the PDB70 database, downloaded 13 May 2020 (<https://wwwuser.gwdg.de/~compbiol/data/hhsuite/databases/hhsuite_dbs/>).

We show experimental structures from the PDB with accession numbers [6Y4F](http://doi.org/10.2210/pdb6Y4F/pdb)[77](/articles/s41586-021-03819-2#ref-CR77 "Jiang, W. et al. MrpH, a new class of metal-binding adhesin, requires zinc to mediate biofilm formation. PLoS Pathog. 16, e1008707 (2020)."), [6YJ1](http://doi.org/10.2210/pdb6YJ1/pdb)[78](/articles/s41586-021-03819-2#ref-CR78 "Dunne, M., Ernst, P., Sobieraj, A., Pluckthun, A. & Loessner, M. J. The M23 peptidase domain of the Staphylococcal phage 2638A endolysin. PDB
                  https://doi.org/10.2210/pdb6YJ1/pdb

                 (2020)."), [6VR4](http://doi.org/10.2210/pdb6VR4/pdb)[79](/articles/s41586-021-03819-2#ref-CR79 "Drobysheva, A. V. et al. Structure and function of virion RNA polymerase of a crAss-like phage. Nature 589, 306–309 (2021)."), [6SK0](http://doi.org/10.2210/pdb6SK0/pdb)[80](/articles/s41586-021-03819-2#ref-CR80 "Flaugnatti, N. et al. Structural basis for loading and inhibition of a bacterial T6SS phospholipase effector by the VgrG spike. EMBO J. 39, e104129 (2020)."), [6FES](http://doi.org/10.2210/pdb6FES/pdb)[81](/articles/s41586-021-03819-2#ref-CR81 "ElGamacy, M. et al. An interface-driven design strategy yields a novel, corrugated protein architecture. ACS Synth. Biol. 7, 2226–2235 (2018)."), [6W6W](http://doi.org/10.2210/pdb6W6W/pdb)[82](/articles/s41586-021-03819-2#ref-CR82 "Lim, C. J. et al. The structure of human CST reveals a decameric assembly bound to telomeric DNA. Science 368, 1081–1085 (2020)."), [6T1Z](http://doi.org/10.2210/pdb6T1Z/pdb)[83](/articles/s41586-021-03819-2#ref-CR83 "Debruycker, V. et al. An embedded lipid in the multidrug transporter LmrP suggests a mechanism for polyspecificity. Nat. Struct. Mol. Biol. 27, 829–835 (2020).") and [7JTL](http://doi.org/10.2210/pdb7JTL/pdb)[84](/articles/s41586-021-03819-2#ref-CR84 "Flower, T. G. et al. Structure of SARS-CoV-2 ORF8, a rapidly evolving immune evasion protein. Proc. Natl Acad. Sci. USA 118, e2021785118 (2021).").

For MSA lookup at both the training and prediction time, we used UniRef90 v.2020\_01 (https://ftp.ebi.ac.uk/pub/databases/uniprot/previous\_releases/release-2020\_01/uniref/), BFD (<https://bfd.mmseqs.com>), Uniclust30 v.2018\_08 (<https://wwwuser.gwdg.de/~compbiol/uniclust/2018_08/>) and MGnify clusters v.2018\_12 (<https://ftp.ebi.ac.uk/pub/databases/metagenomics/peptide_database/2018_12/>). Uniclust30 v.2018\_08 was also used as input for constructing a distillation structure dataset.

Code availability
-----------------

Source code for the AlphaFold model, trained weights and inference script are available under an open-source license at <https://github.com/deepmind/alphafold>.

Neural networks were developed with TensorFlow v.1 (<https://github.com/tensorflow/tensorflow>), Sonnet v.1 (<https://github.com/deepmind/sonnet>), JAX v.0.1.69 (<https://github.com/google/jax/>) and Haiku v.0.0.4 (<https://github.com/deepmind/dm-haiku>). The XLA compiler is bundled with JAX and does not have a separate version number.

For MSA search on BFD+Uniclust30, and for template search against PDB70, we used HHBlits and HHSearch from hh-suite v.3.0-beta.3 release 14/07/2017 (<https://github.com/soedinglab/hh-suite>). For MSA search on UniRef90 and clustered MGnify, we used jackhmmer from HMMER v.3.3 (<http://eddylab.org/software/hmmer/>). For constrained relaxation of structures, we used OpenMM v.7.3.1 (<https://github.com/openmm/openmm>) with the Amber99sb force field.

Construction of BFD used MMseqs2 v.925AF (<https://github.com/soedinglab/MMseqs2>) and FAMSA v.1.2.5 (<https://github.com/refresh-bio/FAMSA>).

Data analysis used Python v.3.6 (<https://www.python.org/>), NumPy v.1.16.4 (<https://github.com/numpy/numpy>), SciPy v.1.2.1 (<https://www.scipy.org/>), seaborn v.0.11.1 (<https://github.com/mwaskom/seaborn>), Matplotlib v.3.3.4 (<https://github.com/matplotlib/matplotlib>), bokeh v.1.4.0 (<https://github.com/bokeh/bokeh>), pandas v.1.1.5 (<https://github.com/pandas-dev/pandas>), plotnine v.0.8.0 (<https://github.com/has2k1/plotnine>), statsmodels v.0.12.2 (<https://github.com/statsmodels/statsmodels>) and Colab (<https://research.google.com/colaboratory>). TM-align v.20190822 (<https://zhanglab.dcmb.med.umich.edu/TM-align/>) was used for computing TM-scores. Structure visualizations were created in Pymol v.2.3.0 (<https://github.com/schrodinger/pymol-open-source>).

References
----------

1. Thompson, M. C., Yeates, T. O. & Rodriguez, J. A. Advances in methods for atomic resolution macromolecular structure determination. *F1000Res*. **9**, 667 (2020).

[Article](https://doi.org/10.12688%2Ff1000research.25097.1)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXis1OgtrzK)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Advances%20in%20methods%20for%20atomic%20resolution%20macromolecular%20structure%20determination&journal=F1000Res.&doi=10.12688%2Ff1000research.25097.1&volume=9&publication_year=2020&author=Thompson%2CMC&author=Yeates%2CTO&author=Rodriguez%2CJA)
2. Bai, X.-C., McMullan, G. & Scheres, S. H. W. How cryo-EM is revolutionizing structural biology. *Trends Biochem. Sci*. **40**, 49–57 (2015).

[Article](https://doi.org/10.1016%2Fj.tibs.2014.10.005)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2cXhvVCktLnK)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25544475)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=How%20cryo-EM%20is%20revolutionizing%20structural%20biology&journal=Trends%20Biochem.%20Sci.&doi=10.1016%2Fj.tibs.2014.10.005&volume=40&pages=49-57&publication_year=2015&author=Bai%2CX-C&author=McMullan%2CG&author=Scheres%2CSHW)
3. Jaskolski, M., Dauter, Z. & Wlodawer, A. A brief history of macromolecular crystallography, illustrated by a family tree and its Nobel fruits. *FEBS J*. **281**, 3985–4009 (2014).

[Article](https://doi.org/10.1111%2Ffebs.12796)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2cXhsFKnsbnM)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=24698025)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6309182)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=A%20brief%20history%20of%20macromolecular%20crystallography%2C%20illustrated%20by%20a%20family%20tree%20and%20its%20Nobel%20fruits&journal=FEBS%20J.&doi=10.1111%2Ffebs.12796&volume=281&pages=3985-4009&publication_year=2014&author=Jaskolski%2CM&author=Dauter%2CZ&author=Wlodawer%2CA)
4. Wüthrich, K. The way to NMR structures of proteins. *Nat. Struct. Biol*. **8**, 923–925 (2001).

[Article](https://doi.org/10.1038%2Fnsb1101-923)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=11685234)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20way%20to%20NMR%20structures%20of%20proteins&journal=Nat.%20Struct.%20Biol.&doi=10.1038%2Fnsb1101-923&volume=8&pages=923-925&publication_year=2001&author=W%C3%BCthrich%2CK)
5. wwPDB Consortium. Protein Data Bank: the single global archive for 3D macromolecular structure data. *Nucleic Acids Res*. **47**, D520–D528 (2018).

[Article](https://doi.org/10.1093%2Fnar%2Fgky949)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXhs1Clu7zL)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Protein%20Data%20Bank%3A%20the%20single%20global%20archive%20for%203D%20macromolecular%20structure%20data&journal=Nucleic%20Acids%20Res.&doi=10.1093%2Fnar%2Fgky949&volume=47&pages=D520-D528&publication_year=2018)
6. Mitchell, A. L. et al. MGnify: the microbiome analysis resource in 2020. *Nucleic Acids Res*. **48**, D570–D578 (2020).

[CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXhs1GltrjN)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31696235)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=MGnify%3A%20the%20microbiome%20analysis%20resource%20in%202020&journal=Nucleic%20Acids%20Res.&volume=48&pages=D570-D578&publication_year=2020&author=Mitchell%2CAL)
7. Steinegger, M., Mirdita, M. & Söding, J. Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold. *Nat. Methods* **16**, 603–606 (2019).

[Article](https://doi.org/10.1038%2Fs41592-019-0437-4)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXht1eku7vJ)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31235882)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Protein-level%20assembly%20increases%20protein%20sequence%20recovery%20from%20metagenomic%20samples%20manyfold&journal=Nat.%20Methods&doi=10.1038%2Fs41592-019-0437-4&volume=16&pages=603-606&publication_year=2019&author=Steinegger%2CM&author=Mirdita%2CM&author=S%C3%B6ding%2CJ)
8. Dill, K. A., Ozkan, S. B., Shell, M. S. & Weikl, T. R. The protein folding problem. *Annu. Rev. Biophys*. **37**, 289–316 (2008).

[Article](https://doi.org/10.1146%2Fannurev.biophys.37.092707.153558)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BD1cXnsVGlurw%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=18573083)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2443096)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=1988PhRvC..37..289D)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20protein%20folding%20problem&journal=Annu.%20Rev.%20Biophys.&doi=10.1146%2Fannurev.biophys.37.092707.153558&volume=37&pages=289-316&publication_year=2008&author=Dill%2CKA&author=Ozkan%2CSB&author=Shell%2CMS&author=Weikl%2CTR)
9. Anfinsen, C. B. Principles that govern the folding of protein chains. *Science* **181**, 223–230 (1973).

[Article](https://doi.org/10.1126%2Fscience.181.4096.223)
 [CAS](/articles/cas-redirect/1:CAS:528:DyaE3sXkvVygtbc%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=4124164)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=1973Sci...181..223A)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Principles%20that%20govern%20the%20folding%20of%20protein%20chains&journal=Science&doi=10.1126%2Fscience.181.4096.223&volume=181&pages=223-230&publication_year=1973&author=Anfinsen%2CCB)
10. Senior, A. W. et al. Improved protein structure prediction using potentials from deep learning. *Nature* **577**, 706–710 (2020).

[Article](https://doi.org/10.1038%2Fs41586-019-1923-7)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXis1SisL0%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31942072)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2020Natur.577..706S)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Improved%20protein%20structure%20prediction%20using%20potentials%20from%20deep%20learning&journal=Nature&doi=10.1038%2Fs41586-019-1923-7&volume=577&pages=706-710&publication_year=2020&author=Senior%2CAW)
11. Wang, S., Sun, S., Li, Z., Zhang, R. & Xu, J. Accurate de novo prediction of protein contact map by ultra-deep learning model. *PLOS Comput. Biol*. **13**, e1005324 (2017).

[Article](https://doi.org/10.1371%2Fjournal.pcbi.1005324)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=28056090)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5249242)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2017PLSCB..13E5324W)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2sXovVykurk%3D)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Accurate%20de%20novo%20prediction%20of%20protein%20contact%20map%20by%20ultra-deep%20learning%20model&journal=PLOS%20Comput.%20Biol.&doi=10.1371%2Fjournal.pcbi.1005324&volume=13&publication_year=2017&author=Wang%2CS&author=Sun%2CS&author=Li%2CZ&author=Zhang%2CR&author=Xu%2CJ)
12. Zheng, W. et al. Deep-learning contact-map guided protein structure prediction in CASP13. *Proteins* **87**, 1149–1164 (2019).

[Article](https://doi.org/10.1002%2Fprot.25792)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXhsFKgsrnK)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31365149)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6851476)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep-learning%20contact-map%20guided%20protein%20structure%20prediction%20in%20CASP13&journal=Proteins&doi=10.1002%2Fprot.25792&volume=87&pages=1149-1164&publication_year=2019&author=Zheng%2CW)
13. Abriata, L. A., Tamò, G. E. & Dal Peraro, M. A further leap of improvement in tertiary structure prediction in CASP13 prompts new routes for future assessments. *Proteins* **87**, 1100–1112 (2019).

[Article](https://doi.org/10.1002%2Fprot.25787)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXhsFaqs77K)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31344267)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=A%20further%20leap%20of%20improvement%20in%20tertiary%20structure%20prediction%20in%20CASP13%20prompts%20new%20routes%20for%20future%20assessments&journal=Proteins&doi=10.1002%2Fprot.25787&volume=87&pages=1100-1112&publication_year=2019&author=Abriata%2CLA&author=Tam%C3%B2%2CGE&author=Dal%20Peraro%2CM)
14. Pearce, R. & Zhang, Y. Deep learning techniques have significantly impacted protein structure prediction and protein design. *Curr. Opin. Struct. Biol*. **68**, 194–207 (2021).

[Article](https://doi.org/10.1016%2Fj.sbi.2021.01.007)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3MXks1Ghtr4%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=33639355)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep%20learning%20techniques%20have%20significantly%20impacted%20protein%20structure%20prediction%20and%20protein%20design&journal=Curr.%20Opin.%20Struct.%20Biol.&doi=10.1016%2Fj.sbi.2021.01.007&volume=68&pages=194-207&publication_year=2021&author=Pearce%2CR&author=Zhang%2CY)
15. Moult, J., Fidelis, K., Kryshtafovych, A., Schwede, T. & Topf, M. Critical assessment of techniques for protein structure prediction, fourteenth round. *CASP 14 Abstract Book* <https://www.predictioncenter.org/casp14/doc/CASP14_Abstracts.pdf> (2020).
16. Brini, E., Simmerling, C. & Dill, K. Protein storytelling through physics. *Science* **370**, eaaz3041 (2020).

[Article](https://doi.org/10.1126%2Fscience.aaz3041)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXisVOmu7vF)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=33243857)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7945008)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Protein%20storytelling%20through%20physics&journal=Science&doi=10.1126%2Fscience.aaz3041&volume=370&publication_year=2020&author=Brini%2CE&author=Simmerling%2CC&author=Dill%2CK)
17. Sippl, M. J. Calculation of conformational ensembles from potentials of mean force. An approach to the knowledge-based prediction of local structures in globular proteins. *J. Mol. Biol*. **213**, 859–883 (1990).

[Article](https://doi.org/10.1016%2FS0022-2836%2805%2980269-4)
 [CAS](/articles/cas-redirect/1:CAS:528:DyaK3cXkvFCgs7Y%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=2359125)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Calculation%20of%20conformational%20ensembles%20from%20potentials%20of%20mean%20force.%20An%20approach%20to%20the%20knowledge-based%20prediction%20of%20local%20structures%20in%20globular%20proteins&journal=J.%20Mol.%20Biol.&doi=10.1016%2FS0022-2836%2805%2980269-4&volume=213&pages=859-883&publication_year=1990&author=Sippl%2CMJ)
18. Šali, A. & Blundell, T. L. Comparative protein modelling by satisfaction of spatial restraints. *J. Mol. Biol*. **234**, 779–815 (1993).

[Article](https://doi.org/10.1006%2Fjmbi.1993.1626)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=8254673)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Comparative%20protein%20modelling%20by%20satisfaction%20of%20spatial%20restraints&journal=J.%20Mol.%20Biol.&doi=10.1006%2Fjmbi.1993.1626&volume=234&pages=779-815&publication_year=1993&author=%C5%A0ali%2CA&author=Blundell%2CTL)
19. Roy, A., Kucukural, A. & Zhang, Y. I-TASSER: a unified platform for automated protein structure and function prediction. *Nat. Protocols* **5**, 725–738 (2010).

[Article](https://doi.org/10.1038%2Fnprot.2010.5)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC3cXksVahs74%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=20360767)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=I-TASSER%3A%20a%20unified%20platform%20for%20automated%20protein%20structure%20and%20function%20prediction&journal=Nat.%20Protocols&doi=10.1038%2Fnprot.2010.5&volume=5&pages=725-738&publication_year=2010&author=Roy%2CA&author=Kucukural%2CA&author=Zhang%2CY)
20. Altschuh, D., Lesk, A. M., Bloomer, A. C. & Klug, A. Correlation of co-ordinated amino acid substitutions with function in viruses related to tobacco mosaic virus. *J. Mol. Biol*. **193**, 693–707 (1987).

[Article](https://doi.org/10.1016%2F0022-2836%2887%2990352-4)
 [CAS](/articles/cas-redirect/1:CAS:528:DyaL2sXitV2ksL8%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=3612789)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Correlation%20of%20co-ordinated%20amino%20acid%20substitutions%20with%20function%20in%20viruses%20related%20to%20tobacco%20mosaic%20virus&journal=J.%20Mol.%20Biol.&doi=10.1016%2F0022-2836%2887%2990352-4&volume=193&pages=693-707&publication_year=1987&author=Altschuh%2CD&author=Lesk%2CAM&author=Bloomer%2CAC&author=Klug%2CA)
21. Shindyalov, I. N., Kolchanov, N. A. & Sander, C. Can three-dimensional contacts in protein structures be predicted by analysis of correlated mutations? *Protein Eng*. **7**, 349–358 (1994).

[Article](https://doi.org/10.1093%2Fprotein%2F7.3.349)
 [CAS](/articles/cas-redirect/1:CAS:528:DyaK2cXitFWqtbs%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=8177884)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Can%20three-dimensional%20contacts%20in%20protein%20structures%20be%20predicted%20by%20analysis%20of%20correlated%20mutations%3F&journal=Protein%20Eng.&doi=10.1093%2Fprotein%2F7.3.349&volume=7&pages=349-358&publication_year=1994&author=Shindyalov%2CIN&author=Kolchanov%2CNA&author=Sander%2CC)
22. Weigt, M., White, R. A., Szurmant, H., Hoch, J. A. & Hwa, T. Identification of direct residue contacts in protein–protein interaction by message passing. *Proc. Natl Acad. Sci. USA* **106**, 67–72 (2009).

[Article](https://doi.org/10.1073%2Fpnas.0805923106)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BD1MXltF2jug%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=19116270)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2009PNAS..106...67W)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Identification%20of%20direct%20residue%20contacts%20in%20protein%E2%80%93protein%20interaction%20by%20message%20passing&journal=Proc.%20Natl%20Acad.%20Sci.%20USA&doi=10.1073%2Fpnas.0805923106&volume=106&pages=67-72&publication_year=2009&author=Weigt%2CM&author=White%2CRA&author=Szurmant%2CH&author=Hoch%2CJA&author=Hwa%2CT)
23. Marks, D. S. et al. Protein 3D structure computed from evolutionary sequence variation. *PLoS ONE* **6**, e28766 (2011).

[Article](https://doi.org/10.1371%2Fjournal.pone.0028766)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC3MXhs1KhurnJ)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=22163331)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3233603)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2011PLoSO...628766M)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Protein%203D%20structure%20computed%20from%20evolutionary%20sequence%20variation&journal=PLoS%20ONE&doi=10.1371%2Fjournal.pone.0028766&volume=6&publication_year=2011&author=Marks%2CDS)
24. Jones, D. T., Buchan, D. W. A., Cozzetto, D. & Pontil, M. PSICOV: precise structural contact prediction using sparse inverse covariance estimation on large multiple sequence alignments. *Bioinformatics* **28**, 184–190 (2012).

[Article](https://doi.org/10.1093%2Fbioinformatics%2Fbtr638)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC38Xht1agurg%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=22101153)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=PSICOV%3A%20precise%20structural%20contact%20prediction%20using%20sparse%20inverse%20covariance%20estimation%20on%20large%20multiple%20sequence%20alignments&journal=Bioinformatics&doi=10.1093%2Fbioinformatics%2Fbtr638&volume=28&pages=184-190&publication_year=2012&author=Jones%2CDT&author=Buchan%2CDWA&author=Cozzetto%2CD&author=Pontil%2CM)
25. Moult, J., Pedersen, J. T., Judson, R. & Fidelis, K. A large-scale experiment to assess protein structure prediction methods. *Proteins* **23**, ii–iv (1995).

[Article](https://doi.org/10.1002%2Fprot.340230303)
 [CAS](/articles/cas-redirect/1:STN:280:DyaK287oslCntw%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=8710822)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=A%20large-scale%20experiment%20to%20assess%20protein%20structure%20prediction%20methods&journal=Proteins&doi=10.1002%2Fprot.340230303&volume=23&pages=ii-iv&publication_year=1995&author=Moult%2CJ&author=Pedersen%2CJT&author=Judson%2CR&author=Fidelis%2CK)
26. Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K. & Moult, J. Critical assessment of methods of protein structure prediction (CASP)-round XIII. *Proteins* **87**, 1011–1020 (2019).

[Article](https://doi.org/10.1002%2Fprot.25823)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXitVSlt7%2FO)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31589781)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6927249)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Critical%20assessment%20of%20methods%20of%20protein%20structure%20prediction%20%28CASP%29-round%20XIII&journal=Proteins&doi=10.1002%2Fprot.25823&volume=87&pages=1011-1020&publication_year=2019&author=Kryshtafovych%2CA&author=Schwede%2CT&author=Topf%2CM&author=Fidelis%2CK&author=Moult%2CJ)
27. Zhang, Y. & Skolnick, J. Scoring function for automated assessment of protein structure template quality. *Proteins* **57**, 702–710 (2004).

[Article](https://doi.org/10.1002%2Fprot.20264)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BD2cXhtVaqtLvI)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=15476259)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Scoring%20function%20for%20automated%20assessment%20of%20protein%20structure%20template%20quality&journal=Proteins&doi=10.1002%2Fprot.20264&volume=57&pages=702-710&publication_year=2004&author=Zhang%2CY&author=Skolnick%2CJ)
28. Tu, Z. & Bai, X. Auto-context and its application to high-level vision tasks and 3D brain image segmentation. *IEEE Trans. Pattern Anal. Mach. Intell*. **32**, 1744–1757 (2010).

[Article](https://doi.org/10.1109%2FTPAMI.2009.186)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=20724753)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Auto-context%20and%20its%20application%20to%20high-level%20vision%20tasks%20and%203D%20brain%20image%20segmentation&journal=IEEE%20Trans.%20Pattern%20Anal.%20Mach.%20Intell.&doi=10.1109%2FTPAMI.2009.186&volume=32&pages=1744-1757&publication_year=2010&author=Tu%2CZ&author=Bai%2CX)
29. Carreira, J., Agrawal, P., Fragkiadaki, K. & Malik, J. Human pose estimation with iterative error feedback. In *Proc. IEEE Conference on Computer Vision and Pattern Recognition* 4733–4742 (2016).
30. Mirabello, C. & Wallner, B. rawMSA: end-to-end deep learning using raw multiple sequence alignments. *PLoS ONE* **14**, e0220182 (2019).

[Article](https://doi.org/10.1371%2Fjournal.pone.0220182)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXhvFamur7E)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31415569)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6695225)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=rawMSA%3A%20end-to-end%20deep%20learning%20using%20raw%20multiple%20sequence%20alignments&journal=PLoS%20ONE&doi=10.1371%2Fjournal.pone.0220182&volume=14&publication_year=2019&author=Mirabello%2CC&author=Wallner%2CB)
31. Huang, Z. et al. CCNet: criss-cross attention for semantic segmentation. In *Proc. IEEE/CVF International Conference on Computer Vision* 603–612 (2019).
32. Hornak, V. et al. Comparison of multiple Amber force fields and development of improved protein backbone parameters. *Proteins* **65**, 712–725 (2006).

[Article](https://doi.org/10.1002%2Fprot.21123)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BD28XhtFWqt7fM)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=16981200)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4805110)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Comparison%20of%20multiple%20Amber%20force%20fields%20and%20development%20of%20improved%20protein%20backbone%20parameters&journal=Proteins&doi=10.1002%2Fprot.21123&volume=65&pages=712-725&publication_year=2006&author=Hornak%2CV)
33. Zemla, A. LGA: a method for finding 3D similarities in protein structures. *Nucleic Acids Res*. **31**, 3370–3374 (2003).

[Article](https://doi.org/10.1093%2Fnar%2Fgkg571)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BD3sXltVWjtbk%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=12824330)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC168977)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=LGA%3A%20a%20method%20for%20finding%203D%20similarities%20in%20protein%20structures&journal=Nucleic%20Acids%20Res.&doi=10.1093%2Fnar%2Fgkg571&volume=31&pages=3370-3374&publication_year=2003&author=Zemla%2CA)
34. Mariani, V., Biasini, M., Barbato, A. & Schwede, T. lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests. *Bioinformatics* **29**, 2722–2728 (2013).

[Article](https://doi.org/10.1093%2Fbioinformatics%2Fbtt473)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC3sXhs1CisrfK)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=23986568)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3799472)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=lDDT%3A%20a%20local%20superposition-free%20score%20for%20comparing%20protein%20structures%20and%20models%20using%20distance%20difference%20tests&journal=Bioinformatics&doi=10.1093%2Fbioinformatics%2Fbtt473&volume=29&pages=2722-2728&publication_year=2013&author=Mariani%2CV&author=Biasini%2CM&author=Barbato%2CA&author=Schwede%2CT)
35. Xie, Q., Luong, M.-T., Hovy, E. & Le, Q. V. Self-training with noisy student improves imagenet classification. In *Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition* 10687–10698 (2020).
36. Mirdita, M. et al. Uniclust databases of clustered and deeply annotated protein sequences and alignments. *Nucleic Acids Res*. **45**, D170–D176 (2017).

[Article](https://doi.org/10.1093%2Fnar%2Fgkw1081)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1cXhslWgsb8%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=27899574)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Uniclust%20databases%20of%20clustered%20and%20deeply%20annotated%20protein%20sequences%20and%20alignments&journal=Nucleic%20Acids%20Res.&doi=10.1093%2Fnar%2Fgkw1081&volume=45&pages=D170-D176&publication_year=2017&author=Mirdita%2CM)
37. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In *Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies* 1, 4171–4186 (2019).
38. Rao, R. et al. MSA transformer. In *Proc. 38th International Conference on Machine Learning* PMLR 139, 8844–8856 (2021).
39. Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. *Nature* <https://doi.org/10.1038/s41586-021-03828-1> (2021).
40. Kuhlman, B. & Bradley, P. Advances in protein structure prediction and design. *Nat. Rev. Mol. Cell Biol*. **20**, 681–697 (2019).

[Article](https://doi.org/10.1038%2Fs41580-019-0163-x)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXhsFyksL7J)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31417196)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7032036)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Advances%20in%20protein%20structure%20prediction%20and%20design&journal=Nat.%20Rev.%20Mol.%20Cell%20Biol.&doi=10.1038%2Fs41580-019-0163-x&volume=20&pages=681-697&publication_year=2019&author=Kuhlman%2CB&author=Bradley%2CP)
41. Marks, D. S., Hopf, T. A. & Sander, C. Protein structure prediction from sequence variation. *Nat. Biotechnol*. **30**, 1072–1080 (2012).

[Article](https://doi.org/10.1038%2Fnbt.2419)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC38Xhs1elt7bM)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=23138306)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4319528)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Protein%20structure%20prediction%20from%20sequence%20variation&journal=Nat.%20Biotechnol.&doi=10.1038%2Fnbt.2419&volume=30&pages=1072-1080&publication_year=2012&author=Marks%2CDS&author=Hopf%2CTA&author=Sander%2CC)
42. Qian, N. & Sejnowski, T. J. Predicting the secondary structure of globular proteins using neural network models. *J. Mol. Biol*. **202**, 865–884 (1988).

[Article](https://doi.org/10.1016%2F0022-2836%2888%2990564-5)
 [CAS](/articles/cas-redirect/1:CAS:528:DyaL1MXhtlWksb0%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=3172241)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Predicting%20the%20secondary%20structure%20of%20globular%20proteins%20using%20neural%20network%20models&journal=J.%20Mol.%20Biol.&doi=10.1016%2F0022-2836%2888%2990564-5&volume=202&pages=865-884&publication_year=1988&author=Qian%2CN&author=Sejnowski%2CTJ)
43. Fariselli, P., Olmea, O., Valencia, A. & Casadio, R. Prediction of contact maps with neural networks and correlated mutations. *Protein Eng*. **14**, 835–843 (2001).

[Article](https://doi.org/10.1093%2Fprotein%2F14.11.835)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BD38XjtVentA%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=11742102)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Prediction%20of%20contact%20maps%20with%20neural%20networks%20and%20correlated%20mutations&journal=Protein%20Eng.&doi=10.1093%2Fprotein%2F14.11.835&volume=14&pages=835-843&publication_year=2001&author=Fariselli%2CP&author=Olmea%2CO&author=Valencia%2CA&author=Casadio%2CR)
44. Yang, J. et al. Improved protein structure prediction using predicted interresidue orientations. *Proc. Natl Acad. Sci. USA* **117**, 1496–1503 (2020).

[Article](https://doi.org/10.1073%2Fpnas.1914677117)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXhsFKrsLg%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31896580)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6983395)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Improved%20protein%20structure%20prediction%20using%20predicted%20interresidue%20orientations&journal=Proc.%20Natl%20Acad.%20Sci.%20USA&doi=10.1073%2Fpnas.1914677117&volume=117&pages=1496-1503&publication_year=2020&author=Yang%2CJ)
45. Li, Y. et al. Deducing high-accuracy protein contact-maps from a triplet of coevolutionary matrices through deep residual convolutional networks. *PLOS Comput. Biol*. **17**, e1008865 (2021).

[Article](https://doi.org/10.1371%2Fjournal.pcbi.1008865)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3MXosFSms78%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=33770072)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8026059)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deducing%20high-accuracy%20protein%20contact-maps%20from%20a%20triplet%20of%20coevolutionary%20matrices%20through%20deep%20residual%20convolutional%20networks&journal=PLOS%20Comput.%20Biol.&doi=10.1371%2Fjournal.pcbi.1008865&volume=17&publication_year=2021&author=Li%2CY)
46. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In *Proc. IEEE Conference on Computer Vision and Pattern Recognition* 770–778 (2016).
47. AlQuraishi, M. End-to-end differentiable learning of protein structure. *Cell Syst*. **8**, 292–301 (2019).

[Article](https://doi.org/10.1016%2Fj.cels.2019.03.006)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXosVyhtb0%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31005579)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6513320)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=End-to-end%20differentiable%20learning%20of%20protein%20structure&journal=Cell%20Syst.&doi=10.1016%2Fj.cels.2019.03.006&volume=8&pages=292-301&publication_year=2019&author=AlQuraishi%2CM)
48. Senior, A. W. et al. Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13). *Proteins* **87**, 1141–1148 (2019).

[Article](https://doi.org/10.1002%2Fprot.25834)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXitFartb%2FK)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31602685)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7079254)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Protein%20structure%20prediction%20using%20multiple%20deep%20neural%20networks%20in%20the%2013th%20Critical%20Assessment%20of%20Protein%20Structure%20Prediction%20%28CASP13%29&journal=Proteins&doi=10.1002%2Fprot.25834&volume=87&pages=1141-1148&publication_year=2019&author=Senior%2CAW)
49. Ingraham, J., Riesselman, A. J., Sander, C. & Marks, D. S. Learning protein structure with a differentiable simulator. in *Proc. International Conference on Learning Representations* (2019).
50. Li, J. Universal transforming geometric network. Preprint at <https://arxiv.org/abs/1908.00723> (2019).
51. Xu, J., McPartlon, M. & Li, J. Improved protein structure prediction by deep learning irrespective of co-evolution information. *Nat. Mach. Intell*. **3**, 601–609 (2021).

[Article](https://doi.org/10.1038%2Fs42256-021-00348-5)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=34368623)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8340610)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Improved%20protein%20structure%20prediction%20by%20deep%20learning%20irrespective%20of%20co-evolution%20information&journal=Nat.%20Mach.%20Intell.&doi=10.1038%2Fs42256-021-00348-5&volume=3&pages=601-609&publication_year=2021&author=Xu%2CJ&author=McPartlon%2CM&author=Li%2CJ)
52. Vaswani, A. et al. Attention is all you need. In *Advances in Neural Information Processing Systems* 5998–6008 (2017).
53. Wang, H. et al. Axial-deeplab: stand-alone axial-attention for panoptic segmentation. in *European Conference on Computer Vision* 108–126 (Springer, 2020).
54. Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M. & Church, G. M. Unified rational protein engineering with sequence-based deep representation learning. *Nat. Methods* **16**, 1315–1322 (2019).

[Article](https://doi.org/10.1038%2Fs41592-019-0598-1)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXitVSlsbnJ)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31636460)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7067682)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Unified%20rational%20protein%20engineering%20with%20sequence-based%20deep%20representation%20learning&journal=Nat.%20Methods&doi=10.1038%2Fs41592-019-0598-1&volume=16&pages=1315-1322&publication_year=2019&author=Alley%2CEC&author=Khimulya%2CG&author=Biswas%2CS&author=AlQuraishi%2CM&author=Church%2CGM)
55. Heinzinger, M. et al. Modeling aspects of the language of life through transfer-learning protein sequences. *BMC Bioinformatics* **20**, 723 (2019).

[Article](https://doi.org/10.1186%2Fs12859-019-3220-8)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXisVGjsLbK)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31847804)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6918593)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Modeling%20aspects%20of%20the%20language%20of%20life%20through%20transfer-learning%20protein%20sequences&journal=BMC%20Bioinformatics&doi=10.1186%2Fs12859-019-3220-8&volume=20&publication_year=2019&author=Heinzinger%2CM)
56. Rives, A. et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. *Proc. Natl Acad. Sci. USA* **118**, e2016239118 (2021).

[Article](https://doi.org/10.1073%2Fpnas.2016239118)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3MXovVantro%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=33876751)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8053943)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Biological%20structure%20and%20function%20emerge%20from%20scaling%20unsupervised%20learning%20to%20250%20million%20protein%20sequences&journal=Proc.%20Natl%20Acad.%20Sci.%20USA&doi=10.1073%2Fpnas.2016239118&volume=118&publication_year=2021&author=Rives%2CA)
57. Pereira, J. et al. High-accuracy protein structure prediction in CASP14. *Proteins* <https://doi.org/10.1002/prot.26171> (2021).

[Article](https://doi.org/10.1002%2Fprot.26171)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=34387010)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8881082)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=High-accuracy%20protein%20structure%20prediction%20in%20CASP14&journal=Proteins&doi=10.1002%2Fprot.26171&publication_year=2021&author=Pereira%2CJ)
58. Gupta, M. et al. CryoEM and AI reveal a structure of SARS-CoV-2 Nsp2, a multifunctional protein involved in key host processes. Preprint at <https://doi.org/10.1101/2021.05.10.443524> (2021).
59. Ingraham, J., Garg, V. K., Barzilay, R. & Jaakkola, T. Generative models for graph-based protein design. in *Proc. 33rd Conference on Neural Information Processing Systems* (2019).
60. Johnson, L. S., Eddy, S. R. & Portugaly, E. Hidden Markov model speed heuristic and iterative HMM search procedure. *BMC Bioinformatics* **11**, 431 (2010).

[Article](https://doi.org/10.1186%2F1471-2105-11-431)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=20718988)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2931519)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC3cXhtVKqsbrF)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Hidden%20Markov%20model%20speed%20heuristic%20and%20iterative%20HMM%20search%20procedure&journal=BMC%20Bioinformatics&doi=10.1186%2F1471-2105-11-431&volume=11&publication_year=2010&author=Johnson%2CLS&author=Eddy%2CSR&author=Portugaly%2CE)
61. Remmert, M., Biegert, A., Hauser, A. & Söding, J. HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment. *Nat. Methods* **9**, 173–175 (2012).

[Article](https://doi.org/10.1038%2Fnmeth.1818)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC3MXhs1OltbnO)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=HHblits%3A%20lightning-fast%20iterative%20protein%20sequence%20searching%20by%20HMM-HMM%20alignment&journal=Nat.%20Methods&doi=10.1038%2Fnmeth.1818&volume=9&pages=173-175&publication_year=2012&author=Remmert%2CM&author=Biegert%2CA&author=Hauser%2CA&author=S%C3%B6ding%2CJ)
62. The UniProt Consortium. UniProt: the universal protein knowledgebase in 2021. *Nucleic Acids Res*. **49**, D480–D489 (2020).

[Article](https://doi.org/10.1093%2Fnar%2Fgkaa1100)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3MXntFCit7s%3D)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=UniProt%3A%20the%20universal%20protein%20knowledgebase%20in%202021&journal=Nucleic%20Acids%20Res.&doi=10.1093%2Fnar%2Fgkaa1100&volume=49&pages=D480-D489&publication_year=2020)
63. Steinegger, M. & Söding, J. Clustering huge protein sequence sets in linear time. *Nat. Commun*. **9**, 2542 (2018).

[Article](https://doi.org/10.1038%2Fs41467-018-04964-5)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29959318)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6026198)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2018NatCo...9.2542S)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1cXht1Cns7rO)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Clustering%20huge%20protein%20sequence%20sets%20in%20linear%20time&journal=Nat.%20Commun.&doi=10.1038%2Fs41467-018-04964-5&volume=9&publication_year=2018&author=Steinegger%2CM&author=S%C3%B6ding%2CJ)
64. Steinegger, M. & Söding, J. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. *Nat. Biotechnol*. **35**, 1026–1028 (2017).

[Article](https://doi.org/10.1038%2Fnbt.3988)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2sXhs1GqsLzE)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=29035372)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=MMseqs2%20enables%20sensitive%20protein%20sequence%20searching%20for%20the%20analysis%20of%20massive%20data%20sets&journal=Nat.%20Biotechnol.&doi=10.1038%2Fnbt.3988&volume=35&pages=1026-1028&publication_year=2017&author=Steinegger%2CM&author=S%C3%B6ding%2CJ)
65. Deorowicz, S., Debudaj-Grabysz, A. & Gudyś, A. FAMSA: fast and accurate multiple sequence alignment of huge protein families. *Sci. Rep*. **6**, 33964 (2016).

[Article](https://doi.org/10.1038%2Fsrep33964)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC28XhsF2qs7fN)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=27670777)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5037421)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2016NatSR...633964D)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=FAMSA%3A%20fast%20and%20accurate%20multiple%20sequence%20alignment%20of%20huge%20protein%20families&journal=Sci.%20Rep.&doi=10.1038%2Fsrep33964&volume=6&publication_year=2016&author=Deorowicz%2CS&author=Debudaj-Grabysz%2CA&author=Gudy%C5%9B%2CA)
66. Steinegger, M. et al. HH-suite3 for fast remote homology detection and deep protein annotation. *BMC Bioinformatics* **20**, 473 (2019).

[Article](https://doi.org/10.1186%2Fs12859-019-3019-7)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31521110)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6744700)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1MXhsl2hurbK)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=HH-suite3%20for%20fast%20remote%20homology%20detection%20and%20deep%20protein%20annotation&journal=BMC%20Bioinformatics&doi=10.1186%2Fs12859-019-3019-7&volume=20&publication_year=2019&author=Steinegger%2CM)
67. Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B. & Wu, C. H. UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches. *Bioinformatics* **31**, 926–932 (2015).

[Article](https://doi.org/10.1093%2Fbioinformatics%2Fbtu739)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC28Xht1Gntb7F)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25398609)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=UniRef%20clusters%3A%20a%20comprehensive%20and%20scalable%20alternative%20for%20improving%20sequence%20similarity%20searches&journal=Bioinformatics&doi=10.1093%2Fbioinformatics%2Fbtu739&volume=31&pages=926-932&publication_year=2015&author=Suzek%2CBE&author=Wang%2CY&author=Huang%2CH&author=McGarvey%2CPB&author=Wu%2CCH)
68. Eddy, S. R. Accelerated profile HMM searches. *PLOS Comput. Biol*. **7**, e1002195 (2011).

[Article](https://doi.org/10.1371%2Fjournal.pcbi.1002195)
 [MathSciNet](http://www.ams.org/mathscinet-getitem?mr=2859646)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC3MXhsVCku7rL)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=22039361)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3197634)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2011PLSCB...7E2195E)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Accelerated%20profile%20HMM%20searches&journal=PLOS%20Comput.%20Biol.&doi=10.1371%2Fjournal.pcbi.1002195&volume=7&publication_year=2011&author=Eddy%2CSR)
69. Eastman, P. et al. OpenMM 7: rapid development of high performance algorithms for molecular dynamics. *PLOS Comput. Biol*. **13**, e1005659 (2017).

[Article](https://doi.org/10.1371%2Fjournal.pcbi.1005659)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=28746339)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5549999)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1cXivVWhur0%3D)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=OpenMM%207%3A%20rapid%20development%20of%20high%20performance%20algorithms%20for%20molecular%20dynamics&journal=PLOS%20Comput.%20Biol.&doi=10.1371%2Fjournal.pcbi.1005659&volume=13&publication_year=2017&author=Eastman%2CP)
70. Ashish, A. M. A. et al. TensorFlow: large-scale machine learning on heterogeneous systems. Preprint at <https://arxiv.org/abs/1603.04467> (2015).
71. Reynolds, M. et al. Open sourcing Sonnet – a new library for constructing neural networks. *DeepMind* <https://deepmind.com/blog/open-sourcing-sonnet/> (7 April 2017).
72. Harris, C. R. et al. Array programming with NumPy. *Nature* **585**, 357–362 (2020).

[Article](https://doi.org/10.1038%2Fs41586-020-2649-2)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXitlWmsbbN)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32939066)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7759461)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2020Natur.585..357H)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Array%20programming%20with%20NumPy&journal=Nature&doi=10.1038%2Fs41586-020-2649-2&volume=585&pages=357-362&publication_year=2020&author=Harris%2CCR)
73. Van Rossum, G. & Drake, F. L. *Python 3 Reference Manual* (CreateSpace, 2009).
74. Bisong, E. in *Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners* 59–64 (Apress, 2019).
75. TensorFlow. XLA: Optimizing Compiler for TensorFlow. <https://www.tensorflow.org/xla> (2018).
76. Wu, T., Hou, J., Adhikari, B. & Cheng, J. Analysis of several key factors influencing deep learning-based inter-residue contact prediction. *Bioinformatics* **36**, 1091–1098 (2020).

[Article](https://doi.org/10.1093%2Fbioinformatics%2Fbtz679)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXisVOrtbvJ)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=31504181)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Analysis%20of%20several%20key%20factors%20influencing%20deep%20learning-based%20inter-residue%20contact%20prediction&journal=Bioinformatics&doi=10.1093%2Fbioinformatics%2Fbtz679&volume=36&pages=1091-1098&publication_year=2020&author=Wu%2CT&author=Hou%2CJ&author=Adhikari%2CB&author=Cheng%2CJ)
77. Jiang, W. et al. MrpH, a new class of metal-binding adhesin, requires zinc to mediate biofilm formation. *PLoS Pathog*. **16**, e1008707 (2020).

[Article](https://doi.org/10.1371%2Fjournal.ppat.1008707)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXhs1Glt7fI)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32780778)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7444556)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=MrpH%2C%20a%20new%20class%20of%20metal-binding%20adhesin%2C%20requires%20zinc%20to%20mediate%20biofilm%20formation&journal=PLoS%20Pathog.&doi=10.1371%2Fjournal.ppat.1008707&volume=16&publication_year=2020&author=Jiang%2CW)
78. Dunne, M., Ernst, P., Sobieraj, A., Pluckthun, A. & Loessner, M. J. The M23 peptidase domain of the Staphylococcal phage 2638A endolysin. *PDB* <https://doi.org/10.2210/pdb6YJ1/pdb> (2020).
79. Drobysheva, A. V. et al. Structure and function of virion RNA polymerase of a crAss-like phage. *Nature* **589**, 306–309 (2021).

[Article](https://doi.org/10.1038%2Fs41586-020-2921-5)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXitlOgs7jI)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=33208949)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2021Natur.589..306D)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Structure%20and%20function%20of%20virion%20RNA%20polymerase%20of%20a%20crAss-like%20phage&journal=Nature&doi=10.1038%2Fs41586-020-2921-5&volume=589&pages=306-309&publication_year=2021&author=Drobysheva%2CAV)
80. Flaugnatti, N. et al. Structural basis for loading and inhibition of a bacterial T6SS phospholipase effector by the VgrG spike. *EMBO J*. **39**, e104129 (2020).

[Article](https://doi.org/10.15252%2Fembj.2019104129)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXotFCnu7s%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32350888)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7265238)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Structural%20basis%20for%20loading%20and%20inhibition%20of%20a%20bacterial%20T6SS%20phospholipase%20effector%20by%20the%20VgrG%20spike&journal=EMBO%20J.&doi=10.15252%2Fembj.2019104129&volume=39&publication_year=2020&author=Flaugnatti%2CN)
81. ElGamacy, M. et al. An interface-driven design strategy yields a novel, corrugated protein architecture. *ACS Synth. Biol*. **7**, 2226–2235 (2018).

[Article](https://doi.org/10.1021%2Facssynbio.8b00224)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC1cXhsFyqtbnP)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=30148951)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=An%20interface-driven%20design%20strategy%20yields%20a%20novel%2C%20corrugated%20protein%20architecture&journal=ACS%20Synth.%20Biol.&doi=10.1021%2Facssynbio.8b00224&volume=7&pages=2226-2235&publication_year=2018&author=ElGamacy%2CM)
82. Lim, C. J. et al. The structure of human CST reveals a decameric assembly bound to telomeric DNA. *Science* **368**, 1081–1085 (2020).

[Article](https://doi.org/10.1126%2Fscience.aaz9649)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXhtFSqt7fI)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32499435)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7559292)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2020Sci...368.1081L)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20structure%20of%20human%20CST%20reveals%20a%20decameric%20assembly%20bound%20to%20telomeric%20DNA&journal=Science&doi=10.1126%2Fscience.aaz9649&volume=368&pages=1081-1085&publication_year=2020&author=Lim%2CCJ)
83. Debruycker, V. et al. An embedded lipid in the multidrug transporter LmrP suggests a mechanism for polyspecificity. *Nat. Struct. Mol. Biol*. **27**, 829–835 (2020).

[Article](https://doi.org/10.1038%2Fs41594-020-0464-y)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3cXhsVKkt7bE)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=32719456)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7951658)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=An%20embedded%20lipid%20in%20the%20multidrug%20transporter%20LmrP%20suggests%20a%20mechanism%20for%20polyspecificity&journal=Nat.%20Struct.%20Mol.%20Biol.&doi=10.1038%2Fs41594-020-0464-y&volume=27&pages=829-835&publication_year=2020&author=Debruycker%2CV)
84. Flower, T. G. et al. Structure of SARS-CoV-2 ORF8, a rapidly evolving immune evasion protein. *Proc. Natl Acad. Sci. USA* **118**, e2021785118 (2021).

[Article](https://doi.org/10.1073%2Fpnas.2021785118)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BB3MXhsVCitb4%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=33361333)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Structure%20of%20SARS-CoV-2%20ORF8%2C%20a%20rapidly%20evolving%20immune%20evasion%20protein&journal=Proc.%20Natl%20Acad.%20Sci.%20USA&doi=10.1073%2Fpnas.2021785118&volume=118&publication_year=2021&author=Flower%2CTG)

[Download references](https://citation-needed.springer.com/v2/references/10.1038/s41586-021-03819-2?format=refman&flavour=references)

Acknowledgements
----------------

We thank A. Rrustemi, A. Gu, A. Guseynov, B. Hechtman, C. Beattie, C. Jones, C. Donner, E. Parisotto, E. Elsen, F. Popovici, G. Necula, H. Maclean, J. Menick, J. Kirkpatrick, J. Molloy, J. Yim, J. Stanway, K. Simonyan, L. Sifre, L. Martens, M. Johnson, M. O’Neill, N. Antropova, R. Hadsell, S. Blackwell, S. Das, S. Hou, S. Gouws, S. Wheelwright, T. Hennigan, T. Ward, Z. Wu, Ž. Avsec and the Research Platform Team for their contributions; M. Mirdita for his help with the datasets; M. Piovesan-Forster, A. Nelson and R. Kemp for their help managing the project; the JAX, TensorFlow and XLA teams for detailed support and enabling machine learning models of the complexity of AlphaFold; our colleagues at DeepMind, Google and Alphabet for their encouragement and support; and J. Moult and the CASP14 organizers, and the experimentalists whose structures enabled the assessment. M.S. acknowledges support from the National Research Foundation of Korea grant (2019R1A6A1A10073437, 2020M3A9G7103933) and the Creative-Pioneering Researchers Program through Seoul National University.

Author information
------------------

Author notes1. These authors contributed equally: John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Demis Hassabis

### Authors and Affiliations

1. DeepMind, London, UK

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli & Demis Hassabis
2. School of Biological Sciences, Seoul National University, Seoul, South Korea

Martin Steinegger
3. Artificial Intelligence Institute, Seoul National University, Seoul, South Korea

Martin Steinegger

Authors1. John Jumper[View author publications](/search?author=John%20Jumper)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=John%20Jumper) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22John%20Jumper%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
2. Richard Evans[View author publications](/search?author=Richard%20Evans)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Richard%20Evans) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Richard%20Evans%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
3. Alexander Pritzel[View author publications](/search?author=Alexander%20Pritzel)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Alexander%20Pritzel) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Alexander%20Pritzel%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
4. Tim Green[View author publications](/search?author=Tim%20Green)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Tim%20Green) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Tim%20Green%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
5. Michael Figurnov[View author publications](/search?author=Michael%20Figurnov)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Michael%20Figurnov) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Michael%20Figurnov%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
6. Olaf Ronneberger[View author publications](/search?author=Olaf%20Ronneberger)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Olaf%20Ronneberger) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Olaf%20Ronneberger%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
7. Kathryn Tunyasuvunakool[View author publications](/search?author=Kathryn%20Tunyasuvunakool)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Kathryn%20Tunyasuvunakool) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Kathryn%20Tunyasuvunakool%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
8. Russ Bates[View author publications](/search?author=Russ%20Bates)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Russ%20Bates) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Russ%20Bates%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
9. Augustin Žídek[View author publications](/search?author=Augustin%20%C5%BD%C3%ADdek)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Augustin%20%C5%BD%C3%ADdek) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Augustin%20%C5%BD%C3%ADdek%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
10. Anna Potapenko[View author publications](/search?author=Anna%20Potapenko)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Anna%20Potapenko) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Anna%20Potapenko%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
11. Alex Bridgland[View author publications](/search?author=Alex%20Bridgland)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Alex%20Bridgland) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Alex%20Bridgland%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
12. Clemens Meyer[View author publications](/search?author=Clemens%20Meyer)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Clemens%20Meyer) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Clemens%20Meyer%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
13. Simon A. A. Kohl[View author publications](/search?author=Simon%20A.%20A.%20Kohl)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Simon%20A.%20A.%20Kohl) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Simon%20A.%20A.%20Kohl%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
14. Andrew J. Ballard[View author publications](/search?author=Andrew%20J.%20Ballard)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Andrew%20J.%20Ballard) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Andrew%20J.%20Ballard%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
15. Andrew Cowie[View author publications](/search?author=Andrew%20Cowie)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Andrew%20Cowie) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Andrew%20Cowie%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
16. Bernardino Romera-Paredes[View author publications](/search?author=Bernardino%20Romera-Paredes)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Bernardino%20Romera-Paredes) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Bernardino%20Romera-Paredes%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
17. Stanislav Nikolov[View author publications](/search?author=Stanislav%20Nikolov)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Stanislav%20Nikolov) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Stanislav%20Nikolov%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
18. Rishub Jain[View author publications](/search?author=Rishub%20Jain)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Rishub%20Jain) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Rishub%20Jain%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
19. Jonas Adler[View author publications](/search?author=Jonas%20Adler)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Jonas%20Adler) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Jonas%20Adler%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
20. Trevor Back[View author publications](/search?author=Trevor%20Back)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Trevor%20Back) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Trevor%20Back%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
21. Stig Petersen[View author publications](/search?author=Stig%20Petersen)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Stig%20Petersen) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Stig%20Petersen%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
22. David Reiman[View author publications](/search?author=David%20Reiman)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=David%20Reiman) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22David%20Reiman%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
23. Ellen Clancy[View author publications](/search?author=Ellen%20Clancy)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Ellen%20Clancy) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Ellen%20Clancy%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
24. Michal Zielinski[View author publications](/search?author=Michal%20Zielinski)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Michal%20Zielinski) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Michal%20Zielinski%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
25. Martin Steinegger[View author publications](/search?author=Martin%20Steinegger)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Martin%20Steinegger) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Martin%20Steinegger%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
26. Michalina Pacholska[View author publications](/search?author=Michalina%20Pacholska)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Michalina%20Pacholska) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Michalina%20Pacholska%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
27. Tamas Berghammer[View author publications](/search?author=Tamas%20Berghammer)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Tamas%20Berghammer) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Tamas%20Berghammer%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
28. Sebastian Bodenstein[View author publications](/search?author=Sebastian%20Bodenstein)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Sebastian%20Bodenstein) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Sebastian%20Bodenstein%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
29. David Silver[View author publications](/search?author=David%20Silver)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=David%20Silver) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22David%20Silver%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
30. Oriol Vinyals[View author publications](/search?author=Oriol%20Vinyals)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Oriol%20Vinyals) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Oriol%20Vinyals%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
31. Andrew W. Senior[View author publications](/search?author=Andrew%20W.%20Senior)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Andrew%20W.%20Senior) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Andrew%20W.%20Senior%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
32. Koray Kavukcuoglu[View author publications](/search?author=Koray%20Kavukcuoglu)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Koray%20Kavukcuoglu) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Koray%20Kavukcuoglu%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
33. Pushmeet Kohli[View author publications](/search?author=Pushmeet%20Kohli)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Pushmeet%20Kohli) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Pushmeet%20Kohli%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
34. Demis Hassabis[View author publications](/search?author=Demis%20Hassabis)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Demis%20Hassabis) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Demis%20Hassabis%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
### Contributions

J.J. and D.H. led the research. J.J., R.E., A. Pritzel, M.F., O.R., R.B., A. Potapenko, S.A.A.K., B.R.-P., J.A., M.P., T. Berghammer and O.V. developed the neural network architecture and training. T.G., A.Ž., K.T., R.B., A.B., R.E., A.J.B., A.C., S.N., R.J., D.R., M.Z. and S.B. developed the data, analytics and inference systems. D.H., K.K., P.K., C.M. and E.C. managed the research. T.G. led the technical platform. P.K., A.W.S., K.K., O.V., D.S., S.P. and T. Back contributed technical advice and ideas. M.S. created the BFD genomics database and provided technical assistance on HHBlits. D.H., R.E., A.W.S. and K.K. conceived the AlphaFold project. J.J., R.E. and A.W.S. conceived the end-to-end approach. J.J., A. Pritzel, O.R., A. Potapenko, R.E., M.F., T.G., K.T., C.M. and D.H. wrote the paper.

### Corresponding authors

Correspondence to
 [John Jumper](mailto:jumper@deepmind.com) or [Demis Hassabis](mailto:dhcontact@deepmind.com).

Ethics declarations
-------------------

### Competing interests

J.J., R.E., A. Pritzel, T.G., M.F., O.R., R.B., A.B., S.A.A.K., D.R. and A.W.S. have filed non-provisional patent applications 16/701,070 and PCT/EP2020/084238, and provisional patent applications 63/107,362, 63/118,917, 63/118,918, 63/118,921 and 63/118,919, each in the name of DeepMind Technologies Limited, each pending, relating to machine learning for predicting protein structures. The other authors declare no competing interests.

Additional information
----------------------

**Peer review information** *Nature* thanks Mohammed AlQuraishi, Charlotte Deane and Yang Zhang for their contribution to the peer review of this work.

**Publisher’s note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information
-------------------------

### [Supplementary Information](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_MOESM1_ESM.pdf)

Description of the method details of the AlphaFold system, model, and analysis, including data pipeline, datasets, model blocks, loss functions, training and inference details, and ablations. Includes Supplementary Methods, Supplementary Figures, Supplementary Tables and Supplementary Algorithms.

### [Reporting Summary](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_MOESM2_ESM.pdf)

### [Supplementary Video 1](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_MOESM3_ESM.mp4)

Video of the intermediate structure trajectory of the CASP14 target T1024 (LmrP) A two-domain target (408 residues). Both domains are folded early, while their packing is adjusted for a longer time.

### [Supplementary Video 2](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_MOESM4_ESM.mp4)

Video of the intermediate structure trajectory of the CASP14 target T1044 (RNA polymerase of crAss-like phage). A large protein (2180 residues), with multiple domains. Some domains are folded quickly, while others take a considerable amount of time to fold.

### [Supplementary Video 3](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_MOESM5_ESM.mp4)

Video of the intermediate structure trajectory of the CASP14 target T1064 (Orf8). A very difficult single-domain target (106 residues) that takes the entire depth of the network to fold.

### [Supplementary Video 4](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-021-03819-2/MediaObjects/41586_2021_3819_MOESM6_ESM.mp4)

Video of the intermediate structure trajectory of the CASP14 target T1091. A multi-domain target (863 residues). Individual domains’ structure is determined early, while the domain packing evolves throughout the network. The network is exploring unphysical configurations throughout the process, resulting in long ‘strings’ in the visualization.

Rights and permissions
----------------------

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit <http://creativecommons.org/licenses/by/4.0/>.

[Reprints and Permissions](https://s100.copyright.com/AppDispatchServlet?title=Highly%20accurate%20protein%20structure%20prediction%20with%20AlphaFold&author=John%20Jumper%20et%20al&contentID=10.1038%2Fs41586-021-03819-2&copyright=The%20Author%28s%29&publication=0028-0836&publicationDate=2021-07-15&publisherName=SpringerNature&orderBeanReset=true&oa=CC%20BY)

This article is cited by
------------------------

* ###
[Deep generative model for drug design from protein target sequence](https://doi.org/10.1186/s13321-023-00702-2)

	+ Yangyang Chen
	+ Zixu Wang
	+ Tetsuya Sakurai*Journal of Cheminformatics* (2023)
* ###
[Insights into the structure-function relationship of the NorQ/NorD chaperones from Paracoccus denitrificans reveal shared principles of interacting MoxR AAA+/VWA domain proteins](https://doi.org/10.1186/s12915-023-01546-w)

	+ Maximilian Kahle
	+ Sofia Appelgren
	+ Pia Ädelroth*BMC Biology* (2023)
* ###
[Formation and characterization of BMP2/GDF5 and BMP4/GDF5 heterodimers](https://doi.org/10.1186/s12915-023-01522-4)

	+ Gregory R. Gipson
	+ Kristof Nolan
	+ Thomas B. Thompson*BMC Biology* (2023)
* ###
[Prediction of protein solubility based on sequence physicochemical patterns and distributed representation information with DeepSoluE](https://doi.org/10.1186/s12915-023-01510-8)

	+ Chao Wang
	+ Quan Zou*BMC Biology* (2023)
* ###
[Definition of the transcriptional units of inherited retinal disease genes by meta-analysis of human retinal transcriptome data](https://doi.org/10.1186/s12864-023-09300-w)

	+ Karla Alejandra Ruiz-Ceja
	+ Dalila Capasso
	+ Sandro Banfi*BMC Genomics* (2023)

Comments
--------

By submitting a comment you agree to abide by our [Terms](/info/tandc.html) and [Community Guidelines](/info/community-guidelines.html). If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Highly accurate protein structure prediction with AlphaFold | Nature

John Jumper
Richard Evans
Alexander Pritzel
Tim Green
Michael Figurnov
Olaf Ronneberger
Kathryn Tunyasuvunakool
Russ Bates
Augustin Žídek
Anna Potapenko
Alex Bridgland
Clemens Meyer
Simon A. A. Kohl
Andrew J. Ballard
Andrew Cowie
Bernardino Romera-Paredes
Stanislav Nikolov
Rishub Jain
Jonas Adler
Trevor Back
Stig Petersen
David Reiman
Ellen Clancy
Michal Zielinski
Martin Steinegger
Michalina Pacholska
Tamas Berghammer
Sebastian Bodenstein
David Silver
Oriol Vinyals
Andrew W. Senior
Koray Kavukcuoglu
Pushmeet Kohli
Demis Hassabis