Folding@home

Original author(s)	Vijay Pande
Developer(s)	Pande laboratory, Sony, Nvidia, ATI, Cauldron Development[1]
Initial release	October 1, 2000
Stable release	Windows and Linux: 7.1.52[2] Mac OS X: 6.29.3[3] PlayStation 3:    Life with PlayStation: 1.4[4]
Operating system	Microsoft Windows, Mac OS X, Linux
Platform	Cross-platform
Available in	English
Type	Distributed computing
License	Partially GPL, partially proprietary[5]
Website	folding.stanford.edu

Folding@home (FAH or F@h) is a distributed computing project for simulation of protein folding, computational drug design, and other molecular dynamics for disease research.^[6] It primarily attempts to determine how proteins reach their final three-dimensional structure, which is of significant academic interest and has major implications for research into Alzheimer's disease, Huntington's disease, and many forms of cancer, among other diseases. To a certain extent, Folding@home also tries to predict that final structure and to determine how other molecules may interact with it, which has applications in drug design.^[7][8] Folding@home is developed and operated by the Pande laboratory at Stanford University, under the leadership of Vijay Pande, and is shared by various scientific institutions and research laboratories across the world in a collaboration known as the Folding@home Consortium.^[1]

Folding@home is powered by the idle processing resources of thousands of volunteered personal computers and PlayStation 3s. As part of the project's client-server architecture, these systems receive simulation Work Units, complete them, and return them to database servers where they are compiled into an overall simulation. Volunteers can track their contributions on the Folding@home website, which can make participation competitive and encourages long-term involvement. The project has pioneered the uses of GPUs, PlayStation 3s, and Message Passing Interface (used for computing on multi-core processors) for distributed computing and scientific research. This large-scale computing network has allowed Folding@home to simulate protein folding at timescales thousands of times longer than previously achieved.

The project uses simulation methodology that represents a paradigm shift from traditional computational approaches.^[9] Since its launch on October 1, 2000,^[10] the Pande lab has produced ninety-six scientific research papers as a direct result of the project.^[11] Folding@home operates at approximately seven petaFLOPS, a higher computational performance than all distributed computing projects under BOINC combined, and it remains one of the world's fastest computing systems. This computing power makes Folding@home the most powerful molecular dynamics simulator, and allows it to run computationally expensive atomic-level simulations over biologically relevant timescales. These simulations have demonstrated accuracy to observations in laboratory research, which is a challenge in computational biology.

  Project significance

  Illustration of a protein before and after folding

Proteins are an essential component to many biological functions and participate in virtually all processes within cells. They often act as enzymes, performing biochemical reactions including cell signaling, molecular transportation, and cellular regulation. As structural elements, some proteins act as a type of skeleton for cells, and as antibodies, other proteins participate in the immune system. Before a protein can take on these roles, it must fold into a functional three-dimensional structure, a process that often occurs spontaneously and is dependent on interactions within its amino acid sequence. Protein folding is driven by the search to find the most energetically favorable conformation of the protein, i.e. its native state. Thus, understanding protein folding is critical to understanding what a protein does and how it works, and is considered a "holy grail" of computational biology.^[12][13] Despite folding occurring within a crowded cellular environment, it typically proceeds smoothly. However, due to a protein's chemical properties or other factors, proteins may misfold — that is, fold down the wrong pathway and end up misshapen. Unless cellular mechanisms are capable of destroying or refolding such misfolded proteins, they can subsequently aggregate and cause a variety of debilitating diseases.^[14] Laboratory experiments studying these processes can be limited in scope and atomic detail, leading scientists to use physics-based computational models that, when complementing experiments, seek to provide a more complete picture of protein folding, misfolding, and aggregation.^[15][16]

Due to the complexity of proteins' conformation space and limitations in computational power, all-atom molecular dynamics simulations have been severely limited in the timescales which they can study. While most proteins typically fold in the order of milliseconds,^[15][17] prior to 2010 simulations could only reach nanosecond to microsecond timescales.^[18] General-purpose supercomputers have been used to simulate protein folding, but such systems are intrinsically expensive and typically shared between many different research groups, and because the computations in kinetic models are serial in nature, strong scaling of traditional molecular simulations to these architectures is exceptionally difficult.^[19][20] Additionally, as the protein folding process is stochastic, a limited number of long simulations are not sufficient for comprehensive views of protein folding.^[21]

Protein folding does not occur in a single step.^[14] Instead, a significant portion of the folding time is spent "waiting" in various intermediate conformational states, where each state represents a local free energy minimum in the protein's energy landscape. Folding@home restarts simulations from these states, and calculates the rates and probabilities of transitions between sets of conformations in parallel. This approach explores protein's phase space while avoiding much of the computation inside the local-minimum state itself and achieves near-linear parallelization, leading to a significant reduction in overall serial calculation time.^[20] The conformational states and the short simulations between them are then compiled into a statistical Markov state model (MSM), which essentially serves as a map of the protein's energy landscape and kinetic and equilibrium thermodynamic properties. Once constructed, each MSM illustrates folding events and pathways and can represent its conformational states at an arbitrary resolution. It can also reveal which transitions are limiting the model's accuracy, which allows for specific follow-up simulations to improve the thoroughness of the model. Using this adaptive sampling technique, the amount of time it takes to construct an accurate Markov state model is inversely proportional to the number of parallel simulations run, i.e. the number of processors available. Folding@home has used these MSMs to simulate folding at biologically relevant timescales, to reveal how proteins misfold, and to quantitatively compare simulations with experiments.^[9][20][21]

In 2002, Folding@home used Markov state models to complete approximately a million CPU days of simulations over the span of several months,^[22] and in 2011, MSMs parallelized another simulation that required an aggregate 10 million CPU hours of computation.^[23] In January 2010, Folding@home used MSMs to simulate the dynamics of the slow-folding 32-residue NTL9 protein out to 1.52 milliseconds, a timescale that is a thousand times longer than previously achieved. The model consisted of many individual trajectories, each two orders of magnitude shorter.^[9][18] This was the first demonstration that MSMs were capable of statistically capturing folding events that could not be seen by conventional simulation methods.^[21] In 2010, Folding@home researcher Greg Bowman was awarded the Thomas Kuhn Paradigm Shift Award from the American Chemical Society for the instrumental development of the software used to automatically build these MSMs and for attaining quantitative agreement between theory and experiment.^[24] For his work, Pande was awarded the 2012 Michael and Kate Bárány Award for Young Investigators for "developing field-defining and field-changing computational methods to produce leading theoretical models for protein and RNA folding"^[25] as well as the 2006 Irving Sigal Young Investigator Award "for his unique approach to employing advances in algorithms that make optimal use of distributed computing, which places his efforts at the cutting edge of simulations. The results have stimulated a re-examination of the meaning of both ensemble and single-molecule measurements, making Dr. Pande’s efforts pioneering contributions to simulation methodology."^[26]

  Biomedical research

Protein misfolding is a component in the development of a variety of diseases, including alpha 1-antitrypsin deficiency, Alzheimer's disease, autism,^[27] cancer, sickle-cell anaemia, Creutzfeldt–Jakob disease, cystic fibrosis, Huntington's disease, osteogenesis imperfecta, Parkinson's disease, and type II diabetes.^[28][6][14] Once it is understood how a protein misfolds, therapeutic intervention can follow, which can use engineered molecules to alter the production of a certain protein, to help destroy a misfolded protein, or to assist in the folding process.^[29] Cellular infection by viruses such as HIV and influenza also involve folding events within cellular membranes.^[30] Computer-assisted drug design has the potential to expedite drug discovery.^[31] The combination of computational molecular modeling and experimental analysis has the possibility of fundamentally shaping the future of molecular medicine and the rational design of therapeutics.^[16]

Even though simulations run on Folding@home are used in conjunction with laboratory experiments,^[21] researchers can use Folding@home to study how folding in vitro differs from folding in native cellular environments. In 2011 Folding@home continued simulations of folding inside a ribosomal exit tunnel, to help scientists better understand how natural confinement and crowding might influence the folding process.^[32][33] Researchers can further use Folding@home to study aspects of folding, misfolding, and its relationship to disease that are exceptionally difficult to observe experimentally. For example, scientists typically employ chemical denaturants to unfold proteins from their stable native state. It is difficult to experimentally determine if these denatured states contain residual structures which may influence folding behavior, but Folding@home has been used to study this denatured state and the denaturation mechanism.^[34] Folding@home is dedicated to producing significant amounts of results about protein folding and the diseases that result from protein misfolding. It is also used to develop novel computational methods for drug design.^[6] The goal of the first five years of the project was to make significant advances in understanding folding, while the current goal is to understand misfolding and related disease, especially Alzheimer's disease.^[35] Between 2000 and 2010, the timescales over which Folding@home simulates protein folding have increased a million fold.^[36]

The Pande lab is a non-profit organization and does not sell the results generated by Folding@home.^[37] The large data sets from the project are freely available for other researchers to use upon request and some can be accessed from the Folding@home website.^[38][39] The Pande lab has collaborated with other molecular dynamics systems such as the Blue Gene supercomputer.^[40] They also share Folding@home's key software with other researchers, so that the algorithms which benefited Folding@home may aid other scientific areas.^[38] In 2011 they released the open-source Copernicus software, which is based on Folding@home's MSM and other parallelization techniques and aims to significantly improve the efficiency and scaling of molecular simulations on large Computer clusters or supercomputers.^[41] Summaries of all of the scientific findings from Folding@home are posted on the Folding@home website after publication.^[11] The full publications are available online from an academic library.^[39]

  Alzheimer's disease

  Enzymes act on the amyloid precursor protein and cut it into Aβ fragments, which then aggregate to form senile plaques characteristic of Alzheimer's patients.

Alzheimer's disease is an incurable form of dementia which most often affects the elderly. Its cause remains largely unknown, but the disease is identified as a protein misfolding disease and is associated with toxic aggregations of the amyloid beta (Aβ) peptide, a fragment of the larger amyloid precursor protein.^[42] High concentrations of misfolded Aβ cause protein oligomer growth that in turn contribute to Aβ misfolding. This cyclic process appears to be toxic and leads to neurodegeneration. The oligomer aggregates then collect into dense formations known as senile plaques, a pathological marker of Alzheimer's.^[43][44] The severity of the disease depends not only on the amount of Aβ, but also on how it misfolds. However, due to the heterogeneous nature of these aggregates, experimental studies of the oligomer structure in atomic detail are difficult,^[43][44] and simulations of oligomer aggregates are extremely computationally demanding due to their size and complexity.^[45][46]

In 2008 Folding@home simulated Aβ oligomerization in atomic detail over timescales of the order of tens of seconds. This was significant as previous simulations had been forced to use simplified models and been limited to several hundreds of microseconds: six orders of magnitude short of experimentally relevant timescales.^[47] This study helped prepare the Pande lab for future aggregation studies and for further research to find a small peptide which may stabilize the aggregation process.^[45] This is regarded as a promising approach to the development of therapeutic drugs for treating Alzheimer's patients.^[48] The Pande lab is focusing their research on Alzheimer's with the goal of predicting the aggregate structure for drug design approaches as well as developing methods to stop the oligomerization process.^[6] In 2008 Folding@home found several small drug candidates which appear to inhibit the toxicity of Aβ.^[49] In 2010, these drug leads went from the test tube to testing on living tissue and, in close cooperation with the Nanomedicine Center for Protein Folding, continue to be refined.^[6] In 2011, Folding@home completed simulations of several Aβ mutations that appear to stabilize the aggregate formation, which could aid in the development of therapeutic drug approaches to the disease as well as greatly assisting with experimental NMR spectroscopy studies of the oligomers.^[46][50] In the same year, Folding@home began simulations of various Aβ fragments in order to determine how various natural enzymes affect the structure and folding of Aβ.^[51][52]

  Huntington's disease

Huntington's disease is a neurodegenerative genetic disorder that is also associated with protein misfolding and aggregation. Excessive repeats of the glutamine amino acid at the N-terminus of the Huntingtin protein cause aggregation, and although the behavior of the repeats is not completely understood, it does lead to the cognitive decline associated with the disease.^[53] As with other aggregates, there is difficulty in experimentally determining its structure.^[54] Scientists are using Folding@home to study Huntingtin protein aggregate structure as well as to predict how the aggregate forms, assisting with rational drug design approaches to stop the aggregate formation.^[6] The N17 fragment of the Huntingtin protein accelerates this aggregation, and while there have been several proposed mechanisms, its exact role in this process remains largely unknown.^[55] Folding@home has simulated this and other fragments in order to elucidate their roles in the disease.^[56] Since 2008, the Pande lab has applied the drug design approaches used in Alzheimer's disease to Huntington's, and in 2010, Folding@home researcher Veena Thomas proposed a novel therapeutic strategy for Huntington's which may be funded by the National Institutes of Health. This strategy could be used to bring the results from Folding@home directly to a therapeutic drug.^[6]

  Cancer

More than half of all known cancers involve mutations of p53, a tumor suppressor protein present in every cell which regulates the cell cycle and signals for cell death in the event of damage to DNA. Specific mutations in p53 can disrupt these functions, allowing an abnormal cell to continue growing unchecked, resulting in the development of tumors. These deleterious mutations may differ between the various types and locations of cancer, and analysis of these mutations is important for understanding the root causes of p53-related cancers.^[57] In 2004, Folding@home was used to perform the first molecular dynamics study of the refolding of p53's protein dimer in explicit water which revealed insights that were previously unobtainable,^[58] and from it produced the first peer reviewed publication on cancer from a distributed computing project.^[59] The following year, they developed a method to identify the amino acids that are crucial for the stability of a given protein, which was then used to study mutations of p53. The method demonstrated reasonable success in identifying cancer-promoting mutations and determined the effects of specific mutations which could not otherwise be measured experimentally.^[60] Following these studies, the Pande lab expanded their efforts to other p53-related diseases.^[6]

Folding@home is also being used to study protein chaperones,^[6] which act as heat shock proteins and assist with protein folding inside the crowded and chemically stressful intracellular environment, and which have essential roles in cell survival. Rapidly growing cancer cells rely on specific chaperones for their function, and some chaperones play key roles in chemotherapy resistance. Inhibiting these specific chaperones are seen as potential modes of action for efficient antineoplastic drugs or for reducing the spread of cancer.^[61] Using Folding@home and working closely with the Protein Folding Center, the Pande lab hopes to find a drug which inhibits those chaperones involved in cancerous cells.^[62] Researchers are also using Folding@home to study other molecules related to cancer, such as the enzyme Src kinase and certain forms of the Engrailed homeodomain.^[63][64] In 2011, Folding@home began simulations of the dynamics of the small knottin protein EETI, which can identify carcinomas in imaging scans by binding to surface receptors of cancer cells. From simulations of this protein, they hope to accelerate research efforts to modify it to identify other diseases or to bind to drugs.^[65][66]

Interleukin 2 (IL-2) is a protein which plays crucial roles in helping T cells of the immune system attack pathogens and tumors. Unfortunately, its use as a cancer treatment is restricted due to serious side effects such as pulmonary edema. IL-2 binds to these pulmonary cells differently than it does to T cells, so IL-2 research involves understanding the differences between these binding mechanisms. In 2012, Folding@home assisted with the discovery of a form of IL-2 which is three hundred times more effective in its immune system role but carries fewer side effects. In experiments, this altered form significantly outperformed natural IL-2 in impeding tumor growth. Pharmaceutical companies have expressed interest in the mutant molecule, and the National Institutes of Health is testing it against a large variety of tumor models in the hopes of accelerating its development as a therapeutic.^[67][68]

  Osteogenesis imperfecta

Osteogenesis imperfecta, also known as brittle bone disease, is a incurable genetic bone disorder which can be lethal. Those with the disease are unable to make functional connective bone tissue. This is most commonly due to a mutation in Type-I collagen,^[69] which is the most abundant protein in mammals and fulfills a variety of structural roles.^[70] The mutation causes a deformation in collagen's triple helix structure, which if not naturally destroyed, leads to abnormal and weakened bone tissue.^[71] In 2005 the Pande lab produced a publication on a quantum mechanical technique that improved upon previous simulations methods, and which may be useful for future computational studies of collagen.^[72] Although researchers have used Folding@home to study collagen folding and misfolding, the interest stands as a pilot project compared to Alzheimer's and Huntington's research.^[6]

  Viruses

The Pande lab is using Folding@home to study certain viruses such as influenza and HIV, with a focus on preventing the virus from entering the cell.^[6] Influenza in particular has been responsible for periodic high-mortality pandemics, such as the 1918 flu pandemic which may have killed up to 100 million people worldwide.^[73][74] Membrane fusion is an essential event for viral infection and involves conformational changes of viral fusion proteins and protein docking. A virus may enter after this process, or a virus may envelop itself in the cell's membrane.^[30] Membrane fusion is also crucial to a wide range of biological functions and controlling it has pharmaceutical implications, but the exact molecular mechanisms behind fusion remain largely unknown.^[75] Fusion events may involve the interactions of over a half million atoms for hundreds of microseconds. This complexity and timescale makes standard computer simulations exceptionally difficult, which are typically limited to about ten thousand atoms over tens of nanoseconds: a difference of several orders of magnitude.^[47] Moreover, such mechanisms are difficult to analyse experimentally.^[76] However, in 2006 scientists applied Markov state models and the Folding@home network to gain detailed mechanistic insights into the fusion process.^[47]

Using Folding@home for detailed simulations of vesicle fusion, in 2007 the Pande lab introduced a new technique for measuring fusion intermediate topology.^[77] In 2009, researchers used Folding@home to study mutations of influenza hemagglutinin, a protein that attaches a virus to its target cell and assists with viral entry. Mutations to hemagglutinin affect the binding affinity to the cell surface receptor glycan of a target species, which determines the infectivity of the virus strain to that species. Knowledge of the effects of hemagglutinin mutations assists in the development of antiviral drugs.^[78][79] In 2011 Folding@home began simulations of the dynamics of the enzyme RNase H, a key component of HIV, in the hopes of designing drugs to deactivate it.^[80] As of 2012, Folding@home continues to simulate the folding and interactions of hemagglutinin, complementing experimental studies at the University of Virginia.^[6][81]

  Drug design

Drugs function by binding to specific locations on target molecules and causing a certain desired change, such as disabling a pathogen. Ideally, a drug should act very specifically and bind only to its target without interfering with other biological functions. However, it is difficult to precisely determine where and how tightly two molecules will bind. Due to limitations in computational power, current in silico approaches usually have to trade speed for accuracy; e.g. use rapid protein docking methods instead of computationally expensive free energy calculations. Folding@home allows researchers to use both, to evaluate these techniques, and to find ways to improve their efficiency.^[35][82][83] An accurate prediction of these binding affinities has the potential to significantly lower the development costs of new drugs.^[84]

Folding@home has been used to study prime binding locations on protein surfaces by testing the interactions of different molecules with known binding sites.^[84][85] Folding@home took part in SAMPL's 2011 blind experiment which assessed current computational protein and ligand modeling methods by having various researchers attempt to predict which of a set of ligands would attach to a target protein and to estimate their associated binding energies.^[86][87] The Pande lab has used Folding@home to study how bacteria develop an immunity to vancomycin, an antibiotic of "last resort",^[88] as well as to study the dynamics of beta-lactamase, a protein that plays important roles in drug resistance, in the hope of being better able to design drugs to deactivate it.^[89]

Approximately half of all known antibiotics interfere with the workings of a bacteria's ribosome, a large and complex biochemical machine that performs protein biosynthesis by translating messenger RNA into proteins. Macrolide antibiotics clog the ribosome's exit tunnel, preventing synthesis of essential bacterial proteins. In 2007 the Pande lab received a grant to study and design new antibiotics.^[6] In 2008 they used Folding@home to study the interior of this tunnel and how specific molecules may affect it.^[90] The full structure of the ribosome has only been recently determined, and Folding@home has also simulated ribosomal proteins, as the functions of many of them remain largely unknown.^[91] Ribosomal research has helped the Pande lab prepare for larger and more complex biomedical problems.^[6]

  Participation

  Folding@home's computing power from November 2006 until September 2007. Note the large spike in performance following March 23, when the PlayStation 3 client was released.

In addition to reporting active processors, Folding@home also determines its computing performance as measured in FLOPS based on the actual execution time of its calculations. Originally this was simply native FLOPS, that is, the raw performance from each given type of processing hardware.^[92] In March 2009 Folding@home began reporting the performance in both native and x86 FLOPS:^[93] the latter being an estimation of how many FLOPS the calculation would take on the standard x86 architecture, which is commonly used as a performance reference. Specialized hardware such as GPUs can efficiently perform certain complex functions in a single FLOP which would otherwise require multiple FLOPS on the x86 architecture. This x86 measurement attempts to even out these hardware differences.^[92] Despite using conservative conversions, for the GPU and PS3 clients x86 FLOPS are consistently much greater than their native FLOPS and comprise a large majority of Folding@home's FLOP performance.^[94][95]

In 2007 Guinness recognized Folding@home as the most powerful distributed computing network in the world.^[96] As of June 12, 2012, the project has 274,005 active CPUs, 23,366 active GPUs, and 23,674 active PS3s, for a total of 4.533 native petaFLOPS (6.612 x86 petaFLOPS).^[94] At the same time, the combined efforts of all distributed computing projects under BOINC totals 6.094 petaFLOPS from 544,444 active hosts.^[97] Using the Markov state model approach, Folding@home achieves strong scaling across its user base and gains a near-linear speedup for every additional processor.^[20][98] This large and powerful network allows Folding@home to do work not possible any other way.^[40]

Active participation in Folding@home has grown steadily since its launch.^[99][100] In March 2002 Google co-founder Sergey Brin launched Google Compute as add-on for the Google Toolbar, which allowed Windows users to participate in the project.^[101] Although limited in functionality and scope, it increased participation in Folding@home from 10,000 up to about 30,000 active CPUs.^[102] The program ended in October 2005 in favor of the official Folding@home clients, and is no longer available for the Toolbar.^[103] Folding@home also gained participants from Genome@home, another distributed computing project from the Pande lab and a sister project to Folding@home. The goal of Genome@home was protein design and associated applications, but was officially concluded in March 2004. Following its completion, users were asked to donate computing power to Folding@home instead.^[104]

  PetaFLOPS milestones

On September 16, 2007, due in large part to the participation of Playstation 3s, the Folding@home project officially attained a sustained performance level higher than one native petaFLOPS, becoming the first computing system of any kind in the world to do so.^[110][111] On May 7, 2008, the project attained a sustained performance level higher than two native petaFLOPS,^[112] followed by the three and four native petaFLOPS milestones on August 20^[113][114] and September 28, 2008 respectively.^[115] It was the first computing project to do so.^[116] Then on February 18, 2009, Folding@home achieved a performance level of just above five native petaFLOPS.^[117][118] Most recently, on November 10, 2011, Folding@home crossed the six native petaFLOPS barrier with the equivalent of nearly eight x86 petaFLOPS.^[119]

  Points

Similarly to other distributed computing projects, Folding@home quantitatively assesses user computing contributions to the project through a credit system.^[120] Each user receives points for completing every Work Unit, but for reliably and rapidly completing units which are exceptionally computationally demanding, or are of great scientific priority, users who opt-in may be non-linearly awarded additional bonus points.^[121] All units from a given protein project have uniform base credit, which is determined by benchmarking one or more Work Units from that project on an official reference machine before the project is released.^[122] This generates a fair system of equal pay for equal work, and attempts to align credit with the value of the scientific results.^[120][122] The points can foster friendly competition between individuals and teams to compute the most for the project.^[120][123] Users may receive credit for their work by clients on multiple machines.^[37] Users can use a passkey to securely protect their contributions, as they not only allow for the receipt of bonus points, but they also separate a user from any policy issues arising from another using that username.^[124]

Users can register their contributions under a team, which combine the points of all their members. A user can start their own team, or they can join an existing team,^[3] but existing points cannot be transferred to a new team or username.^[125] In some cases, a team may have their own community-driven sources of help or recruitment such as an Internet forum.^[126] Rivalries between teams benefit the folding community,^[127] and members can have intra-team competitions for top spots.^[128] Individual and team statistics are posted on the Folding@home website.^[120]

  Software

Folding@home software at the user's end involves three primary components: Work Units, cores, and a client.

  Work Units

A Work Unit is the protein data that the client is asked to process. Work Units are a fraction of the simulation between the states in a Markov state model. After the Work Unit has been completely processed, it is returned and the respective credit points are awarded, and this cycle then repeats automatically.^[123] All Work Units have associated deadlines, and if this deadline is exceeded, the user may not get credit and the unit will be automatically reissued to another participant. As protein folding is serial in nature and many Work Units are generated from their predecessors, this allows the overall simulation process to proceed normally if one is not returned after a certain period of time. Due to these deadlines, the minimum system requirement for Folding@home is a Pentium 3 450 MHz CPU with SSE or newer.^[37] However, Work Units for high performance clients have a much shorter deadline than those for the uniprocessor client, as a major part of the scientific benefit is dependent on rapidly completing simulations.^[129]

Before public release, Work Units go through several quality assurance steps to keep problematic ones from becoming fully available. These stages include internal testing, closed beta testing, and open beta testing, before a final full release across all of Folding@home.^[130] Folding@home's Work Units are normally processed only once, except in the rare event that errors occur during processing. If this occurs for three different users, it is automatically pulled from distribution.^[131][132] The Folding@home support forum can be used to differentiate between problematic hardware and a bad Work Unit.^[133]

  Cores

Specialized scientific computer programs, referred to as "FahCores" and often abbreviated "cores," perform the calculations on the Work Unit behind the scenes.^[123] Folding@home's cores are modified and optimized versions of molecular dynamics programs, including GROMACS, AMBER, TINKER,^[5] ProtoMol, CPMD, SHARPEN, and Desmond.^[134][135] Most of Folding@home's cores use GROMACS,^[123] one of the fastest and most popular molecular dynamics software packages available, which largely consists of manually-optimized assembly code.^[136][137] Some of these cores perform explicit atom-by-atom molecular dynamics calculations, while others perform implicit solvation methods, which treat atoms as a mathematical continuum.^[138][139] While these cores use open-source software, Folding@home uses a closed-source license and is not required to release the cores' source code.^[5] The same core can be used by various versions of the client, and separating the core from the client enables their scientific methods to be updated automatically as needed without a client update. The cores also periodically create calculation checkpoints so that if they are interrupted they can resume work from a checkpoint upon startup.^[123]

  Client

Folding@home participants install client programs on their personal computer or on the PlayStation 3 gaming console. The user interacts with the client, which manages the other software components behind the scenes. Through the client, the user may pause the folding process, open an event log, check the work progress, or view personal statistics.^[140] The clients run continuously in the background of the user's computer at an extremely low priority, utilizing otherwise unused processing power so that normal computer usage is unaffected.^[37][141] The maximum CPU utilization can also be adjusted through client settings.^[140][142] The client connects to a Folding@home server and retrieves a Work Unit and may also download an appropriate core, depending on client settings, operating system, and underlying hardware architecture. Computer clients tailor to uniprocessor and multi-core processors systems, as well as graphics processing units. While these latter clients use significantly more resources, the diversity and power of each hardware architecture provides Folding@home with the ability to efficiently complete many different types of simulations in a timely manner, (in a few weeks or months rather than years) which is of significant scientific value. Together, these clients allow researchers to study biomedical questions previously considered impossible to tackle computationally.^[35][129]

Significant work goes into minimizing security issues in all of Folding@home's software. For example, clients can be downloaded only from the official Folding@home website or its commercial partners.^[37][143] Each client will upload and download data only from Stanford's Folding@home data servers (over port 8080, with 80 as an alternative) using 2048-bit digital signatures for verification and will only interact with Folding@home computer files.^[37][144] Folding@home's End-User License Agreement forbids public access to the client source code for security and scientific integrity reasons.^[143][145] Thus from a security standpoint it behaves in a similar fashion to a web browser, but is even more secure.^[102][144]

Folding@home's first client was a screensaver, which would run Folding@home while the computer was not otherwise in use.^[146] Starting in 2004 the Pande lab collaborated with David Anderson to test a supplemental client on the open source BOINC framework.^[147] This client was released to closed beta in April 2005; however, the approach became unworkable and was abandoned in June 2006.^[148] BOINC's fixed architecture limits the types of project it can accommodate and thus was not appropriate for Folding@home.^[123]

  Graphics processing units

The specialized hardware of GPUs is designed to accelerate rendering of 3D graphics applications such as video games and can significantly outperform CPUs for certain types of calculations. Although limited in generality, this makes GPUs one of the most powerful and rapidly growing computational platforms. As such, general purpose GPU computing is the pursuit of many scientists and researchers. However, GPU hardware is difficult to utilize for non-graphics tasks and usually requires significant algorithm restructuring and an advanced understanding of the underlying architecture.^[149] Such customization is challenging, especially to researchers with limited software development resources. To achieve hardware-independence, the Pande lab's open source OpenMM library serves as a high-level API, allowing molecular simulation software to run efficiently on varying architectures without significant modification. Lower-level APIs interface the higher-level API with the underlying platform. This flexible approach delivers performance nearly equal to hand-tuned GPU code, and greatly outperforms CPU implementations.^[138][150] GPUs remain Folding@home's most powerful platform in terms of FLOPS; as of June 2012 GPU clients account for 72% of the entire project's x86 FLOP throughput.^[94]

Prior to 2010 the computational reliability of GPGPU consumer-grade hardware had remained largely unknown, and circumstantial evidence related to the lack of built-in error detection and correction in GPU memory raised reliability concerns. The Pande lab then conducted the first large-scale test of GPU scientific accuracy on over 20,000 hosts on the Folding@home network. Soft errors were detected in the memory subsystems of two-thirds of the tested GPUs. The study found that the error rate was most dependent on board architecture, but concluded that reliable GPGPU computing was very feasible as long as attention is paid to the hardware characteristics.^[151]

The first generation of Folding@home's Windows GPU client (GPU1) was released to the public on October 2, 2006,^[148] delivering a 20-30X speedup for certain calculations over its CPU-based GROMACS counterparts.^[152] It was the first time GPUs had been used for either distributed computing or major molecular dynamics calculations.^[153][154] Pande lab gained significant knowledge and experience with the development of GPGPU software, but citing a need to improve scientific accuracies over DirectX, it was succeeded by GPU2, the second generation successor of the client on April 10, 2008.^[152][155] Following its introduction, GPU1 was officially retired on June 6.^[152] Compared to GPU1, GPU2 was more scientifically reliable and productive, ran on ATI and CUDA-enabled Nvidia GPUs, and supported more advanced algorithms, larger proteins, and real-time visualization of the protein simulation.^[156][157] Following this, the third generation of Folding@home's GPU client (GPU3) was released on May 25, 2010. While backwards compatible to GPU2, GPU3 is comparatively more stable and efficient, is more flexible for additional scientific capabilities,^[158] and uses OpenMM on top of an OpenCL framework.^[158][159] Although the GPU client does not natively support the Linux operating system, it can be run under WINE for users with Nvidia graphics cards.^[160][161]

  PlayStation 3

  The PlayStation 3's Life With PlayStation client displays a 3D animation of the protein being folded

Folding@home can also take advantage of the computing power of PlayStation 3s. At the time of its inception and for certain calculations, its main streaming Cell processor delivered a 20x speed increase over PCs, processing power which could not be found on other systems such as the Xbox 360.^[35][102] The PS3's high speed and efficiency introduced other opportunities for worthwhile optimizations, and significantly changed the tradeoff between computational efficiency and overall accuracy, allowing for the utilization of more complex molecular models at little extra computational cost.^[162] This allowed Folding@home to run biomedical calculations that would otherwise be computationally infeasible.^[163] The PS3 also has the ability to stream data quickly to its GPU, and is capable of real-time atomic detail visualizations of the protein dynamics.^[162]

The PS3 client was developed in a collaborative effort between Sony and the Pande lab and was first released as a standalone client on March 23, 2007.^[35][164] Its release made Folding@home the first distributed computing project to utilize PS3s.^[165] On September 18 of the following year, the PS3 client became a channel of Life with PlayStation on its launch.^[166][167] In terms of the types of calculations it can perform, the client takes the middle ground between a CPU's flexibility and a GPU's speed.^[148] However, unlike CPUs and GPUs, users cannot perform other activities on their PS3 while running Folding@home.^[163] The PS3's uniform console environment makes support easier and makes Folding@home more user friendly.^[35]

  Multi-core processing client

Folding@home can also utilize the parallel processing capabilities of modern multi-core processors. The ability to use several CPU cores simultaneously allows completion of the overall folding simulation much faster. Working together, these CPU cores complete single Work Units proportionately faster than the standard uniprocessor client,^[129] which reduces the traditional difficulties of scaling a large simulation to many processors. While this approach is not only scientifically valuable, the resulting publications would not have been possible without this computing power.^[168]

In November 2006 first generation symmetric multiprocessing (SMP) clients were publicly released for open beta testing, referred to as SMP1.^[148] These clients used Message Passing Interface (MPI) communication protocols for parallel processing, as at that time the GROMACS cores were not designed to be used with multiple threads.^[129] This was the first time a distributed computing project had utilized MPI, as it had previously been reserved only for supercomputers.^[169] SMP1 represented a landmark in the simulation of protein folding.^[168] Although the clients performed well in Unix-based operating systems such as Linux and Mac's OS-X, they were particularly troublesome in Windows.^[168][169] On January 24, 2010, SMP2, the second generation of the SMP clients and the successor to SMP1, was released as an open beta and replaced the complex MPI with a more reliable thread-based implementation.^[121][170]

SMP2 also supports a trial of a special category of "bigadv" Work Units, designed for simulating proteins that are unusually large and computationally intensive but have a great scientific priority. These units originally required a minimum of eight CPU cores,^[171] but on February 7, 2012 this was increased to sixteen CPU cores.^[172] Compared to standard units run on SMP2, these also require more system resources such as RAM and Internet bandwidth, but users who run these are rewarded with a 20% increase over SMP2's bonus point system.^[173] The bigadv category allows Folding@home to run particularly demanding simulations on long timescales that had previously required the use of supercomputing clusters and could not be performed anywhere else on Folding@home.^[171]

  V7

  A sample image of the V7 client in Novice mode running under Windows 7. In addition to a variety of controls and user details, V7 also presents Work Unit information, such as its state, calculation progress, ETA, credit points, identification numbers, and description.

The V7 client is the seventh and latest generation of the Folding@home client software, and is a complete rewrite and unification of the previous clients for Microsoft Windows, Mac OS X and Linux operating systems.^[174][175] Like its predecessors, V7 can also run Folding@home in the background at a very low priority, allowing other applications to use CPU resources as they need. It is designed to make the installation, start-up, and operation user-friendly for novices, as well as offers greater scientific flexibility than previous clients.^[176] V7 uses Trac for managing its bug tickets so that users can see its development process and provide feedback.^[175] It was officially released on March 22, 2012.^[177]

V7 consists of three elements. The user interacts with V7's GUI, known as FAHControl. It has Novice, Advanced, and Expert user interface modes, and has the ability to monitor, configure, and control many remote folding clients from a single computer.^[175] FAHControl can monitor and direct FAHClient, which runs behind the scenes and in turn manages each FAHSlot (or "slot"). These slots act as replacements for the previously distinct Folding@home computer clients, as they may be of Uniprocessor, SMP, or GPU type. Each slot also contains a core and data associated with it, and can download, process, and upload Work Units independently. The FAHViewer function, modeled after the PS3's viewer, displays a real-time 3D rendering, if available, of the protein currently being processed.^[174]

  Comparison to other molecular systems

Rosetta@home is a distributed computing project aimed at protein structure prediction and is one of the most accurate tertiary structure predictors available.^[178][179] As Rosetta only predicts the final folded state, and not how proteins fold, Rosetta@home and Folding@home address very different molecular questions.^[180] The Pande lab can use the conformational states from Rosetta's software in a Markov state model as starting points for Folding@home simulations.^[181] Conversely, structure prediction algorithms can be improved from thermodynamic and kinetic models and the sampling aspects of protein folding simulations.^[182][183] Thus, Folding@home and Rosetta@home perform complementary work.^[184]

Anton is a special-purpose supercomputer constructed for molecular dynamics simulations. It is unique in its ability to produce individual ultra-long molecular trajectories on biological timescales. These simulations, while computationally expensive, contain more phase space than any one of Folding@home's many shorter trajectories. Like Folding@home, it has also improved several long-held theories of protein folding.^[185] As of October 2011 Anton and Folding@home are the two most powerful molecular dynamics systems,^[186] and Anton has also run individual simulations out to the millisecond range.^[187][188] In 2011 the Pande lab built a Markov state model from a 200-μs Anton simulation. The publication demonstrated that an MSM built from serial data revealed folding information unobtainable with traditional approaches, and that there was little difference between Markov models constructed from Anton's fewer long trajectories or one assembled from Folding@home's many shorter trajectories.^[185] Starting in June 2011 Folding@home began additional sampling of an Anton simulation in an effort to better determine how its techniques compare to Anton's more traditional methods.^[189][190] It is probable that a combination of Anton's and Folding@home's simulation methods would provide a well-sampled simulation over long timescales.^[185]

  See also

• Blue Gene
• Computational biology
• Foldit
• List of distributed computing projects
• Molecular dynamics
• Molecular modeling on GPUs
• Rosetta@home
• Software for molecular modeling
• Storage@home

  Notes

Note 1: Supercomputer FLOP performance is assessed by running the legacy LINPACK benchmark. This short-term testing has difficulty in accurately reflecting sustained performance on real-world tasks because LINPACK more efficiently maps to supercomputer hardware. Computing systems also vary in architecture and design, so direct comparison is difficult. Despite this, FLOPS remain the primary speed metric used in supercomputing.^[191][192] Folding@home measures its work using wall clock time, a more accurate method of determining actual performance.^[193]

  References

  External links

• Folding@home homepage
• Folding@home news blog
• Folding@home forum
• Folding@home statistics
• List of Folding@home publications
• "Futures In Biotech" episode (for newcomers)
• Video of 1.5ms folding of NTL9
• Pande lab's OpenMM molecular dynamics library