There is an ongoing need to develop strategic combinations of therapeutic agents to prevent type 1 diabetes (T1D) or to preserve islet β-cell mass in new-onset disease. Although clinical trials using candidate therapeutics are commonly based on preclinical studies, concern is growing regarding the reproducibility as well as the potential clinical translation of reported results using animal models of human disorders. In response, the National Institutes of Health Immune Tolerance Network and JDRF established a multicenter consortium of academic institutions designed to assess the efficacy and intergroup reproducibility of clinically applicable immunotherapies for reversing new-onset disease in the NOD mouse model of T1D. Predicated on prior studies, this consortium conducted coordinated, prospective studies, using joint standard operating procedures, fixed criteria for study entry, and common reagents, to optimize combined anti-CD3 treatment plus interleukin-1 (IL-1) blockade to reverse new-onset disease in NOD mice. We did not find that IL-1 blockade with anti–IL-1β monoclonal antibody or IL-1trap provided additional benefit for reversing new-onset disease compared with anti-CD3 treatment alone. These results demonstrate the value of larger, multicenter preclinical studies for vetting and prioritizing therapeutics for future clinical use.
An ongoing objective for the treatment of type 1 diabetes (T1D) is to preserve residual islet β-cell survival and function after new-onset disease (1). Clinical trials are commonly based on results of testing candidate therapies in preclinical animal models of disease. However, there is an alarmingly growing concern regarding the reproducibility and clinical relevance of therapeutic agents tested in preclinical models (2–5), as exemplified by surprisingly low rates of reproducibility in animal models of neurologic diseases (2,6). Such discrepancies called for more scrutiny and rigor in the design, execution, and reporting of animal studies (4–6).
This important concern extends to preclinical studies intended to assess therapeutics for preventing T1D or preserving islet β-cell mass in recent-onset disease (3). Some therapies effective in NOD mice, such as anti-CD3 and anti-CD20, have translated to a degree of clinical benefit (7–12). However, other treatments, such as interleukin (IL)-2 plus rapamycin treatment (13), proved ineffective and possibly accelerated disease in human subjects (14). At present, it is uncertain whether such variability in clinical translation of results is due to intrinsic differences in disease mechanisms in NOD mice versus patients with T1D or may be related to the design and rigor of preclinical studies that are usually analogous to single-center pilot clinical trials.
To address these issues, the Immune Tolerance Network and JDRF assembled a preclinical consortium involving four participating academic institutions to evaluate whether rigorous design, execution, and reporting of animal studies leads to increased validation of results achieved in NOD mice. To attain this end, this multicenter consortium collaboratively assesses candidate combinational therapies for their relative efficacy in reversing new-onset disease in the NOD mouse model of T1D. Predicated on prior promising results (15), we set out to determine optimal conditions for using anti-CD3 plus IL-1 blockade to promote disease reversal. That is, in an attempt to guide potential clinical trials, this study formed the rationale for our consortium to refine clinically relevant protocols of combined anti-CD3 plus IL-1 blockade and to determine the efficacy, intersite reproducibility, and durability of combined treatment to reverse new-onset disease in NOD mice.
Research Design and Methods
These studies were undertaken at the University of Florida, La Jolla Institute for Allergy and Immunology, University of Colorado Denver, and Yale University. Specific sites are blinded in data presented and are referred to as sites 1–4.
NOD/ShiLtJ mice were purchased from The Jackson Laboratory, except at performance site 1, where the NOD mice were bred in-house from the NOD/Bdc subline or were purchased from The Jackson Laboratory. All mice were housed under specific pathogen-free conditions and provided bedding and chow that was in standard use at each of the study sites.
Disease Monitoring and Definition
Female NOD mice, 10–26 weeks of age, were monitored for diabetes onset three times per week. Mice were entered into a predetermined treatment group on the second of two consecutive daily blood glucose value (BGV) readings ≥250 mg/dL (day 1 of study). A portable blood glucose monitor was used to monitor morning BGVs of treated mice twice per week and was determined from tail venous blood. The study was run for 60–62 days, at which point mice were killed. At study termination, mice were categorized as cured if their BGV reading was <250 mg/dL or diabetic if >250 mg/dL. Some animals were removed from the study and killed before day 60–62 if their body weight decreased below levels permitted by the local institutional animal care and use committees (IACUCs) or three consecutive maximum BGV readings as permitted by each site's IACUC.
Treatment With Antibodies and Fusion Proteins
Hamster anti-mouse CD3 145-2C11 monoclonal antibody (mAb) F(ab)2 fragments (BioXCell, West Lebanon, NH) or control hamster F(ab)2 fragments (BioXCell) were injected intraperitoneally (i.p.) for 5 consecutive days (5 µg/day). This dose was less than that used in previous studies of anti-CD3 mAb (15) to reduce potential toxicity and to enhance the identification of synergistic effects with anti–IL-1 agents. When indicated, anti–IL-1β mAb and isotype antibodies (a gift from Novartis) were given at a dosage of 75 µg/day i.p. Alternatively, the fusion protein IL-1trap (Rilonacept; a gift from Regeneron) was injected at 60 mg/kg i.p.
Study Trial Design and Enrollment
Two separate study trials were performed as follows:
Study 1: Anti-CD3 Plus Anti–IL-1β Antibody Treatment
To minimize the effects of subjective bias and to ensure a uniform population of treatment groups over time, each center enrolled animals sequentially in a revolving format into treatment arms. With the exception of group 2, recipients also received one LinBit sustained-release insulin pellet (LinShin Canada, Inc., Toronto, Ontario, Canada) implanted subcutaneously and were allocated to the following groups:
Group 1: Simultaneous anti-CD3 (5 µg/day) plus anti–IL-1β (75 µg/day) (day 1, 3, and 5)
Group 2: Simultaneous anti-CD3 (5 µg/day) plus anti–IL-1β (75 µg/day) (day 1, 3, and 5); no insulin
Group 3: Early (day 1, 3, and 5) anti-CD3 (5 µg/day) treatment, followed by delayed (day 5, 7, and 9) anti–IL-1β treatment (75 µg/day)
Group 4: Anti-CD3 (5 µg/day) plus early isotype control antibody (day 1, 3, and 5)
Group 5: Anti-CD3 (5 µg/day) (day 1, 3, and 5), followed by delayed isotype control antibody (day 5, 7, and 9)
Group 6: Simultaneous control F(ab)2 plus isotype control antibody (day 1, 3, and 5)
Group 7: Early control F(ab)2 (day 1, 3, and 5) plus delayed isotype control antibody (day 5, 7, and 9)
Group 8: Early anti–IL-1β alone (75 µg/day) (day 1, 3, and 5)
Group 9: Delayed anti–IL-1β alone (75 µg/day) (day 5, 7, and 9).
Study 2: Pilot Study of Anti-CD3 Plus IL-1trap (Rilonacept) Treatment
In a second abbreviated pilot study approach, center sites 1, 2, and 4 tested the efficacy of disease reversal using combined anti-CD3 treatment (5 µg/day) with IL-1 blockade by the fusion protein IL-1trap (60 mg/kg), without exogenous insulin therapy, according to the following treatment groups:
Group 1: Untreated diabetic control mice
Group 2: Early anti-CD3 alone
Group 3: Simultaneous anti-CD3 plus IL-1trap treatment
Group 4: Early anti-CD3 treatment with delayed IL-1trap treatment.
Animals were entered into treatment groups using the same enrollment scheme as described for study 1 above.
The first protocol (anti-CD3 plus anti–IL-1β) was 80% powered to determine a 50% improvement in the reversal of diabetes in the combination versus the anti-CD3 mAb arms using anti–IL-1β mAb. A two-way ANOVA test was used for analyses of parameters at onset. Diabetes incidences were assessed using the log-rank test. Graphs were plotted and statistics calculated with GraphPad Prism 4 and 5 software.
These studies were performed on all sites upon approval of their respective IACUCs.
Standard Operating Procedures
A detailed version of all methods and standard operating procedures (SOPs) for these studies and all individual BGV data can be found at https://www.itntrialshare.org/.
Study 1: Multicenter Study of Combining Anti-CD3 Treatment With Anti–IL-1β mAb Therapy
In this initial trial, this multicenter study combined anti–IL-1β mAb therapy at varied timing (simultaneous or delayed) relative to a fixed anti-CD3 treatment. We evaluated nine treatment groups that included placebo controls for anti-CD3 (control F[ab]2 fragments) and/or for anti–IL-1β (isotype control antibody). In addition, we determined whether exogenous insulin treatment early after disease onset affected the efficacy of disease reversal by simultaneous anti-CD3 plus anti–IL-1β mAb treatment (groups 1 and 2).
Anti-CD3 but Not Anti–IL-1β mAb Results in Significant Disease Reversal at All Sites
Because the age of onset or the severity of initial hyperglycemia could possibly contribute to variability in the ability to reverse new-onset disease, initial BGV and the age of new-onset disease were determined for enrolled animals. We found no significant difference in these initial disease characteristics between treatment groups or across centers (Supplementary Fig. 1). Thus, neither the baseline age nor the BGV of new-onset NOD mice were considered to bias study results.
Relative to isotype control-treated groups, anti-CD3 treatment combined with isotype control mAb led to significant early and sometimes sustained disease reversal in a proportion of animals at all sites (P < 0.05 for each site; Fig. 1). Interestingly, although low-dose anti-CD3 treatment resulted in significant disease reversal, this effect generally decayed over the 60-day observation period with a gradual reversion to hyperglycemia. This gradual decline in disease reversal is readily apparent when viewing the pooled composite data of anti-CD3–treated animals from all sites (Fig. 2B and Supplementary Fig. 2). In contrast, anti–IL-1β mAb treatment without anti-CD3 mAb did not result in significant disease reversal at any site (Supplementary Fig. 3). That is, although exogenous insulin therapy alone via an implanted insulin pellet resulted in transient reversion from hyperglycemia in most animals, neither early nor delayed anti–IL-1β mAb treatment without anti-CD3 led to prolonged disease reversal at any treatment site, consistent with previous results (15).
Intrasite Consistency but Intersite Variability of Anti-CD3 Treatment With Simultaneous or Sequential Administration of Isotype Antibody
Despite considerable effort to generate study uniformity, we nevertheless saw surprising variability in treatment outcomes among the centers (Fig. 2). For example, although all sites demonstrated a clear benefit of anti-CD3 treatment for achieving disease reversal (P < 0.05 for all sites), the actual frequency and duration of successful reversion to euglycemia was greater in sites 2 and 3 relative to site 1 (Fig. 2), regardless of the timing of isotype control antibodies (P < 0.03 in all cases).
IL-1β Blockade by Anti–IL-1β mAb Does Not Improve the Efficacy of Anti-CD3 Treatment
Importantly, despite differences in the rate of disease reversal induced by anti-CD3 found between treatment sites, three of the four participating centers failed to detect significant improvement in disease reversal using combined anti-CD3 plus anti–IL-1β mAb therapy relative to anti-CD3 treatment alone (Fig. 2A). At sites 1, 3, and 4 neither simultaneous nor delayed addition of anti–IL-1β mAb improved rates of disease reversal from relative to anti-CD3 plus isotype control mAb (P = NS). At site 2, however, anti-CD3 plus sequential anti–IL-1β mAb treatment showed significantly improved disease reversal relative to simultaneous anti-CD3 plus anti–IL-1β mAb (P = 0.02) or to sequential anti-CD3/isotype control mAb (P = 0.01). Nevertheless, results of composite data pooled from all centers indicate a clear distinction in the rate of disease reversal based only on the presence or absence of anti-CD3 treatment, whether or not anti–IL-1β mAb is added (Supplementary Fig. 2).
Effect of Insulin Therapy on Disease Reversal
Another important issue is whether exogenous insulin therapy given to relieve potential hyperglycemic-related islet injury (16) improved the short- or longer-term efficacy of immunotherapies. This was tested in animals receiving simultaneous anti-CD3 and anti–IL-β mAb, with or without insulin therapy (groups 1 and 2). Early exogenous insulin treatment greatly improved the initial success of achieving disease reversal at sites 1, 2, and 4 (Fig. 3A). However, this benefit of insulin treatment waned over time. That is, none of the performing sites observed differences in long-term (60 days) disease reversal, with or without early insulin treatment (Fig. 3A), as was also reflected in the pooled data from all sites (Fig. 3B and Supplementary Fig. 2).
Study 2: Pilot Study of Anti-CD3 Plus IL-1trap (Rilonacept) Treatment
Given the unexpected inability of anti–IL-1β mAb therapy to improve the efficacy of anti-CD3 treatment, a second trial was conducted to determine if a different approach to IL-1 blockade would improve disease reversal. In this case, we used the IL-1 receptor fusion protein IL-1trap (17) as an agent to inhibit IL-1 in vivo in new-onset NOD mice. However, in this second trial, we opted to perform a more targeted pilot study approach in which we focused on the key question of whether the addition of simultaneous or delayed IL-1trap treatment to anti-CD3 therapy would improve the rate of disease reversal in new-onset NOD mice relative to anti-CD3 treatment alone. Three sites (1, 2, and 4) performed this second trial (Fig. 4). Results from this pilot study indicated that neither simultaneous (Fig. 4C) nor delayed (Fig. 4D) IL-1trap treatment improved the rate of disease reversal relative to anti-CD3 alone at any site (Fig. 4B). Although all treatment groups showed significant disease reversal relative to untreated control animals (P < 0.04 for all groups), neither simultaneous nor delayed IL-1trap treatment improved disease reversal relative to anti-CD3 alone (P = NS). Again, although results showed intersite variability, composite results clearly indicated that IL-1trap did not improve disease reversal relative to anti-CD3 treatment alone (Fig. 4E).
We established a multicenter, preclinical consortium to test the efficacy of and intersite reproducibility of candidate combinations of therapeutic agents for their ability to reverse new-onset disease in the NOD mouse model of T1D. Importantly, this group used joint SOP methodologies and shared therapeutic agents and oversight to perform robust, prospective animal studies (18). A key translational goal of this program is to scrutinize clinically applicable therapeutic agents in the NOD model to guide clinical trial design for the preservation of islet β-cell mass in patients with new-onset T1D. Our initial goal was to determine if the timing of IL-1 blockade relative to anti-CD3 therapy had an effect on the efficacy of reversing disease with this combined therapeutic approach. This issue has direct practical relevance for guiding clinical trials that might be performed using a combination of agents. For example, studies in NOD mice indicate that adjunct rapamycin treatment can impair (19) or improve (20) the efficacy of anti-CD3 treatment, depending on the timing of rapamycin treatment relative to anti-CD3 therapy. Although significant reversal of disease occurred with anti-CD3 mAb at all sites, we found limited improvement with additional anti–IL-1β mAb treatment, with only one treatment group at one site suggesting efficacy for this combination. This latter result is significant and illustrates the potential bias toward positive results that has been a significant concern raised in a number of commentaries regarding smaller, single-center small-animal studies (2,4,6). Interestingly, site 2 found that delayed (but not simultaneous) anti–IL-1β mAb could improve disease reversal; however, this was not observed at other sites (Supplementary Fig. 1). Thus, one subgroup suggesting a positive effect of adjunct anti–IL-1β with anti-CD3 was tempered by the results from a larger sample size and multiple performance sites.
Several noteworthy features of the study design differ from practices used in most preclinical studies in this model. We designed the study to involve robust sample sizes and multiple sites that used the same reagent lots and SOPs for designation of diabetes, drug dosing, and an animal enrollment scheme designed to ensure unbiased allotments to study groups. In total, data from these studies did not independently support the use of IL-1 blockade as an adjunct therapy to improve the efficacy of disease reversal achieved by anti-CD3. We must emphasize that although we used the same reagents used in a prior study combining anti-CD3 and anti–IL-1β in new-onset NOD mice (15), this study was not designed to directly replicate these earlier results. Rather, we intentionally reduced anti-CD3 dosing to alleviate potential dose-related toxicity and to increase the likelihood of achieving synergism with additional IL-1 blockade. We also varied the timing of anti–IL-1β mAb relative to anti-CD3 treatment as a means of potentially guiding the development of future clinical studies. Importantly, if subtle differences in the treatment regimens using the same agents between studies can have a major effect on disease reversal in the NOD mouse model, this issue should arguably create pause in proceeding with clinical trials without further preclinical testing.
Finally, although anti-CD3 mAb alone consistently showed efficacy at all sites, there was nevertheless a substantial variability in the rate of successful disease reversal between sites, with or without IL-1 blockade. This was not due to any detectable differences among the sites in the initial age or degree of hyperglycemia in new-onset NOD mice (Supplementary Fig. 1). At present, we cannot identify specific factors that influenced intersite variability. However, some potential differences among the sites may include subtle differences between NOD sublines (e.g., between the NOD/ShiLtJ and NOD/Bdc lines) and housing or diet conditions with a subsequent effect on disease pathogenesis, such as potential differences in the NOD colony microbiome (21,22). Similar variability in outcomes has been observed in other animal models of human disease (23). However, despite some intersite variability, we did not note significant differences in outcome trends among the centers. That is, anti-CD3 alone was significantly effective at reversing disease at all sites (albeit to differing degrees), and adjunct IL-1 blockade provided little or no additional benefit at any site. In our view, the variability in results found among the sites enhances the rationale for this type of multicenter, preclinical approach to testing therapeutics in NOD mice. We would assert that validating results among sites enhances the confidence in a specific study in NOD mice and greatly lessens the likelihood of a false-positive (or -negative) outcome based on a small, single-center study.
Acknowledgments. The authors thank Dr. Hermann Gram (Novartis) and Dr. Desper Gromada (Regeneron) for kindly providing anti–IL-1β mAb and IL-1trap reagents, respectively. We also thank Dr. Richard Insel (JDRF) for careful review of the manuscript and Dr. Katarzyna Bourcier (NIAID), Dr. James McNamara (NIAID), Dr. Julia Greenstein (JDRF), and Barbara Droker (BRI) for helpful discussion.
Funding. Research reported in this article was performed as a project of the ITN-JDRF Type 1 Diabetes Preclinical Consortium supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under award number UM1AI109565 and JDRF grant 9-2012-22 and the National Institute of Diabetes and Digestive and Kidney Diseases.
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Duality of Interest. No potential conflicts of interest relevant to this article were reported.
Author Contributions. R.G.G., M.A.A., K.C.H., and M.v.H. designed studies and prepared and reviewed the manuscript. P.P.P. performed studies, prepared data, and prepared the manuscript. T.K. researched and prepared data and reviewed and edited the manuscript. C.H.W. designed studies and prepared data and reviewed and edited the manuscript. S.D., A.P., Y.M., and A.B.H. researched and prepared data. L.S. was involved in project management and data analysis. P.B. organized data and reviewed and edited the manuscript. T.S. designed studies and contributed to discussion. M.R.E. designed studies and reviewed the manuscript. G.T.N. designed studies and reviewed and edited the manuscript. R.G.G., M.A.A., K.C.H., and M.v.H. are joint guarantors of this study and, as such, had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
This article contains Supplementary Data online at http://diabetes.diabetesjournals.org/lookup/suppl/doi:10.2337/db15-0492/-/DC1.
See accompanying article, p. 1161.
- Received April 12, 2015.
- Accepted December 22, 2015.
- © 2016 by the American Diabetes Association. Readers may use this article as long as the work is properly cited, the use is educational and not for profit, and the work is not altered.