DOI: 10.2337/db05-1635 © 2006 by the American Diabetes Association
Diabetic RetinopathySeeing Beyond Glucose-Induced Microvascular Disease
1 Department of Cellular and Molecular Physiology, Penn State College of Medicine, Hershey, Pennsylvania Address correspondence and reprint requests to Thomas W. Gardner, Departments of Ophthalmology and Cellular & Molecular Physiology, Penn State College of Medicine, 500 University Dr., HU19, Hershey, PA 17033. E-mail: tgardner{at}psu.edu
Abbreviations:
DCCT, Diabetes Control and Complications Trial; IRS, insulin receptor substrate; PI, phosphatidylinositol; PKC, protein kinase C; VEGF, vascular endothelial growth factor
Diabetic retinopathy remains a frightening prospect to patients and frustrates physicians. Destruction of damaged retina by photocoagulation remains the primary treatment nearly 50 years after its introduction. The diabetes pandemic requires new approaches to understand the pathophysiology and improve the detection, prevention, and treatment of retinopathy. This perspective considers how the unique anatomy and physiology of the retina may predispose it to the metabolic stresses of diabetes. The roles of neural retinal alterations and impaired retinal insulin action in the pathogenesis of early retinopathy and the mechanisms of vision loss are emphasized. Potential means to overcome limitations of current animal models and diagnostic testing are also presented with the goal of accelerating therapies to manage retinopathy in the face of ongoing diabetes. Despite years of clinical and laboratory investigation, diabetic retinopathy remains the leading cause of vision impairment and blindness among working-age adults, yet the fundamental cause(s) remains uncertain. Retinal photocoagulation to reduce neovascularization and macular edema was developed in the 1950s and is still the standard of care (1). The number of people worldwide at risk of developing vision loss from diabetes is predicted to double over the next 30 years (2), so it is imperative to develop better means to identify, prevent, and treat retinopathy in its earliest stages rather than wait for the onset of vision-threatening lesions. Progress in these areas requires a new perspective on the problem that includes the roles of the neural retina, impaired insulin action, and inflammation. In this way, established neurobiological principles can inform us how diabetes impairs vision, and knowledge of metabolism, inflammation, and regenerative medicine may lead to new treatments. This perspective will discuss how the unique anatomy and physiology of the retina may render it vulnerable to the metabolic derangements of diabetes and lead to impaired vision. The intent of this unconventional approach is to encourage consideration of new opportunities for investigations that will advance the field.
Topographic and cellular organization of the retina. It is instructive to consider the functional organization of the retina (literally a network) to better understand the impact of diabetes (http://webvision.med.utah.edu ). The retina is a transparent layer of neural tissue between the retinal pigmented epithelium and the vitreous body. Normal vision depends on intact cell-cell communication among the neuronal, glial, microglial, vascular, and pigmented epithelial cells of the retina. The fundamental functions of the retina are to capture photons, convert the photochemical energy into electrical energy, integrate the resulting action potentials, and transmit them to the occipital lobe of the brain, where they are deciphered and interpreted into recognizable images. The retina is partitioned from the systemic circulation by the blood-retinal and blood-aqueous barriers and receives its nutritional supply from the retinal and choroidal circulations and perhaps from the ciliary body by diffusion through the vitreous gel (3). As discussed below, the unique features of retinal anatomy and physiology that allow it to function so efficiently may also be its Achilles heel under the stressful metabolic conditions of diabetes.
The retina operates in bright light to near darkness, so photoreceptor sensitivity varies over nine orders of magnitude (4). The macula operates in moderate- to bright-light conditions to subserve detailed acuity and color perception, whereas the peripheral retina operates in dim-light conditions, detects motion, and peripheral vision (Fig. 1A). The macula,
The macula contains cone and rod photoreceptors; the center of the macula contains the fovea (pit), a 1-mm cone-dominated depressed region, and the central-most 200-µm foveola is specialized for highest spatial resolution. The foveola contains only cone photoreceptors (first-order neurons), and the inner retinal cells are displaced so as not to interfere with light transmission. Outside the fovea, rod photoreceptors dominate and second- and third-order neurons (bipolar, amacrine, and ganglion cells) are present. The lamellar cellular architecture of the retina (Fig. 1B) has alternating layers of neurons (outer and inner nuclear layers and ganglion cell layer) interposed with two plexiform layers, where neurons communicate at synapses between dendrites and between axons and dendrites. The retina includes five major cell types that perform sensory, regulatory, nutritional, and immunomodulatory functions. The neurons (photoreceptors, bipolar, horizontal, amacrine, and ganglion cells) perform sensory functions and define color perception, spatial resolution, and contrast discrimination (6). Müller cells and astrocytes, two types of glial (glue) cells, provide nutritional and regulatory support for neurons. Müller cells span the retina from the pigment epithelium to the internal limiting membrane, a basement membrane formed by Müller cell end-feet that interfaces with the vitreous gel. Müller cells contact neurons and blood vessels in the plexiform and nerve fiber layers, and astrocytes envelop blood vessels in the nerve fiber and ganglion cell layers and contact ganglion cells and amacrine cells. Müller cells and astrocytes convey substrates, including lactate and amino acids, from the circulation to neurons and regulate blood-retinal barrier properties (7) and synaptic function (8). Müller cells also store glycogen for conversion to lactate, synthesize retinoic acid from retinol, regulate extracellular ion concentrations to modulate plasma membrane polarization/depolarization, participate with neurons in the glutamate/glutamine cycle to control neurotransmission, and protect neurons from glutamate excitotoxicity (9). Glial cells are the interface between the neurons and the vasculature and are thus key regulators of neuronal nutrition and metabolism. The pigmented epithelial cell layer serves also as a selective conduit of substrates as the outer blood-retinal barrier and allows oxygen diffusion from the choroidal circulation to the outer retina. It removes retinal lactic acid and phagocytoses shed photoreceptor outer segments, comprises the outer blood-retinal barrier, absorbs light, secretes trophic factors such as pigment epithelium–derived factor, and, with the photoreceptors, participates in the cycling of the vitamin A isoforms retinol and retinal (10). Thus, this epithelial cell layer plays a crucial role in vision, although its role in diabetic retinopathy is not clear. Immunomodulatory functions are provided by a fourth class of cells, microglia. They are a heterogenous population of resident macrophages that monitor the local environment by interacting with neurons, glia, and endothelium and that react to stresses, including infection, trauma, and retinal detachment, by release of proinflammatory cytokines (11) and clearance of necrotic or apoptotic cells via phagocytosis (12). Microglial cells become activated and help to resolve local injury (13,14), but unrelenting stresses cause persistent inflammatory responses. The fifth class of cells includes vascular endothelial cells and pericytes. They provide nutritional support and waste product removal for the inner retina and have been the focus of much research in diabetic retinopathy. It is likely that their function depends on as-yet-undefined signals from the neural retina. Blood vessels are the only structures that are visible by clinical examination because they convey erythrocytes containing the visible pigment hemoglobin. Despite their prominent appearance by clinical examination, the vasculature constitutes less than 5% of the retinal mass, so the retina is a vascularized neural tissue (15).
Retinal physiology may underlie its vulnerability to diabetes. The retina is further specialized because, like the brain, its metabolic functions are compartmentalized between glia and neurons. In glia of the inner retina, glucose metabolism occurs mostly via glycolysis, whereas cells of the outer retina fully oxidize glucose to CO2 and water via oxidative phosphorylation (24). In the inner retina, metabolic substrates, such as glucose, flow from vascular endothelium to astrocytes to neurons. In the outer retina, substrates reach Müller cells and photoreceptors from the choroid via the pigmented epithelium. Thus, glial cells are vital to neuronal function because they convey circulating glucose into the retina for ATP production and provide intermediary compounds such as lactate. The functional anatomy of the retina is illustrated in Fig. 2.
Taken together, these findings strongly suggest that the unique anatomic and physiologic specialization required for vision demands intact cell-cell communication. This specialization may, in turn, predispose the retina to diabetes-induced damage if the metabolic derangements typical of diabetes interfere with the generation of neurotransmitters, macromolecule synthesis, or induce proapoptotic or proinflammatory responses.
Numerous investigators have suggested that the pathogenesis of diabetic retinopathy includes glucose-mediated microvascular damage. Previously implicated pathways related to excess glucose include oxidative stress, activation of protein kinase C (PKC), and activation of advanced glycation end products and their receptor (25–29). Mechanisms of vascular injury include increased vascular permeability due to tight junction disassembly (30) and endothelial cell–mediated leukostasis (31). These studies have led to potential therapeutic approaches, including PKC inhibitors, corticosteroids, and soluble receptor for advanced glycation end product inhibitors (25); of these, PKC inhibition has been shown to be effective in randomized clinical trials (32). However, recent work strongly suggests that diabetic retinopathy involves more than elevated glucose and microvascular lesions, so evidence for alterations of the neural retina and insulin action are presented below.
Evidence for neural retinal involvement in diabetic retinopathy. At the cellular level, diabetes alters the function and structure of all retinal cell types. Postmortem human diabetic retinas exhibit increased markers of apoptosis in ganglion cells (44). Animal studies show accelerated apoptosis of retinal neurons (45,46), glial activation (47–49), impaired glial cell metabolism (50–52), and microglial cell activation (11,53,54). Together, these studies leave little doubt that neural retinal defects are among the earliest detectable changes in diabetes. Regardless of whether the initial events begin in blood vessels or neural cells, the clinical stages of diabetic retinopathy manifest cellular, histologic, and functional features of a retinal neuropathy (15,55,56). To the best of our knowledge, there is no evidence that a primary, selective defect in vascular cells is sufficient to cause diabetic retinopathy. Clearly, it is essential to treat both the vascular and neural elements of the retina to preserve vision. This concept permits a new paradigm for understanding the mechanism of vision impairment in diabetes and provides therapeutic targets that are directly linked to vision (57,58).
A feed-forward concept of diabetic retinopathy involving neural and vascular lesions. If diabetes exerts its primary damage on vascular cells and increases permeability or vascular occlusion, neuronal and glial cell integrity would be compromised by the entry of circulating macrophages, antibodies, inflammatory cytokines/chemokines, excitotoxic amino acids, or fatty acids into the retina. On the other hand, if diabetes primarily affects the neural retina, it could compromise vascular integrity by loss of normal barrier-inducing functions of glia or increased expression of proinflammatory cytokines or reactive oxygen species that promote vascular leakage or occlusion. At this point, it is not known whether vascular or neural cell defects occur first; most likely they are interdependent. Thus, we propose a feed-forward concept of vascular-neural dysfunction that begins shortly after the onset of diabetes and increases over time, creating a widening vortex of retinal injury (Fig. 3). Eventually, accumulated injury and failing reparative responses lead to the clinically evident features of diabetic retinopathy.
Inflammation in diabetic retinopathy. Inflammation is a prominent component of many diseases, including primary retinal degenerations, insulin resistance, and diabetes (59). Diabetic retinopathy was initially termed, "diabetic retinitis" (60), although this usage fell out of favor in the 1970s. Chronic inflammation is characterized by increased vascular permeability, edema, inflammatory cell infiltration, cytokine and chemokine expression, tissue destruction, neovascularization, and attempts at repair (61), and diabetic retinopathy exhibits most of these features (Table 1). Microglia associate intimately with neurons that express molecules such as CX3CL1 (fractalkine) (62) and CD200 (63) that negatively regulate microglial activation through their respective receptors. As such, perturbation of expression of ligand or receptor during stress would activate microglia to produce proinflammatory cytokines and acquire an activated morphology (11,64). Activated microglia produce chemokines such as monocyte chemoattractant protein-1 (11), inducing expression of adhesion molecules, which can promote the leukostasis of neutrophils, on endothelium (65) and potentially inducing the extravasation of inflammatory macrophages (66). In addition, elevated levels of complement and reduced levels of complement inhibitors (67) and acute-phase proteins (68) are likely key events in the phagocytic clearance of necrotic and apoptotic neurons (12). Recent evidence strongly suggests that inflammation involving vessels and neural tissue occurs early in experimental (11,64,65,67,69) and human (70) retinopathy, involving humoral and cellular components of innate immunity.
Physiologic repair processes that help retinal cells survive stress include increased expression of numerous growth factors and cytokines, including vascular endothelial growth factor (VEGF), IGF-1, interleukin-1ß, and tumor necrosis factor- (rev. in 71). These proteins, which have been implicated in the development of retinopathy, also provide neurotrophic functions to support retinal cell survival (71–73). Diabetes-induced upregulation of these growth factors may be an attempt to reestablish a prosurvival-state equilibrium while simultaneously altering the blood-retinal barrier and inducing inflammatory responses (Fig. 4). In the short term, increased cytokine/chemokine expression may serve an adaptive function to maintain neuronal function but, over time, becomes maladaptive, with progressive vascular damage, ultimately resulting in macular edema and neovascularization. The feed-forward process thereby perpetuates both vascular and neuronal damage and culminates in the clinical features of diabetic retinopathy.
In this light, the issue is not simply whether diabetes incites a retinal inflammatory process but whether the inflammatory response contributes to initiation, propagation, or resolution of the injury. Impaired wound healing is a long-recognized feature of diabetes, and recent evidence indicates that diabetic animals have impaired reparative responses to central nervous system metabolic stress. For example, hypoxic/ischemic stress in normal mice brain increases expression of the antiapoptotic genes bcl-2 and blf-1 in microglial cells. However, diabetic db/db mice do not mount this reparative response and have larger brain infarcts than control animals (74). Thus, the normal inflammatory response is designed to limit tissue injury, and diabetes may disrupt the ability of tissues to respond appropriately and recover. Further characterization of the temporal sequence and balance of these injurious and reparative responses in the retina is required to understand how to minimize tissue injury. Therefore, if the upregulation of growth factors and cytokines, such as VEGF, serves to support cell survival during diabetes, it is possible that their inhibition by pharmacologic means could aggravate retinal neuronal or vascular cell death while apparently improving the clinical picture if the retinal cells that express the growth factors/cytokines are dying (75). Nevertheless, from a therapeutic point, it is likely that identifying and halting key steps of the inflammatory process contributing to tissue destruction will help to control retinopathy. Taken together, recognition of how diabetes disrupts the entire retina also illustrates how diabetes alters the parenchyma of other tissues, including adipose, skeletal muscle, kidney, heart, brain, peripheral nerve, and bone. In fact, the concept of "microvascular" complications does not adequately describe diabetic retinopathy, nephropathy, or neuropathy and should be reevaluated.
Evidence for impaired insulin action in diabetic retinopathy. The clinical case for a central role of glucose is based on the strong statistical association between HbA1c (A1C) levels and retinopathy (78–80). However, data from the Diabetes Control and Complications Trial (DCCT) also show a lower risk of retinopathy for the same A1C levels in patients who received intensive control compared with those who received conventional treatment (81). Thus, either the consistency of euglycemia and/or fewer periods of relative insulin deficiency may account for the benefits of intensive therapy. In the DCCT, the reduced risk of retinopathy was achieved by giving insulin more frequently to patients in the intensive-treatment groups (82). Hence, the DCCT might reasonably be viewed as an insulin dose-response trial (83). Further clinical evidence for a role of insulin is that systemic insulin resistance is a risk factor for retinopathy in patients with type 1 diabetes in the DCCT (84), the EURODIAB study (85), and the Pittsburgh Epidemiology of Diabetes Complications study (86). In type 2 diabetes, insulin deficiency is also an independent risk factor for the presence of retinopathy (87,88). These data strongly suggest a clinical role for systemically administered insulin and raise the question of whether the retina, like other tissues, is a direct target of insulin.
Constitutive retinal insulin receptor signaling is reduced in diabetes.
Diabetes disrupts insulin signaling in peripheral tissues. Recent experimental diabetes causes rapid and progressive loss of the constitutive retinal insulin receptor kinase activity and downstream PI 3-kinase
Reduced insulin action leads to neurodegeneration. The findings that diabetes impairs insulin receptor signaling in retina, brain (106–109), and peripheral nerve (110), as well as in classic insulin-sensitive tissues, suggest that the metabolic impact of diabetes on "complications"-prone tissues and peripheral tissues forms a continuum. Whereas muscle and adipose respond acutely to fluctuating insulin levels and change rapidly after diabetes onset, retina and brain insulin action has a higher set point and responds less rapidly after diabetes onset. Plasma insulin penetrates the eye and brain more slowly than peripheral tissues, and with saturable kinetics (111–114). Thus, it is important to consider the role of impaired insulin action in the development of complications, even in tissues where insulin does not regulate glucose uptake. "Complications," then, are the manifestations of altered diabetes metabolism in organs that are associated with clinical impairment.
Retinopathy in the absence of or before onset of hyperglycemia. Overall, the data support the idea that impaired insulin action, the primary defect of diabetes, directly impacts the retina and may initiate retinal dysfunction. This leads us to question 1) whether this change results from loss of pancreatic insulin and 2) whether retinopathy is due to this alteration alone or if retinopathy results from combinatorial insults. Potential factors that may alter the retinal homeostatic equilibrium include glucose, lipids, hypertension, counterregulatory hormones (glucagon, glucocorticoids, adipokines), or inflammation due to insulin resistance.
Impaired vision in patients with diabetes is most frequently associated with macular edema, macular ischemia, epiretinal membranes that distort or elevate the macula, or vitreous hemorrhages that obscure the ocular media. For example, leakage of retinal capillaries contributes to macular edema, and it is generally thought that the clinically apparent vascular leakage accounts for the impaired vision. However, the specific cellular mechanisms by which macular edema reduces visual acuity have not been well defined. From an optical perspective, macular cysts scatter light within the retina (T.W.G., personal observation) such that it is not focused on photoreceptors, decreasing the image quality. From a cellular standpoint, visual function could decline if fluid accumulation within the retina 1) alters extracellular ionic concentrations required for action potentials, 2) physically compresses retinal neurons, 3) compromises the normal interchange of glutamate and glutamine between glial cells and neurons required for neurotransmission, or 4) neurons have increased susceptibility to excitotoxic amino acids, antibodies, or inflammatory cells that reach the retina via leaking vessels. Vascular leakage could thus impair neuronal function and vision in the absence of clinically detectable macular edema. Occluded capillaries near the fovea may also cause retinal neurons to suffer ischemic damage. Table 2 describes possible mechanisms of vision impairment in diabetes.
Direct diabetes damage to glial cell or neuronal metabolism would directly impact neurotransmission (38,122) and may lead to apoptosis of retinal neurons and visual field defects. Indeed, retinal axons are lost before the onset of visible vascular lesions (123,124). Recent reports also demonstrate that impaired local responses on multifocal electroretinograms predict subsequent development of vascular lesions (125). Vision depends on neuronal function, so in the final analysis, all forms of vision impairment with clear ocular media (macular edema, macular ischemia, traction retinal detachment) must, by definition, include neuronal dysfunction. Further work is needed to determine how alterations in vascular, glial, microglial, and neuronal cell interactions reduce the quality of vision.
Retinopathy is diagnosed clinically with the onset of ophthalmoscopic signs such as microaneurysms, hemorrhages, and cotton-wool spots, but as discussed above, functional defects often precede these signs. If diabetic peripheral neuropathy is diagnosed on the basis of subtle electrophysiologic and sensory defects before the onset of clinical symptoms, should retinopathy not be categorized before the onset of vascular lesions? This interval should provide the best window of opportunity for treatment while visual function remains intact. Individuals who have not developed clinically evident retinopathy represent the greatest therapeutic opportunity to preserve vision because they constitute the majority of the patients and respond better to intensive therapy (79). Therefore, development of sensitive, predictive measures of retinal function that could be employed in practice and clinical trials would have great utility.
Diabetic retinopathy remains a plague of modern society. Advanced vascular lesions of diabetic retinopathy have been treated by destructive panretinal photocoagulation and vitrectomy surgery for the last 5 decades. In comparison, peptic ulcer disease was once a common cause of disability, at a time when the diagnosis depended on radiographic studies and treatment was surgical repair of bleeding ulcers. Endoscopy now allows early diagnosis, and research has elucidated the role of gastric Helicobacter pylori infections and acid overproduction; antibiotic and acid-blocking medications have substantially reduced the mortality and morbidity of peptic ulcers. We are looking for opportunities to make detection and treatment paradigms as effective as those for peptic ulcer disease. Of the major blinding retinal diseases (macular degeneration, retinitis pigmentosa, and diabetic retinopathy), retinopathy is the only one for which a specific medical intervention (intensive insulin therapy) has been shown to retard both its development and progression. Improved medical care over the past 3 decades has reduced the risk of vision-threatening retinopathy (126); therefore, there is ample reason for optimism that retinopathy can be prevented. Nevertheless, several factors limit the progress toward prevention or cure of retinopathy in the face of imperfectly controlled diabetes. A key step includes development of clinically practical means to diagnose retinal dysfunction before the onset of vascular lesions and symptomatic vision loss. Retinal function tests that can be utilized clinically, such as short-wave automated or frequency-doubling perimetry or dark adaptation, might predict disease onset and progression. Development of such tests could shorten the duration of therapeutic clinical trials by providing end points acceptable to the U.S. Food and Drug Administration, unlike the current threshold of three lines of visual acuity change or three-step change on the Early Treatment of Diabetic Retinopathy Study retinopathy scale. The models available for in vivo retinopathy studies also restrain progress. Although the retina is easily visible, it is, ironically, the only major tissue affected by diabetes that cannot be biopsied in humans. Vitreous samples from patients with proliferative retinopathy reflect late-stage changes but are not readily available in the early stages of disease. Rodent models of diabetes, mostly streptozotocin-induced rats and mice (127) and Ins2Akita mice (92), reveal retinal vascular and neural alterations. These models are relatively inexpensive and can provide detailed molecular information, including acute and chronic changes in gene expression and cell signaling. Conditional gene knockout studies in mice could provide substantial insights by mimicking or altering disease progression. However, evolutionary distance from humans and inbreeding may not accurately predict disease in genetically diverse humans. For example, the roles of the polyol pathway and aldose reductase inhibitors predicted by rodent studies were not borne out by human studies of sorbinil or tolrestat. The small eyes of rodents also do not have maculas or permit validation of pharmacokinetic data needed to develop drug delivery devices for patients. Therefore, investment to determine the potential of large animals such as pigs or nonhuman primates for retinopathy research may be efficient and cost-effective over the long term. Clinical and histopathologic signs of retinopathy develop in type 2 diabetic monkeys (128–131), but further investment is needed to develop large animal models of type 1 and type 2 diabetes for complications research (132). Parallel studies in humans should permit correlation of the systemic metabolic profiles and retinal phenotypes of animal models so the information gained in animals can be translated more readily to human studies. The development of new treatments for retinopathy is also hampered by the slow course of the clinical vascular changes. However, evidence that a ruboxistaurin, a PKC inhibitor (32), and pancreatic transplants (133) can modify the course of established retinopathy indicates that progression of the disease can be modified. Intensive insulin treatment is the ideal means to prevent and treat mild retinopathy, but hypoglycemia limits the ability of many patients to achieve the degree of control needed to prevent retinopathy (134). Most patients are not candidates for pancreatic transplantation, and islet cell or stem cell transplants are not widely available. The key to future treatments lies in elucidating the biological processes that lead to retinopathy. Therapeutic strategies may involve dual arms that 1) identify and augment deficient neurotrophin pathways and 2) inhibit key proinflammatory/proapoptotic pathways to restore normal vascular and neuronal function (Fig. 4). This dual approach may lead to improved management of retinopathy in the face of continued diabetes.
The intent of this perspective is to synthesize recent clinical and laboratory findings and to engender new approaches to the diagnosis and treatment of diabetic retinopathy, with increased emphasis on prevention and early intervention. Better understanding of the biology of diabetic retinopathy will require genetic dissection of the molecular pathways in rodents, determination of the interrelationships between vascular and neural cells, and proof-of-principle studies in large animal models. Additional work is required to understand how the early cellular and biochemical mechanisms of retinal injury and stress responses impair vision, which molecular components can be targeted for therapeutic intervention, and how best to introduce drugs to the retina with minimal systemic impact. Enhanced collaboration between groups of laboratory and clinical investigators with better-defined criteria for moving drugs tested in animals into clinical trials will accelerate preservation of vision in people with diabetes.
Supported by the JDRF (Juvenile Diabetes Research Foundation) Diabetic Retinopathy Center at Penn State, by grants from the American Diabetes Association (to A.J.B. and T.W.G.), by National Institutes of Health Grants EY12021 and EY16413 (to D.A.A.) and EY015800 (to M.K.), and by the Pennsylvania Lions Sight Conservation & Eye Research Foundation. This project is funded in part under a grant with the Pennsylvania Department of Health using Tobacco Settlement Funds. The Department specifically disclaims responsibility for any analyses, interpretations, or conclusions. T.W.G. is the Jack and Nancy Turner Professor of Ophthalmology, and L.S.J. is an Evan Pugh Professor. Received for publication December 17, 2005 and accepted in revised form May 3, 2006
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||