Article Text
PDF
Abstract
Objectives This study aimed to characterise the prescribing patterns and evaluate the appropriateness of the prescribed proton pump inhibitors (PPIs) in adult patients via a review of electronic medical records in a single-centred hospital.
Design All patients admitted to the outpatient department of Jinshan Hospital, Fudan University, Shanghai, between 1 January 2018 and 31 December 2018 were evaluated. Individuals aged 18 years or above and with at least one dispensing for PPIs were identified as PPI users. New PPI users were defined as a subject who did not receive any dispensing for PPIs in the year prior to the index date. Baseline characteristics of PPI users and their therapies were described by treatment indication, economic indicators and co-prescription, overall and separately.
Setting The prescription database was retrieved from the hospital information system of Jinshan Hospital, Fudan University.
Results Among 18 435 identified PPI users in 2018, 14 219 patients (aged 18 years or above) who had at least one dispensing PPIs were new users (77%), and among them, men accounted for 47%. The mean treatment duration was 23 days. Omeprazole was the most commonly prescribed drug. PPIs are inappropriately prescribed in 50% (13 589/25 850) of prescriptions. Prescription appropriateness analysis indicated that the unapproved indications for PPI new users accounted for 47%; among them, the proportion of gastritis diagnosis was 34%. The proportion of PPI new users with co-prescription of glucocorticosteroids (GCs) who have risk factors accounted for 24% and lower than other co-prescription. A majority of PPI users (73%) reported high-dose PPI prescription. The defined daily dose of oral pantoprazole was the highest, and injectable omeprazole had the highest defined daily cost. In contrast, only the drug utilisation index value of oral esomeprazole was less than 1.0.
Conclusion The results indicate the challenge of PPI use was accompanied by unapproved indications, frequent inappropriate co-prescription with GCs and excessive dosages. Efforts should be paid to promote rational use and ensure the choice of suitable PPI therapy in the future.
- epidemiology
- adult gastroenterology
- preventive medicine
- health informatics
- information management
- health economics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- epidemiology
- adult gastroenterology
- preventive medicine
- health informatics
- information management
- health economics
Strengths and limitations of this study
This is the most recent and comprehensive study on proton pump inhibitor (PPI) utilisation based on data from a big third-tier hospital to reflect the trends and general problems in PPI usage.
The study covered multifaceted information on PPIs including different types, dispensing frequency, duration, prophylactic use, defined daily dose, defined daily cost and drug utilisation index values, and dose intensity which would provide a comprehensive view for PPI usage in clinical practice.
Appropriateness of PPI usage was also evaluated and discussed.
The study lacks comparison with other prehospital systems and follow-up data on evaluation after the intervention.
This is a cross-sectional study with small sample size and did not include the data from the private healthcare and community hospitals; further randomised clinical trials may be needed to provide causative explanations.
Introduction
Proton pump inhibitors (PPIs) are essentially H+-K+-ATPase inhibitors suppressing gastric acid secretion. These drugs tend to be used for the management of acid-related diseases, such as peptic ulcer disease (PUD), gastro-oesophageal reflux disease (GORD), gastrointestinal (GI) bleeding and Helicobacter pylori infection, or the prevention of gastric ulcers in patients who are taking non-steroidal anti-inflammatory drugs (NSAIDs), glucocorticosteroids (GCs), antiplatelet and anticoagulants.1
The currently marketed main PPIs include omeprazole, esomeprazole, lansoprazole, pantoprazole and rabeprazole. PPI usage has dramatically increased since the introduction in the late 1980s. Nowadays, they have become one of the most commonly prescribed and used drugs in the world.2 For instance, in the UK, nearly 59 million PPIs were dispensed annually, and the total usage doubled since 2007.3 In one of the largest teaching hospitals in the southwest of China, an appreciable increase in PPI utilisation was observed rising about 10.4-fold between 2004 and 2013.4 Meanwhile, urgent concern about the overutilisation of PPIs has been growing. It has been estimated that between 25% and 70% of the PPI prescriptions in the USA have no appropriate indication.2 5 What is more, PPIs have recently been associated with a variety of severe adverse effects such as osteoporosis and associated fractures, hypomagnesemia, community-acquired pneumonia, Clostridium difficile colitis and cardiovascular morbidity.6–11
Even though extensive studies have been investigated on the appropriateness of PPIs in Western countries, such data from China are still very limited. Thus, there is an urgent need to characterise the prescribing patterns and drug utilisation research, and to evaluate the appropriateness of the initiative of prescribing PPIs, so that rationality and cost-effectiveness could be improved in the future.
Methods
Data source and sample selection
Electronic medical records were retrieved from the hospital information system of Jinshan Hospital Affiliated to Fudan University. The prescription contains the medical card number, prescription number, age, gender, diagnosis (based on the International Classification of Diseases 10th edition), the specialty of the primary prescriber, generic name of PPIs and dosage. Individuals aged 18 years or above in outpatient service with at least one dispensing for PPIs in 2018 were identified as PPI users and were included in the study. The first dispensing date for PPIs was defined as the index date. PPI users were defined as new users if they did not receive any dispensing for PPIs in the year prior to the index date. We excluded patients with a missing medical card number or an unknown date of birth or gender and patients who changed their prescribed PPIs in a calendar year.
Different types of PPIs and frequency of dispensing
Different types of PPIs including omeprazole (10 mg, 20 mg), esomeprazole (20 mg), lansoprazole (15 mg, 30 mg), pantoprazole (40 mg) and rabeprazole (20 mg) prescription were analysed. Patients were deemed to be prescribed PPIs once when they received any type and dosage of PPIs. The frequency of dispensing was evaluated according to the prescription dates.
Duration of PPI prescription
We defined treatment duration by the number of PPI tablets dispensed. Treatment discontinuation was confirmed by the absence of a new prescription within the estimated dispensing duration plus a 30-day grace period. Treatment duration was categorised as less than a month, 1–3 months, 3–6 months and more than 6 months. PPI users after the initial prescription lasting more than 6 months were considered as long-term users.
Prophylactic use of PPIs
First, PPI co-prescription included NSAID therapy, antiplatelet therapy, anticoagulant therapy and GC therapy for the prevention of gastric ulcers in the survey. Then, based on PPI co-prescription, recommendations for PPI prophylaxis should include at least one risk factor: (1) age >65 years, (2) antiplatelet/NSAID/anticoagulant/GCs use and (3) history of peptic ulcer disease.
DDD, DDC and DUI values of different PPIs
Drug utilisation research was assessed using the prescription number, proportion and economic indicators. The defined daily dose (DDD) value for PPIs including oral or injectable form was the assumed average dose advocated by WHO. The defined daily cost (DDC) value was calculated by the total sales of PPIs (China yuan) divided by the DDD value, whereas the drug utilisation index (DUI) value was described as the ratio of the DDD value and the actual days of PPI use. To evaluate the rationality of PPI use, DUI values exceeding 1.0 indicate the existence of inappropriate use.
PPI dose intensity
Low dose, standard and high dose were categorised as per definition referred to12 and DDD value (as shown in table 1). If the product dose was equal to and higher than the DDD, it was distinguished as standard and high dose; otherwise, it was classified as low dose.
Definition of low-dose, standard and high-dose proton pump inhibitors (PPIs)
Pattern of prescription and definition of appropriateness
The pattern of prescription was evaluated by types of PPIs and frequency of dispensing, duration and prophylactic use of PPIs, drug utilisation research (DDD, DDC and DUI) and dose intensity. A prescription was deemed appropriate in case (1) the approved indications were defined as PUD, GORD, GI bleeding and H. pylori infection; (2) the duration of therapy was 4–8 weeks following the diagnosis of PUD, 4–8 weeks for GORD, 6–8 weeks for GI bleeding or 2 weeks for H. pylori infection according to the most recent recommendations of the Expert consensus on optimal application of PPIs13 and Expert consensus on the rational use of PPIs in the elderly in China14; and (3) prophylactic use of PPIs meets the condition defined previously.
Statistical analysis
The demographic and clinical characteristics of patients were summarised by descriptive statistics. Categorical variables including frequency and percentage were described across all respondents and evaluated using the χ2 or Fisher’s exact test as appropriate. Continuous variables were expressed as mean±SD or as median and range, and were compared using Student’s t-test (two groups) or one-way ANOVA. Differences were considered statistically significant when p values were less than 0.05.
Patient and public involvement
We did not directly involve patients or the public in our study.
Results
Patient characteristics
The total number of outpatient visits during the year of study was 1 281 769. We identified 18 435 PPI users (from 38 076 prescriptions) and 14 219 new users (from 25 850 prescriptions) (47% men, 53% women, men vs women: 0.9/1.0), which accounted for 3.0% and 2.0% of all outpatient visits, respectively. Groups aged between 18 and 65 years reached up to 84% (11 921/14 219) (table 2). There was a significant difference in gender across the two age groups (p<0.05). Patients’ prescriptions were mostly from the Department of Gastroenterology (82%) and General Internal Medicine (10%).
Characteristics of new proton pump inhibitor (PPI) users
Types, frequency and duration of PPIs
Among all new users, omeprazole was the most frequently dispensed PPI at initiation (33%) followed by rabeprazole (31%) and pantoprazole (27%). Omeprazole accounts for 38% of new users aged >65 years. There were differences in the proportion of patients in the two age groups using different drugs (p<0.05). The average number of dispensing was 1.8 (±1.5), and 59% of new users received only one dispensing. In contrast, 12% of new users in the group aged >65 years received four or more dispensing (p<0.05).
The duration of PPIs was divided into four categories: less than 1 month (83%), 1–3 months (14%), 3–6 months (2.0%) and more than 6 months (0.4%). The mean treatment duration was 23 days and was higher among the group aged >65 years than that among those aged 18–65 years (28 days vs 22 days). PPI therapy prolonged >6 months accounted for 0.4% of new users and was higher in the group aged >65 years when compared with the group aged 18–65 years (1.0% vs 0.3%). In terms of duration for approved indication, duration within 1 month accounted for the majority, while the PPIs used for long term (>6 months) were very few (online supplemental table S1).
Supplemental material
Clinical diagnosis for PPI prescriptions
From the total 25 850 PPI prescriptions in 2018, the approved indications for PPI new users included H. pylori eradication (34%), PUD (9.3%), GORD (5.5%) and GI bleeding (0.8%) (figure 1). Moreover, the PPI prescriptions with a diagnosis of gastritis-related and dyspepsia accounted for 34% and 1.0%, respectively. Pantoprazole was the primarily used PPI to treat the preceding approved indications (39%), followed by rabeprazole (32%), omeprazole (19%), lansoprazole (6.9%) and esomeprazole (2.5%).
Prescription number of different proton pump inhibitors (PPIs) with approved indications. GORD, gastro-oesophageal reflux disease; GI, gastrointestinal; PUD, peptic ulcer disease.
Characteristics of co-prescription
The co-prescription with GCs accounted for the majority (n=314, 2.2%) (table 3). The average age of PPI users with co-prescription of GCs (49±17) was significantly smaller than co-prescription with antiplatelet drugs (67±13) (p<0.05). Among the commonly used PPIs with co-prescription of antiplatelet, NSAID and GC therapy, omeprazole was the most commonly prescribed (figure 2). In contrast, pantoprazole was mainly used in combination with anticoagulant drugs.
Percentage of different proton pump inhibitors among the co-prescription drugs. GC, glucocorticosteroid; NSAID, non-steroidal anti-inflammatory drug.
Characteristics of new proton pump inhibitor (PPI) users with co-prescription of antiplatelet, NSAID, GCs and anticoagulant
The average number of dispensing with co-prescription of NSAID was 1.1, and a large proportion (90%) of the prescriptions contained only one dispensing (p<0.05). A higher dispensing rate was observed among PPI new users with co-prescription of anticoagulant (1.8). In addition, we found over four dispensing with co-prescription of GCs and anticoagulants, which accounted for 10.19% and 12% of all PPI new users, respectively. The shortest mean treatment duration (17 days) was observed in co-prescription with NSAID. Only a few patients among PPI co-prescription users were treated with PPIs for more than 3 months (p>0.05).
DDDs, DDC and DUI values of different PPIs
We analysed the DDDs, DDC and DUI values of different PPIs. Table 4 shows that the DDD value of oral pantoprazole was the highest, while the lowest DDD value was observed in injectable omeprazole. The minimum value of DDC was obtained in oral lansoprazole. Only the DUI value of oral esomeprazole was less than 1.0, and the DUI value of oral pantoprazole was the highest (1.8).
DDD, DDC and DUI values of different PPIs
PPI intensity utilisation patterns
The high-dose PPI prescription comprises the majority (73%) (table 5). The high dose of lansoprazole, rabeprazole and pantoprazole accounted for 78%, 69% and 81%, respectively. In contrast, the standard dose of esomeprazole accounted for 63%, followed by rabeprazole (31%). The low dose of total PPI prescription accounted for 3.6%.
Proton pump inhibitor (PPI) intensity utilisation patterns according to different PPI prescriptions
Assessment of PPIs appropriateness
Overall, PPIs are inappropriately prescribed in 50% (13 589/25 850) of prescriptions. In order to reveal the main categories of inappropriate drug use, PPI appropriateness was further evaluated by the approved indications, the duration of therapy for different diseases and prophylactic use of PPIs, respectively. In detail, the rate of appropriateness according to the approved indications was 53%. In contrast, a high rate of appropriate prescriptions was observed for the duration (PUD: 99.6%, GORD: 99.7%, H. pylori eradication: 95.3% and GI bleeding: 99.5%). According to the recommendations for PPI prophylaxis, the proportion of PPI new users with co-prescription of GCs who have risk factors was lower when compared with that of anticoagulant (24% vs 91%), while the proportions of PPI new users with co-prescription of NSAID was 43% (table 6).
Number and proportion of appropriate PPI prescriptions
Discussion
Based on hospital electronic medical records data, our study provides important insights into the patterns of utilisation among new PPI users. We found that 3.0% of outpatient visits were PPI users, and 77% of them were new users. Given that PPIs are the most frequently used drugs worldwide, its overutilisation has become more common in recent years. For example, in one study from the USA, PPIs were prescribed in 4% of outpatient visits in 2002 and increased to 9.2% in 2009.15 Besides, similar increases in prescription rates have also been observed in many other countries.16–19 Although our result revealed that the prevalence of PPI use was less than that of other reports, an appreciable increase in PPI utilisation has been witnessed in other regions in China20 which could be reflected in a study conducted, in particular, for injectable PPIs.4 These facts pose serious queries and concerns on the inappropriate use of PPIs, the occurrence of potential side effects and the increase in healthcare costs.
Omeprazole was developed in 1979 and has been on the WHO list of essential medication.21 Our analysis found that omeprazole was commonly prescribed, especially among those aged >65 years, which was similar to the findings of one study in New Zealand.22 Also, prescription preferences on omeprazole have been reflected in pharmacoepidemiological studies conducted in Scotland, New Zealand and Australia.22–24 Despite omeprazole, rabeprazole and pantoprazole being among the most frequently prescribed drugs in our study, at present, at least five available PPIs including esomeprazole, lansoprazole, omeprazole, pantoprazole and rabeprazole are mainly prescribed for patient care. So, clinicians should choose the most appropriate PPIs according to different pharmacokinetic characteristics, tissue selectivity, binding specificity and potential interaction with the cytochrome P450 enzyme system. Meanwhile, we observed that more than half of new users (59%) had only one dispensing. The average number of dispensing was 1.8, and a little bit higher than that value from a nationwide PPI utilisation study in France.25 However, the mean treatment duration was significantly lower (23 days vs 41 days). The proportion of patients with prolonged (>6 months) PPI therapy accounted for only 0.4% of new users overall, and 1.0% of those were aged over 65 years. The result is also comparable with the finding of a study in France. These results present a relatively low frequency and short treatment duration, which may be necessary to avoid PPI overuse, potentially serious adverse events and costly PPI treatment.
Despite several guidelines and many published reports highlighting the importance of appropriate use of PPIs and potential drawbacks of inappropriateness, the inappropriate use of PPI remained in clinical practice. For instance, Giannini et al26 reported that PPIs were inappropriately used in about 40% of outpatients, and an even higher inappropriate rate of PPIs was found in 68.8% of patients by Thomas et al.27 In our study, PPIs are inappropriately prescribed in around 50% of prescriptions. Although the indications for PPI use have expanded, the PPI prescriptions used for inappropriate indications were very common,2 which also could be reflected in a study conducted in more than 45 hospitals in China.21 These facts alert the increasing worries regarding the cost and also safety, especially for long-term use.
Clinical diagnosis is the foundation in support of prescribing appropriate medication to initiate certain therapy. In our survey, the overall approved indication prescription ratio does not exceed 50%; for instance, the prescription for H. pylori eradication accounted for 34%. Of note, the prescription for gastritis-related diagnosis accounted for 34% among all prescriptions and is similar to Jie Ying’s survey result (34.3%) among 45 hospitals in China.20 Gastritis-related diagnosis is a vague diagnostic description or symptom and is classified as an uncertain indication for PPI use. The PPI was mostly prescribed in the departments of gastroenterology in our study, which could result from the main application of PPI in digestive system diseases. Unfortunately, we could not rule out unreasonable PPI use in gastritis-related diagnosis due to the deficiency and information shortage in our electronic medical records. Therefore, we should strengthen the training of doctors in diagnosis and prescriptions, and reciprocal correspondence between diagnosis and medication to achieve improved diagnostic validity and avoid overuse of PPIs.
GC use can lead to adverse reactions, such as PUD. In addition, other debated potential adverse reactions are also frequently reported.28 However, the occurrence of such adverse reactions is often associated with higher dosages or long-term use. In our study, PPIs were frequently used in co-prescription with GCs (n=314, 50%), but patients with measurable risk factors that would support the use of a gastroprotective therapy accounted for only 24%. Despite the previous studies showing that patients get no additional benefit from PPI prophylaxis by taking systemic GCs without concomitant NSAIDs,29 30 GCs are still considered as a cause of upper GI complications by many prescribers who systematically add PPIs to their prescriptions.31 The occurrence may be also reflected in our result where the risk of GI bleeding was low, but PPI usage was frequently identified. Clinicians should assess the risk factors of bleeding, such as age, whether there was concomitant use of NSAIDs and whether PPIs were used as gastric mucosal protective agents when prescribing glucocorticoids, instead of using PPI combination therapy once glucocorticoids alone are used.
NSAIDs are very effective for the management of pain by various inflammatory conditions and among the most widely prescribed medications. However, a large number of adverse events in the GI tract, particularly in the gastric mucosa, were reported. In our study, only 14% of new users with co-prescription initiated an NSAID therapy and 43% of them with a measurable risk factor of GI complications. This proportion was lower than the result from Marion Lassalle et al’s research (53.5%), in which 80% of PPI new users had co-prescription with NSAIDs without any measurable risk factor.25 About half of the patients in this study did not have high-risk factors but used PPI to prevent NSAID-related ulcers. However, it is not recommended to use gastroprotective strategies with PPIs in younger NSAID users (<65 years) and without other known risk factors, and this is also one of the main causes of NSAID misuse.32 Therefore, clinicians should be trained and guided on the exact indications of PPI to reduce unnecessary use of PPI and avoid serious adverse reactions.
Unexpectedly, we found that omeprazole was mainly prescribed with antiplatelet in our study. Omeprazole is metabolised mainly through CYP2C19, which may interact with other drugs that are metabolically activated by the same enzyme such as clopidogrel. So, omeprazole could lead to the reduced protective role of clopidogrel in cardiovascular events. In March 2010, The Food and Drug Administration (FDA) issued a ‘black box warning’ to warn that the prescription on clopidogrel should avoid concomitant use of omeprazole or esomeprazole if the patients have been identified as CYP2C19 poor metabolisers,33 especially in East-Asian patients for whom the CYP2C19 loss-of-function (LoF) allele is associated with an increased risk of adverse cardiovascular outcomes when treated with clopidogrel.19 In this case, lansoprazole and pantoprazole could be considered as a rational choice.34
In terms of cost-effective prescribing, the DDC results showed that the highest average daily costs of PPIs were injectable omeprazole, followed by oral esomeprazole. Thus, the result indicated that these patients paid a higher price when these drugs were prescribed. The lowest DDC value was for oral lansoprazole, suggesting that it has a price advantage for therapy. However, lansoprazole was one of the least used oral PPIs in our study. Meanwhile, despite oral esomeprazole, the DUI value of all forms PPIs was above 1, and the highest DUI value was 1.8 for pantoprazole and showed that PPI dose was excessive. As mentioned previously, pantoprazole is the most commonly used in the approved indications, indicating that the overdose use of this type of PPIs should be strictly managed in the future.
A Canadian population study has shown that high-dose PPI use is becoming more prevalent.35 In our study, we found that high-dose therapy was most frequently prescribed (73%) compared with standard (23%) and low-dose therapy (3.6 %), which was similar to the findings of the result in Australia in 201716 but higher than other studies.12 The effect of dose and type of PPI research has shown that high doses of PPIs could increase the risk of chronic kidney disease at an impressive level (92%) for any kind of PPI exposure, and this risk increased even more when doing the follow-up.36 There are pieces of evidence to suggest that high-dose PPIs can lead to the development of hypomagnesemia,37 and increase the risk of osteoporotic fractures38 and serious cardiovascular disease.39 Furthermore, it also increased the risk of iron deficiency in renal transplant recipients,40 and even 1-year mortality in older inpatients needed acute care.41 In daily clinical practice, the common PPI dose may be doubled when common-dose PPIs cannot control and alleviate symptoms. Thus, once symptoms were adequately relieved, a ‘step down’ therapy is recommended to reduce the dose to the lowest effective dose through a periodic reassessment of patients.12 42 Together, these collective shreds of evidence highlight the importance of doctors in collaborative practices with supervisor pharmacists to develop innovative practices to provide economic, effective and safe pharmacotherapy.
Although our study found some characteristics of PPI utilisation and pointed out irrational drug prescribing, our study also has several limitations. First, the data in our article were obtained from just one hospital and did not include the data from the private healthcare and community hospitals. Although the number of patients included is estimated to be relatively small, it also reflects the trends in PPI usage and general problems in clinical practice. Second, the conclusion was based on an observational study, and a further randomised clinical trial may be needed to provide causative explanations. Third, the actual prevalence of PPIs may be underestimated because the information on over-the-counter drugs was lacking. Thus, we suggest future studies should be conducted with a longitudinal cohort design based on more rigorous standards for initial, randomised clinical trials and ongoing follow-up to provide more extensive investigation and more convincing effective therapy.
Conclusion
The results of this study showed that PPI utilisation was accompanied by unapproved indications and excessive dosages. There was a lack of indication for combined medication. Inappropriate co-prescription with GCs was frequent. So, overuse of PPIs and the high rate of inappropriate prescriptions reflect a lack of concern for optimising PPI use. Efforts should be made to limit PPI treatments to appropriate indications and ensure the choice of suitable PPIs promoting rational use. The educational activities would be useful to address concerns of PPI indication, adverse effects and novel indication. Healthcare practitioners should assess the risk and benefit while prescribing the PPIs to identify the actual need and to choose the most rational alternative.
References
Footnotes
Contributors YL and JZ conceived and designed the study. YL, XZ and RL analysed the data and drafted the manuscript. YL and JZ collected and organised the patient data. FZ critically revised the manuscript. All authors read and approved the final manuscript.
Funding The work was funded by the Key Construction Project on Clinical Pharmacy (1229).
Disclaimer The funding sources had no involvement in study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethical approval was not required for this study. The present study retrospectively analysed data abstracted from the hospital information system of Jinshan Hospital without interventions or disruptions to patients’ lives, and no direct human participants were involved, thus informed consent was not required and waived by the Ethics Committee of Jinshan Hospital Affiliated to Fudan University.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.