Assessing the Quality of Pharmacoeconomic Studies

Another alternative to sensitivity analysis is probabilistic sensitivity analysis.

Probabilistic sensitivity analysis allows to assess the overall uncertainty on all parameters in the model simultaneously based on a set of input parameter values randomly selected from the distribution of that parameter. The choice of distribution depends on the nature of the input parameters. After selecting new values, the model is rerun to calculate the corresponding output results. The rerun of the model is repeated many times (usually 1,000-10,000 times), corresponding to different analysis results and is presented on the cost-effectiveness plane and compared with the payment threshold. The set of cost-effective output results will give the percentage of cost-effectiveness of the interventions. The curve showing the correlation between WTP and the percentage of cost-effectiveness of the interventions is called the cost-effectiveness acceptability curve [68], [79]. Based on the cost-effectiveness curve decision makers can decide to approve or reject an intervention.

1.3.4. Quality assessment of Pharmacoeconomics research

Nowadays, Pharmacoeconomic studies play an important role in providing evidence and basis for health policy makers to make decisions in selecting drugs for payment list or developing treatment regimens. Therefore, the process of conducting research and presenting the results of Pharmacoeconomic studies must be rigorous, scientific and reliable. Similar to clinical research, Pharmacoeconomic studies also have differences in scale and quality of research. Therefore, evaluating the quality of research is necessary.

Some toolkits to support writers of Pharmaceutical Economics reports in self-assessing and improving the quality of research, as well as supporting reviewers or editorial boards in assessing the quality of research include: QHES, CHEC, BMJS and CHEERS [82]. Of these, the CHEERS (Consolidated Health Economic Evaluation Reporting Standards) [86] toolkit is currently the most updated toolkit and is recommended for use in the Guidelines for preparing Economic Analysis Reports.

Maybe you are interested!

Vietnam Pharmacy was published by the Department of Health Insurance - Ministry of Health in 2019 [8] for researchers and editors-in-chief, editors and reviewers when evaluating the report results. The questionnaire includes 6 sections with 24 "yes/no" assessment questions, the sections include: title/abstract, introduction, methods, results, discussion and other content including funding sources and conflicts of interest.

In Pharmacoeconomics studies, decision analysis models play an important role in providing a framework for value in health economic evaluations [68], the results of the model depend largely on the selection of data and evidence to be included in the model as well as the construction of assumptions. Therefore, decision models need to be carefully considered and evaluated, and model results should not be used to make decisions without verification. Although currently, according to the Vietnamese Pharmacoeconomic Reporting Guidelines, there is no recommendation to evaluate the quality of decision models, researchers can self-evaluate based on the Good Practice Guidelines for Decision Analysis Modeling developed by Philips et al. in 2004 [119]. This assessment table includes 3 main categories (model structure, data and model consistency), 15 subcategories and 57 assessment questions. This checklist is also recommended for use in Pharmacoeconomics monographs worldwide [58], [59], [68]. The authors also recommend that this checklist is model-related and cannot replace the Quality Assessment Checklist for Health Economic Evaluation Studies in general, and therefore, the report writer should use both checklists at the same time.

1.3.5. The role and necessity of Pharmaceutical Economics research in the current context in the world and Vietnam

The increasing demand for health services coupled with the rising cost of health care has posed a major challenge to the health sector. With limited resources, it is challenging for health professionals to provide quality care to patients at minimal cost. Therefore, the evaluation of drugs and health services today must not only assess safety and effectiveness but also consider cost-effectiveness and

budget impact on the Health Insurance Fund [44]. Therefore, in many countries around the world, health technology assessment in general and pharmacoeconomic assessment in particular are considered important tools in the process of setting priorities and planning health policies, especially in building health service packages within the scope of benefits of health insurance participants in countries [7]. Especially in Vietnam, implementing universal health insurance with a health insurance coverage rate of 90%, the Health Insurance Fund must increasingly cover more health services (especially treatment drugs) while health resources are becoming increasingly scarce. Thus, health technology assessment or pharmacoeconomic assessment needs to be developed and applied in the process of health policy planning, planning, management and clinical practice. This has been clearly stated in the national strategy for protecting, caring for and improving people's health for the period 2011-2020, with a vision to 2030 (Issued with Decision No. 122/QD-TTg dated January 10, 2013), stating that in the coming period, one of the important tasks of the health system is to "Develop standards and implement health technology assessment to identify effective, efficient, low-cost health interventions, ensuring the quality of health services" [39]. And recently, the Ministry of Health issued Decision No. 5315/QD-BYT dated August 31, 2018 on "Principles and criteria for developing the List attached to the Circular promulgating the List and rates and payment conditions for modern medicines within the scope of benefits for health insurance participants", which mentioned the use of evidence of medical technology assessment or pharmacoeconomic assessment in recommending the addition of new drugs to the health insurance payment list [9].

1.4. Systematic review and quality assessment of studies related to the topic

In recent years, studies on the economic efficiency of NL drugs for the treatment of BCMDT in the world are gradually gaining attention. Specifically, in a systematic review study conducted by author Tran Thi Thu Ha [22] published in 2015, 4 studies were found evaluating the economic efficiency of NL in step 1 treatment (2 articles) and step 2 treatment (2 articles). However, the search only stopped in 2014 and was limited to 2 database sources, and there was no detailed analysis of economic efficiency.

in first-line treatment and to evaluate the quality of selected studies. Therefore, in order to update the studies and address the above issues, the topic conducted a systematic review on 3 database sources: Pubmed, Cochrance and ScienceDirect. A total of 10 articles on the cost-effectiveness assessment of NL compared to IM in first-line treatment of BCMDT were recorded through the database search process with keywords related to the topic including: cost, cost-utility, cost-effectiveness, economic analysis, first-line, chronic myeloic leukemia, imatinib, nilotinib; most of the studies were conducted in Europe. In Vietnam, there are currently many topics evaluating the cost-effectiveness of many drugs but there are no similar studies. The process of searching and screening studies according to the Prisma diagram is presented in Figure 1.9.

Pubmed (n=48) Cochrane (n=16) Sciencedirect (n=126)

Screening

Study after deduplication (n=182)

Select

Total number of studies screened (n=10)

172 articles were excluded because:

-Step 2 treatment BCMDT (n=30)

-Not NC KTD (n=96)

-Evaluation on other drugs (n=33)

-Not a complete NC KTD (n=5)

-Not BCMDT disease (n=1)

-No effectiveness assessment (n=4)

-Overview NC (n=2)

-No cost-effectiveness assessment of the 2 drugs (n=1)

Full text studies included in the synthesis (n=4)

Abstracts/posters included in synthesis (n=6)

Synthetic

Total number of studies included in the study quality assessment (n=4)

Figure 1.9. Flowchart of search and selection of studies

After synthesizing studies conducted worldwide on the cost-effectiveness of NL versus IM in the first-line treatment of BCMDT, some results (Appendix 1) on research methods and research results were obtained as follows:

1.4.1. Characteristics and methods of conducting research

In 10 studies, the main research subjects were newly diagnosed patients with BCMDT, in the chronic stage, with or without Ph+; and the main research methods used in the studies were cost-effectiveness analysis, modeling, and sensitivity analysis.

Regarding the modeling method, Markov models were used in all studies with a similar structure including the main disease states (chronic, accelerated, and acute), bone marrow transplant status, and death related or unrelated to BCMDT. The studies by Martin in Colombia [127] and Shin in Korea [131] divided the chronic phase into two separate disease states: treatment response and partial response. The model run time was mostly over the entire patient's life (5/10 studies). The study by Shin M et al. [131] in Korea had a model run time ranging from 46-82 years old, by Li N et al. [101] in China ran for 20 years, by Aiello et al. in Italy [45] ran for 15 years and the study by Inocencio et al. in the US [87] ran for 72 months and 30 years. The models had cycles of 3 months (2 studies), 1 month (2 studies) and 1 year (3 studies). The remaining three studies mentioned the model cycle.

For clinical data sources, the models were built based on two main clinical data sources: the ENESTnd trial (Evaluating Nilotinib Efficacy and Safety in clinical Trials-newly diagnosed patients) and the IRIS trial (International Randomized Study of Interferon and STI571).

Regarding model uncertainty, most studies used simple sensitivity analysis to assess model parameter uncertainty, 5/10 studies used probabilistic sensitivity analysis. Regarding structural uncertainty, only 3 studies by Rochau [125], [126] and Aiello [45] used scenario analysis to assess.

The studies found were conducted from the societal perspective (2 studies), the payer perspective (2 studies), the healthcare provider perspective (3 studies), the third-party perspective (2 studies), and 1 study did not mention the perspective. Thus, from the research perspective, most studies only assessed direct costs, except for 2 studies in Austria and Hong Kong, which assessed direct and indirect costs from the perspective of the whole society. One study did not mention the research perspective, but this study did not calculate indirect costs, so it would not be from the societal perspective.

Most authors chose the LYG and QALY efficiency indexes as the output results, except for the Colombian study which chose the progression-free life-years index. This is one of the limitations mentioned in this study, when the author stated that it is necessary to add the QALY index when conducting the study, because this index helps determine whether the results are reproducible or not. However, to date, no study has been conducted on this index in the treatment of BCMDT in Colombia. Regarding QALY, most studies estimated using patient-derived data based on the IRIS trial (using the EQ-5D tool), 2 studies [125], [126] chose the method of consulting clinical experts through the use of standard gambling techniques.

Over the study period, monetary values and benefits may change, so studies use discount rates to bring future values to the present. To simplify the calculation process, most studies use the same discount rate for both costs and benefits. Of these, a 3% discount rate is used in the majority of studies (5 studies), 1 study uses a 3.5% rate; 2 studies use a 5% rate, and 2 studies do not mention the information.

1.4.2. Analysis of research results

To facilitate the evaluation of research results, the ICER index of studies with the same QALY output index was converted to the same year 2019 based on the consumer price index (CPI) and to the same US dollar currency [155]. The conversion results showed that the ICER index ranged from absolute cost-effectiveness (higher efficiency, lower cost) to 934,541

$US/QALY, respectively (Figure 1.10).

According to Figure 1.10, the study recorded a huge difference in results with the ICER index in the study of Inocencio et al. [87], which was 2 times higher than the study of Rochau et al. [125] in the same country (in the US) (934,541 $US/QALY compared to 497,584 $US/QALY); 2.5 times higher than the study conducted in the UK on the same treatment regimen [107] and nearly 40 times higher than the study in Hong Kong [137], the country with the lowest ICER index after the UK (the country that demonstrated that NL achieved a superior level compared to IM [107]).

Notes: a: NL regimen → dasatinib → stem cell transplantation; b: NL regimen → stem cell transplantation; c: model run time 72 months; d: model run time 30 years

Figure 1.10. Comparison of ICERs across countries

Comparing the ICER index based on the Markov model running time shows that studies with short Markov time have higher ICER index than those with long time.

Long Markov (over the entire patient's life cycle). Specifically, the study Inocencio et al. [87] conducted in the US showed that the ICER in the 72-month model had a value of 934.541

$US/QALY is higher than that of the 30-year model (839,967$US/QALY). The study by Shin M and colleagues in Korea [ 131] on a 36-year Markov time gave an ICER index 7 times higher than that over the entire patient's lifetime in the study by author Tse V and colleagues in Hong Kong [137].

Mildred et al.'s second-line regimen with dasatinib

[107] in the UK shows that NL is outstanding. Meanwhile, in Austria and the US the ICER is 136,193$US and 497,584$US, respectively.

Based on the payment threshold of each country, 7/10 studies showed that using NL in the first-line treatment of BCMDT is cost-effective compared to IM. The study in Korea by author Shin.M and the study in the US by author Inocencio concluded that NL is not cost-effective. The study by author Li N et al conducted in China concluded that IM is superior to NL in cost-effectiveness.

1.4.3. Assessment of report quality and decision analysis model quality

The results of the assessment of the quality of the reports based on the 24-question table of the ISPOR author group (CHEERS) showed that the quality of the reports of the studies was very high, all studies ensured more than 90% of the required reporting content, including 2 studies by the same author U.Rochau [125], [126] and the study by author Li N.

[101] met the required items for the report higher than the study of author M.Romero [127]. The items of title/summary, introduction, discussion and other items, the studies all ensured 100% of the required content, except for the item of research results, the requirement to specifically describe the inconsistency between groups in the study was not implemented in the studies (Figure 1.11).

Comment