Issuu

How Evidence is Used to Inform Health Policy in Viet Nam April 2022

Oxford University Clinical Research Unit

How Evidence is Used to Inform Health Policy in Viet Nam

TABLE OF CONTENTS Lists

1. List of Figures 2. List of Tables 3. List of Abbreviations

About Us

About the Project

Acknowledgements

Summary

Chapter 1. Introduction

1. Background 2. Objectives

Chapter 2. Methods

1. Literature Review 2. In-depth Interviews

Chapter 3. Results

2. In-depth Interviews 2.1. Demographic Information 2.2. Perceived Concepts 2.3. Successes of Viet Nam in Policy Development 2.4. COVID-19 2.5. Policy Development 2.6. Evidence Use Related Issues 2.7. Research Related Issues 2.8. Engagement between Policy Makers and Researchers 2.9. OUCRU’s Achievements and Gaps 2.10. Recommendations for OUCRU’s Engagement with Policy Makers

Chapter 4. Discussion

1. Evidence Use and Citation 2. Policy Stakeholders’ Joining in Research Study 3. OUCRU’s Policy Engagement Plan

Chapter 5. Conclusion

Appendix 1. Question Guide for In-depth Interviews with External Key Stakeholders

References

1. Literature Review 1.1. Document Categorisation and Requirements on Evidence Use in Policy 1.2. MoH Documents 1.3. Non-MoH Documents

How Evidence is Used to Inform Health Policy in Viet Nam

Lists List of Abbreviations

List of Figures 1. Figure 1. Three frameworks for policy development 2. Figure 2. Kingdon’s Multiple Streams Framework 3. Figure 3. The I - I - I framework 4. Figure 4. Outline of Vietnamese health system14 5. Figure 5. MoH documents search and selection flow diagram 6. Figure 6. Summary of the review of MoH documents 7. Figure 7. Non-MoH document search and selection flow diagram 8. Figure 8. Driving factors for policy changes 9. Figure 9. Factors increasing likelihood of policy makers' accepting research findings 10. Figure 10. Key stakeholders in policy development 11. Figure 11. Viet Nam's successes in policy development 12. Figure 12. COVID-19 Evidence Sources 13. Figure 13. COVID-19 Diagnosis and Treatment Guideline Development 14. Figure 14. Policy development confidentiality and sharing 15. Figure 15. Policy development determinants 16. Figure 16. The process of policy development 17. Figure 17. Stakeholders’ concepts about the availability of evidence 18. Figure 18. Stakeholders’ concepts about evidence sources 19. Figure 19. Research and policy engagement gaps 20. Figure 20. Stakeholders’ recommendations for OUCRU’s enhancing engagement with policy makers 21. Figure 21. Stakeholders’ ideas on establishing a Policy Stakeholder Advisory Board 22. Figure 22. The continuous cycle of systemic policy engagement

10 11 12 13 23 24 25 27 28 29 34 37 39 40 41 47 49 50 56 61 64 69

List of Tables Table 1. Demographic information of interviewees Table 2. Perceived concepts on policy and policy makers Table 3. Policy development gaps and proposed solutions

32 32 44

AECOPD AMR ASP BMH CAP CCT COPD CRH CSO DOH Dx and Tx EBPs EPPI Centre GDPM HIPAA HSPI HUPH LMICs MoH MSA MeSH NGO NHTD NICE OAB OUCRU PCDC QH QS TIHE US-CDC VAAC VIDS VNCH WHO

Acute Exacerbation of Chronic Obstructive Pulmonary Disease Antimicrobial resistance Antimicrobial stewardship program Bach Mai Hospital Community Acquired Pneumonia Centre-based Compulsory Drug Treatment Chronic Obstructive Pulmonary Disease Cho Ray Hospital Civil Society Organiszation Department of Health Diagnosis and Treatment Evidence-based Best Practices Evidence for Policy and Practice Information and Coordinating Centre General Department of Preventive Medicine, Ministry of Health Health Insurance Portability and Accountability Act Health Strategy and Policy Institute Ha Noi University of Public Health Low and middle income countries Ministry of Health Medical Service Administration, Ministry of Health Medical Subject Headings Nongovernmental Organisation National Hospital for Tropical Diseases The National Institute for Health and Care Excellence Outbreak Advisory Board Oxford University Clinical Research Unit Provincial Centre for Disease Control National Assembly Quality Standards Tay Nguyen Institute of Hygiene and Epidemiology United States - Centres for Disease Control and Prevention Viet Nam Administration of HIV/AIDS Control Viet Nam Infectious Diseases Society Viet Nam National Children’s Hospital World Health Organisation 2

How Evidence is Used to Inform Health Policy in Viet Nam

About Us The Oxford University Clinical Research Unit (OUCRU) is a largescale clinical and public health research unit with site offices in Viet Nam, Indonesia, and Nepal.

OUCRU-NP Kathmandu, Nepal

Part of the Centre for Tropical Medicine and Global Health at the University of Oxford (UK), OUCRU was first established in Ho Chi Minh City in 1991, hosted by the Hospital for Tropical Diseases (HTD), Viet Nam. In 2003, OUCRU-NP was established in Kathmandu, Nepal, hosted by Patan Hospital and the Patan Academy of Health Sciences. OUCRU Ha Noi was established in 2006 in partnership with the National Hospital for Tropical Diseases (NHTD) and the National Institute of Hygiene and Epidemiology (NIHE), Viet Nam. In 2008, the Eijkman-Oxford Clinical Research Unit (EOCRU) was established in Jakarta, Indonesia, in partnership with the Eijkman Institute for Molecular Biology and Faculty of Medicine University of Indonesia.

OUCRU Ha Noi Viet Nam

OUCRU Ho Chi Minh City Viet Nam

Our vision is to have a local, regional and global impact on health by leading a locally-driven research programme on infectious diseases in Southeast Asia. Our research programme covers clinical and laboratory research with hospital and community-based patient populations, including epidemiology, immunology, host and pathogen genetics, molecular biology, microbiology and virology, mathematical modelling, bioinformatics, biostatistics, and social science. This work is supported by an extensive clinical trials unit and data management centre compliant with national and international regulations and comprehensive management, finance, public engagement, and administrative support offices. OUCRU receives considerable support from Wellcome as part of the Africa and Asia Programmes. Together with our partners, we have led a highly successful effort in enhancing the infrastructure and capacity to perform clinical trials and basic scientific research in Viet Nam, Indonesia, and Nepal. Website: www.oucru.org

Eijkman-Oxford Clinical Research Unit Jakarta, Indonesia

How Evidence is Used to Inform Health Policy in Viet Nam

About the Project Since our its establishment in 1991, OUCRU Viet Nam has been actively engaging with policy stakeholders as a leading clinical and public health research unit. Over the last 30 years, OUCRU has had many successes, and achieved some remarkable impacts on health policy in Viet Nam and in the region. This report is carried out as one of the key outcomes of the Project “Establishing systemic policy engagement at OUCRU: A pilot project” which was conducted from October 2019 to September 2021 by OUCRU Viet Nam. This report is authoured by Katrina Lawson and Nghiem Nguyen Minh Trang. Other authors include Prof. Guy Thwaites, Assoc. Prof. H Rogier van Doorn, Dr. Jennifer Van Nuil, Dr. Hugo Turner, Nguyen Kim Ngan, and Ngo Phan Bao Tran.

Acknowledgements The project team would like to express our great appreciation to OUCRU Director Prof. Guy Thwaites and OUCRU Ha Noi Director Prof. Rogier van Doorn for their continuous support and valuable advice to improve the effectiveness of this project. To all external interviewees who agreed to participate in our interviews, share their experience and knowledge, and actively provide recommendations on how to better OUCRU's engagement with policy stakeholders – thank you. The project team have tried our best with all dedication and enthusiasm to conduct the review and carry out the best results in the hope of contributing, to some extent, to the enhancement of OUCRU’s policy engagement system.

Photo: OUCRU Ha Noi working with Nam Dinh CDC

How Evidence is Used to Inform Health Policy in Viet Nam

Summary Background Evidence-based medicine and health policies have long been considered key elements in improving health systems globally1. Health professionals have critical contributions to such processes. Their role in conducting policy relevant health research, disseminating actionable results, encouraging the use of evidence in policies, and developing partnerships has been well recognised2. OUCRU, with more than 30-year experience in conducting health research in Viet Nam, has set a new vision of having local, regional, and global impact on health by leading a locally driven research program on infectious diseases in Southeast Asia. In order to develop a locally driven research programme, OUCRU has recognised the need to understand the role, the position of, and the need for policy development in Viet Nam. We, therefore, conducted a study which looked at how policy makers used research evidence in policies in Viet Nam. The main goal of this study was to understand how policy makers in Viet Nam used scientific evidence to inform health policies and decision making, and then to determine where there are opportunities for the scientific community (and OUCRU specifically) to better engage with policy makers to facilitate that process.

Methods In the study, we conducted a bi-lingual literature review of guidelines published within the past five years by the Ministry of Health (MoH), and also of published case studies, reports and reviews by national and international agencies within the same period. We also interviewed key stakeholders in the

policy environment about their experience of, and attitudes towards, engaging with researchers and research evidence in order to generate knowledge on policy development processes, evidence use in decision-making, challenges and opportunities to policy and research engagement, as well as how to better engage policy and research communities.

Results How evidence had been used in policy development was not commonly described in the literature or very rarely shown in the MoH documents. However, the literature did provide information on different aspects that relate to or can have influence on policy formulation. Factors that were said to increase the likelihood of policy makers’ accepting research findings included the timeliness of research, relevance of topics, accessibility of research results, unequivocal nature of findings, quality of research methods and long-term trusted collaborations3. Apart from government officials and local authorities, other key informants like international organisations, local non-governmental organisations, civil society organisations, or activists also had a role in each step of the policy development process4. Building capacity and understanding of policy processes, engaging early on with policy makers in the whole course of a study, using explicit and systematic advocacy approaches and frequent communication were considered core components of successful advocacy approaches5.

Policy development in Viet Nam was a continuous process with the participation of both scientists having expertise in related topics and management institutions. Viet Nam has made achievements in policy development which were shown by improvements in evidence use in comparison with ten years ago, more involvement of scientists in the whole process and more application of research results into practices. Policy development was faster and Viet Nam had better access to updated information globally. However, obstacles still existed. Time pressure, lack of information, limited capacity, and lack of standard procedures for policy development caused difficulties for the policy formulation process. Policies in Viet Nam were confirmed by the study participants to be evidence-informed. Yet, while evidence played a very important role, so did other factors. Interviewees explained that developing a policy also depended on contextual factors and available resources. Policy makers selected evidence for decision-making based on the level of evidence, the quality of research studies, the appropriateness with the context of Viet Nam, and the source of evidence. In order to stimulate evidence use in policy formulation, developing better mechanisms was considered a key to success. A suggestion of developing a guideline on developing evidenceinformed guidelines was raised as one potential solution to address existing problems of evidence use in policy development. In evidence-based policy-making, the engagement between researchers and policy makers should be close. However, this was not practically the case in Viet Nam. The two communities saw things differently and had different priorities. Researchers’ unwillingness to share data, policy makers’ time

constraints and high rate of staff turnover were commonly mentioned as obstacles for engagement. Particularly for OUCRU, in order to better enhance policy engagement, OUCRU should engage with policy makers from the very beginning of research idea formulation. OUCRU should also further engage with the MoH, at both the level of department leadership and with specialists. Enhancing regular communication with policy stakeholders, via both formal and informal approaches, was also suggested as another approach to better engage with policy actors.

Conclusion Policy development in Viet Nam is a continuous process involving both scientists and management officials, starting with prioritising issues for change, establishing committees, then writing drafts, organising meetings, to the final steps of finalising and endorsing documents. Evidence is clearly consulted and used during the process. How evidence is incorporated does not merely depend on the evidence itself. In order to enhance evidence use in decision-making, it is necessary to establish better mechanisms, systems and guidelines on the development of evidence-informed guidelines as well as policy and research engagement. For better engagement, research institutions should enhance regular communication, further consolidate available collaborations, andinitiate new partnerships with policy actors. Involving policy makers into research studies at the very beginning step and throughout the whole course of a study was highly recommended. How to involve policy makers into a study should be strictly considered in order to ensure policy makers’ valued contribution as well as ensure the objectivity of the study.

Chapter 1. Introduction 1. Background The World Health Organisation stated the importance of the application of relevant high-quality scientific research evidence into the formulation of health policies. Evidence-based medicine and health policy have an important role as a key strategy to improve the health system globally1. In the process of formulating policies, health professionals play critical roles of conducting policy-relevant health research, disseminating actionable results, encouraging the use of evidence in policies and building partnerships2. Thus, there are opportunities for researchers to contribute to the development and implementation of policies to improve health and health care services. Clancy et al. stated in their article “From research to health policy impact” that there were four potential ways in which researchers could influence policy via identification of critical problems, detection of benefits and harms of policy solutions, estimation of costs and consequences of policy proposals and active participation in the policy processes to provide support in decision-making6. In different areas of the health sector, the policy development processes might not be the same7. Understanding theoretical frameworks used in support to health policy development could help to identify factors facilitating successes in enhancing policy engagement and thus, creating better chances for impacts on decision-making. Kingdon’s Multiple Streams Theory (MSF)8–10, Weiss’ Ideology, Interest and Information framework (I-I-I)3,11 and Lomas’ Linkage and Exchange framework (L-E)3,12 were applied in different aspects of agenda setting, policy development, and implementation. Within each of these frameworks, there was scope for engagement between the research and policy communities. The key aspects of each theory are summarised in Figure 1 below. I-I-I*

MSF* •

•

Problem streams: Evidence (data) to reflect the situation requesting for actions to improve the situation Policies streams: Powerful policy champions/ entrepreneur with proposed interventions to address the problems Politics streams: Power of relevant national and international events

• • •

Ideology: philosophy, principles, values and political orientations Interest: self-interests of policy makers Information: current state of affairs, seriousness of problems, why things happen, and which progress if possible

L-E* • Bring researchers and policy makers into one platform • Setting priorities • Funding programs • Assessing applications • Conducting research • Communicating findings • Planning for evaluation

Figure 1. Three frameworks for policy development

CHAPTER 1. INTRODUCTION

* MSF: Kingdon’s Multiple Streams Theory8–10 I – I – I: Ideology, Interests and Information3,11 L – E: Linkage and Exchange3,12 10

How Evidence is Used to Inform Health Policy in Viet Nam

Within each of these frameworks, there was also scope for engagement between the research and policy communities. For example, when discussing agenda-setting for decision-making to inform policies, Kingdon’s Multiple Streams Framework (MSF) was referred to10,13. The MSF describes three streams of influence: problem, policy and politics streams (defined in Figure 1 and Figure 2) and at the convergence of those streams, there is a window of opportunity for creating sound policy10. Asa Knaggard, when refining the MSF, emphasised that the problem stream should be positioned at the same level of importance as the policy and politics streams and knowledge might not need to be scientific but scientific knowledge should be positioned at a higher value than experience and professional knowledge9. Therefore, the role of evidence and data from scientific research, reflecting the situation and essential actions for improvement, should be raised up to its appropriate level of importance.

Ideology

Interest

Information

Problem stream

Policy stream

Broad issues faced by societies

Proposed interventions to address the problems

Global or national political Politics stream events that are associated with the problems

Policy window: Convergence of these 3 streams

Figure 3. The I - I- I Framework

Window or opportunity to place a particular issue on a policy agenda

Figure 2. Kingdon’s Multiple Streams Framework

The Ideology – Interest – Information (I-I-I) framework was often mentioned during the discussion on scientists’ dissemination and policy makers’ uptake of research results. This theory has its basis in social sciences in particular, and the ways that social sciences can influence policy making. With this theory, the author strongly claimed the importance of understanding the nature of political decision-making in order to know the position of research in such process. The I-I-I framework points out that research plays a very important role in the process of policy development as research can be served as a tool to provide better understanding of problems. Research could help for developing better intervention strategies. However, research is not the only kind of information that is used by policy makers to inform decision-making: they also rely on ideological information, and their own self-interest. Ultimately, research results do not speak for themselves, they exist within a context of competing interests. When ideologies and interests are aligned, making the information relevant to policy making, then research is better able to provide significant contributions. Decision-making in almost all issues also needs to have compromises and trade-offs in order to avoid losing support from other groups3. 11

The I-I-I framework can be useful as a diagnostic tool for certain situations – if the research is not having the expected impact on policy, it is possible or likely that the reason is because either the research information itself or the way it is communicated is not effectively aligned with the ideology or interest of the stakeholders11. Effective engagement occurs at the intersection of ideology, interest, and information (see Figure 3). The Linkage and Exchange Theory (L-E) focuses on the relationship between policy making and the conduct of research. To formulate this theory, Lomas aggregated different viewpoints from various sources regarding the involvement of researchers in policy making and vice versa, policy makers in research processes. Policy makers joining in research could help to promote the relevance and applicability of research studies, and researchers joining in policy development, with their analysis capacity, could help to enhance consensus and reduce uncertainty in decision-making. Consistent communication between researchers and decision makers could be considered the most efficient way in providing knowledge from scientific studies to decision makers. Engagement with a researcher would not be only equal to establishing relationship with that person only but also linking to extensive knowledge of a researcher’s community. Based on what had been aggregated, Lomas built the Linkage and Exchange principle, which involved the participation of both researchers and policy makers in different activities including setting priorities, funding programs, assessing applications, conducting research, communicating findings, and evaluation12. Getting to know the health system of Viet Nam and how it works would also help to explore factors related to health policy development as well as identify at which level researchers should target to create influence on policy. The health system of Viet Nam in general covers four levels of administration which are central, provincial, district, and communal levels (see Figure 4). Facilities at different levels are assigned with different rights, responsibilities, and roles dealing with different issues of the health sector covering from the national to the grassroots levels14. Within the government of Viet Nam, the MoH is assigned to take responsibilities of all activities related to health issues nationwide, including taking the leadership in developing national action plans, strategic plans, and policies of the health sector15.

How Evidence is Used to Inform Health Policy in Viet Nam Government

Ministry of Health

Other ministries

Health services belonging to other ministries

The Decree No.75 issued in 2017 by the government15 defines that the Ministry of Health of Viet Nam is a government institution performing the function of state management of health, including:

Provincial People's Committee

National hospitals Institutes Medical universities Colleges Units in MoH Provincial health services

District People's Committee

Provincial hospitals Centre for preventative medicine, health education Medical high schools District general hospitals Polyclinics

District health bureau Commune People's Committee Commune health centre Village

Village health workers

Official management Professional supervision

Figure 4. Outline of Vietnamese health system14 13

Preventive medicine Diagnosis, treatment, rehabilitation Forensic examination, mental forensics Traditional medicine, pharmacy Reproductive health Medical equipment Pharmacy, cosmetics Food safety Health insurance Population Government management of public services under the management of the ministry.

In Viet Nam, ‘legal documents’ are classified into 15 categories consisting of: 1. Constitution 2. Codes and Laws, Resolutions of the National Assembly 3. Ordinances, Resolutions of Standing Committee of the National Assembly; Joint Resolutions between Standing Committee of the National Assembly and Management Board of Central Committee of Vietnamese Fatherland Front 4. Orders, Decisions of the President 5. Decrees of the Government; Joint Resolutions between the Government and Management Board of Central Committee of Vietnamese Fatherland Front 6. Decision of the Prime Minister 7. Resolutions of Judge Council of the People’s Supreme Court 8. Circulars of executive judge of the People’s Supreme Court; Circulars of the Chief Procurator of the Supreme People’s Procuracy; Circulars of Ministers, Heads of ministerial agencies; Joint Circulars between executive judge of the People’s Supreme Court and the Chief Procurator of the Supreme People’s Procuracy; Joint Circulars between Ministers, Heads of ministerial agencies and executive judge of the People’s Supreme Court, the Chief Procurator of the Supreme People’s Procuracy; Decisions of State Auditor General 9. Resolutions of the People’s Councils of central-affiliated cities and provinces 10. Decisions of the People’s Committees of provinces 11. Legislative documents of local governments in administrative - economic units 12. Resolutions of the People’s Councils of districts, towns and cities within provinces 13. Decisions of the People’s Committees of districts 14. Resolutions of the People’s Councils of communes, wards and towns within districts 15. Decisions of the People’s Committees of communes16. As a ministerial institution, the MoH is responsible for developing and submitting to the government proposals of different levels of policies including ‘legal documents’ and ‘technical documents’ related to health and population. The MoH also leads, guides and implements policies related to the areas under their responsibilities15,17. 14

How Evidence is Used to Inform Health Policy in Viet Nam OUCRU has been operating in the research area in Viet Nam for over 30 years with a focus on infectious diseases. Over the past 30 years, OUCRU has developed tight collaboration with multiple stakeholders playing key roles in the health sector of Viet Nam including hospitals and research institutes as well as the MoH. However, these pieces of engagement work are just project- or issue-focused and were not performed under a systemic engagement mechanism. In the context of a global interest in research uptake in policy and practice18, OUCRU’s ten-year vision is to have local, regional and global impact on health by leading a locally driven research programme on infectious diseases in Southeast Asia. In order to develop a locally driven research programme and establish the systemic engagement mechanism, OUCRU has recognised the need to understand the role, the position of, and the need for research in policy development in Viet Nam. To realise the short-term goal of effective engagement with policy stakeholders, and the long-term goal of having an impact on health, OUCRU also needs to understand different components that contribute to policy development in Viet Nam, and how to form effective partnerships with stakeholders in the policy making environment. We, therefore, conducted a study which looked at how policy makers used research evidence in policies in Viet Nam. In the study, we conducted a bi-lingual literature review of guidelines published within the past five years by the MoH, and also of published case studies, reports and reviews by national and international agencies within the same period. We also interviewed key stakeholders in the policy environment about their experience of, and attitudes toward engaging with researchers and research evidence in order to generate knowledge on policy development processes, evidence use in decisionmaking, challenges and opportunities to policy and research engagement, as well as how to better engage policy and research communities. In this report, ‘policy’ refers to all kinds of approved documents that stakeholders in the policy environment create – at local, national, and international levels. ‘Policy stakeholder’ refers to people who are directly involved in policy development process.

2. Objectives The main goal of the study is to understand how policy makers in Viet Nam use scientific evidence to inform health policies and decision-making, and then to determine where there are opportunities for the scientific community (and OUCRU specifically) to better engage with policy makers to facilitate that process. Specific objectives are: • To understand policy development in Viet Nam and the key stakeholders involved in the policy development process; • To investigate and document various ways in which research evidence is used in making policies in Viet Nam; • To investigate challenges and opportunities of policy and research engagement to inform health policy; • To determine how to better engage researchers with policy makers.

Photo: UK- Viet Nam Expert Meeting "Infectious Diseases at the Animal - Human Interface" in January 2011 in Ho Chi Minh City, Viet Nam

Chapter 2. Methods We conducted a literature review of guidelines issued by the MoH of Viet Nam and published case studies over the past 5 years and conducted in-depth interviews with key stakeholders.

1. Literature Review We looked at a number of different official information sources (listed below). The sources all have different search algorithms, and so we needed to use different search strategies on each one. •

Inclusion criteria -

•

Exclusion criteria -

•

Time: Published from January 01, 2015 to March 04, 2020 Issues: health related issues including clinical treatment, preventive medicine, laboratory procedures, evidence-based healthcare/practice/medicine, policy development Venue: Viet Nam

Issues: issues related to marketing strategies, alcohol industry, digital health products, bidding procedures, hospital audit, implementation of decree or circular, administrative procedures, surveys, pricing, correspondence, plan, invitation… Status: Expired documents

Information sources -

Ministry of Health portal (https://emohbackup.moh.gov.vn/publish/home), Medical Service Administration portal (www.kcb.vn/vanban/huong-dan), Viet Nam Administration of HIV/AIDS Control portal (www.vaac.gov.vn/vanban_detail), PubMed, Cochrane, EPPI Centre, Health Evidence.

CHAPTER 2. METHODS 18

How Evidence is Used to Inform Health Policy in Viet Nam

2. In-depth Interviews •

Search strategy

We used the advanced search functions on different portals. The setting was in Viet Nam, and the publication dates selected were from January 01, 2015 to March 04, 2020. Keywords of “Hướng dẫn” (guideline), “Điều trị” (treatment), “Dự phòng” (preventive), and “Phòng chống” (prevention) were used to search for guidelines issued by the MoH of Viet Nam. MeSH terms were used to filter publications on PubMed, Cochrane, and EPPI Center. The terms used for searching were: •

evidence-based healthcare evidence-based practice evidence-based medicine evidence used in policy policy development

Study records

a. Data management: Zotero software was used to manage records and data for the review. Microsoft Excel was used to keep track of all extractions from the publications under the review. b. Data collection process: Microsoft excel was used to manage extracted data from included sources of evidence. Main information of each publication was recorded, including: •

Title Authors Year of publication Country of origin Aims/objectives Study population/sample size Methods Key findings related to the research question on evidence use and policy development

Review of documents

•

Study participants

Key stakeholders from the MoH, government institutions, hospitals, universities, international organisations, private sector, and professional associations identified to have experience in policy-making process were approached and invited to participate in the study. •

Sampling

Purposive sampling method was used to develop the initial list of potential study participants based on the literature results, internal engagement reviews and our current relationships with key stakeholders. We selected potential participants based on work position, their expertise, and experiences in the development of policies related to clinical, preventive and laboratory issues. During the interviews, we also used snowball sampling to identify further respondents from interviewees. •

Consent process

The researcher approached potential participants and invited them to participate in the interviews. Prior to the interviews, each participant was provided a copy of the Participant Information Sheet, which explained the study and details of study participation. The researcher then explained the information and answered any questions from the participants. When the participants agreed to participate, they were then asked to sign the Informed Consent Form. •

Ethical consideration

The study had obtained ethical approval from the Oxford Tropical Research Ethics Committee (OxTREC) and the Ha Noi University of Public Health (HUPH), Ha Noi, Viet Nam before the in-depth interviews were conducted. •

Interviews

All the interviews were conducted using a semi-structured guide which included questions exploring interviewees’ demographic information, their roles in policy development, their perceptions and knowledge on policy development process, on evidence use, and on issues related to engagement between researchers and policy makers (for more details, see Appendix 1. Question guide for in-depth interviews with external key stakeholders). Audio was recorded for most of the interviews when interviewees provided consent to be recorded; otherwise, note-taking was used to record key information of the interviews.

After reviewing the MoH documents, we also searched for references cited in the documents for crosschecking to see how policy makers applied evidence in developing their documents.

•

Given the limited number of published case studies focusing explicitly on evidence use in policy development in Viet Nam, the study team decided to look at the documents that had emerged from the search, to see if we could implicitly deduce how evidence was used to inform those publications. During the process of reviewing published case studies, we also tried to look for policy analysis frameworks to understand the theoretical approach to policy development that were being used.

Audio recordings and note recordings were then transcribed for data analysis. We used a thematic analysis approach to analyse data generated from the interviews. We followed the following steps for the thematic analysis: familiarising with data, coding, generating initial themes, reviewing themes, defining and naming themes, and writing up21. We used NVivo 12 Pro software to manage the coding of the data and further analysis.

Data analysis

Chapter 3. Results 1. Literature Review 1.1. Document Categorisation and Requirements on Evidence Use in Policy In Viet Nam, the National Assembly issued the law No 80 on June 22nd, 2015 on endorsement of legal documents16 and the MoH issued the decision No 4068 dated July 29th, 2016 regarding the guideline on developing diagnosis and treatment clinical/care pathways22. The Law on the endorsement of legal documents clearly defines the system of legal documents of Viet Nam which includes 15 categories of documents at different levels, but does not include the decisions issued at the ministerial level. This means that the MoH decisions regarding technical guidelines do not fall under any of the categories of ‘legal documents’. The law indicates that institutions, organisations, National Assembly delegates are responsible for conducting scientific research studies on related issues, and reviewing international information, materials, treaties, serving for building up proposals on law and ordinance development. Information from other countries should be referred to and if needed, Vietnamese institutions and individuals in charge will need to submit related documents and information16. According to the MoH decision regarding the guideline on development of diagnosis and treatment clinical/ care pathways (decision No 4068), “clinical data used in professional procedures should be cited from available clinical guidelines, protocols, technical procedures, nursing caring procedures, … for the purpose of reference searching. Original documents should not be included in the proposals…” The guideline also provides clear definitions of clinical guidelines, protocol, medical procedures, nursing care procedures, standard operating procedures and clinical/care pathways22.

1.2. MoH Documents

CHAPTER 3. RESULTS

In total, we identified 87 eligible documents related to preventive medicine, lab procedures, diagnosis, and treatment. A document was considered for eligibility screening if it included references, or citations, or both. At this stage, we found 43 documents with a list of reference and 44 documents without. We included only those documents with citations for analysis in the literature review (n=11). Figure 5 describes details of our search and selection process.

How Evidence is Used to Inform Health Policy in Viet Nam

Included

Eligibility

Screeening

Identification

All the MoH documents were in Vietnamese language. 20 documents under the review included the list of editorial committee members, committee chair/chief editor, and secretariat. In such documents, the editorial committee members were all Vietnamese. They were specialists and leaderships from the MoH and experts in related areas from hospitals, universities, international organisations, and other related institutions. Some documents also acknowledged the technical or financial support of international organisations and pharmaceutical companies. MoH portal searching with keywords: Guideline (n = 114) Treatment (n = 35) Prevention (n = 64)

MSA portal searching with keywords: Dx and Tx guideline Technical procedure Preventive medicine (n = 54)

VAAC portal searching by year: from 2015 to 2020 (n = 84)

Within the documents themselves, the most common sections where we found citations was in the introduction/background, and causes and summaries of clinical symptoms. It was much less common to see citations in the sections that discussed diagnosis and treatment. When specific data were mentioned, they were normally cited, but this was also more likely to occur in the introduction and background sections of the documents24–32. For technical procedures, user manuals of test kits were often cited. However, when process recommendations were included in those technical procedures, key steps in the process did not include any citation26.

Documents after duplicates removed (n = 198)

Documents screened to find technical documents related to preventive medicine, Dx and Tx and lab procedures (n = 153)

Documents excluded for irrelevance (n = 66)

Documents screened for eligibility (including list of reference, or citation, or both) (n = 87)

Documents excluded as for no citation included in the documents (n = 76)

Documents included in the final review (n = 11)

Figure 5. MoH documents search and selection flow diagram 23

Half of the documents (n = 43) identified under the review included reference lists but not any citation while only 11 documents included citation in some parts of the documents. Other guidelines/regulations/circulars related to the topic and issued by the MoH were often referred to in almost all of the documents reviewed – indicating that the MoH often reviews and consults with its own documents before issuing new ones. Both English and Vietnamese textbooks were often cited in these documents, but were not accessible online for cross-checking. WHO guidelines related to the topics were commonly cited – indicating that WHO is a significant resource in developing these kinds of documents for the MoH. One document had citations but did not include a reference list, so we could not compare with the reference to check how evidence was used in the guideline23.

Among 11 documents reviewed, only one included citations in all parts of the document33. When comparing the published documents with the cited references in the document, we could see similarities between them. This indicates that the reference material was genuinely consulted, and the reference material was accurately used.

Citation was rare: • 11 out of 87 documents included citation • 1 out of 11 documents included citation in all parts of the documents

Frequently cited and referred documents: • Other MoH documents • WHO guidelines and textbooks

Citation included in: • Introduction/ background • Causes or symptoms

Comparison between MoH documents and references: Similarities -> References may had been genuinely consulted and accurately used

Figure 6. Summary of the review of MoH documents 24

How Evidence is Used to Inform Health Policy in Viet Nam

1.3. Non-MoH Documents

Database searching with keywords: evidencebased medicine/ evidence-based practice/ evidence-based healthcare/evidence used in policy/policy development and Viet Nam (n = 67)

Additional documents identified through reviewing publications (n = 3)

Documents screened for eligibility (n = 70)

Screeening

Identification

After conducting searches on PubMed, Cochrane, EPPI Center, and Health Evidence sources, we found duplicates among those sources and PubMed covered all documents that were available on the other three sources. Therefore, we finally included all related documents from PubMed searches. We reviewed published case studies, reports, and reviews by national and international agencies to identify information about how scientific evidence informed public health policy in Viet Nam. Using the search strategy outlined above, we found 67 documents, and identified three additional records while reviewing publications (total n = 70). We checked the abstracts of all documents, and then excluded 16 publications from full-text analysis because they were either not relevant to the study topic, or the setting was not in Viet Nam. Finally, we reviewed full text of 54 documents to understand different aspects related to evidence use and policy formulation.

Among the 54 documents reviewed, 36 were aimed at providing evidence and recommendations to healthcare managers and policy-makers about the burden of diseases, existing barriers, cost-effectiveness of different interventions, people’s knowledge, and recommendations on what should be done in order to push the development of new policy or changes of existing policies for a better policy implementation34–69. When selecting search criteria, we looked for documents that could demonstrate or explain how evidence is used to inform policy making in Viet Nam. It was immediately clear that there were few publications that focused on this topic explicitly. We found two documents that included evidence use in policy making as one of the main focuses3,70. For one of these two documents, the use of evidence in policy making was a secondary focus – with implementation of policy being the primary. However, they described how decrees were developed based on the situation where the implementations were taking place and survey results after implementation evaluation and before the development of a compulsory decree70. In this section, we will present findings related to: -

Provision of evidence, Issues related to the application of EBPs, Use of evidence in policy formulation, and Roles of stakeholders and advocacy approaches in policy making.

1.3.1. Evidence Provision For two thirds of the documents we identified in the literature search (n=36), the purpose of the documents was to provide evidence to the government. They did not explicitly discuss how evidence is used to inform policy making, or they did not explicitly or implicitly discuss the policy making process. For the most part, they were documents that discussed the author’s own research and research findings. Therefore, these documents did not reveal much about evidence informed policy making practice in Viet Nam.

Documents excluded (n = 16)

Included

Eligibility

1.3.2. Evidence Based Practice Application

Full-text documents assessed (n = 54)

Evidence provision (n = 36)

EBP application (n = 4)

Evidence used in policy (n = 2)

Stakeholders' roles and advocacy approaches (n = 12)

Figure 7. Non-MoH document search and selection flow diagram

Evidence-based practice (EBPs) is an emerging theme in hospitals in Viet Nam, and the four documents that were included in this review explored some of the issues related to EBPs71–74 . These issues are not identical to facilitating evidence informed policy making. As reported by Hiep Thi Dao et al., 91.6% of participants in her study (348 out of 380 participants) thought that EBP was necessary and useful, and 89,8% (342/381 participants) agreed that EBP could help in improving health care for patients71. Also, there was agreement that it was important to understand the knowledge, the willingness, the skills and attitudes of those who were direct actors of EBPs in hospitals in order to enhance practices74. Additionally, some of the barriers and challenges to implementing EBPs in hospitals are languages of documents, accessibility to the information, time availability, organisational support, skills, and existing perceptions71,73.

How Evidence is Used to Inform Health Policy in Viet Nam

1.3.3. Evidence Use in Policy Two of the publications that came out of the literature search clearly indicated the inclusion of evidence in decision-making to inform health policies in Viet Nam3,70. We looked closely at both publications to explore how they presented the use of evidence in policy development process in more detail below. Vuong et al.: Strengthening advocacy efforts with empirical evidence3 In the area of illegal drug treatment, Thu Vuong et al. described different approaches used for engaging with local authorities and policy makers in order to apply their study findings into practical interventions. Their research was a cost-effectiveness study focusing on the comparison between center-based compulsory treatment with community-based voluntary methadone maintenance treatment in Viet Nam. In this article, the authors explained details of each step they took in their advocacy and engagement with authorities and policy makers. Policy makers were invited to participate in the very beginning steps of the study and throughout the whole course of the study implementation. Thu Vuong et al. explored the L-E and the I-I-I frameworks respectively in looking at how they should engage with policy makers in the whole course of their research study and how different factors should contribute to the impacts of their research findings dissemination on decision-making3. They were also concerned about balance – and looked at ways that they could maintain study objectivity and still follow the L-E Framework’s requirement of different actors’ participation in the policy making process. One strategy that was implemented to mitigate this risk was to establish a review committee that included representatives from local stakeholders, and had a very clear mandate: ‘to provide advice and support, rather than direct the research findings’3. They had some recommendations to consider with respect to policy engagement and influence towards treatment of illegal drug users in Viet Nam. They argued that it was important to capture all factors impacting decision-making to define the interventions and advocacy approach to facilitate effective incorporation of evidence in decision-making. They pointed out that all of the factors in the I-I-I framework were important (ideology, interest, information), but it was also necessary to identify the challenges faced by policy makers and support in the uptake of evidence into policies to plan for a dissemination strategy. Context was also a very important component contributing to the success of advocacy for the incorporation of evidence into policy changes. For their study, the availability of legal frameworks, the emergence of policy champions in drug treatment, and Viet Nam’s transition from a low-income country to a middle-income country (which led to withdrawal of funding from international donors) required changes in drug-related policies. At the same time, the global pressure for a more humane treatment for drug users was also a factor pushing the government of Viet Nam to change their policies. Viet Nam's transition to a LMIC Legal framework and global pressure

Changes in drug related policies

Emergence of policy champions

Figure 8. Driving factors for policy changes

Involving policy makers and local authorities in the whole course of the research study helped persuade policy makers to use study findings in the development of policy changes. The study findings were disseminated in different workshops to different stakeholders helping them to understand the situation and to use evidence-based proposed approaches to solve the problems. From the results of their own literature review, they concluded that “factors increasing likelihood of policy makers accepting research findings included timeliness of research, relevance of topic, rigour of research methods, unequivocal nature of the findings, the accessibility of the results presented in policy-accessible language, long-term trusting relationships between the individual researchers and policy makers”3 (see Figure 9 below).

Factors increasing likelihood of policy makers' accepting research findings

Timeliness of research

Relevance of topics

Accessibility of results presented in policy-accessible language

Unequivocal nature of findings

Rigour of research methods

Long-term trusting relationship

Figure 9. Factors increasing likelihood of policy makers' accepting research findings Tran et al.: Micronutrient deficiency control in Viet Nam from policy and research to implementation70 Micronutrient deficiency was a nutritional issue in Viet Nam. Realising the situation of micronutrient deficiency, in 2001, the government of Viet Nam took actions to improve the situation, including the fish sauce fortification program. To support the implementation of this program, the National Institute of Nutrition (NIN) received funding from the Global Alliance for Improved Nutrition (GAIN) to support production of fortified fish sauce and education programs about micronutrient deficiency and to call for fortification via mass-communication campaigns. The food safety law was approved and endorsed by the National Assembly in 2010 which included the content of mandatory fortification. The Ministry of Health was then assigned the task of developing a mandatory decree and they needed updated data on the population-level for nutritional status, consumption surveys, and micronutrient surveys to inform the mandatory decree70. In this instance, the policy development of the Ministry of Health is an evidence-based process. In this publication, the authors also described the development of legal frameworks, the application of regulations into research, and the implementation of regulations into practices in the area of micronutrient deficiency control. After the government had issued new policies and guidelines, the national survey programs were conducted in order to capture data about the status of micronutrient deficiency in Viet Nam and to foster efforts to reduce micronutrient deficiency rates. Government ownership and leadership played a key role in facilitating the success of the implementation of these interventions70.

How Evidence is Used to Inform Health Policy in Viet Nam

1.3.5. Roles of Stakeholders and Advocacy Approaches As reflected in the theories of policy making outlined above, the policy making processes in health in Viet Nam involved the participation of different key stakeholders. In a paper by Gomez and Ruger, “The Global and Domestic Politics of Health Policy”, they examined the international and domestic linkages related to health policy making in selected LMICs, and the roles of international and domestic institutions in those countries. Their study showed that apart from public officials and international donors, other key informants like activists, local non-government organisations, civil society organisations (CSOs), multilateral health agencies, or local authorities also had a role in each step of the process. In Viet Nam, in particular, the authors discussed the strong role of CSOs in influencing policy for selected infectious diseases interest groups such as HIV, tuberculosis, and malaria, and the authors pointed out a change in Vietnamese government policy which had sanctioned the establishment of these CSOs, leading eventually to changes in policy such as passing the 2006 Law on HIV/AIDS Prevention and Control4. It is clear that non-governmental actors can and do have a direct influence on policy in Viet Nam, when they are working in partnership with the government and within the frameworks that the government has in place to support that engagement and influence. Politics also played a role. In the conduct of research studies on sensitive issues like HIV/AIDS control and drug treatment, political challenges were unavoidable in Viet Nam. Thu Vuong published a series of manuscripts about compulsory drug treatment in Viet Nam3,46,55,75, and one paper specifically discussed the political challenges that arose from this kind of research75. Their study faced a lot of challenges throughout the whole process of the research, including design, approval of local authority, ethical approval, data collection, and analytic strategy. As a result, the team had to navigate a path between science and politics. Partnership and engagement with local authorities during the study processes were useful tools to overcome such challenges faced by the study team. Other important factors contributing to the success of the program were the progressiveness of the local leadership, the government’s trust in the organisation, and the longterm technical and financial support of the organisation to the locality75. Taking advantage of long-term relationships with policy makers, the organisation promoted interactions between their researchers and policy makers, at both institutional and individual levels3.

Public officials

Local authorities

International donors

Advocacy approach was mentioned in two papers; one focused on improving breastfeeding policies76 and another aimed at enhancing advocacy for policy change in nutrition5. The use of explicit advocacy approaches, the creation of a strategic group of actors, and the realisation of critical tasks were advocated for the improvement of breastfeeding programs. France Bégin et al. described the establishment of the Global Breastfeeding Collective (in short, called Collective) in 2018 led by UNICEF and WHO with the participation of organisations all over the world. Aimed at enhancing breastfeeding, the Collective had different strategic advocacy approaches with national governments to create influence on their actions. They created seven key policy actions focusing efforts to promote breastfeeding that countries needed to follow to achieve the World Health Assembly’s breastfeeding targets. The Global Breastfeeding Scorecard was one of the advocacy tools developed by the Collective to be used for collecting information related to the implementation status of these seven policy actions in all countries globally76. This can be linked to the MSF framework, which indicated the impacts of global events on policy agenda setting under the politics stream9. France Bégin et al. used Viet Nam as an example of how global events impacts policy agenda setting to promote breastfeeding. In particular, they highlighted the extensive knowledge exchange between China and Viet Nam regarding the policies on maternity leave, workplace lactation support and breastfeedingfriendly health systems. Apart from the role of global events, this article also emphasised the importance of engagement with relevant sectors and groups to increase the voice of the organisation for policy impacts76. Isabelle Michaud-Letourneau et al. also discussed how to improve advocacy for policy change5. They pointed out the key components that could help strengthen advocacy for policy change or development. The core components of successful advocacy approaches for policy changes or development they mentioned included: • • • • • •

overcoming the fear of engaging in advocacy, building capacity and understanding of policy processes, engaging early on with policy makers and engagement during the whole course of the initiatives, using explicit and systematic advocacy approaches, supporting evidence generation and strategic use of data, and frequent communication with and among actors5.

Review of both MoH and non-MoH documents shows that there has been limited knowledge generated explicitly on how evidence has been used in policy development in Viet Nam. However, we have been able to learn about different aspects related to or having certain influence on policy formulation, which might help us to develop our policy engagement plan at OUCRU. Thanks to the two publications focusing on the application of evidence into policies in Viet Nam, we are aware that policies in Viet Nam might be evidence-based, but there is not enough evidence in the literature to show this definitively. We therefore explored more about this aspect during our interviews with key stakeholders, of which the results will be presented in the following section.

Key stakeholders Multilateral health agencies

Local NGOs and CSOs

Figure 10. Key stakeholders in policy development

How Evidence is Used to Inform Health Policy in Viet Nam N = 18 Gender Current position experience

Photo: Workshop to Review implementation Phase 1 of National Action Plan on AMR in 2017

Expertise

2. In-depth Interviews This section will focus on findings analysed from 18 in-depth interviews conducted within the framework of the study. In this section, we will present content related to the following 9 categories: • • • • • • • • • •

Demographic information Perceived concepts Successes of Viet Nam in policy development COVID-19 Policy development Evidence use related issues Research related issues Engagement between policy makers and researchers OUCRU’s achievements and gaps Recommendations for OUCRU’s engagement with policy makers.

Educational level

Male

Female

1 - 3 years

4 - 6 years

7 - 10 years

Unknown

Animal Health

Clinical Pharmacy Education

Epidemiology

Laboratory

Medical Education

Management

Pediatrics

Infectious Diseases

PhD.

Master and Medical Doctor

Table 1. Demographic information of interviewees

2.2. Perceived Concepts We explored what the stakeholders thought a policy was, and who they thought policy makers were. We did not ask these as explicit questions, but analysing the language that they used and the ways that they discussed these concepts in the interviews helped us gain a better understanding of their perspectives.

2.1. Demographic Information Between May and September 2020, we conducted 18 interviews with key stakeholders from the MoH, hospitals, institutes, private sector, international organisations, and professional associations who we identified to have experience in policy development processes. Among 18 study participants, 16 participants provided consent for recording and two just agreed with note-taking. 4 of these 18 study participants were from the ministry level, 8 held the role of leadership at institutional level, 4 from international organisations, 1 from a professional association, and 1 from the private sector. Brief demographic information about work places, their roles, gender, experience, expertise, and educational level is described in Table 1. The respondents’ areas of expertise varied, but as for the focus of the study, most of the interviewees were experts in infectious diseases. They were all senior, and 15 out of 18 interviewees had a PhD level of education.

Terms

Concepts

Policy

Legal documents such as laws, decrees, and circulars Technical documents such as diagnosis and treatment guidelines, drug use guidelines, or standard operation procedure.

Policy maker

People who signed decisions, or People working at the higher level (MoH), not themselves, or People who participated and chaired the policy development process

Table 2. Perceived concepts on policy and policy maker 31

How Evidence is Used to Inform Health Policy in Viet Nam •

What is a policy?

When we raised the term ‘policy’ with our interviewees, most of them considered it to be too general, and asked us to specify exactly what kind of policies we would like them to focus on. The stakeholders informed us that there were different types of official documents that can be considered ‘policy’. Stakeholders tended to separate legal documents such as laws, decrees and circulars from technical documents such as diagnosis and treatment guidelines, drug use guidelines, or standard operating procedures. These ‘legal documents’ are developed by multi-ministries, not just by the MoH, but health related ‘technical documents’ are mostly developed by the MoH and the institutions and hospitals under the MoH. Some stakeholders spoke about both the ‘legal’ and ‘technical’ documents interchangeably. However, many stakeholders only had practical experience with ‘technical documents’, and in our interviews we most focused on these kinds of documents, or policies.

“In fact, if you ask about policy in general, it would be too broad…. In terms of technical guidelines… you should distinguish a little bit. When mentioning policy, when you mentioned policy at the beginning, I was confused because I thought of legal documents…” (Senior specialist) In spite of this, there was still some disagreement among stakeholders about what constituted a ‘legal document’ versus a ‘technical document’. A technical guideline was defined by a representative of a professional association as a legal document, while defined as more technical, not a legal document by a senior specialist.

“And when the MoH issues their decisions, they are legal documents. So this is the request from the MoH, the Government and we must follow it… But when the Government makes a decision, it becomes a legal document of Viet Nam, so all organisations must follow it…” (Institutional leadership) “Technical guideline is a little bit different. It is not as legal as the other documents, but it is more technical. Because it is called guideline, it is not mandated but it is used to encourage people to follow.” (Senior specialist) •

Policy maker was defined by our key stakeholders working at institutional levels as people who signed decisions, or people working in the higher level, particularly in the ministry level.

“In general, with policy makers, scientists whenever raise up any issues need to get solutions, only so should they raise up any issue. If you just raise up the issue without any solution, it would take… That’s right for an issue. Policy makers just need to sign…” (Department leadership)

2.3. Successes of Viet Nam in Policy Development

Awareness of strengths and weaknesses of policy development

Improvements in evidence use

More involvement of scientists in the whole process

Quick action

Timely exposure to updated information

More application of research studies into practices

Figure 11. Viet Nam's successes in policy development

Who is a policy maker?

Five of our senior stakeholders were fully aware of their roles as policy makers.

“For all of the guidelines [….], I’m the committee’s chairperson.” (Institutional leadership) However, it was interesting to observe that some of our stakeholders (n = 9) did not consider themselves to be policy makers, despite being clearly involved in the policy making process. They were the ones to develop the procedures but they thought policy makers were the ones to whom they needed to submit the procedures and who provided approval.

“… The procedures we have developed are trusted by policy makers. They did not revise much…” (Institutional leadership) 33

Nine of our stakeholders mentioned that Viet Nam has made achievements in policy development. They had made awareness about strengths and weaknesses of policy development, improvement in evidence use, and had encouraged more involvement of scientists in policy development and had provided quick actions.

“Generally speaking, we had the MoH’s guideline within some days. Why is this guideline valuable? As it got opinions from scientists specialising in that area, it was not that someone copied it from somewhere and made it up.” (Institutional leadership)

How Evidence is Used to Inform Health Policy in Viet Nam The interviewees mentioned that Viet Nam had been exposed to updated information at the same speed as other countries, that research happened in Viet Nam, and research outcomes were applied in Vietnamese treatment practice.

“In fact, normally, currently, diagnostic tools for […] currently are quite similar and the information of Viet Nam has been updated at the same level with the whole world.” (Institutional leadership) “In fact, at universities and institutes, lecturers have also conducted lots of scientific research studies. And they have done quite a lot and also applied it into reality. I’ll take one of the Viet Nam’s improvements as an example. For example, in terms of hand foot mouth diseasei, Viet Nam seems to have gone ahead of the world. The […] Hospital has applied dialysis for patients with hand foot mouth disease and succeed. In dengue treatment as well, there have been no recommendations on using synthetic colloid solutions in treatment of dengue shock, but Viet Nam has used it and saved lots of patients.” (Institutional leadership) Four stakeholders mentioned that policy development in Viet Nam had improved and it was faster than in the past.

“I’ve seen the difference compared with 10 years ago. 10 years ago, sometimes diseases were everywhere, but it took a very long time to have an MoH’s guideline. But the MoH’s response has been very quick recently… Like the recent case, Minh Chay Pâté caseii. In Viet Nam, this disease was rare, so there was no guideline for treating botulism. When there was a series of cases, within less than a week, the MoH issued a temporary guideline…” (Department leadership) One respondent in particular mentioned that in the past, policy makers had included evidence just based on their preference – looking for the evidence that agreed with their preferred strategy – whereas now the approach was more objective. Stakeholders joining in developing policies needed to review existing literature and prepare materials. Committee members could ask for reference sources, especially when they had different opinions. And if a reference could not be shown, the idea might be rejected.

“Basically I think it’s totally different from the agreements in the old days. In the old days, the Health Insurance invited experts and it was totally based on the experts’ consensus. And the agreements were totally based on their own knowledge, points of view, the experts’ points of view. There were not agreements based on preparation of materials. But nowadays, building those lists relies on a base, even things like I’ve said, not all of them have research, reports, but you prepare materials and you reach agreements based on those materials. If you don’t agree, you have to show which part in those materials you have different information. Some participants are professors, but when they give their personal opinions, because we have prepared materials, we have the right to ask them where they get their information and if they cannot tell you, their opinions are not accepted. It’s not as in the old days. In the old days, I just smiled and said that when I looked at the participants, I would know what drugs would be approved, what drugs would not.” (Department leadership) As the interviews were mostly conducted in May and June 2020, between the first and the second and third clusters of COVID-19 cases in Viet Nam, many respondents used examples from COVID-19 to illustrate the policy development process. Stakeholders considered Viet Nam to have been very successful in controlling COVID-19 and that policy making was one of the key factors contributing to that success. Stakeholders mentioned that hospitals and institutes played crucial roles in developing emergency policy recommendations – demonstrating the decentralised decision-making, and high levels of trust in the Vietnamese health policy making process.

“The thing is that the MoH completely trusts [us] as a core team in fighting against epidemics. So when we produce a guideline, the MoH will completely agree and when they agree, all hospitals need to follow. And that’s how we succeed.” (Institutional leadership)

2.4. COVID-19 The COVID-19 outbreak challenged the whole world including the health sector of all nations. The development of guidelines in the context of an outbreak was also challenging to policy makers in Viet Nam. The first policy responses, including treatment guidelines happened before the first case appeared in Viet Nam. Therefore, at that time policy makers had no access to direct evidence, no diagnostic tools and as a new disease, there were no pre-existing treatment guidelines. However, with experience in controlling outbreaks (including SARS), Viet Nam had been able to develop laboratory procedures, surveillance guidelines, and diagnosis guidelines very early. The experiences of clinicians, laboratory staff and preventive medicine departments, as well as the MoH themselves in dealing with previous similar public emergencies such as SARS, and influenza were considered a source of information that could be used for the development of COVID-19 related guidelines.

“Viet Nam had issued a guideline before WHO did” (Institutional leadership) “Hand, foot and mouth disease (HFMD) is a common infectious disease that occurs most often in children, but can also occur in adolescents and occasionally in adults. In most cases, the disease is mild and self-limiting, with common symptoms including fever, painful sores in the mouth, and a rash with blisters on hands, feet and buttocks. However, more severe symptoms such as meningitis, encephalitis and polio-like paralysis may occur.”77 ii Minh Chay Pâté case: a series of patients with botulism following having pâté made by the company called Minh Chay. i

“The scientific evidence for that disease was not much. But we had the evidence serving for our preventing an emerging epidemic like COVID-19, which was a dangerous emerging disease. Correct? It’s not the evidence for that disease, but for other similar diseases. We had that because we had much reinforcement on the system of prevention of infectious diseases and research on infectious diseases, then… it’s not all of a sudden that we could do isolation. For example, we did isolate SARS in the past, if it was not an evidence, what would be? We were able to fight epidemics, we have combatted many epidemics, from SARS to H7N9, to other epidemics, so we just went ahead and applied it.” (Institutional leadership)

How Evidence is Used to Inform Health Policy in Viet Nam

COVID-19 Evidence Sources

Experience

Available samples

Publications

COVID-19

SARS and MERS

National database

Official sources

Domestic cases

National statistics

“….the data on population, the General Office just keeps the data. Right? Up to now, it has been very difficult to ask for data. But during COVID-19 outbreak, it was the first time that we have had access to that database… And data related to movement, for example, the telecommunication sector has it. We do not need phone numbers, we just need to know how people move to evaluate the risk of different areas and the risk of transmitting diseases from this area to another. We could not access. But thanks to this outbreak, we could access this database…” (Department leadership) Stakeholders did rely heavily on guidelines, standard operating procedures (SOPs), and recommendations issued by international bodies such as WHO or US-CDC in the health policy development process. However, there was also a requirement that SOPs developed internationally must be tested in Viet Nam before they are implemented on a national basis.

“A: COVID-19 is also a problem even though for COVID-19, the procedure has been available even before the first case in Viet Nam.

Figure 12. COVID-19 Evidence Sources Later in the pandemic, once COVID-19 cases were being recorded in Viet Nam and other countries, Vietnamese stakeholders looked to WHO guidelines, or guidelines issued by professional associations, or governments of other countries. These were considered official sources of evidence and were commonly used in the development or revision of Vietnamese guidelines related to COVID-19. Another source was scientific evidence from research studies and publications on not only COVID-19 but also SARS and MERS (see Figure 12).

“Not mentioning severe or minor COVID-19, it’s mainly about the fact that for severe COVID-19, the International ICU Associations in Europe and North America have issued a recommendation and in the recommendation, they have also relied on evidence of research, clinical research and they have provided the level of recommendation based on the level of evidence. That’s one thing we have referred to in order to add to our guideline. (Department leadership) Information from domestic cases was also used to update diagnosis and treatment guidelines.

“For other epidemics later in Viet Nam, this COVID-19 outbreak, in fact, later there were some cases in Ho Chi Minh city, so we did make reference to introduce it into the guideline.” (Department leadership) Stakeholders used data from multiple sources to inform policy making in response to COVID-19. Notably, one stakeholder mentioned that they were exceptionally granted access to specific datasets in order to facilitate their risk assessment and rapid response in policy development. This could be done thanks to the wellstructured working mechanism of the government with detailed task assignment, rights, and responsibilities of each committee member and sub-committee. It is also an example of the multi-sectoral efforts in mobilising data and evidence for decision-making in rapid outbreak response.

Q: Yes. You’ve just said before applying any procedures of WHO and US-CDC, they must be tried on our samples. A: Right. Q: What about for new diseases? What do you do? A: Yes, for new diseases for example like COVID-19, it is very lucky that we did have … we did have the samples from SARS in the past which we can use. But for other new diseases which we haven’t had any evaluation on… Firstly, when we haven’t know how to evaluate sensitivity, we should evaluate its specificity to see whether it has cross-activity with other types or not. If it doesn’t have it, we could feel comfortable that it could run but our sample is negative.” (Institutional leadership) This indicates that international sources are valuable, but that local context is also important to the policy making process in Viet Nam. However, there is also flexibility in the process, as indicated by the use of SARS samples to test the COVID-19 SOPs before any SARS-CoV-2 samples were available. The development process of COVID-19 diagnosis and treatment guideline (see Figure 13) was developed based on our stakeholders’ description. In general, stakeholders described the MoH as the focal point for the diagnosis and treatment guidelines process – the ‘technical documents’ described earlier. They set the agenda, convened the committees, and selected particular members/organisations/individuals to contribute. But they did not direct the writing of the actual policy – that was the responsibility of the final committee once convened. The role of the MoH was to finally approve the guidelines, and then to ensure that they would be implemented. However, the responsibility for writing the guidelines was devolved to the institutions/hospitals/ organisations/specialist contributors who were part of those committees. The committee chair approved the document before submitting it to the MoH for final approval and endorsement. The structure of the editorial committee was established including the chair of the VIDS being the committee chair, committee members from hospitals, VIDS and ICU Association and the secretariat from the MoH, representatives from key hospitals like BMH, NHTD and VNCH. WHO and US-CDC played a role of providing technical support to the whole development process. 38

How Evidence is Used to Inform Health Policy in Viet Nam Editorial Committee established by the MoH

Committee chair and members (clinicians, professional associations' members): - Aggregate information - Aggregate feedback

2.5. Policy Development Technical suppport from international organisations (WHO and US-CDC)

Secretariat: - Aggregate information - Aggregate feedback - Coordinate activities

One objective of the study is to understand the full process of policy development and factors that might have an influence in policy decision-making. From the interviews with stakeholders, we were able to identify four main components which stakeholders perceived were related to policy formulation: 1. Policy development confidentiality 2. Policy development determinants 3. Policy development gaps 4. Policy development roles and stages. Each of these components will be discussed in more detail below.

Presentation, comments and revision Approval by Committee Chair

2.5.1. Policy Development Confidentiality We were interested to observe that stakeholders had varying opinions about the confidentiality and sensitivity of both the policy making process, and the policy making agendas with which they were engaged. There was a clear split between the opinions of international versus national stakeholders as shown in Figure 14.

MoH's approval and endorsement

Figure 13. COVID-19 Diagnosis and Treatment Guideline Development

Policy making confidentiality

Stakeholders described this process and their participation in it as the basic process that was generally applied for other diseases. And – for COVID-19 – the same process was followed. The point here is that because they had an existing process and procedure, it was relatively straightforward (perhaps sped up!) to put these committees in place, and do the work for developing COVID-19 diagnosis and treatment guidelines. Specifically for OUCRU in responding to COVID-19, our stakeholders suggested that with our strengths in laboratory testing and research capacity, OUCRU should draw more focus on conducting serologic studies, supporting the MoH with OUCRU’s technical expertise in case definition, treatment guidelines and vaccine related information.

“I probably would say is very carefully designed serologic studies so that when there is another wave in the future, those serologic studies would be implemented in a way they get reliable information… Take this peace time we have right now, design a really really high quality studies, so that when the next round comes, we will implement them without having to spend so much of time planning and discussing.” (Institutional leadership) “… like OUCRU, you are much related to prevention of infectious diseases, have expertise so we might request the provision of COVID-19 related information, for example, on COVID-19 vaccine, on treatment, or other advancement, or treatment regimen of other countries which we can take reference from and apply in Viet Nam. Or there is any definition on cases or definition of surveillance cases, for example. Such information on technical issues would depend on the strengths of international organisations.” (Institutional leadership)

Confidential

International organisations

No regulation on sharing

Open

International organisations

Committee Chair

Committee members

Figure 14. Policy development confidentiality and sharing 40

How Evidence is Used to Inform Health Policy in Viet Nam International stakeholders were much less likely to share information about what policy drafting processes they were involved in with others – despite not being advised that they needed to keep processes or drafts confidential.

“But at the same time, we also need a kind of… also need to know because the guideline sometimes is very confidential information be shared. So there is probably no regulation, but still much information we can know, not others.” (Senior specialist) Meanwhile, the stakeholders taking the role of committee chairman or committee members all said that there was nothing confidential, the whole process was open and they shared the policy drafts with their colleagues in other institutions or international organisations to get their comments and to ask for more references or related supporting documents.

“Yes, it was totally open, we just shared it so that we could write the most complete and comprehensive draft, we just shared it. It was very open and the final goal was to make it the most precise to have the best guideline. Those things were not restricted.” (Department leadership)

2.5.2. Policy Development Determinants

Necessity of changes

Social pressure

Morbidity patterns

Policy development determinants

Determination of actors

Legality to protect some groups

Financial resources

Healthcare system Personnel

“Then we build up a guideline based on the situation of Viet Nam, experience, typical characteristics of models, diseases, response ability, facilities, update of materials, etc.” (Department leadership)

2.5.3.

Policy Development Gaps

Stakeholders identified some constraints that could act as gaps in the policy development process, such as time constraints and financial limitations. For the context of an outbreak like COVID-19, the pressure of emergency, and thus time constraint, was very significant. In the case of emerging diseases, lack of information was another obstacle that our key stakeholders had to overcome. Lack of a standardised procedure for application of evidence in policy or for policy development was another issue that caused difficulties for stakeholders.

“… in emergency, like COVID-19 for sure, they have like a… they work 24/7 and come up to something every day or something.” (Senior specialists)

Infrastructure Available resources

A determinant is a factor contributing to the decision of developing or revising a policy. Stakeholders identified a number of different determinants that can contribute to foster the initiation of policy development (see Figure 15). All stakeholders confirmed that the ‘technical documents’ development process was triggered by the situation in country: e.g., when something was escalating, or when something got more public concerns. When something was identified as needing a new policy or revision of current policy, then policy makers would initiate the process of policy changes. Particularly, they would initiate developing a policy or revising an existing policy. Perspectives of multiple actors contributed to identifying these needs: not only the leadership but also the healthcare workers, lecturers, clinicians, etc. A strong and common determinant that was raised was the need of legal protection for healthcare workers by having formalised guidelines - when a risk was identified, that could trigger the process. During COVID-19, for example, preventive medicine played a very important role; however, in Viet Nam, there had not been any policies to protect or support preventive medicine staff, which urged them to think of developing a new policy to cover this matter. Developing any policy also depended on available resources in Viet Nam; thus, financial resources, infrastructure, healthcare system, personnel, morbidity patterns, etc., were also identified as important determinants of policy development.

Figure 15. Policy development determinants

“... when they provide a recommendation, they gather a panel of many experts in the related area. They will search for materials, analyse them and aggregate information and they’ll provide evidence. We lack that stage. We have done it, but it’s not standardised.” (Department leadership) Six of our respondents mentioned gaps caused by human factors: for example, people did not want to change, or things were decided by subjective opinions of a committee chair. Consensus was considered important, and so another gap identified was where there were many contrasting opinions and comments which would take time to resolve, and might delay the whole process.

“… whenever there’s a change even with good scientific evidence, it’s still not easy. There are always obstacles. It would be so strange if changes don’t face challenges. Changes always face obstacles which are unwillingness to change and fear to change. It’s normal. It’s everywhere and it’s reality.” (Institutional leadership) 42

How Evidence is Used to Inform Health Policy in Viet Nam One of our respondents mentioned that policies were frequently developed by healthcare experts working at higher levels while guidelines need to be applied at a very large scale across the whole country. This can produce another perceived gap or challenge, as people working at higher or more centralised levels may not be fully aware of the situation in the lower or more devolved levels.

“Yes, difficulties and barriers like different conditions in different areas. Sometimes, when we work in big hospital, we can’t imagine or can’t understand all difficulties of people working at primary level. That’s the issue that we are the most concerned about. When we issue any guideline, will they be able to do it at primary level?” (Department leadership) Unlike many other countries where technical guidelines are approved and published by professional associations, the institution providing approval for technical guidelines in the health sector of Viet Nam is the MoH, which is a management institution.

“But who exactly in the MoH approved them? Doctors in hospitals do surgery, why does the MoH approve it? However, the regulation is like that.” (Institutional leadership) The writing group needed to present their drafts and explain the evidence they had used, allowing for discussions and comments back and forth. Sometimes, the secretariat struggled to take note all of the comments and feedback, and this was identified as another gap.

“For some formulas, when they were presented, specialists would say that they relied on this or that research project with this outcome, so they chose this to add into it. Then the committee would come to agreement. The problem was that even the one taking meeting minutes, they couldn’t take note with such many details and nowhere to record them all. When the final guideline was issued, it couldn’t be seen in it.” (Institutional leadership) Two of our respondents also mentioned the gaps existing with legal documents which were the complicated system of legal documents and difficult language used in the legal system and thus it was very hard to implement such documents.

“… for one decree, one circular after being published, no one understands. In Viet Nam, it is very common that it is very difficult to understand decrees, circulars or legal documents for which there are always documents to instruct. Not only one document to instruct, but there would be instruction of instruction, instruction of instruction of instruction, and only after that will it be understood for the implementation.”(Department leadership) Our respondents proposed some solutions to fill these gaps, such as:

• Institutions conducting research studies within their capacity:

“It is not okay to say that we can’t do anything as we don’t have money. We still have to conduct research with our capacity” (Institutional leadership) • Leadership mechanism: Two of our respondents also talked about leadership, and suggested more leadership roles for those coming from local facilities, like hospitals or research institutions, during the policy development process, as well as some standardised processes to follow:

“… the work will be more effective if high level leaders are the ones who come from local facilities. That’s one advantage as they know the work. However, it’s not always like that. That’s the reality.” (Institutional leadership) • Separated groups with different functions in the development process:

“… to avoid doing non-standard things, for international organisations such as WHO or professional societies, when they provide a recommendation, they gather a panel of many experts in the related area. They will search for materials, analyse them and aggregate information and they’ll provide evidence.” (Department leadership) The information related to policy development gaps and proposed solutions was summarised in the table below:

Areas Finance

Financial limitation

Human resources

Personal opinions/decision Persons don’t want to change Contrasting opinions Capacity Systematic issues Time constraint

Emergency

• Building standards for policy development:

“I mean there must be standards or criteria to base on so they can add or remove or change things. I feel that this issue has not been standardised nowadays.” (Department leadership)

Proposed solutions -

Enhance evidence use Seeking technical support Separated groups with different functions in the development process Leadership mechanism Develop standard procedures Institutions conducting research within their capacity

Time pressure Lack of information

Procedure

Approval and endorsing institution

• Seeking for support from experts:

“… in order to change it, we would find other sources such as WHO or opinions from experts. And we respect the community’s benefits …” (Institutional leadership)

Gaps

No standard procedure Legal documents

Complicated system Hard to understand and thus hard to be implemented

Table 3. Policy development gaps and proposed solutions 44

How Evidence is Used to Inform Health Policy in Viet Nam

2.5.4.

Policy Development Roles and Stages

There was strong consensus among the 17 interviewees, that the policy development process was a continuous process with the participation of multiple stakeholders (see Figure 15). In the remaining interview, we did not discuss policy development process because the respondent only helped the government to implement interventions. From the stakeholders’ viewpoints, there were three groups who played the most important roles in the policy development process: the editorial committee chairiii, the secretariativ, and the writing groupv.

“In a committee, usually, the writers, the chairperson and the secretary play the most important roles.” (Institutional leadership) According to the participants, the initiation of developing a new policy or revising a current policy depended on real life situations in the country. There were multiple opinions discussed in the interviews: Some mentioned the initiation was from the MoH, some said the ideas were bottom-up, some said case by case, or it was more dependent on the situation, for example, for COVID-19, the initiation of COVID-19 diagnosis and treatment guideline came from the MoH.

“The idea here must come from the MoH as the MoH is the organisation with the highest level and responsible for…for the Vietnamese people’s health, if they see that it’s necessary and it’s not in Viet Nam yet, they will have to build it up.” (Department leadership) “90% policies are bottom-up.” (Institutional leadership) “There are diagnosis and treatment guidelines for almost all specialties which must have standards and be approved by the MoH for application. They are the basis for specialtyrelated and legal issues. But who raises them up? Of course, specialty institutions raise them up. However, for example, to develop guidelines such as COVID-19 to respond to situations, it was the request from the MoH. Take COVID-19 as an example, COVID-19 has happened nationwide, the request did not come from any institution or any association, but it must come from… The management of treatment-related policies currently belongs to management institutions.” (Department leadership) Our respondents informed us that before deciding which policy to be formulated, they had a committee to consider which topic should be prioritised for revision or development, or they would need to conduct situation analysis to evaluate the necessity of such issue.

“… when they have any issue related to health which they think should be prioritised, but how it could be prioritised, they would have another committee.” (Department leadership)

“… first, it needs to be based on a survey, depending on the content of that guideline, the survey scale should be big or small, the research method should be different depending on the level of the guideline. After having scientific evidence, now all is about evidence-based as people often mention nowadays, right? The process is quite the same. After having the aggregation, specialists here will make a report to the leaders for them to make a plan to approve the development of that guideline. Then we will need to submit it to the leadership of the ministry to get approval in terms of strategy, we will need to present all results, evidence to show its necessity to solve this issue, coming from actual situation or from demands of each related side. After that, to solve any problem related to the law regulated but we haven’t got any regulations, for example, after that, the Ministry leadership will approve the strategy, and at the same time, approve the establishment of the editing committee for that guideline, for that document.” (Senior specialist) When a policy proposal was raised up by a local institution, it would need to include recommendations on how to solve the problems, otherwise, the proposal would be ignored by higher level.

“When you raise up the issue, you have to propose the solution for them to evaluate whether your solution is appropriate or not. The leaders and policy makers are somehow like that. If you come to them and ask how this issue should be solved, they would ask for your opinions. If you do not have any ideas, why you come to ask them? That’s it.” (Department leadership) The focal point for any health related guidelines was specific departments in the MoH depending on the focus of the guideline, e.g., for diagnosis and treatment guidelines, the focal point was often the Medical Service Administration (MSA), MoH; for preventive related guidelines, the focal point was the General Department of Preventive Medicine (GDPM), MoH. The focal point would be in charge of managing and coordinating the whole process. And in some particular cases, the contact point would be allocated by the focal point to specialised institutions or hospitals. Depending on particular focus of the guidelines, the focal point would send invitation to related institutions, hospitals, professional associations, provincial department of health, management institutions to call for specialists’ participation as the committee members. For example, in terms of preventive medicine, NIHE, TIHE, Pasteur Institutes would be invited, and in terms of infectious diseases, NHTD, BMH, VNCH, CRH, VIDS, ICU association would be involved as committee members. “The Ministry leaders will approve the decision of establishing the editorial board to make it fast. The editorial board needs to have related members, apart from government’s management units’ representatives from related departments under the ministry, we also need the participation of experts in related area depending on the level of that guideline.” (Senior specialist)

Editorial Committee Chair: a leading expert in a particular area to play a role of chairing editorial committee meetings and providing final decision about a guideline’s content. iv Secretariat: coordinating the activities of the editorial committee, aggregating comments and feedbacks. v Writing group: established from the committee members, in charge of aggregating, analysing and selecting evidence and then developing the document iii

How Evidence is Used to Inform Health Policy in Viet Nam

The writing group established from members of the editorial committee would then be assigned to be in charge of writing particular parts of the guidelines. Before starting to draft the documents, the writing group would need to aggregate, analyse and select evidence to be used. International organisations, most commonly mentioned WHO and US-CDC, provided technical support to the development process.

“For some decrees, circulars and guidelines from the MoH, I joined as a member of the editorial committee. I was even assigned tasks such as finding materials, updating information and reading and writing.” (Department leadership) “And this sometimes we don’t work for the writing group, actually, we […] work in this meeting to give out framework or technical inputs, and then when we agree, we put a lot of comments.” (Senior specialist) “… Even for macro management policies, even when we join in developing policies related to organisations supporting clinical trial research, we also invited a lot of experts from international organisations to the conferences to share their opinions. It was not just Vietnamese organisations alone.” (Department leadership) Provinces or hospitals or institutions who were the ones to implement guidelines would provide comments and feedback to the drafts sent around by the secretariat. The secretariat would collect comments and feedback either via sending around the drafts or via workshops. The committee chair was always a senior expert in the area, often the chairman of the particular professional association like the VIDS, ICU, etc. As a committee chair, they would be responsible for the final version to be submitted to the MoH for final approval, decision and endorsement.

“After making a decision on developing or revising a certain policy, we will assign the focal point to a certain department, usually to MSA. After that, a writing team will be formed and responsible for finding materials, situation analysis and writing a draft. Selecting members of the team depends on topics. After the team finishes the draft, it will be sent to the editorial committee for comments. After having the committee’s comments, the writing team will revise it. Then workshops will be organised to collect experts’ opinions. The draft is also sent to all facilities for their opinions. After that, it’s also posted on the website of the MoH and Government to collect comments. All of the procedures follow the Law and regulations of writing a legal document.” (Institutional leadership) In short, the whole process of policy development in the health sector of Viet Nam could be summarised and visualised as in Figure 16 :

Figure 16. The process of policy development

2.6. Evidence Use Related Issues As reflected in the literature review, we found that policies in Viet Nam might be evidence-informed, and out of 18 interviewees, 17 interviewees confirmed that all policies had been developed using evidence. It reflects that in Viet Nam, the value of evidence has been well recognised and evidence-informed decision-making has attained more attraction as a key component in the conceptual framework of policy formulation. Meanwhile, only 1 interviewee showed reluctance in affirming that policies were evidence-informed because the evidence use was case by case. This interviewee took three specific examples related to health insurance, tobacco, and COVID-19 to explain the differences in using evidence to serve for policy formulation. For the first two examples of which the effects did not come right away, the interviewee affirmed that evidence had often been neglected while policies related to COVID was evidence-informed. In the opinion of this interviewee, COVID-19 policies were more likely to be evidence informed because of the seriousness and danger of COVID-19.

“Normally if it doesn’t cause death, it would be ignored, no one would take care of it, right? Smoking is often mentioned, but smoking doesn’t cause death right away so no one cares while if smoking is banned, they will lose a lot of tax on cigarettes, which brings benefits to many people. But for COVID, if they were not isolated, and if they went around, it would cause a lot of deaths, so they have been put in quarantine.” (Senior Specialist) 47

How Evidence is Used to Inform Health Policy in Viet Nam

2.6.1. Evidence Sources, Availability and Selection Criteria

Pros

Seven of our stakeholders pointed out barriers to evidence accessibility like language barriers, limited quantity and quality of evidence in LMICs, and difficulties accessing publications from Vietnamese journals, which were not often available online or not stored systematically in house.

“How to find …material resources which were really meaningful. We didn’t always find out something good from Nature or some journals or there were some obstacles, difficulties in terms of languages.” (Department leadership)

Cons

Available evidence of similar diseases

Evidence from institutions

Good availibility

Language barriers

Low quality and quantity in LMICs

The respondents stated that they aggregated information and evidence for policy development from various sources, mainly from international guidelines and publications, meetings/workshops/seminars, experience and national databases. Out of 17 interviewees confirming evidence-informed policies of Viet Nam, 14 ranked WHO and US-CDC guidelines as the most commonly used official sources of evidence. The 3 other informants did not discuss the development of technical documents related to health issues, but about education, or technical support. National domestic evidence sources were listed such as the government’s official information, data of research studies conducted by institutions of Viet Nam, publications in Vietnamese journals and National Pharmacopeia. For evidence from national sources, data from surveillance, clinical trials and situation analysis were mostly mentioned by interviewees.

Publications on Vietnamese journals

Where to find meaningful information

Figure 17. Stakeholders’ concepts about the availability of evidence

For our key stakeholders working at the institutional level, the availability and accessibility of evidence that they needed was basically good. They could find information that they needed. But for information of indepth studies, they might seek for help from their PhD students who had accounts to access their university library in other countries.

Experience

Other institutes, NGOs, etc.

International guidelines

Evidence sources Workshops, meetings, seminars

National sources

“... I think with our faculty’s working conditions and with the current needs of information on drugs, our access is basically ok. When we need to find in-depth information for our in-depth research for example, it depends.” (Department leadership) “...Generally, it is enough” (Institutional leadership)

Government's official information National Pharmacopeia Prestige Vietnamese journals Research data conducted in Viet Nam

Publications

Figure 18. Stakeholders’ concepts about evidence sources 49

How Evidence is Used to Inform Health Policy in Viet Nam Our respondents also listed experience as one source of evidence used to inform policies.

“The difficulty in Viet Nam is that we don’t have any strict regulations in terms of science, it is not compulsory to use scientific evidence. That’s why the experts usually use many evidence sources in Viet Nam and rely a lot on their own experience” (Institutional leadership) With different sources and availability of evidence, our key stakeholders stated that evidence had been selected based on different criteria. Five stakeholders discussed how, before selecting any evidence to be used, they often evaluated the research studies, focusing on research method, study design, and sample size. Evidence was also selected based on the level of evidencevI (by four respondents), on the appropriateness with the context of Viet Nam (by six respondents) and on sources of evidence (by six respondents). Evidence from official sources would be always prioritised.

“For evidence, we mainly rely on evidence-based medicine, on the levels of evidence… it’s very clear… according to recommendations and evidence levels.” (Department leadership) “Additionally, we also screen research studies just published but we should focus carefully on the issue of the publication, on the design of the research. When we see the study design is appropriate, we can take it as the basis for further comments. Additionally, we also have our own experience about data. On malaria or antibiotic use, for example, we need to base on microbiological data of the hospital. And there should be interaction among hospitals during specialised workshops from which we could know their data from different areas. It is a kind of the basis for us to have evaluation for decision-making.” (Department leadership)

2.6.2. Evidence Use in Policy Development The stakeholders informed us that evidence had been aggregated, evaluated and selected by the writing group before they defended the draft with the editorial committee. International organisations were also the ones to aggregate evidence and to submit to the government of Viet Nam for consideration for changes.

“Or each partner can bring their own evidence to the table or whatever… what they know and what they think will be feasible in Viet Nam. And then we can help to put together that evidence into a policy brief and then propose to the government or whatever level. It depends on the objective of what level.” (Senior specialist)

Levels of evidence: “are assigned to studies based on the methodological quality of their design, validity, and applicability to patient care” with evidences from randomised controlled trials at level 1, downward to authorities’ or experts’ opinions at the lowest level78

According to our stakeholders, the decision to use evidence or not was the choice of the writers, and there were some motivational factors driving them to use evidence in policy development. For example, when the stakeholders needed to figure out the need for a policy, or when they saw the necessity of using evidence and the persuasiveness of evidence itself. Other drivers for evidence use were listed as available regulations and official requests or support from the higher level, especially from the MoH, to push up the application of evidence. In such cases, the use of evidence in policy development would be better enforced. How to refer to evidence also depended on each individual, whether they wanted to have references in the document or not.

“Now if you want them to use evidence, the higher level will need to request them to use evidence. … Let’s take COVID-19 as an example. They would use evidence when they needed. They did not know how to do, they were afraid of one death case, it would make a mess so they had to use evidence.” (Senior Specialist) One respondent said that the capacity to use real data in making policies was limited, while our stakeholders working in clinical fields confirmed that guidelines were updated based on data from actual cases in Viet Nam. Ideas about citation were also different between different groups of interviewees. Stakeholders working at institutional levels confirmed that citation was included in all draft versions, but not in the published guidelines. Whereas, according to our interviewees working at the ministerial level, there was no citation even in the draft version submitted. And the issue of citation was considered a shortcoming of the health system which would need to be fixed.

“I always cite references. In any of the procedures that I had developed, I always cited reference just like for international publications. However, when we submitted to them, they always had editors, but we didn’t know how they would edit.” (Institutional leadership)'' “Yes , in recommendations and guidelines or treatment guidelines, there is no citation of references and this is the limit I want to change. I’m looking for an example for you to know how a document looks like, no citation. You can see a list of reference materials here, but you cannot see any citation in the whole document. And this is a shortcoming as I cannot show that this content has been cited from which materials or research… They don’t make any reference in the drafts. They just write without citation. There’s only a list of reference materials.” (Institutional leadership) The study participants also recognised the limitations regarding citation in policies as one of the weaknesses in policy development of Viet Nam. They thought that with citation, the guidelines would be more persuasive towards clinicians, it would be easier for others to find out why such therapies had been recommended in the guidelines. They also suggested having standard mechanisms for developing policies which included who do what, compulsory components included in policies and that citation should be included. They wanted to have a standard guideline on how to develop evidence-based guidelines because they believed that such guideline would help to solve weaknesses of policy development in Viet Nam.

In terms of regulations on evidence use in policy development and citation in policy documents, there were contrasting ideas. Eight of the interviewees said it was regulated that legal documents must be evidence-based while only one respondent affirmed that there was no regulation on such. As discussed earlier, we have indicated different concepts of our stakeholders on the ‘legal documents’ and ‘technical documents’. Even when a technical guideline is not officially a legal document, it can’t be completely excluded from the category of ‘legal documents’.

“A technical guideline is not a legal document; it is impossible to separate technical guideline out of legal documents as technical guideline must be published together with a decision by the Ministry of Health” (Senior Specialist) However, for the format of legal documents, citation was not included in the whole document. Our interviewees stated that a legal document was not a publication and thus there was no need for citation; just a list of references might be included. “A legal document is not a publication… It doesn’t have any citation while it has a list of references at the end of the document” (Department leadership, Institutional leadership) Three of our interviewees stated that Viet Nam did not have any strict regulations on the use of science in policies.

“The difficulty in Viet Nam is that we don’t have any strict regulations in terms of science; it is not compulsory to use scientific evidence.” (Institutional leadership) With existing problems of evidence use in the policy making process and with the hope to improve policy development, our key stakeholders emphasised the need for changes, focusing on how to enhance evidence use, as well as other recommendations discussed in the session of policy development above. Recommendations from stakeholders included: • There should be better mechanisms to enforce the use of scientific evidence in policy development, • The editorial committee should review scientific evidence at the beginning of the policy development process, • An independent group in charge of collecting evidence, analysing and evaluating evidence should be established, who would then provide recommendations for the writing group to develop documents informed by such evidences. A suggestion of developing a guideline on developing evidence-informed guidelines and piloting these evidence-use guidelines with some infectious diseases was also mentioned.

“… it would be better if there’s a person who is in charge of collecting evidence, then a group will assess and analyse the evidence and produce recommendations. Then finally, a person will write the proposal based on that.” (Department leadership) “… to pilot building some evidence-based guidelines according to the levels of evidencebased medicine. The reference levels will be recognised. If we can do so, when using guidelines doctors will know the value of evidence and appreciate those guidelines better. In order to do that, we need to change our method of building a policy from the first steps. In other countries, they have an editorial committee who is responsible for reviewing scientific evidence from the beginning to understand the situation, and after that they assign the members of the board to write it.” (Institutional leadership)

How Evidence is Used to Inform Health Policy in Viet Nam

2.7. Research Related Issues When talking about scientific research, our key stakeholders pointed out the meanings of research studies. There might be two types of research: translational research - essential at the moment for policy development, and discovery research - providing knowledge which may be useful in the future. Research would also be beneficial to the diagnosis and treatment guidelines, would help to estimate the risks, or should be used to serve for vaccine development. Research would also provide knowledge on situation description.

“You don’t know the future, maybe useful in the future, understanding about the nature around us is already important. But there is another stream of research that, you know, that, for example what disinfection to use for African swine fever. This kind of research would be something that we can use right away, testing, comparing disinfection, comparing diagnostic methods…” (Senior Specialist) We wanted to explore policy makers’ willingness to participate in scientific research and how they perceived their contributions and roles in the research process. When asking about which stages of research policy makers should be engaged or consulted, the research team found that eight of our stakeholders recommended that policy makers should participate in suggesting research ideas or formulating research questions because they knew the situation and they knew what they needed.

“… You know, I think a lot of the time research has to do with asking good questions, right? And framing the question in a way that the answer would be useful, right? So those you need policy makers so that you make sure that your question would answer what they are interested in or what they need. In the process of answering that question, you know, I think researchers are in a better position to decide.” (Senior specialist) “… they know real situations in many provinces, they know the situation of many hospitals and places. Therefore, when they consult you at the designing stage, the research scale will be big, selecting research sites will… I mean you can go to sites and hot spots better.” (Department leadership) “I think yes, they need to join research at the stage of ordering. As they are the ones who propose questions, they know what they need to order scientists to do. They are the ones who propose questions so that scientists can go on the right track.” (Institutional leadership) Five stakeholders suggested involving policy makers in study evaluation or in supporting the study implementation periods to help it run smoothly. They could provide support by requesting the study area authorities or institutions in charge to provide support to the implementation of the study.

“They can support the study implementation, but just support. For example, when you need to implement a certain activity at a site, or interview drug stores or healthcare facilities, if you contact them by yourself, it would be hard to persuade them. If there’s a letter from the MoH to DOHs asking for their support, your research will run smoothly, correct? They only support research to be implemented smoothly; they don’t interfere in research methods or procedures.” (Institutional leadership) 54

How Evidence is Used to Inform Health Policy in Viet Nam

Some of the strengths identified in OUCRU’s internal policy engagement review were long-term engagement with stakeholders, and having multi-level stakeholders involved in the research implementation79 – which aligns with the L-E theory. However, some OUCRU researchers also expressed reluctance to engage with stakeholders throughout the conduct of the research itself – because they believe that engagement should happen only when they have results to share. OUCRU researchers also raised the issue of balance, and the need to maintain their integrity and ownership of their research79. Some of our respondents in this study, our external policy engagement review, also pointed out their concern about bias of policy makers’ joining in research. For example, if a policy maker working in the MoH in charge of providing approval for research studies joined a research study, would that be a potential bias? Another respondent from the clinical area thought research studies must be independent.

“The more important thing is that when you do a study, it must be completely independent. The outcome must be able to convince researchers. If a policy maker joins in a certain study or a research team, when you have the outcome, is there any bias? That’s certainly a conflict of interest. It’s not simple.” (Department leadership)

“In terms of barriers, the nature of it is a barrier. One person specialises in making policies and another focuses on their specialty. The one working on their specialty can’t imagine the whole story of an issue, for example, when he raises an issue that all children after being born need to be vaccinated. The specialised person just wants this. But the policy maker would think differently. Like when this issue is mentioned, who would do the injection, what if they are born at home… Something like that.” (Department leadership) “Well, because you… you see … I think a lot of policy makers see things differently. If you want policy makers to use your research, you may need to see things their way. Because I think their thinking of things is very different.” (Senior specialist) “Another barrier is that the mindsets are different, right?” (Institutional leadership)

Data sharing

• Unwillingness to share • Data ownership and availability

2.8. Engagement between Policy Makers and Researchers Our stakeholders tended to agree that the engagement between policy makers and researchers should be very close because policy should be evidence-informed.

“In theory, it should be close as policies have to rely on… it’s called evidence-based, basing on reality to issue a policy. However, is it close or not nowadays? It is, but there are still limitations. The proof for those things is that there are a lot of policies issued but they cannot be implemented as they were not initiated from reality.” (Institutional leadership) Stakeholders indicated that if they could see the benefits of the program or of engaging with researchers, then they would be willing to engage. Some stakeholders thought that engagement with policy makers would help to ensure the sustainability of the intervention after funding withdrawal. While six interviewees thought that the engagement between researchers and policy makers were good, others mentioned the differences in priorities, objectives, and mindsets or the current system for engagement which caused difficulties for engagement between researchers and policy makers (see Figure 19). Stakeholders identified that problems existed from both researchers and policy makers. Both researchers and policy makers did not have any clear plan for engagement, and stakeholders felt that researchers did not have any information about policy priorities. The barriers for engagement came firstly from the nature of research and policy making - they were naturally different, and thus their priorities were also different. Research and policy engagement should work effectively when there is a convergence between researchers’ and policy makers’ priorities. This is well recognised by our key stakeholders and emerged as a theme in the interviews.

• Gaps caused • by policy • makers • •

Gaps caused by researchers

• • • •

Time constraints Staff turnover Consensus Natural difficulties Lack of connection with researchers

Lack of capacity Financial issues Most concerns for publications Knowledge on regulations

• No clear plan • No information about priorities in policy Mindsets or development priorities • Natural differences of researchers and policy makers • Different priorities between the two communities

Figure 19. Research and policy engagement gaps 55

How Evidence is Used to Inform Health Policy in Viet Nam In terms of engagement gaps that stakeholders identified as being caused by researchers, it was stated that researchers focused too much on publishing their research findings, and were not willing to share their data. Other issues were related to finance, researchers’ limited knowledge on the government’s regulations, or limited capacity. Limited financial capacity caused difficulties for institutions to initiate research studies even when they had research ideas in mind. One interviewee also complained that some organisations had not done any engagement with policy makers, which might be because they did not know all regulations of the health sector of Viet Nam.

“All scientific studies now are for publication…. But after publication, who would read those papers? Again, only scientists who are very major in the area, or only when they do something similar, they would try to find information about such issues, and search for those papers to take a look.” (Department leadership) Time constraints and staff turnover were mostly listed as obstacles from the side of policy makers. The issues might come from the policy makers themselves, as it was said to be very difficult to work with some leaders in the MoH or very difficult to reach out for their consensus.

“Nowadays, in fact, the engagement between policy makers and scientists is weak and little, I think so; it’s very weak. There may be lots of reasons, one of which is that maybe policy makers are changing positions so quickly. Today they work in this position, tomorrow they are transferred to another position, for example. So sometimes it’s so difficult. Sometimes you have already established relationship with a person in charge of a certain area in the MoH, a while later they would be assigned to work in another area. I think it’s one thing which is still limited and we need to improve it.” (Institutional leadership) In order to have better engagement, two of our respondents planned to develop a project at Ministerial level about open science, or to improve the health informatics system, and to find out the collaboration and coordination mechanism. Also, it was suggested that the engagement between policy makers and researchers could be improved if there were some regulations enforcing the use of evidence in policy formulation.

“To continue, this group is sitting together to develop a policy on open science. It means all groups need to respect each other in terms of the copyright of data but should share data with each other… If we can do that, data would be optimised, which means that the data would be used to select priorities and to answer some questions. Or those data would help to answer the questions. Simply saying, now we want to estimate the outbreak high risk areas, if institutions do not provide data, we can’t do anything.” (Department leadership)

2.9. OUCRU’s Achievements and Gaps OUCRU was recognised by key stakeholders for our long-term collaboration and support to the health sector of Viet Nam, for our high quality research studies, and for our contribution in certain areas related to infectious diseases, especially antimicrobial resistance.

“We highly appreciate OUCRU’s support. I must say that OUCRU has been standing with [us] to develop research projects to provide evidence for writing prevention policies… So we highly appreciate those, and OUCRU’s contribution is so big because apart from being a research unit, OUCRU also joins in doing research to provide scientific evidence which helps the policy making better.” (Institutional leadership) However, as pointed out by our key stakeholders, OUCRU still needed to overcome several weaknesses in terms of engagement, both subjectively and objectively. In terms of objective reasons, there was no official agreement or commitment from governments of the two countries, which challenged OUCRU in terms of engagement with policy makers in Viet Nam.

“OUCRU is an organisation which is quite… not… is a non-governmental organisation, so between government and government there hasn’t been any agreement or any commitment, so there is nothing to tie each other. This would cause difficulties for OUCRU. But if the two governments had signed an agreement or MOU, the story would be very easy.” (Institutional leadership) OUCRU’s slow speed of work, lack of engagement with key partners like the MoH or PCDCs and policy makers’ preconception on OUCRU’s goal were also listed as our weaknesses in engagement with policymakers. In the eyes of some stakeholders, OUCRU was a research organisation, and OUCRU’s research aimed to serve Oxford; or the overall aim of conducting research studies in Viet Nam was just for publications. Stakeholders felt that for such reasons, policy makers might not think of OUCRU when they wanted to involve key research stakeholders into their policy making process.

“OUCRU hasn’t been included in the group providing solutions, it just belongs to the research group.” (Department leadership) “In fact, OUCRU’s problem is about their principles and goals… Firstly, when they established the first research unit in 1992, right, in Viet Nam, was the goal to serve Viet Nam or Oxford? And after nearly 30 years, the site in Ha Noi from 2000 or 2004 until now, after many years, the final goal is to serve mostly… it’s like a clinical site just like a certain site in Europe? Or Africa? Right? To serve Oxford or whom? Of course, it serves the global medical community, which is unarguable. The problem is whether it has brought more to the place where it’s located or not? So that’s what you want, right?” (Department leadership) get the trust by not intending to publish … or… not sharing the national data for only your sake …. So that needs to be clarified with the Ministry of Health but still you can put more like public health… you are the really… your intention is not only the publication…” (Senior specialist)

How Evidence is Used to Inform Health Policy in Viet Nam

2.10. Recommendations for OUCRU’s Engagement with Policy Makers As discussed earlier, gaps for engagement between researchers and policy makers came from natural differences between the two communities. And the policy makers’ impression on OUCRU might also be influenced by such differences. They had a preconception that OUCRU wanted to share the country’s data globally without informing them. However, for researchers, sharing their results should be one of their motivations for work. Policy makers need evidence to support their policy making, but they also have a political agenda, and a public image to control, and may be reluctant to publish findings that don’t support them. It’s a natural conflict that can’t be worked around if not acknowledged.

“They also kind of a little bit… be careful of OUCRU too because information of the country is shared globally without noticing or that is really … information sometimes is very sensitive. So I think you can also get the trust by not intending to publish … or… not sharing the national data for only your sake …. So that needs to be clarified with the Ministry of Health but still you can put more like public health… you are the really… your intention is not only the publication…” (Senior specialist)

2.10.1 General Recommendations The respondents recommended that with the strengths and experiences, OUCRU should support the MoH in building a guideline on how to develop evidence-informed guidelines, and pilot it with some infectious disease guideline development. Additionally, with OUCRU’s expertise in infectious diseases, stakeholders identified that OUCRU would be a very good stakeholder to provide support to the development of diagnosis and treatment guidelines, especially in diagnostic testing area. Those are the two activities which they thought OUCRU could provide direct support to the MoH.

“OUCRU can support the MoH to pilot developing a scientific evidence-based guideline for treatment of some infectious diseases. Maybe you can pilot building up guidelines for about 5 infectious diseases, no need to do more… I think it would be very good if OUCRU can support to develop a standard procedure for developing evidence-based treatment guidelines for some infectious diseases. After that we can recommend that for the rest of infectious diseases, this procedure should be applied in developing guidelines. I think we don’t need to do for many diseases, just pilot for 5 diseases as examples, that’s fine.” (Institutional leadership) For better engagement with policy makers, stakeholders suggested that OUCRU should engage with policy makers at the very beginning point of research idea formulation. OUCRU should also consider the context and leadership of some departments of the MoH to avoid wasting time and effort, as some people were too difficult to work with.

“That’s it, you can invite them to join in the stage of suggesting ideas for research … They know the real situations in many provinces… Therefore, when they consult you at the designing stage, the research scale will be big, selecting research sites will… I mean you can go to sites and hot spots better. Secondly, they also have a strong review committee which is the Ethics committee. The MoH’s committees, then the MoH’s Ethics committee. It means they can contribute a lot for you in designing the research proposal which meets the Viet Nam’s criteria.” (Department leadership)

Photo: Outbreak Advisory Board First meeting on 21 October 2020

With the current close collaborations and relationships with hospitals, research institutes, professional associations and other institutions of the government of Viet Nam, they suggested that OUCRU should maintain and improve relationships - as people coming from these institutions were those who would provide evidence to the policy making process. At the same time, OUCRU should enhance engagement with the MoH, both at the level of the department leadership as well as with specialists, via both formal and informal networks. Engaging with leadership would help to obtain their agreement while engagement with specialists would help to get information, to make the working process with the leadership smoother and there would be a person to do the work.

How Evidence is Used to Inform Health Policy in Viet Nam

“Now you should try to provide impacts at hospital level.” (Senior Specialist) “I mean now you want to improve it, in my opinion, besides conducting scientific research, you can also join in contributing opinions in building policies by enhancing the engagement with the functional departments in the MoH. Then when they invite… they will remember you and invite you.” (Department leadership) Different working approaches (summarised in Figure 20 below) were raised up, for example, providing direct buy-in as a consultant, providing financial support to particular activities, offering to work with key stakeholders, and enhancing communication with policy makers. Regular communication via both formal and informal approaches to send out OUCRU’s communication messages were suggested.

Enhancing Engagement

Policy makers' Buy-in

More Communication

Support Provision

Strategic Plan

“But if engaging with policy becomes a priority, then you should manage time to meet them more often. I think having regular meetings or those kinds of things… just go and have coffee and can talk a lot of other things…” (Senior specialist) According to our respondents, developing both institutional and personal networks would help OUCRU to enhance engagement with policy community. And when OUCRU wanted to establish a partnership with the MoH, OUCRU should send the signals to the MoH that OUCRU wanted to partner with them. In terms of scientific areas, OUCRU had a lot of meaningful activities apart from research studies like the Open Day or Journal Clubs in which OUCRU should invite stakeholders to join.

“OUCRU should enhance communication activities so OUCRU can invite those persons to participate. Activities like OUCRU’s Open Day are very interesting, so you should give them a chance to join. OUCRU has some studies in the community; with those studies, you should enhance providing the study results and its impact” (Institutional leadership)

Consultant

Clear message

Financial support

Various perspectives

Study tour

Appropriate approaches

Technical support

Detailed plans

OUCRU's activities

Regular communication

Send signals

Evaluation

Researchers who are interested in sending signals could try these techniques mentioned by our stakeholders: -

Organising scientific conferences Inviting the MoH to join in OUCRU’s activities Enhancing communicating with policy makers and asking them to work with OUCRU Meeting with the MoH and telling them what OUCRU would like to support.

“… Firstly you can organise scientific conferences, secondly you can invite the MoH to join in something, even in Viet Nam or abroad, so they will know about your position. That’s it. Secondly, you enhance working with them, asking them to work with you. Or you provide experts, for example when the COVID-19 started, you had your experts, you could send a signal or you could go to the MoH to ask them for work, tell them you were this and that.” (Department leadership)

Figure 20. Stakeholders’ recommendations for OUCRU’s enhancing engagement with policy makers 62

How Evidence is Used to Inform Health Policy in Viet Nam

The study participants suggested that with OUCRU’s expertise, OUCRU should also try to detect the issues that the MoH had not been able to solve and offer support to explore solutions. In addition, working attitude and behaviors should also be of consideration. One of the respondents suggested that when working with the government, OUCRU researchers should be ‘hard and persistent’, should keep patient and should not get angry.

2.10.2. Policy Stakeholder Advisory Boards

Policy Stakeholder Advisory Board (PSAB) (n = 10)

“…first, it is the approaching method for them to… it means that we should raise up the offer to support them, support them with their remaining issues that they haven’t been able to solve” (Institutional leadership) “When you can’t work with anyone, just work with their bosses. The principle in working with the government is that I don’t work with you, I work with your boss, and we are not enemies, today we haven’t understood each other but tomorrow we will. You must be very hard and persistent. Working with the government, you mustn’t get angry. I know many people so working with the government, you must be patient.” (Senior Specialist)

Yes (n =5)

Key elements that could help to facilitate engagement between policy makers and OUCRU researchers were listed, such as: better mechanisms for policy makers to work with scientists, shared concerns between the two sides, financial resources, knowledge about the health system and regulations of Viet Nam, strengths of the programs which OUCRU would like to bring up to policy makers for engagement, willingness to provide OUCRU’s technical support to policy stakeholders, and a policy champion – the person who can represent a position confidently in communication with policy actors, who can disseminate, advocate and mobilise resources as well as propose problems into the policy agenda without having to wait for the policy window to emerge10.

“…you need some champion though. But then if you have a person who is championing in some certain issues, you know, then you may not need a committee.” (Senior Specialist) Our respondents thought that when Viet Nam had better regulations enforcing evidence use in the policy cycle, just like other researchers, OUCRU would also benefit from a better legal framework for engagement. And when OUCRU’s priorities were aligned with policy makers’ prioritised concerns, it would be much easier for OUCRU to partner and work with them.

“In fact, there are no regulations on that. But if there’s a better progress to develop evidencebased policies, there will be a better mechanism for policy makers working with scientists.” (Institutional leadership)

Who to be invited

No (n =3)

Depending (n =2)

Another name

Ownership

Time

Too many committees

Too serious

Too many people can't make decision

Figure 21. Stakeholders’ ideas on establishing a Policy Stakeholder Advisory Board We asked our stakeholders what they thought about the idea of establishing a Policy Stakeholder Advisory Board (PSAB), 5 out of 10 interviewees thought it would be a very good idea, but advised that OUCRU should consider who is to be invited. Two interviewees did not provide yes or no answer - it would be depending, either on the activity name, or on the ownership of the PSAB, or on the time that it would happen. Three other interviewees were against the idea of the PSAB, and provided the following reasons: • because the name of PSAB was too serious, • because the MoH had too many committees already, and • because too many people with different expertise in one committee could not make any decisions.

“And if you set up a too general committee, you know, then… to me… sort of you won’t be able to make any decision. It’s [a] too broad, too varied group of people. I think for things to work in a committee, you know, the members should have some mutual interest already, you know, on certain issues or certain areas. So having a general committee [is]… in my opinion, something that inefficient.” (Senior Specialist) 64

Chapter 4. Discussion We have demonstrated earlier in this report that policy development in Viet Nam has improved considerably, especially in terms of evidence use and involvement of specialists in the process. Despite the fact that policies in Viet Nam very rarely contain citations, the study participants strongly confirmed that evidence had been referred to when they developed any policies. The issue of citation in policies was listed as one of the weaknesses of the policy development process in Viet Nam. In order to improve policy development, the interviewees recommended further enhancing evidence use in policy, developing standard procedures for policy development, and separating groups with different functions in the development process. From what we have indicated earlier in the results, there is a considerable gap in engagement between policy makers and researchers. Policy and research communities are naturally different. They have different concerns, different ideologies, and different priorities, and although engagement between the two communities has improved in Viet Nam, there are still things to do. For OUCRU in particular, OUCRU has strengths for better engagement with policy communities like high quality research studies and long-term collaboration with the health sector of Viet Nam. More communication and strategic plan for engagement were highly recommended by our study participants for OUCRU to better enhance engagement with the policy community.

1. Evidence Use and Citation How evidence is judged to be relevant and useful in policy making is a process in itself. For example, stakeholders talked about level of evidence. There are more factors that come into play when selecting evidence than just the research quality score – it can score highly, but not be relevant or adequate for the local context, for example. Stakeholders also draw on their own personal experiences in selecting evidence, and in deciding on what would be relevant or useful in the framework of their current focus.

CHAPTER 4. DISCUSSION

From the review on MoH documents, even though there was a lack of evidence to show that policies in Viet Nam were developed using scientific evidence (because most of the MoH documents did not include citations), we could not come to the conclusion that policy development was not informed by scientific evidence. And, from our interviews with key stakeholders, it is clear that policy makers in Viet Nam believe that the policy making process is evidence-informed to some extent – although there is also some consensus that there could be improvements in the use of evidence in policy making in Viet Nam. As mentioned, most of our interviewees (17 out of 18) strongly asserted that policies in Viet Nam were evidence-informed. All of these caused us to question what regulations for evidence use in policy making existed. We wanted to find regulations that required evidence use to inform policy development, and details of those requirements. We found two documents, which were the Law No 80/2015/QH13 dated June 22nd, 2015 by the National Assembly16, and the decision No 4068/QĐ-BYT dated July 29th, 2016 by the Ministry of Health regarding the guideline on developing diagnosis and treatment clinical/care pathways22. 66

How Evidence is Used to Inform Health Policy in Viet Nam

In both documents, evidence was required, but while the National Assembly’s Law did not mention anything about citation in legal documents, in the MoH’s guideline, citation was obviously required as for “when any clinical data is used in professional procedures, available clinical guidelines, protocols, technical procedures, nursing caring procedures, … reference should be cited for the purpose of reference searching. Original documents should not be included in the proposals…”22. The law on the endorsement of legal documents was often mentioned during our interviews, whereas no respondent mentioned the MoH’s guideline on developing diagnosis and treatment clinical pathway/care procedures. Our respondents even said that Viet Nam did not have strict regulations in terms of scientific evidence use in policy development.

2. Policy Stakeholders’ Joining in Research Study

There is no way, without citation, to be sure about how the referenced documents are used. We could have checked all of the references, and crosschecked against the document itself for similarities in content, but this would have been too time-consuming process, and still, would have entailed too many assumptions from our part. We did check some of the documents on iThenticate text matching software, to see if we could see clear links between the documents and other published articles. However, one strong limitation of this is that iThenticate cannot match documents in different languages. All of the policy documents we selected were published in Vietnamese. We know that the Vietnamese authors would have consulted with literature in Vietnamese and in English while drafting the policies, but iThenticate can only detect like-for-like languages. So, it can detect a match in English, or a match in Vietnamese, but it cannot detect a match in a document that is written in Vietnamese and references text originally written in English. Another limitation of this approach, is that not many Vietnamese publications have been digitised, or made available on the internet. This means that it is quite possible that the MoH have consulted Vietnamese literature in compiling these documents, but there is no way of discovering these links without either citations, or digitisation of the original sources. This issue is also reflected in the stakeholders’ complaints of difficulties in getting publications in Vietnamese journals.

In other studies conducted by Thu Vuong et al. in the area of HIV, a local review committee was established which included a full range of stakeholders. In order to ensure the study’s objectivity, the roles and responsibilities of the committee were clearly defined: The members played the role of supporting, rather than directing, the study findings. And it was clearly stated that the research team would make final decisions on the study protocol, not the committee. However, the research team still had to compromise due to the sensitivity of the data collected3. This shows that even with careful planning, it can be difficult to totally eliminate the risk of bias or conflict of interest in having policy makers involved in earlier stages of research. However, it also shows that some level of compromise may be acceptable, in certain circumstances.

As discussed earlier, early engagement with policy makers has been shown to be beneficial to policy advocacy5. At OUCRU, we would like to establish systemic engagement at all stages of research, and we strongly believe that successful engagement at one stage will facilitate more successful engagements at other stages.

With the L-E framework, Jonathan Lomas showed how they applied the framework in all activities of their foundation, from setting up priorities, funding programs, assessing applications, conducting research, communicating findings and planning for evaluation12. The application of the L-E framework demonstrated the increase of familiarity and connection between the world of research and the world of policy making, at both individual and institutional levels, to promote the use of evidence from scientific research in decision-making and to encourage research that generates evidence useful to policy makers12,80–82. Despite the challenges that still persisted, if influencing decisions was critical to a research study, the involvement of decision-makers in the research process was meaningful, and the objectivity of the research could still be maintained by different approaches. When you apply the L-E framework to a policy engagement system that is working well, you see multiple level communication between researchers and stakeholders – at different stages in the processes of both research and policy making, and among different actors in those processes. When there are problems in the policy engagement system, there can be a lack of communication, and there can be an imbalance of power, or consensus in both the policy making and research processes. Where there is a lack of consensus, poor communication, or imbalance of power, then perceived threats to integrity and autonomy can arise. Therefore, it is important to consider all the aspects that can help facilitate a policy engagement system to work well, in order to maximise the benefits of multi-level communication.

Photo: ViParc participation in international workshop on AMR in October 2019 67

How Evidence is Used to Inform Health Policy in Viet Nam

3. OUCRU’s Policy Engagement Plan OUCRU is committed to actively engaging with policy stakeholders in order to create impact. In order to do so, it is important that OUCRU develops a strategic engagement plan. Evidence from the literature review, corroborated by the interviews conducted both with policy stakeholders in this external review, and our own researchers in our internal review79, show that successful engagement at one stage, can facilitate more successful engagement at other stages, and we can have also have concurrent engagement across all parts of the research life-cycle (see Figure 22 below).

During project design

Program level consultation Systemic engagement

Outcomes dissemination

Continuous throughout the course of research

Research results

Figure 22. The continuous cycle of systemic policy engagement Our initiative of early engagement and continuous engagement during a study is supported by our literature review findings. In the study related to enhancing governance and strengthening advocacy for policy change, Isabelle Michaud-Letourneau et al. pointed out the key components that could help strengthen advocacy for policy change or development, such as engagement in advocacy, engagement with policy makers early and during the whole course of the initiatives, frequent communication with policy actors5.

As defined by our key stakeholders in the interviews, health policy in general consists of ‘legal documents’ and ‘technical guidelines’, and these ‘legal documents’ may be policy developed by multiple ministries, not just by the MoH. Most of our stakeholders mentioned having influence and engagement in the development of technical guidelines, rather than the legal documents. In considering where OUCRU should direct its strategic engagement focus, our aim is therefore to create an engagement programme that facilitates influence on the development of these technical guidelines. The policy analysis frameworks (MSF, L-E and I-I-I) provide a number of ideas for the development of strategic engagement that OUCRU could take into consideration. In the Linkage and Exchange (L-E) principle, both researchers and policy makers participate in different activities including setting priorities, funding programs, assessing applications, conducting research, communicating findings, and evaluation12. With the aim of enhancing engagement with key stakeholders, OUCRU has been and will be able to learn from Lomas’ experience and apply this principle in developing and implementing action plans for engagement. The MSF emphasised the significance of the problem stream, the policy stream and the politics stream and that all the three streams should be positioned at the same level – each stream is equally important9. This agenda setting relates to higher-level, strategic engagement, which may be useful for OUCRU researchers to consider when approaching engagement on high-level or thematic issues with policy stakeholders such as antimicrobial resistance, or issues of international relevance. One of the key aspects of the MSF is the idea of having a powerful policy entrepreneur. Our interviewees talked about “policy champions” – which fits with this idea, and the phrase “policy champion” also appeared in publications about policy impact in Viet Nam3. These policy champions can play a role in each of these three streams, and OUCRU should invest energy in identifying, training and empowering policy champions who can play this role. Lomas suggested that having policy makers as members in an advisory board can help to generate their knowledge, experience, opinions, and ideas82. The L-E framework has been reflected in various ways that OUCRU has created platforms for information exchange and sharing ideas with policy stakeholders. There are many examples, but to list a few: In 2017, to prepare for the establishment of the national serosurveillance network in Viet Nam, a stakeholder meeting was organised to exchange information on available infrastructure and resources, ongoing projects and research regarding serosurveillance in Viet Nam, to learn from experiences in the UK and to discuss options for establishing a national serosurveillance network among key stakeholders in Viet Nam19. And to prepare for the development of the framework for collaborative and complementary antimicrobial stewardship research and implementation, in 2019, OUCRU supported WHO and the MoH to host a stakeholder meeting, with the participation of the MoH, hospitals, private sector and international development partners, to review activities and agree on an AMS framework. Most recently, in 2020, OUCRU established an Outbreak Advisory Board (OAB) with the participation of both researchers and policy makers. The OAB aims to create a platform or channel for researchers and policy makers to exchange knowledge, experience and ideas in the context of outbreaks, strengthening the relationships between researchers and policy makers, and ensuring the capacity of generating information essential for policy making. This helps ensure that OUCRU’s outbreak-related projects are locally-driven, and best able to achieve local, regional and global impact.

How Evidence is Used to Inform Health Policy in Viet Nam

Our respondents suggested OUCRU support the MoH by developing a guideline on how to develop evidenceinformed guidelines, and to pilot these evidence-use guideline with some infectious disease treatment guidelines. For example, in 2016, OUCRU together with NICE International, did support the MoH to develop the Quality Standards (QS) for prescribing for respiratory infections in Viet Nam20. In particular, the QS focused on appropriate antibiotic use in community acquired pneumonia (CAP) and acute exacerbation of COPD (AECOPD). The QS is an example of evidence-based guideline and the process of developing the QS could be considered an example process of developing evidence-based guideline. For the development of the QS, the MoH formally established the editorial committee consisting of members from the MoH, hospitals, and universities, with technical support from OUCRU and the NICE International. All indicators and standards were developed based on evidences from published guidelines. For each of the recommendations in the QS, evidence from each reference source was summarised and cited. The QS was finalised in late 2016 and then in 2018, the QS on CAP was piloted in one hospital in Viet Nam83. As illustrated in the examples above, policy engagement is a growing focus at OUCRU. OUCRU’s engagement with policy makers still remains largely project or issue focused, but OUCRU is now in a position to learn from these previous experiences, and begin to focus on more systemic engagement activities. The suggestion offered by stakeholders of creating a ‘guideline for writing guidelines’ is one example of how best practice guidelines could be developed and implemented within OUCRU to create sustainable practice. The policy development process requires the participation of different stakeholders including government officials, international organisations, activists, local non-government organisations, civil society organisations (CSOs), multilateral health agencies, and local authorities. As shown by the analysis frameworks found during the review (MSF, I-I-I and L-E frameworks) and recommendations from published case studies, we can see that existing long-term relationships with partners, reliable data from high-quality research studies, timely information on international and regional events, and reputable researchers, all contribute to the ability of an organisation to influence policy. And, we can see that OUCRU is able to meet many of these conditions to enhance our engagement activities with the policy community. However, we also have learned, both by our discussions with stakeholders, and by looking at our own track record, that meeting these conditions is not enough. We also need to work on active advocacy to attain policy makers’ attention to specific issues, early and continuous engagement between researchers and policy makers, and developing skills and empowering members of our research community to take the role of policy champions where appropriate. Additionally, we need to further develop and enhance our stakeholder engagement through the use of advisory boards, and explore further opportunities to improve the quality of our engagement with stakeholders by producing guidelines for that engagement and continuing to record and evaluate our engagement impact.

Photo: Expert Meeting on Fluid Management of Dengue Shock Syndrome in January 2018

Chapter 5. Conclusion

The study findings have demonstrated that absence of evidence is not evidence of absence. There is a perception among researchers, as shown in the results of OUCRU’s internal review report, that stakeholders only go looking for evidence that agrees with their pre-adjudicated decisions. However, we cannot show this just by looking at the evidence that does get included in policy. The stakeholder interviews revealed that there is actually significant use of evidence in the policy-making process, however, not all evidence ends up getting included in the final documents. Just because research evidence does not ‘make it’ into the final document, does not mean that the evidence was never consulted. As a matter of logic, supporting evidence is ultimately included. Evidence that is reviewed and deemed not relevant would not be included. This doesn’t mean that the ultimately less relevant or less valued evidence was never consulted by the stakeholders. What makes evidence relevant or valued is multi-factored – as demonstrated both by the evidence use and engagement frameworks discussed, and in the interviews with stakeholders. In order to enhance the use of scientific evidence from OUCRU’s research studies, we should better enhance engagement with and involvement of policy makers into our research studies at early stages with tailored approaches to ensure both the valued contribution of policy makers towards our studies and the objectivity of science. Although scientific evidence plays a very important role in policies, it does not mean all in decisionmaking, and therefore we need to expand our methods of engagement beyond assuming that the scientific evidence we produce will be powerful enough to speak for itself. And as corroborated by literature as well as by the theoretical frameworks in policy analysis, our stakeholders told us that other factors like experiences, context, mechanisms, and infrastructure were also of importance in policy development in Viet Nam, and our engagement approach needs to take these factors into consideration as well. OUCRU’s systemic engagement plan will be developed using the information gathered from this project. OUCRU is well placed to implement this systemic engagement plan, because many of the factors identified by both our stakeholders and the literature as being important to successful engagement are either already in place, or well on the way to being developed within the programme. We have good evidence that our stakeholders are both willing to collaborate and engage with us, and willing to connect us with other stakeholders in the health policy environment in Viet Nam. Our engagement plan will focus on improving early and continuous engagement, training and empowering policy champions, establishing thematic advisory boards and targeting impacts on technical guidelines. Despite challenges to create impact on policy in Viet Nam, there are many opportunities for policy engagement and impact available for OUCRU, and these will help ensure that we achieve our long-term goal of having local, regional and global impact on health.

CHAPTER 5. CONCLUSION

How Evidence is Used to Inform Health Policy in Viet Nam

Appendix 1. Question Guide For In-Depth Interviews With External Key Stakeholders

Appendix 1 (Cont'd)

A. About the stakeholders:

D. Policy-making process:

1. To ask about demographic information: Age, gender, job title, education level/background, career path and how long have you been in that position? 2. What is the main area that you are responsible for? 3. What policies/regulations/decrees… have you participated in making or drafting? (get the document number if possible) 4. What was your role in that process? B. Key stakeholders in policy making process: 5. Who do you think are involved in policy making (both institutions and individuals)? And what is their role? 6. Where it is more than on party involved, who is leading the process, in your experience? 7. Are you a member of any formal networks? If yes, who introduced you? Please list them. If no, please describe your informal relationships. 8. Which individuals have a significant power to influence policy? 9. How do they provide influence? C. Evidence: 10. Thinking about a policy/guideline etc. you have been involved in, what sort of evidence from research did you look for? How did you select the evidence to be included? How did you evaluate the evidence? How easy was it to find good quality evidence? 11. In your area of expertise, what should researchers be focusing on? 12. Have you ever requested evidence from scientists for making/revising policies? If yes, what type of evidence was requested? And who did you reach out to? 13. Is there enough high quality evidence for policy makers to use? 14. Is the evidence provided considered trustworthy? Which researchers do you trust? What make you trust a research group? What are barriers to trust? 15. Have any personally ignored any specific data or research results? Why?

16. If you work on a new policy project, how does it start? What are the steps? How long does it take? What makes it more or less urgent? 17. Who sets the initiatives of making/revising policies? 18. Are there any regulations that must cite scientific literature or consult with different departments? 19. How do you share your activities during the policy making process? And what are the opportunities for others to get involved? 20. What are some of the barriers that you face? E. Policy and research engagement: 21. What do you think about engagement with the research community? How do you think the research community should engage with you? 22. What questions do you have for researchers? 23. Have you been engaged in any research study? If yes, in which part of the studies? 24. At what stages of research should policy makers be involved? 25. What sorts of updates from researchers would you want to receive? And how? 26. While utilising the evidence, did you come back to researchers for questions/comments/clarifications? If yes, what were the most common questions or about which parts of the study? Were you satisfied? 27. When researchers/international organisations engage with you, how do they do it well? How could they do it better? 28. What motivates you to engage with researchers? 29. What are the barriers to engagement? And how should you (or we) overcome these barriers?

References 1. Policy WHOGP on E for H. World Health Organization. Global Programme on Evidence for Health Policy. (2003). Guidelines for WHO guidelines. World Health Organization. https://apps.who.int/iris/ handle/10665/68925. Published online 2003. Accessed May 13, 2021. https://apps.who.int/iris/ handle/10665/68925 2. Centers for Disease Control and Prevention. Office of the Associate Director for Policy and Strategy. Definition of Policy. Centers for Disease Control and Prevention. Published June 18, 2019. Accessed May 14, 2021. https://www.cdc.gov/policy/analysis/process/definition.html 3. Vuong T, Ritter A, Nguyen N, Ali R. Strengthening advocacy efforts with empirical evidence: A case example of the conduct, uptake and utilisation of research in drug policy decision-making in Vietnam. Int J Drug Policy. 2017;49:41-47. doi:10.1016/j.drugpo.2017.07.029 4. Gomez EJ, Ruger JP. The Global and Domestic Politics of Health Policy in Emerging Nations. J Health Polit Policy Law. 2015;40(1):3-11. doi:10.1215/03616878-2854256 5. Michaud-Létourneau I, Gayard M, Mathisen R, Phan LTH, Weissman A, Pelletier DL. Enhancing governance and strengthening advocacy for policy change of large Collective Impact initiatives. Matern Child Nutr. 2019;15(Suppl Suppl 2). doi:10.1111/mcn.12728 6. Clancy CM, Glied SA, Lurie N. From Research to Health Policy Impact. Health Serv Res. 2012;47(1 Pt 2):337-343. doi:10.1111/j.1475-6773.2011.01374.x 7. Ha BTT, Green A, Gerein N, Danielsen K. Health policy processes in Vietnam: A comparison of three maternal health case studies. Health Policy. 2010;98(2-3):178-185. doi:10.1016/j. healthpol.2010.06.015 8. Kingdon JW. Agendas, Alternatives, and Public Policies. Second edition, Pearson new international edition. Pearson; 2014. 9. Knaggård Å. The Multiple Streams Framework and the problem broker: The Multiple Streams Framework and the problem broker. Eur J Polit Res. 2015;54(3):450-465. doi:10.1111/1475-6765.12097 10. Ha BTT, Mirzoev T, Mukhopadhyay M. Shaping the Health Policy Agenda: The Case of Safe Motherhood Policy in Vietnam. Int J Health Policy Manag. 2015;4(11):741-746. doi:10.15171/ijhpm.2015.153 11. Weiss CH. Ideology, Interests, and Information. In: Callahan D, Jennings B, eds. Ethics, The Social Sciences, and Policy Analysis. Springer US; 1983:213-245. doi:10.1007/978-1-4684-7015-4_9 12. Lomas J. Essay: Using ‘Linkage And Exchange’ To Move Research Into Policy At A Canadian Foundation: Encouraging partnerships between researchers and policymakers is the goal of a promising new Canadian initiative. Health Aff (Millwood). 2000;19(3):236-240. doi:10.1377/hlthaff.19.3.236 13. Kane S. The Health Policy Process in Vietnam: Going Beyond Kingdon’s Multiple Streams Theory. Int J Health Policy Manag. 2016;5(7):435-437. doi:10.15171/ijhpm.2016.45 14. Le DC, Kubo T, Fujino Y, Pham TM, Matsuda S. Health Care System in Vietnam: Current Situation and Challenges. Asian Pac J Dis Manag. 2010;4:23-30. doi:10.7223/apjdm.4.23

How Evidence is Used to Inform Health Policy in Viet Nam

15. Government of Viet Nam. Decree No 75/2017/NĐ-CP: Decree regulating functions, responsibilities, rights and organization structure of Ministry of Health.Viet Nam. 2017. Accessed September 27, 2021. https://moh.gov.vn/documents/20182/174449/75.signed.pdf/14b2085e-9a5c-46cf-8721a54fb74669c8 16. National Assembly. Decision No 80/2015/QH13: Law on the endorsement of legal documents. National Assembly: Ha Noi, Viet Nam. 2015. 17. Government of Viet Nam. Decree No 123/2016/NĐ-CP: Decree regulating functions, responsibilities, rights and organizational structure of ministries and ministerial agencies. Viet Nam. 2016. 18. Nyström ME, Karltun J, Keller C, Andersson Gäre B. Collaborative and partnership research for improvement of health and social services: researcher’s experiences from 20 projects. Health Res Policy Syst. 2018;16(1):46. doi:10.1186/s12961-018-0322-0 19. Anh DD, Choisy M, Clapham HE, et al. Plans for Nationwide Serosurveillance Network in Vietnam. Emerg Infect Dis. 2020;26(1). doi:10.3201/eid2601.190641 20. Li R, van Doorn HR, Wertheim HFL, et al. Combating antimicrobial resistance: quality standards for prescribing for respiratory infections in Vietnam. Lancet Glob Health. 2016;4(11):e789. doi:10.1016/ S2214-109X(16)30267-4 21. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101. doi:10.1191/1478088706qp063oa 22. Ministry of Health of Viet Nam. Decision No 4068/QĐ-BYT: Guideline on Developing Diagnosis and Treatment Technical Procedures. Ha Noi, Viet Nam. 2016. 23. Viet Nam Administration of HIV/AIDS Control. Ministry of Health. Decision No 146/QĐ-AIDS. Interventions on HIV/AIDS Prevention and Control for MSM group. Ha Noi. Viet Nam. 2019. 24. Ministry of Health. Decision No 2831/QĐ-BYT. Pediatrics Technical Procedures. Hanoi. Viet Nam. 2019. Accessed March 4, 2020. http://kcb.vn/wp-content/uploads/2019/07/210-QTKT-ban-h%C3%A0nhk%C3%A8m-Q%C4%90-2831.pdf 25. Ministry of Health. Decision No 4290/QĐ-BYT. Guideline on sanitation for surgical area surface. Ha Noi. Viet Nam. 2015. 26. Ministry of Health. Decision No 3336/QĐ-BYT. Technical procedures on Hematology - Blood transfusion - Immunology - Genetics - Molecular biology. Ha Noi. Viet Nam. 2017. Accessed March 5, 2020. https:// kcb.vn/wp-content/uploads/2017/09/H%C6%B0%E1%BB%9Bng-d%E1%BA%ABn-quy-tr%C3%ACnhk%E1%BB%B9-thu%E1%BA%ADt-huy%E1%BA%BFt-h%E1%BB%8Dc-truy%E1%BB%81n-m%C3%A1u. pdf 27. Ministry of Health. Decision No 3916/QĐ-BYT. Guidelines on infection control in health facilities. Ha Noi. Viet Nam. 2017. 28. Ministry of Health. Decision No 4487/QĐ-BYT. Guideline on acute malnutrition diagnosis and treatment for children from 0 to 72 months old. Ha Noi. Viet Nam. 2016.

References (Cont'd) 29. Ministry of Health. Decision No 4928/QĐ-BYT. Guideline on diagnosis and treatment for common retinal vitrenous diseases. Ha Noi. Viet Nam. 2019 30. Ministry of Health. Decision No 5992/QĐ-BYT. Guideline on infection control in neonatal care. Ha Noi. Viet Nam. 2019. 31. Ministry of Health. Decision No 708/QĐ-BYT. Guideline on Antibiotic use. Ha Noi. Viet Nam. 2015. 32. Ministry of Health. Decision No 3931/QĐ-BYT. Guideline on Diagnosis and Treatment for some Nephrology diseases. Ha Noi. Viet Nam. 2015. 33. Ministry of Health. Decision No 3780/QĐ-BYT: Clinical guidance on uterine fibroids. Ha Noi. Viet Nam. 2019. 34. Van Minh H, Van Thuan T, Shu XO. Scientific Evidence for Cancer Control in Vietnam. Cancer Control J Moffitt Cancer Cent. 2019;26(1). doi:10.1177/1073274819866450 35. Pham T, Bui L, Kim G, Hoang D, Tran T, Hoang M. Cancers in Vietnam—Burden and Control Efforts: A Narrative Scoping Review. Cancer Control J Moffitt Cancer Cent. 2019;26(1). doi:10.1177/1073274819863802 36. Nguyen T, Le KK, Cao HTK, et al. Association between in-hospital guideline adherence and postdischarge major adverse outcomes of patients with acute coronary syndrome in Vietnam: a prospective cohort study. BMJ Open. 2017;7(10):e017008. doi:10.1136/bmjopen-2017-017008 37. Quach DT, Hiyama T, Gotoda T. Identifying high-risk individuals for gastric cancer surveillance from western and eastern perspectives: Lessons to learn and possibility to develop an integrated approach for daily practice. World J Gastroenterol. 2019;25(27):3546-3562. doi:10.3748/wjg.v25.i27.3546 38. Ho-Pham LT, Nguyen TV. Association between trabecular bone score and type 2 diabetes: a quantitative update of evidence. Osteoporos Int J Establ Result Coop Eur Found Osteoporos Natl Osteoporos Found USA. 2019;30(10):2079-2085. doi:10.1007/s00198-019-05053-z 39. Turk T, Elhady MT, Rashed S, et al. Quality of reporting web-based and non-web-based survey studies: What authors, reviewers and consumers should consider. PloS One. 2018;13(6):e0194239. doi:10.1371/journal.pone.0194239 40. Yahya N, Roslan N. Estimating radiotherapy demands in South East Asia countries in 2025 and 2035 using evidence-based optimal radiotherapy fractions. Asia Pac J Clin Oncol. 2018;14(5):e543-e547. doi:10.1111/ajco.12831 41. Utley JM, Affuso O, Rucks AC. Adolescent obesity in contextual settings: a scoping study of multilevel and hierarchical examinations: Scoping study of multilevel adolescent obesity research. Clin Obes. 2016;6(5):296-304. doi:10.1111/cob.12163 42. Mendelson M, Røttingen JA, Gopinathan U, et al. Maximising access to achieve appropriate human antimicrobial use in low-income and middle-income countries. The Lancet. 2016;387(10014):188-198. doi:10.1016/S0140-6736(15)00547-4 43. O’Connell TS, Bedford KJA, Thiede M, McIntyre D. Synthesizing qualitative and quantitative evidence on non-financial access barriers: implications for assessment at the district level. Int J Equity Health. 2015;14:54. doi:10.1186/s12939-015-0181-z

How Evidence is Used to Inform Health Policy in Viet Nam

44. Turk T, Quang ND, Nga TT, Phuong H, Tung LVA, Trang VH. A rapid assessment and response approach for socially marketed nutrition commodities in Viet Nam. Asia Pac J Clin Nutr. 2017;26(1):182-189. doi:10.6133/apjcn.072016.12 45. Arnold JB, Tu CG, Phan TM, et al. Characteristics of postoperative weight bearing and management protocols for tibial plateau fractures: Findings from a scoping review. Injury. 2017;48(12):2634-2642. doi:10.1016/j.injury.2017.10.040 46. Vuong T, Shanahan M, Nguyen N, et al. Cost-effectiveness of center-based compulsory rehabilitation compared to community-based voluntary methadone maintenance treatment in Hai Phong City, Vietnam. Drug Alcohol Depend. 2016;168:147-155. doi:10.1016/j.drugalcdep.2016.09.008 47. An DTM, Kibria N, Huy NV, Hai PT, Stillman F. Establishing smoke-free hospitals in Vietnam: A pilot project. Glob Public Health. 2015;10(sup1):S5-S20. doi:10.1080/17441692.2014.986155 48. Gomes MGM, Oliveira JF, Bertolde A, et al. Introducing risk inequality metrics in tuberculosis policy development. Nat Commun. 2019;10. doi:10.1038/s41467-019-10447-y 49. Policy and priorities for national cancer control planning in low- and middle-income countries: Lessons from the Association of Southeast Asian Nations (ASEAN) Costs in Oncology prospective cohort study. Eur J Cancer. 2017;74:26-37. doi:10.1016/j.ejca.2016.12.014 50. Berger J, Roos N, Greffeuille V, Dijkhuizen M, Wieringa F. Driving Policy Change to Improve Micronutrient Status in Women of Reproductive Age and Children in Southeast Asia: The SMILING Project. Matern Child Health J. 2019;23(S1):79-85. doi:10.1007/s10995-018-02730-z 51. Huy NV, Bang K, Xuan HTA, et al. The roles of, activities of, and competencies for, community nursing services in rural Vietnam: Implications for policy decisions. Int J Health Plann Manage. 2018;33(4):e1147-e1159. doi:10.1002/hpm.2600 52. Arensberg MB. Population aging: opportunity for business expansion, an invitational paper presented at the Asia-Pacific Economic Cooperation (APEC) International Workshop on Adaptation to Population Aging Issues, July 17, 2017, Ha Noi, Viet Nam. J Health Popul Nutr. 2018;37(1):7. doi:10.1186/s41043-0180138-0 53. Michaud-Létourneau I, Gayard M, Pelletier DL. Translating the International Code of Marketing of Breastmilk Substitutes into national measures in nine countries. Matern Child Nutr. 2019;15(S2):e12730. doi:10.1111/mcn.12730 54. Farrell PC, Hunter C, Truong B, Bunning M. Control of highly pathogenic avian influenza in Quang Tri province, Vietnam: voices from the human-animal interface. Rural Remote Health. 2015;15(3):3044. 55. Economic Evaluation Comparing Center-Based Compulsory Drug Rehabilitation (CCT) with CommunityBased Methadone Maintenance Treatment (MMT) in Hai Phong City, Vietnam (English, Vietnamese). FHI 360. Accessed May 23, 2021. https://www.fhi360.org/resource/economic-evaluation-comparingcenter-based-compulsory-drug-rehabilitation-cct-community 56. Nguyen CL, Hoang DV, Nguyen PTH, et al. Low Dietary Intakes of Essential Nutrients during Pregnancy in Vietnam. Nutrients. 2018;10(8). doi:10.3390/nu10081025

References (Cont'd) 57. Tan SY, Melendez-Torres GJ. Do prospective payment systems (PPSs) lead to desirable providers’ incentives and patients’ outcomes? A systematic review of evidence from developing countries. Health Policy Plan. 2018;33(1):137-153. doi:10.1093/heapol/czx151 58. Hort K, Jayasuriya R, Dayal P. The link between UHC reforms and health system governance: lessons from Asia. J Health Organ Manag. 2017;31(3):270-285. doi:10.1108/JHOM-11-2016-0220 59. Minh HV, Giang LM, Cashin C, Hinh ND. Health system research in Vietnam: Generating policy-relevant knowledge. Glob Public Health. 2015;10(sup1):S1-S4. doi:10.1080/17441692.2014.986179 60. Dao TMA, Hoang VM, Le TH, et al. Attitudes toward nonsmoking policies and tobacco tax increases: a cross-sectional study among Vietnamese adults. Asia Pac J Public Health. 2015;27(2):NP947-957. doi:10.1177/1010539512460568 61. Nguyen TS, Nguyen TLH, Pham TTV, Hua S, Ngo QC, Li SC. Impact of pharmaceutical care in the improvement of medication adherence and quality of life for COPD patients in Vietnam. Respir Med. 2019;153:31-37. doi:10.1016/j.rmed.2019.05.006 62. Linh NN, Huong NT, Thuy HT. Evolving trade policy and the Trans-Pacific Partnership Agreement: Does it threaten Vietnam’s access to medicine and its progress towards scaling up HIV prevention, treatment and care? Glob Public Health. 2015;10(sup1):S149-S160. doi:10.1080/17441692.2014.981829 63. Waleewong O, Laslett AM, Chenhall R, Room R. Harm from others’ drinking-related aggression, violence and misconduct in five Asian countries and the implications. Int J Drug Policy. 2018;56:101-107. doi:10.1016/j.drugpo.2018.03.015 64. Safarnejad A, Pavlova M, Son VH, Phuong HL, Groot W. Criteria for prioritization of HIV programs in Viet Nam: a discrete choice experiment. BMC Health Serv Res. 2017;17(1):719. doi:10.1186/s12913-0172679-0 65. Congenital rubella syndrome (CRS) in Vietnam 2011-2012--CRS epidemic after rubella epidemic in 2010-2011 - PubMed. Accessed May 23, 2021. https://pubmed.ncbi.nlm.nih.gov/26087296/ 66. The impact of health insurance for children under age 6 in Vietnam: A regression discontinuity approach | Elsevier Enhanced Reader. doi:10.1016/j.socscimed.2014.08.012 67. Nakhongsri A, Crow G. Virtual nursing grand rounds: Improving practice overseas. Nursing2020. 2015;45(5):18-21. doi:10.1097/01.NURSE.0000463666.68476.82 68. Mirzoev T, Poudel AN, Gissing S, et al. Is evidence-informed urban health planning a myth or reality? Lessons from a qualitative assessment in three Asian cities. Health Policy Plan. 2019;34(10):773-783. doi:10.1093/heapol/czz097 69. Hung BT, Long NP, Hung LP, et al. Research trends in evidence-based medicine: a joinpoint regression analysis of more than 50 years of publication data. PloS One. 2015;10(4):e0121054. doi:10.1371/ journal.pone.0121054

How Evidence is Used to Inform Health Policy in Viet Nam

70. Tran VK, Spohrer R, Le TD, Poonawala A, Monech-Pfanner R. Micronutrient Deficiency Control in Vietnam from Policy and Research to Implementation: Keys for Success, Challenges and Lessons Learned. J Nutr Sci Vitaminol (Tokyo). 2015;61 Suppl:S198-200. doi:10.3177/jnsv.61.S198 71. Dao HT, Pichaiyongwongdee S, Sullivan PE, Prasertsukdee S, Apinonkul B. Are physical therapists in Viet Nam ready to implement evidence-based practice? A survey. BMC Med Educ. 2018;18. doi:10.1186/ s12909-018-1428-3 72. Bergström A, Skeen S, Duc DM, et al. Health system context and implementation of evidence-based practices—development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings. Implement Sci IS. 2015;10. doi:10.1186/s13012-015-0305-2 73. Nguyen TNM, Wilson A. Hospital readiness for undertaking evidence-based practice: A survey: Readiness for evidence-based practice. Nurs Health Sci. 2016;18(4):465-472. doi:10.1111/nhs.12293 74. Nguyen TNM, Wilson A. Knowledge, skills, and attitudes to implementing best practice in hospitals in Central Vietnam: Int J Evid Based Healthc. 2016;14(4):142-149. doi:10.1097/ XEB.0000000000000081 75. Vuong T, Nguyen N, Le G, Shanahan M, Ali R, Ritter A. The political and scientific challenges in evaluating compulsory drug treatment centers in Southeast Asia. Harm Reduct J. 2017;14(1):2. doi:10.1186/ s12954-016-0130-1 76. Bégin F, Lapping K, Clark D, et al. Real-time evaluation can inform global and regional efforts to improve breastfeeding policies and programmes. Matern Child Nutr. 2019;15(Suppl Suppl 2). doi:10.1111/ mcn.12774 77. Hand, Foot and Mouth Disease. Accessed November 16, 2021. https://www.who.int/westernpacific/ emergencies/surveillance/archives/hand-foot-and-mouth-disease 78. Pruka A. Research Hub: Evidence Based Practice Toolkit: Levels of Evidence. Accessed November 16, 2021. https://libguides.winona.edu/ebptoolkit/Levels-Evidence 79. OUCRU Policy Engagement Team. Policy Engagement Internal Report. OUCRU. Viet Nam. 2021 (Unpublished Internal Report). 80. Conklin A. Briefing on Linkage and Exchange: Facilitating Diffusion of Innovation in Health Services. :24. 81. Green A, Gerein N, Mirzoev T, et al. Health policy processes in maternal health: A comparison of Vietnam, India and China. Health Policy. 2011;100(2-3):167-173. doi:10.1016/j.healthpol.2010.11.016 82. Goering P, Butterill D, Jacobson N, Sturtevant D. Linkage and exchange at the organizational level: A model of collaboration between research and policy. J Health Serv Res Policy. 2003;8(2_suppl):14-19. doi:10.1258/135581903322405126 83. Do NTT, Li R, Dinh HTT, et al. Improving antibiotic prescribing for community-acquired pneumonia in a provincial hospital in Northern Vietnam. JAC-Antimicrob Resist. 2021;3(2):dlab040. doi:10.1093/ jacamr/dlab040

How Evidence is Used to Inform Health Policy in Viet Nam

Oxford University Clinical Research Unit Website: oucru.org For more information, contact us at: communications@oucru.org