University of Toronto ·  Trinity College


Charting Compliance with the 2007 Port of Spain Summit Commitments on Non-communicable Diseases

Final Research Report [pdf]

John Kirton, Caroline Bracht, Julia Kulik and Madeline Koch
Global Health Diplomacy Program, University of Toronto
November 27, 2015
(Partial findings published September 24, 2015)

We gratefully acknowledge the valuable research assistance provided in the production of this research report by Alissa Wang, Marissa Young, Rowena Symss, Brittaney Warren, Kayla Jacobs, Aislin Flynn, Sarah Scott and Becky Carpenter.

Introduction

In September 2007, the heads of the governments in the Caribbean Community (CARICOM) met in Trinidad to discuss the prevention and control of non-communicable diseases (NCDs). In the first year following this historic Port of Spain Summit, how much did the CARICOM members comply with the 27 commitments they had made? The answer is important for several reasons. Compliance with commitments — made by the leaders themselves — is critical to solving this major health problem in the Caribbean and to lowering the substantial economic cost to its countries, territories, companies, communities and citizens alike. The collective commitments made at summits add a comprehensiveness, coherence and authority to the control effort that no other actor can offer. Commitments made by leaders indicate a high priority. The effective control of NCDs requires top-level, whole-of-government commitment. Compliance in the first year following the summit should be strong, for that is when leaders and their citizens believe that that they are able and willing to comply with their new and fresh commitments, remember the commitment and expect effective action to take place, before conditions change and other problems and priorities arise to shift attention elsewhere.

Assessing compliance with the leaders' commitments is the most authentic approach for evaluating the effectiveness of the Port of Spain Summit. This approach starts with the exact text of the commitments made by the political leaders themselves, rather than using subsequently selected indicators chosen by and familiar to health professionals, valuable through such indicators are even if their implementation can be more readily traced (Samuels, Kirton and Guebert 2014). Assessing compliance with commitments fills an important empirical gap, because the existing work on implementing key indicators from the Port of Spain Summit has no data for the first year after the summit (from 2007 to 2008), and there is thus no baseline for assessing advances made immediately after the summit. Using the leaders' commitments as the referent allows for the application of the proven methodology for assessing compliance with summit commitments, developed since 1991 by George von Furstenberg and Joseph Daniels (1992), Ella Kokotsis (1999) and the G8 Research Group (2007). It thus permits the comparison of the results with those from two decades of analysis of the compliance performance of other plurilateral summit institutions, notably the G7, G20 and BRICS. It also does so embryonically with those of the Port of Spain progeny — the United Nations High Level Meeting (HLM) on the Prevention and Control of NCDs in September 2011 and its review HLM in 2014.

To chart CARICOM members' first-year compliance with their Port of Spain commitments, the Global Health Diplomacy Program (GHDP) at the University of Toronto conducted top-down, internet-based, retroactive compliance assessments of all of the 27 commitments it had identified for the period beginning with the summit until September 2008 (see Appendix A).

It followed the same method used for its regular annual and special compliance assessments of priority commitments made by the G7/8, G20 and BRICS. All 27 commitments were assessed for compliance and are used in the analysis in this report. These findings are sufficient to judge the feasibility and value of applying the global summit compliance methodology to a regional summit institution and to assess any empirical patterns that arise.

Methodologically, the study showed that it was feasible to conduct such research, at the regional level even eight years after the commitments had been made. Data were obtained for almost all 15 full CARICOM members and some of the five associate members on all 27 commitments. The validity and reliability were similar, and the comprehensiveness was superior, to the two existing sets of Port of Spain implementation indicator data — the indicator grid with 20 components used since 2008 and the more recent indicator implementation data set from the Caribbean Cooperation in Health Phase III strategy (CCH-3), which is more extensive but incomplete. The grid and CCH-3 omit the cross-cutting Port of Spain commitment on gender. The GHDP compliance research showed the value of moving beyond self-reported data to develop definitions for each commitment or indicator, interpretive guidelines and a scoring metric for what constitutes compliance or implementation, as well as produce detailed empirical reports on each member's behaviour — all publicly reported and thus open to stakeholder feedback, empirical enhancement, and public and scientific use.

Substantively, the results showed that CARICOM members got off to a slow start in complying with the leaders' Port of Spain commitments (see Appendix B). At the end of the first year, overall compliance with the 27 assessed commitments averaged only −0.21 (40%), far lower than almost all G7/8 summits and all G20 and BRICS ones. No single commitment had full or no compliance. Only six commitments were positive, three neutral and 18 negative in their compliance. Among the 20 members, none had full or no compliance. Only three members had positive compliance: Trinidad and Tobago, Jamaica and Grenada. There were 17 members with negative compliance.

Methodology

To chart first-year compliance with Port of Spain commitments, during the summer of 2015 nine analysts working under the direction of John Kirton and Caroline Bracht conducted compliance assessments of all 27 commitments, using data available online on implementing actions carried out by governments between the summit in September 2007 until September 2008. The analysts followed the same methods used for the regular annual and special compliance assessments conducted by the G7 Research Group, the G20 Research Group and the BRICS Research Group (G8 Research Group 2007). In a few cases, contact was made with the Toronto consulates of CARICOM members to aid in the research.

This research contributes to the larger field of assessing compliance with summit and international institutions' commitments in several ways.

First, it was the first systematic application of the standard compliance assessment methodology, developed for the G7/8 and similar global plurilateral summit institutions, to a regional summit institution, that of CARICOM, which, like the G7/8 holds annual summits and sometimes inter-sessional ones. Previously only very select, diagnostic work had been conducted in relation to regional institutions such as the trilateral Security and Prosperity Partnership Summit with the leaders of the United States, Canada and Mexico.

Second, it was the first application of the methodology to a special, ad hoc, subject-specific summit, in this case on NCDs.

Third, it was the first application of the methodology to an institutionalized group consisting of members small in population, geography and, for most, gross domestic product. All were recent or current territories of current G7 members. The five associate members of CARICOM that United Kingdom Overseas Territories (UKOTS) posed the methodological challenge of assessing whether they or their imperial governments were legally and operationally responsible for the compliance that they had made.

Fourth, it was the first application of the methodology to a summit with a dedicated secretariat, and in this case with a relatively large one.

Fifth, it was the first deeply retroactive compliance assessment, requiring searching for eight-year-old compliance data about small governments with limited statistical and reporting capacity. It succeeded in securing sufficient data to provide scores for all the commitments for almost all 15 CARICOM members and many of the five UKOTS.

Sixth, it was the first application a complete set of a summit's commitments — all 27 that the Port of Spain Summit made. Previous work had assessed only a small subset of selected priority commitments. A special compliance study had been conducted of all 24 development commitments made by the G20 Seoul Summit in November 2010 (Kirton, Bracht and Rasmussen 2012). Yet although this offered complete coverage of an issue area, it was only a minority of all the commitments produced by the multi-subject G20 summit.

Seventh, it allowed for a focused look at commitments containing international institutional "mandates" and a comparison with those that had none (Larionova, Rakhmangulov and Shelepov 2015). Of the 27 Port of Spain commitments, six had explicit institutional organizational mandates embedded in the commitment, while the remaining 21 did not.

Compliance Overall

Port of Spain Summit compliance got off to a very slow start across all 27 commitments assessed after the first year, with an average of −0.21 (40%). Among the 20 members, the average was −0.22 (39%). This could suggest that members were two fifths of the way there after the first year, but only based on the unlikely assumption that all were starting from a zero baseline, having done nothing before the summit was held.

These −0.21 and −0.22 first-year averages are far lower than the +0.50 (75%) average of the major democratic powers in the G7/8 for the first year after compliance since its summitry started in 1975. Only in three years, based on very partial compliance data, did the G7 summit have a negative score: 1983 at −0.11 (44%), 1988 at −0.48 (26%) and 1990 at −0.14 (43%). Since 1991, the G7/8 summit always had a compliance performance in the positive range. All summits of both the G20 and the BRICS summits have had positive compliance since their start in 2008 and 2009 respectively. This suggests that size counts. Bigger countries have a greater ability to comply, assuming that the willingness to comply, the ambition of the commitments and the difficulty of compliance are constant across all summit bodies.

This first-year compliance data can be usefully compared with the second-year implementation data for some of the 20 indicators previously used to assess the effectiveness of the Port of Spain Summit (Samuels, Kirton and Guebert 2014).

Compliance by Commitment

Compliance by commitment varied widely. No commitment had either complete compliance or none at all. Six commitments had positive scores, three were neutral and 18 had negative scores (see Appendix C). The commitment with the highest compliance at +0.88 (94%) was on mandating the reintroduction of physical education in schools. The second highest at +0.40 (70%) was a commitment to provide incentives and resources to support the reintroduction of physical education in schools. The third highest score, at +0.35 (68%), was a commitment to declare the second Saturday in September "Caribbean Wellness Day." The commitment with the lowest compliance, at −0.90 (5%), was on accounting for the gender dimension in all programs to prevent and control of NCDs. The second lowest at −0.71 (14%) was a commitment to ban advertising tobacco products to children. The third lowest at −0.70 (15%) was on increasing physical activity in the entire Caribbean population.

The cluster of commitments categorized under education and promotion achieved the highest compliance scores, with four commitments above the median score of −0.25 (38%). The cluster of commitments on tobacco follows, also with three above the median, but with slightly lower scores. Commitments on nutrition follow, with three compliance scores falling closer to the median score. The issue area of tobacco has the highest number of scores below the median; however, it also has the highest number of overall commitments identified and assessed for compliance.

Compliance by Member

Compliance by member also varied widely (see Appendix D). None of the 20 members had a complete score and all complied to at least some degree. The score ranged from +0.37 (69%) for Trinidad and Tobago to −0.74 (13%) for Haiti, with an overall median score of −0.22 (39%) and an average of −0.22 (39%). Only three members had positive compliance: Trinidad and Tobago, Jamaica and Grenada. All other members had negative compliance.

Support from International Institutions

Support from international institutions through explicit mandates contained in the commitment appear to increase compliance (see Appendix E). Such institutions include the Pan American Health Organization, Caribbean Agricultural Research and Development Institute and the Caribbean Food and Nutrition Institute, among others. Of the 27 commitments assessed for compliance, six contained mandates to international institutions. These were the commitments on research and surveillance, monitoring and evaluation, fair trade, food security, trans fats and strengthening regional institutions. Only one commitment had a positive score and the overall average was −0.16 (42%), above the overall average of −0.21 (40%).

In contrast, the 21 commitments without such mandates included five with positive scores. The average score for those 21 commitments was −0.22 (39%). Thus, there is a +0.06 difference between those with mandates and those without, as those containing mandates have a higher level of compliance. However, both averages were in the negative range and a +0.06 difference is not very significant on a 200-point scale. Furthermore, the commitments without mandates had a much broader range of compliance, including a much higher score for the top-ranked commitment and a much lower score for the bottom-ranked one. There is thus little evidence for concluding that the presence of an institutional mandate in a commitment increases compliance by a meaningful amount in the first year of compliance.

References

G8 Research Group. 2007. "Background on Compliance Reports: Methodology." http://www.g8.utoronto.ca/evaluations/methodology/g7c2.htm.

Kirton, John J., Caroline Bracht, and Leanne Rasmussen. 2012. "Implementing the G20 Seoul Development and Employment Commitments: An Assessment." Paper produced with the financial support of the International Labour Office through the ILO–Korea Partnership Program, G20 Research Group. http://www.g20.utoronto.ca/biblio/SDC_Assessment.pdf.

Kokotsis, Eleanore. 1999. Keeping International Commitments: Compliance, Credibility, and the G7, 1988–1995. New York: Garland.

Larionova, Marina V., Mark Rakhmangulov, and Andrey Shelepov. 2015. "Assessing G7/8 and G20 Effectiveness in Global Governance." In The G8-G20 Relationship in Global Governance, Marina Larionova and John J. Kirton, eds., pp. 77–108. Farnham: Ashgate.

Samuels, T. Alafia, John J. Kirton, and Jenilee Guebert. 2013. "Monitoring Compliance with High-Level Commitments in Health: The Case of the CARICOM Summit on Chronic Non-Communicable Diseases." World Health Bulletin 2014(92): 270–76B. doi: dx.doi.org/10.2471/BLT.13.126128.

Von Furstenberg, George M., and Joseph P. Daniels. 1992. "Can You Trust G7 Promises?" International Economic Insights 3(5): 24–27. http://www.g8.utoronto.ca/scholar/furstenberg1992/document.html.


[back to top]

Appendix A: 2007 Port of Spain Summit Commitments

[back to top]


Appendix B: Compliance Scores

  Average Anguilla Antigua and Barbuda Bahamas Barbados Belize Bermuda British Virgin Islands Cayman Islands Dominica Grenada Guyana Haiti Jamaica Montserrat Saint Kitts and Nevis Saint Lucia Saint Vincent and Grenadines Suriname Trinidad and Tobago Turks and Caicos
1 −0.25 −1 0 0 0 0 −1 0 −1 0 0 0 −1 0 0 0 −1 +1 0 0 −1
2 −0.07   −1 0 0 0 0   0 0 0 0   0   0 −1 0 0 +1  
3 −0.50   −1 −1 −1 0     +1 −1 −1 0   0   −1 −1 −1 −1 +1  
4 −0.64   −1 −1 −1 −1     +1 −1 −1 −1   0   −1 −1 −1 −1 +1  
5 −0.71   −1 0 −1 −1     0 −1 −1 −1   0   −1 −1 −1 −1 0  
6 −0.57   −1 1 −1 −1     +1 −1 −1 −1   −1   −1 −1 −1 −1 +1  
7 −0.40   −1 0 −1 −1     −1 −1 0   +1   −1 −1 −1 −1 −1 0  
8 0.14   0 0 +1 0 +1     1 0 0   0   0 −1 0 0 0  
9 −0.14   −1 −1 0 0 0     0 −1 0   +1   0 0 0 0 0  
10 −0.45 −1 +1 −1 +1 0 −1 −1 −1 +1 +1 0 −1 −1 0 −1 −1 −1 −1 −1 −1
11 0.88 +1 +1 1 +1 +1 +1 0 +1 +1 +1 +1   +1   +1 +1 +1 0 1 +1
12 0.40 +1 +1 1 +1 +1 +1 −1 +1 +1 0 0 −1 +1 −1 +1 +1 +1 −1 1 −1
13 −0.10 0 0 0 0 0 +1 0 0 0 0 0 −1 0 0 0 0 0 −1 0 −1
14 −0.47 −1 +1 −1 −1 −1 −1 −1 −1 −1 +1 −1 −1 +1 −1 0 +1 0 −1 0 −1
15 −0.32 0 0 0 −1 0 −1 0 −1 0 0 0 −1 0 0 0 −1 0 −1 0 0
16 −0.15 −1 +1 0 0 0 −1 −1 −1 0 +1 0 0 +1 −1 0 0 0 0 0 −1
17 −0.20 −1 0 −1 −1 0 −1 0 −1 0 +1 +1 −1 0 −1 0 0 +1 −1 +1 0
18 −0.70 0 −1 −1 −1 −1 0 −1 −1 −1 −1 −1 −1 +1 −1 −1 −1 −1 −1 +1 −1
19 −0.65 −1 0 −1 −1 −1 −1 −1 −1 −1 −1 +1 −1 +1 −1 −1 −1 −1 −1 +1 −1
20 −0.90 −1 −1 −1 −1 −1 −1 −1 −1 −1 0 −1 −1 −1 0 −1 −1 −1 −1 −1 −1
21 0.10 0 0 −1 +1 +1 +1 0 +1 0 +1 +1 −1 0 −1 0 ++1 −1 0 0 −1
22 0.00 +1 0 0 +1 0 −1 0 0 +1 0 +1 −1 +1 −1 −1 +1 0 −1 0 −1
23 −0.45 0 −1 −1 +1 0 −1 −1 0 −1 +1 −1 −1 +1 −1 0 0 −1 −1 −1 −1
24 0.00 0 0 0 0 0 +1 0 0 0 0 0 −1 +1 −1 −1 0 0 0 +1 0
25 0.25 0 −1 +1 +1 +1 −1 0 −1 0 +1 +1 −1 +1 0 +1 0 +1 +1 +1 −1
26 0.00 −1 0 +1 +1 0 0 −1 −1 0 0 0 0 0 0 0 0 0 0 +1 0
27 0.35 +1 0 0 0 1 0 1 0 0 0 0 0 +1 0 +1 0 0 +1 +1 0
Average by commitment −0.22                                        
Average by member −0.21 −0.21 −0.22 −0.22 −0.07 −0.11 −0.23 −0.42 −0.24 −0.19 0.04 −0.04 −0.74 0.35 −0.58 −0.26 −0.30 −0.22 −0.52 0.37 −0.63

[back to top]


Appendix C: Compliance by Commitment

Commitment Compliance Score Issue Area
11. Mandated physical education in school +0.88 Physical Activity
27. Caribbean Wellness Day +0.35 Education/Promotion
25. Research Surveillance +0.25 Surveillance
8. Fiscal Measures Against Tobacco +0.14 Tobacco
21. Incentives for Public Education on Wellness +0.10 Education/Promotion
22. Incentives for Public Education on Changing Behaviour 0 Education/Promotion
24. Media Partners 0 Education/Promotion
26. Monitoring and Evaluation 0 General
2. Legislate the FCTC −0.07 Tobacco
13. Healthy Meals/Eating Through Education −0.10 Nutrition
9. Revenue from Tobacco and Alcohol −0.14 Tobacco
16. Fair Trade −0.15 Nutrition
17. Food Labelling for Nutrition −0.20 Nutrition
1. Strengthen Regional Institutions Median −0.25 General
15. Trans fats −0.32 Nutrition
7. Warning Labels for Tobacco −0.40 Tobacco
12. Incentives/Resources for Physical Education in Schools +0.40 Physical Activity
10. Screening −0.45 General
23. Incentives for Public Education on NCD Self-Management −0.45 Education/Promotion
14. Food Security −0.47 Nutrition
3. Ban on Smoking in Public Places −0.50 Tobacco
6. Ban on Promoting Tobacco to Children −0.57 Tobacco
4. Ban on Tobacco Sales to Children −0.64 Tobacco
19. Parks for Physical Education −0.65 Physical Activity
18. Mass Physical Education −0.70 Physical Activity
5. Ban on Tobacco Advertising to Children −0.71 Tobacco
20. Gender −0.90 General
Average −0.21

[back to top]


Appendix D: Compliance by Member

Member Compliance Score
Trinidad and Tobago +0.37
Jamaica +0.35
Grenada +0.04
Guyana −0.04
Barbados −0.07
Belize −0.11
Dominica −0.19
Anguilla −0.21
Bahamas −0.22
Antigua and Barbuda −0.22
Median −0. 22
Saint Vincent and the Grenadines −0.22
Bermuda −0.23
Cayman Islands −0.24
Saint Kitts and Nevis −0.26
Saint Lucia −0.30
British Virgin Islands −0.42
Suriname −0.52
Montserrat −0.58
Turks and Caicos −0.63
Haiti −0.74
Average −0.22

[back to top]


Appendix E: Commitments with Mandates

E-1: Mandates to International Institutions
Commitment
Compliance Score
25. Research surveillance
+0.25
26. Monitoring and evaluation
+0.00
16. Fair trade
−0.15
1.   Strengthen regional institutions
−0.25
15. Trans fats
−0.32
14. Food security
−0.47
Average
−0.16
E−2: No Mandates to International Institutions
Commitment
Compliance Score
11. Mandated physical education in school
+0.88
12. Incentives/resources for physical education in schools
+0.40
27. Caribbean Wellness Day
+0.35
8.   Fiscal measures against tobacco
+0.14
21. Incentives for public education on wellness
+0.10
24. Media partners
0
22. Incentives for public education on changing behaviour
0
2.   Legislate the Framework Convention on Tobacco Control
−0.07
13. Healthy meals/eating through education
−0.10
9.   Revenue from tobacco and alcohol
−0.14
17. Food labelling for nutrition
−0.20
7.   Warning labels for tobacco
−0.40
10. Screening
−0.45
23. Incentives for public education on self−management
−0.45
3.   Ban on smoking in public places
−0.50
6.   Ban on promoting tobacco to children
−0.57
4.   Ban on tobacco sales to children
−0.64
19. Parks for physical education
−0.65
18. Mass physical education
−0.70
5.   Ban on tobacco advertising to children
−0.71
20. Gender
−0.90
Average
−0.22