An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countries
Background: Cumulative exposure to critical incidents and life-threatening events leads to significant risk for Veterans and First Responders (VFRs) developing mental ill health and disorders. Philanthropic organisations, Movember and Distinguished Gentleman's Ride, funded 15 organisations to c...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-04-01
|
| Series: | Comprehensive Psychiatry |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S0010440X25000057 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832573250404614144 |
|---|---|
| author | S. Lukersmith L. Salvador-Carulla C. Woods T. Niyonsenga M.R. Gutierrez-Colosia I. Mohanty C.R. Garcia-Alonso D. Diaz-Milanes J.A. Salinas-Perez R. Davey A. Aryani |
| author_facet | S. Lukersmith L. Salvador-Carulla C. Woods T. Niyonsenga M.R. Gutierrez-Colosia I. Mohanty C.R. Garcia-Alonso D. Diaz-Milanes J.A. Salinas-Perez R. Davey A. Aryani |
| author_sort | S. Lukersmith |
| collection | DOAJ |
| description | Background: Cumulative exposure to critical incidents and life-threatening events leads to significant risk for Veterans and First Responders (VFRs) developing mental ill health and disorders. Philanthropic organisations, Movember and Distinguished Gentleman's Ride, funded 15 organisations to conduct early intervention Projects across seven countries. The Projects aim to improve the mental health and wellbeing of VFRs, their families/significant others. This paper describes the novel external evaluation and impact analysis methods to identify effective Projects having positive impact on VFRs and their families, provide return on investment and the overall Grant Program. Methods: We take an ecosystem real-world approach, which recognises the context and aims to manage the complexities involved, uses a complexity and systems perspective, multi-step mixed methods and approaches. The evaluation is from three perspectives of: Projects; Project comparisons; Grant program. Embedded in the evaluation design are methods, knowledge sharing and organisational learning activities for all stakeholders. Data is collected by the Projects and evaluation team on input, throughputs, and output indicators. Analysis tools include Global Impact Analytics Framework, multi-layered statistical analysis, performance evaluation using an efficient decision support approach, Project and Grant program social return on investment, visual linking and data connection platform and assessment of gendered lens approaches. Implications: The complexity and heterogeneity of Projects implemented in the real world continues to present significant evaluation challenges and limitations for project leads, stakeholders, researchers and evaluators. Our ecosystem approach and novel evaluation methodology will reduce the uncertainty around real world implementation, provide key learnings for project stakeholders and more broadly implementation researchers. |
| format | Article |
| id | doaj-art-d3d1f1f2111743f89283afb79ec46587 |
| institution | Kabale University |
| issn | 0010-440X |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Comprehensive Psychiatry |
| spelling | doaj-art-d3d1f1f2111743f89283afb79ec465872025-02-02T05:26:36ZengElsevierComprehensive Psychiatry0010-440X2025-04-01138152578An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countriesS. Lukersmith0L. Salvador-Carulla1C. Woods2T. Niyonsenga3M.R. Gutierrez-Colosia4I. Mohanty5C.R. Garcia-Alonso6D. Diaz-Milanes7J.A. Salinas-Perez8R. Davey9A. Aryani10Health Research Institute, Faculty of Health, University of Canberra, ACT, Australia; Corresponding author at: Health Research Institute, Faculty of Health, University of Canberra, Bruce, ACT, Australia.Health Research Institute, Faculty of Health, University of Canberra, ACT, AustraliaHealth Research Institute, Faculty of Health, University of Canberra, ACT, AustraliaHealth Research Institute, Faculty of Health, University of Canberra, ACT, AustraliaUniversidad Loyola Andalucía, Sevilla Andalucía, SpainHealth Research Institute, Faculty of Health, University of Canberra, ACT, AustraliaUniversidad Loyola Andalucía, Sevilla Andalucía, Spain; Health Research Institute, Faculty of Health, University of Canberra, ACT, AustraliaUniversidad Loyola Andalucía, Sevilla Andalucía, Spain; Health Research Institute, Faculty of Health, University of Canberra, ACT, AustraliaHealth Research Institute, Faculty of Health, University of Canberra, ACT, Australia; Universidad Loyola Andalucía, Sevilla Andalucía, SpainHealth Research Institute, Faculty of Health, University of Canberra, ACT, AustraliaSwinburne University of Technology, Melbourne, Victoria, AustraliaBackground: Cumulative exposure to critical incidents and life-threatening events leads to significant risk for Veterans and First Responders (VFRs) developing mental ill health and disorders. Philanthropic organisations, Movember and Distinguished Gentleman's Ride, funded 15 organisations to conduct early intervention Projects across seven countries. The Projects aim to improve the mental health and wellbeing of VFRs, their families/significant others. This paper describes the novel external evaluation and impact analysis methods to identify effective Projects having positive impact on VFRs and their families, provide return on investment and the overall Grant Program. Methods: We take an ecosystem real-world approach, which recognises the context and aims to manage the complexities involved, uses a complexity and systems perspective, multi-step mixed methods and approaches. The evaluation is from three perspectives of: Projects; Project comparisons; Grant program. Embedded in the evaluation design are methods, knowledge sharing and organisational learning activities for all stakeholders. Data is collected by the Projects and evaluation team on input, throughputs, and output indicators. Analysis tools include Global Impact Analytics Framework, multi-layered statistical analysis, performance evaluation using an efficient decision support approach, Project and Grant program social return on investment, visual linking and data connection platform and assessment of gendered lens approaches. Implications: The complexity and heterogeneity of Projects implemented in the real world continues to present significant evaluation challenges and limitations for project leads, stakeholders, researchers and evaluators. Our ecosystem approach and novel evaluation methodology will reduce the uncertainty around real world implementation, provide key learnings for project stakeholders and more broadly implementation researchers.http://www.sciencedirect.com/science/article/pii/S0010440X25000057EvaluationImpact analysisimplementation scienceMental ill healthveterans first respondersGlobal impact analytics framework |
| spellingShingle | S. Lukersmith L. Salvador-Carulla C. Woods T. Niyonsenga M.R. Gutierrez-Colosia I. Mohanty C.R. Garcia-Alonso D. Diaz-Milanes J.A. Salinas-Perez R. Davey A. Aryani An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countries Comprehensive Psychiatry Evaluation Impact analysis implementation science Mental ill health veterans first responders Global impact analytics framework |
| title | An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countries |
| title_full | An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countries |
| title_fullStr | An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countries |
| title_full_unstemmed | An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countries |
| title_short | An ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and/or early interventions projects for veterans and first responders in seven countries |
| title_sort | ecosystem approach to the evaluation and impact analysis of heterogeneous preventive and or early interventions projects for veterans and first responders in seven countries |
| topic | Evaluation Impact analysis implementation science Mental ill health veterans first responders Global impact analytics framework |
| url | http://www.sciencedirect.com/science/article/pii/S0010440X25000057 |
| work_keys_str_mv | AT slukersmith anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT lsalvadorcarulla anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT cwoods anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT tniyonsenga anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT mrgutierrezcolosia anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT imohanty anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT crgarciaalonso anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT ddiazmilanes anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT jasalinasperez anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT rdavey anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT aaryani anecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT slukersmith ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT lsalvadorcarulla ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT cwoods ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT tniyonsenga ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT mrgutierrezcolosia ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT imohanty ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT crgarciaalonso ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT ddiazmilanes ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT jasalinasperez ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT rdavey ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries AT aaryani ecosystemapproachtotheevaluationandimpactanalysisofheterogeneouspreventiveandorearlyinterventionsprojectsforveteransandfirstrespondersinsevencountries |