Using Collective Intelligence, the power of people of AI to enhance Sensemaking and impact at UNDP

Sharing our new report here

Our role as an innovation team is to undertake continuous research and development to improve our work at the UNDP and to ensure we impact and despite talking a lot about collective intelligence, we actually find it very hard to practice ourselves.

At the UNDP we have a lot of data coming from many places but to bring this data together, to make sense of it and then make decisions from it, is a challenge (as it is for any organisation). We were driven in this project by R&D for Sensemaking, primarily a qualitative process and wanted to know if we could augment with machine intelligence to learn, see connections, and ultimately make different decisions regarding our programming.

• How could we make our Sensemaking process “smarter”? Could we do this by bringing in relevant quantitative data?
• Could we have programmatical data available to support strategy and design making beyond management data (which is very comprehensive in Power BI)?
• Could we find a way to support strategic coherence of projects and programmes by understanding of linkages between programmes, partners, and funding that our people and teams don’t necessarily always, see?

With our partners in the Philippines Country Office and data science company Dataverz we developed the “Portfolio Analysis for Strategic Insights” (PASI) project and today we are sharing the findings of this work.

We are releasing a report (written Pedro Parraguez Ruiz, CEO of Dataverz with support from Shumin Liu from the UNDP) which takes the reader through both the findings from the work but also the technical specifications and a step-by-step technical walk through of our approach. You can find the data at our GitHub page here

For the project we set out these more specific objectives, to:

1. Explore and identify the hidden connections and patterns among the Philippine’s UNDP Country Office’s projects with the semi-structured and unstructured data in the organization.
2. Adopt a data-driven approach to provide useful intelligence for Sensemaking to support the development of a coherent set of programmes that support each other to achieve “north star” goals
3. Accelerate institutional learning at UNDP through collective intelligence and,
4. Enhance the UNDP Country Office’s knowledge and capacity in gaining basic understanding of applying Artificial Intelligence for development work, including how it works, what it can and cannot do, potential bias and limitation and implication of ethical use of AI.

We wrote about this already here “​​Time for Sensemaking 3.0? The potential of AI-powered portfolio analytics to drive impact”, where the authors provided background about the and teased out some early project results, this blog wraps up the pilot project and releases important products for use by UNDP teams and anyone working in global development who can see the potential power of AI to help us make sense of our work.

What did we do?

We worked with the team at the Philippines country office who were visionary enough to work on this project. The Philippines office has a track record of strategic approaches to its work and has embedded some Sensemaking in their regular office practice. We developed the objectives above and then went further to define some key questions we wanted the data to answer:

1. Development challenges: How does the Country Office portfolio look like in terms of the development challenges projects working on by geographic location, partners, investment, and interventions? How does the gender lens is being embedded? Is there a COVID-19 portfolio?
2. How: What are the opportunities for project teams to collaborate with and learn from each other? What are the entry points based on thematic areas, approaches, and capacity?
3. Partners and stakeholders: How do the mapping of stakeholders and partners look like by different types of relationship (such as donors, implementation partners, beneficiaries, other stakeholders)?
4. Effect: What results/impact are the projects aiming to achieve? How do they contribute to the CPD outputs? How does gender equality be reflected in the outcome/outcome measurement?
5. COVID-19: How does COVID-19 impact our portfolio? What are the changes we made to cope with the COVID-19 crisis? What are the lessons learned?

Diagram showing the analytical approach to the project.

As the first pilot to test out the portfolio analysis in UNDP, this project worked on the project documents and administrative reports from the Philippines. In addition to this, and when relevant, we pulled global UNDP project data to test scalability potential and to run comparative analyses. To make sense of the data required a lot of work which is detailed in the report. How we organized and coded the data, how we drew linkages between the data and how we came to our conclusions is detailed there.

What did we find out?

The findings from the Philippines Country Office programme are detailed in the report however several interesting findings emerged including that Gender was across more programming than the Country Office originally anticipated and was very strong especially in the “planet” themes and portfolio. The data showed that in terms of Covid-19 response it was geographically targeted at the south of the country. There were significant connections between donor funded programmes in the data, but the people were not making these connections in person. Overall, the findings and data showed areas for collaboration and connection the Country Office hadn’t seen before.

Findings showing connections (or lack of connection) between different project portfolio areas.

With regards to the method, we have shown how structured and unstructured data can be coded, mined, used in various ways to answer our questions to find linkages and uncover hidden meaning. It is possible to do and usefully shows areas where we can collaborate, connect, and learn from each other. There were significant challenges in coding the data and organizing the data and this required more manual labour than would be desirable. To be able to use data the data needs to be good. This is an area that we could improve on across the organization. The main findings about the method are at the end of the report and document ideal next steps.

What we didn’t do

We worked with the Philippines Country Office and presented the work, these inputs were used during a Sensemaking process in the Philippines Country Office in 2021, however we didn’t map or track if better or different decisions were made because of this work, we also didn’t work out a way to mainstream this into Sensemaking. This was due to changes in personal in the Philippines Country Office and at the Regional Innovation Centre but also funding for this work.

Ultimately this project was a start.

We showed how the data could be gathered and organized in a useful way, we uncovered some unusual patterns that humans alone couldn’t see but we didn’t show how this could be mainstreamed into decision making or into Sensemaking on a more regular basis. How do these processes become part of the management routines which would spurn innovation and connection? We haven’t cracked this one yet!

What’s next?

The Philippines Country office have access to the data and have reviewed it as part of their ongoing Sensemaking. We hope that the various data teams at UNDP are willing to pick up and run with some of our findings.

This blog was written by Kate Sutton based on the amazing work of Pedro Parraguez from Dataverz and Shumin Liu who is now supporting the establishment of a data lab in the Maldives. I take full responsibility for any mistakes in the blog. Thank you so much the leadership of Enrico Gaveglia, Marian Theresia Valera and the Philippines Country Office. Also thank you to Sven Simikin our UNV and anyone else who I inadvertently forgot to thank.