Hello, thanks for your comment. You can try to use sources of public geo-referenced data, check out the Q&A section in Joachim’s blog for more information, https://europeanevaluation.org/2020/08/26/case-study-observing-from-space-when-you-cannot-observe-from-the-field/, as well as Hur and Anupam’s blog: https://europeanevaluation.org/2020/09/10/integrating-geospatial-methods-into-evaluations-opportunities-and-lessons/

@Ian Davies. Yes, that is a funny one and shows the difference between a specialist in GIS and one in public administration. 🙂

When I blue-water sailed (professionally) fifty years ago I collected geospatial data (although the term didn’t yet exist) by observing the sun, moon, other planets and stars, landmarks, clock set to GMT, through a geospatial data collection instrument called a sextant. The data were quantitative, i.e. angles & time. These data allowed me to plot my position on a chart, i.e. to transfer the data to a physical model of reality. I would repeat the exercise a few times a day and at least everyday at noon local. Spatial and temporal variations between quantitative data were made meaningful by translating them into qualitative data on the chart, i.e. you could see on the model where you were, where you had come from and the distance covered.

Other quantitative data such as elapsed time of variation would give an estimate of average speed. These data would allow for an estimation of ETA at waypoints, taking into account personal intuitive estimates of current, wind direction & velocity, and possible use of engine. On a few occasions I came across local fishing folk (some in dugouts 2-3 days from shore) who would give me verbal advice (qualitative data) about how best to navigate to where I wanted to go. Those qualitative data trumped my calculations, plots, etc. each and every time. In other words, context appropriate knowledge is most valuable.

The only difference today, as is the case with your Madagascar example, is that, instead of a model (and all models are wrong) we have photos of the real thing (I have hit reefs that were not on the chart ). And this is of course a key improvement.

Having said this, the rest remains pretty much the same. Interpretation, estimation and judgment, all of which incorporate and express a set of values, or if you wish, a world view, determine at the end of the day the value of the process.

Distinguishing between “a robust quantitative assessment of project effectiveness” and “exploration of the qualitative assessment of the “why” of project effectiveness” appears to me, quite frankly, somewhat obtuse and biased towards the use of numbers and their manipulation. These kinds of statements, that I come across regularly, tend to be ideological and/or market driven, and, unfortunately, they do the exact opposite of what their proponents think, which is to reduce the credibility of the overall message.

I also took a look at your WB blog. There’s a lot there that I would challenge in terms of methodology, however, to be fair, I realise that it’s a high-level summary and that there’s undoubtedly a lot more serious work and thought behind it. Having said this, the kernel of insight resides in your statement: “These factors are often highly contextual and linked with human behavior, which is much more difficult to measure from space.” It is indeed all about messy human beings, who in different and varying collectivities and contexts, incorporate the attributes of complexity.

Finally, as I state repeatedly in various fora, if it doesn’t value it is not an evaluation: The extent to which something has been achieved is a measurement question, not a valuing question. Checking to see whether intended results have been achieved, or not, unintended positive, negative, etc. does not make an evaluation. Evaluative reasoning is required to value.

Young (er) evaluation professionals such as you are key to changing the trajectory of mainstream evaluation practice so that it effects positive change for living systems on this planet. I salute your engagement and willingness to share your perspectives, and I encourage you to keep it up. Discussion, debate, self-reflection and deep thinking are also key to keeping us alive and well.

Thank you for engagement and please don’t hesitate to get in touch with me directly should you so wish.

Best regards,
Ian C. Davies
Credentialed Evaluator