Evaluation increasingly takes place in complex, multi-layered contexts where programs, policies, and systems overlap, and where meaning is rarely stable or shared at the outset. In such settings, evaluators are often asked to move quickly—toward designs, indicators, and judgments—before there’s been time to examine how different people are understanding what’s being built or assessed.

This reflection emerges from practice in those kinds of contexts. It’s not an argument for a particular method or tool, but an exploration of front-end ‘discernment’—the largely invisible work evaluators do to notice, interpret, and hold complexity before formal evaluation begins. In particular it reflects on unit of analysis, not as a technical decision made during design, but as a discipline of attention that shapes what becomes visible, what’s taken for granted, and what kinds of questions seem reasonable to ask.

Differences in interpretation are often attributed to disagreement. This essay suggests another possibility: that people may simply be attending to different levels at the same time—individual, organizational, policy, or system—and drawing conclusions accordingly. Managing those shifts, rather than collapsing them prematurely, is one way that evaluators can demonstrate credibility in uncertain environments.

What follows is a reflection on how attention to scale can support shared sensemaking, and how document-based work can function less as a technical input and more as a visible signal of professional discernment.

Unit of analysis as orientation, not technique

In evaluation, unit of analysis is often treated as a technical decision: whether data are gathered at the level of individuals, programs, organizations, or systems. In practice, however, unit of analysis operates much earlier and more pervasively than that. It shapes how people interpret documents, events, and intentions—often without being named.

Different people can read the same passage, observe the same meeting, or encounter the same policy document and come away with different interpretations. These differences are not necessarily matters of misunderstanding. They often reflect differences in where attention is being placed—at the individual-level, the micro-level of interaction, the meso-level of organizational practice, the macro-level of policy, or the systems-level where patterns and feedback loops emerge.

Seen this way, unit of analysis is not simply a methodological choice. It’s a cognitive and professional orientation that determines what becomes visible, what recedes into the background, and what kinds of judgments feel warranted.

Zooming in, zooming out—and knowing when each matter

Effective evaluation practice requires the ability to move deliberately across units of analysis. Sometimes credibility is built by zooming in—attending carefully to fine-grained detail, lived experience, or localized meaning. In teaching and practice, this kind of focus can be illustrated by the work of Georgia O’Keeffe, where attention to a single form reveals dimensions that disappear at a distance—or, more playfully, by focusing on the eyelash of a flea.

At other times, credibility depends on zooming out—taking in broader patterns, structural constraints, or system-level dynamics. Aerial views from a hot air balloon, or satellite images from much higher above, reveal relationships and risks that are invisible on the ground.

Neither perspective is inherently superior. Each reveal something the others cannot. The professional challenge lies in knowing when to move, when to pause, and what’s lost or gained at each shift in scale. Trouble arises when scale collapses too quickly—when one unit of analysis is privileged prematurely and others are foreclosed.

The front-end problem: premature collapse of scale

In many evaluation contexts, there’s pressure to move quickly toward familiar tools, such as logic models, performance measures, or survey instruments. These tools are valuable, but when introduced too early they can harden assumptions that have not yet been surfaced.

Premature movement toward design often reflects an unexamined collapse of unit of analysis. Decisions get locked in at one level—typically the program or intervention level—before there has been space to examine how different people are interpreting the same documents, goals, or histories. When this happens, evaluation work may proceed efficiently, but not necessarily credibly.

Credibility as demonstrated discernment

Credibility in evaluation is often discussed in terms of methodological rigor or technical competence. Those dimensions matter. But credibility is also assessed relationally and early. Before evidence is collected, people are already forming judgments about whether an evaluator understands the work, sees its complexity, and can be trusted to navigate uncertainty without rushing to closure.

In that sense, credibility is not only claimed; it’s demonstrated. One way it’s demonstrated is through discernment—by showing that multiple units of analysis are being held in view, rather than collapsed prematurely into a single frame.

This is where the distinction between navigator and decider becomes important. A navigator does not choose the destination. The work is not theirs to own. Instead, the navigator helps others see the terrain—including hazards, tradeoffs, and blind spots—so that decisions can be made with greater awareness.

Document models as scale-stabilizing artifacts

Document models can enter the practice at this point—not primarily as front-end tools, but as artifacts that support shared sensemaking across units of analysis.

A document model is a visual or structured representation of an existing document, such as a grant proposal, policy statement, strategic plan, or conceptual framework. Unlike summaries or critiques, document models mirror what’s already there. They surface embedded logic, assumptions, priorities, sequencing, and omissions without judgment. As with any modeling practice, they’re provisional and open to correction.

Used this way, document models do not resolve differences in interpretation. They make those differences discussable. By anchoring attention to what’s written—rather than what’s assumed or intended—document models slow the rush toward design and keep multiple units of analysis in play. They function as credibility artifacts: visible evidence that discernment is being exercised carefully and that attention is being managed deliberately.

Lineage and orientation

This front-end orientation draws from long-standing traditions in program theory and evaluability assessment, including work such as DFID’s Working Paper 40 on planning evaluability assessments, which emphasizes the importance of clarifying intent, assumptions, and coherence before formal evaluation design begins. In those traditions, the evaluator’s task isn’t simply to measure outcomes, but to help ensure that what’s being evaluated is sufficiently coherent, shared, and plausible to warrant evaluation in the first place.

What’s less often named is how much of this work depends on managing unit of analysis—on recognizing when disagreement reflects different levels of attention rather than opposition, and on creating spaces where those differences can be surfaced without being adjudicated too quickly.

Why this matters now

Evaluation work increasingly takes place in complex systems and contested environments, where trust in evidence and expertise is fragile. In such contexts, credibility is shaped less by technical explanation than by early signals of care, restraint, and good faith.

Front-end practices that demonstrate discernment—the ability to notice, shift attention, and pause deliberately—offer one way to meet that challenge. They don’t guarantee trust, but they create conditions under which trust can form.

Seen this way, document models aren’t ends in themselves. They’re one way of making professional discernment visible—of showing that the evaluator sees the terrain, understands what’s at stake at different levels, and is committed to navigation rather than decision-making on others’ behalf.

Bio: Jacqueline H. Singh, MPP, PhD, is an Executive Evaluation & Program Design Advisor based in Indianapolis, USA. Her work focuses on front-end evaluation practice, including program theory, evaluability assessment, and document-based approaches that support sensemaking, credibility, and thoughtful evaluation design in complex contexts.