Classifying images sometimes begins with clarifying one’s own criteria

To sort images according to subjective criteria, a method from Google Research helps users clarify their own rules.

A smoothie image is enough to put an instruction in difficulty. The fruit can justify its classification among healthy foods; its sugar content can lead to its exclusion. Before even evaluating an AI’s response, it is therefore necessary to specify what is expected of it. This problem guides Agile Deliberation, a Google Research method intended to build image classification tools from subjective categories.

In its post dedicated to Agile Deliberation, the team describes an approach in which human criteria evolve through contact with examples. An initial definition serves as a starting point. Ambiguous images then reveal its imprecisions, and the user’s decisions feed an automatic revision of the instructions. The evaluation focuses on healthy food and on images promising misleading rewards online. Defining a category sometimes requires seeing its exceptions

Recognizing a road sign and selecting works deemed inspiring do not pose the same problem. In the second case, the category name alone is not enough to establish a shared rule. Individual preferences, cultural context, or an organization’s rules can change the expected choices. Two people can therefore classify the same image differently while remaining consistent with their own criteria.

The methods described by the researchers often assume that the person annotating the images already has a sufficiently clear definition. Yet some distinctions only appear at the moment of deciding. An instruction that seemed complete may leave an open question in front of an unexpected example. Continuing to assign labels without clarifying this distinction risks conveying to the model a rule that is still uncertain.

The first task proposed consists of breaking the general category down into subcategories and formulating exclusions. For healthy eating, the user may keep salads, cereal bowls, or certain proteins, then rule out processed products. These examples illustrate possible choices within the system, without constituting a universal nutritional definition. Their role is to make explicit the scope this person wants to apply. Ambiguous images are used to evolve the instructions

Once this scope has been established, models capable of processing text and images look for examples positioned at the boundary of the retained criteria. The selection of images targets the ambiguities of the human definition. The goal is not simply to submit the content that the machine classifies with the least certainty: an interesting case must also help the user clarify a useful distinction.

The smoothie mentioned in the post illustrates this work. The presence of fruit may satisfy one criterion, while their transformation or the sugar may bring another into play. The user decides whether to include or exclude the image and may comment on the choice. The example then serves to clarify how several rules should fit together.

This feedback is fed into an automatic prompt-revision loop. The classification instructions evolve with the decisions expressed, as the images examined progress. The person retains responsibility for the criteria; the tool handles their reformulation. This division addresses a difficulty reported in the post: modifying an instruction oneself to correct an error can raise concerns about degrading decisions that had worked up to that point. A comparison with the initial prompt and manual work

To evaluate this approach, the researchers compared it with three methods. The first uses only an initial instruction, with no clarification phase. The second automatically breaks down the concept with the help of an agent, without letting the user guide that breakdown. The third asks the person to search for problematic images themselves and rewrite their instructions.

These comparisons cover two different situations. Healthy eating involves a personal selection of desired content. The other category, called “paid to play” in the publication, concerns moderating images promising unrealistic rewards for online entertainment activities. The system is therefore examined both for retaining content and for spotting content that a rule seeks to exclude.

The demonstration notebook indicated by the authors reproduces the healthy eating case. The post also considers building collections according to artistic tastes or classifying scientific data. These uses are proposed as avenues for adaptation; they are not part of the two evaluated domains. Measured gains on a small group

The scientific article published at CVPR 2026 specifies that the eighteen sessions correspond to nine participants, with no professional moderation experience, recruited within the organization. Each tried the two interactive tools on different concepts. The performances are therefore compared across participants, whose personal definitions may vary.

For misleading rewards, the F1 score rises from 0.48 with the initial prompt to 0.59 after deliberation; for healthy eating, from 0.48 to 0.58. These gaps represent 11 and 10 points on a scale from 0 to 100. This metric combines precision and recall. The images retained are more often relevant, but a larger share of the images sought escapes classification: recall decreases.

All participants preferred Agile Deliberation to manual work. Among the six dimensions of the questionnaire, the statistically significant differences concern the effort required and negative emotions, including stress. This result does not extend to all measures of fatigue or frustration.