CIPHER

Research project · TEXTUAL 2027 workshop at SANER 2027

CIPHER

Contextual Instruction Precedence and Hierarchy Evaluation for Repositories

Coding agents read project instructions from AGENTS.md files, one at the repository root and sometimes more in subfolders. Two of those instructions can look like they disagree: one file asks for four-space indentation, another for two. That is not always a real conflict. A more specific file in the folder being edited may legitimately take precedence, or one instruction may simply not apply to the file or task at hand. CIPHER asks whether automated methods can tell these situations apart, labelling each apparent disagreement as an unresolved conflict, a valid override, disjoint applicability, compatible, or insufficient evidence.

Awaiz Ahmed, Mohammad Umar, Huda Mueen · Al Ain University, United Arab Emirates

Two instructions, one repositoryauthored example
AGENTS.mdUse exactly four spaces per indentation level.
web/AGENTS.mdUse exactly two spaces per indentation level.
Editing web/src/button.ts? web/AGENTS.md is deeper on its folder chain, so it overrides. Editing api/src/server.ts? The web/ rule is never loaded. Try other placements in the Case Lab.

Status: pilot study in progress — results not yet available

Data export not yet available.

Case Lab

Same two rules, different places

Authored illustration of the declared profile, computed in your browser by fixed rules; not a model and not research data.

The wording of both rules never changes. Move Rule B to a different AGENTS.md file, or change which file is being edited, and follow how the decision sequence reaches a label.

Rule A · always in root AGENTS.md
Use exactly four spaces per indentation level.
Wording fixed
Rule B · placement chosen below
Use exactly two spaces per indentation level.
Wording fixed
Where Rule B lives
File being edited (target)

Task: add a helper function to the target file. Decision in question: indentation width.

Resulting label

Repository

Applicability

Precedence

Declared profile

How a label is decided

The study fixes one interpretation of AGENTS.md precedence before any case is labelled. It is grounded in the agents.md description and is a frozen experimental interpretation, not a claim about how every tool behaves.

Profile

  • Only the root AGENTS.md and nested AGENTS.md files participate.
  • The binding loaded set for a target file is the AGENTS.md files on its ancestor chain.
  • A deeper applicable file overrides an incompatible instruction from an ancestor file.
  • Requirements in an ancestor file that the deeper file does not contradict stay in effect.
  • Contradictions between instructions of equal authority (same file or same depth) are not resolved by the order in which they appear.
  • Instructions from unrelated branches of the repository are not loaded.

Decision sequence

  1. Evidence sufficiency. Are the two source spans, the target and the task enough to judge?
  2. Initial joint applicability. Are both instructions active for this target and task?
  3. Incompatibility. Do they demand different outcomes for the same decision?
  4. Priority resolution. Does a deeper file on the ancestor chain settle it?

Labels

  • Unresolved conflictBoth apply, they are incompatible, and no file outranks the other.
  • Valid overrideBoth apply and are incompatible, but the deeper file on the target's chain wins.
  • Disjoint applicabilityOne instruction is inactive for this target or task, for example it lives in an unrelated folder or its condition excludes the task.
  • CompatibleBoth apply and can be satisfied together.
  • Insufficient evidenceThe available spans, target or task do not support a decision.

Methods compared

  • B0 · keyword heuristicA Ruleherd-style lexical check that outputs a binary conflict flag.
  • B1 · direct contextual LLMA local model given the case evidence and asked for a label directly.
  • B2 · structured pipelineThe same model extracts each instruction's requirement, decision, modality and condition; code computes the loaded set and depth precedence; a second call checks incompatibility only when both rules stay active.
  • B1R · equal-call controlB1 followed by one self-review call over the same evidence.

Research outputs

Results, recorded cases and paper

Every number on this site is read from files generated by the analysis pipeline. The site itself computes no research numbers and calls no model.

Results

Tables and figures exported by the analysis pipeline, with sample sizes as reported.

Checking data/results.json…

Open results

Recorded cases

Replay of saved cases: source spans at pinned commits, profile-determined reference labels where available, and recorded B0/B1/B2 outputs.

Checking data/replay.json…

Open recorded cases

Paper

The four-page workshop manuscript as a readable web page, with the compiled PDF when it exists.

Checking build information…

Open paper

Reproducibility

Rebuilding from the repository

Instructions for reproducing the study (environment, pinned repository snapshots, annotation and run commands) are kept with the code, in the repository README.

This website is a static view of the generated files. To rebuild it from a checkout:

python site/build.py
python -m http.server 8000 --directory site/dist

The build copies analysis/exports/*.json and the compiled manuscript from paper/build/ when they exist; when they do not, the pages say so instead of showing placeholder numbers.