Payload
{
"action": "edited",
"issue": {
"url": "https://api.github.com/repos/darkmatter/nixmac/issues/158",
"repository_url": "https://api.github.com/repos/darkmatter/nixmac",
"labels_url": "https://api.github.com/repos/darkmatter/nixmac/issues/158/labels{/name}",
"comments_url": "https://api.github.com/repos/darkmatter/nixmac/issues/158/comments",
"events_url": "https://api.github.com/repos/darkmatter/nixmac/issues/158/events",
"html_url": "https://github.com/darkmatter/nixmac/issues/158",
"id": 4450681072,
"node_id": "I_kwDOSB6EzM8AAAABCUgA8A",
"number": 158,
"title": "Prevent unrelated edits outside the user's prompt scope",
"user": {
"login": "linear-code[bot]",
"id": 222613912,
"node_id": "BOT_kgDODUTRmA",
"avatar_url": "https://avatars.githubusercontent.com/in/1658531?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/linear-code%5Bbot%5D",
"html_url": "https://github.com/apps/linear-code",
"followers_url": "https://api.github.com/users/linear-code%5Bbot%5D/followers",
"following_url": "https://api.github.com/users/linear-code%5Bbot%5D/following{/other_user}",
"gists_url": "https://api.github.com/users/linear-code%5Bbot%5D/gists{/gist_id}",
"starred_url": "https://api.github.com/users/linear-code%5Bbot%5D/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/linear-code%5Bbot%5D/subscriptions",
"organizations_url": "https://api.github.com/users/linear-code%5Bbot%5D/orgs",
"repos_url": "https://api.github.com/users/linear-code%5Bbot%5D/repos",
"events_url": "https://api.github.com/users/linear-code%5Bbot%5D/events{/privacy}",
"received_events_url": "https://api.github.com/users/linear-code%5Bbot%5D/received_events",
"type": "Bot",
"user_view_type": "public",
"site_admin": false
},
"labels": [
{
"id": 10686173040,
"node_id": "LA_kwDOSB6EzM8AAAACfPIPcA",
"url": "https://api.github.com/repos/darkmatter/nixmac/labels/bug",
"name": "bug",
"color": "d73a4a",
"default": true,
"description": "Something isn't working"
},
{
"id": 10936169199,
"node_id": "LA_kwDOSB6EzM8AAAACi9iy7w",
"url": "https://api.github.com/repos/darkmatter/nixmac/labels/juan-sourced",
"name": "juan-sourced",
"color": "9D86FF",
"default": false,
"description": null
}
],
"state": "open",
"locked": false,
"assignees": [],
"milestone": null,
"comments": 0,
"created_at": "2026-05-15T02:11:36Z",
"updated_at": "2026-06-01T06:45:38Z",
"closed_at": null,
"assignee": null,
"author_association": "NONE",
"issue_field_values": [],
"type": null,
"active_lock_reason": null,
"sub_issues_summary": {
"total": 0,
"completed": 0,
"percent_completed": 0
},
"issue_dependencies_summary": {
"blocked_by": 0,
"total_blocked_by": 0,
"blocking": 0,
"total_blocking": 0
},
"body": "## Problem\n\nThe Evolve agent must not make tangential or \"helpful\" changes that the user did not ask for. This is broader than preserving Nix formatting: if the user asks for one change, the resulting diff should stay scoped to that prompt.\n\nThis preserves the distinct acceptance criterion from [ENG-210](https://linear.app/darkmatterlabs/issue/ENG-210/trust-and-safety-ai-should-only-make-changes-directly-related-to-the) while moving it under the broader Evolve quality eval suite.\n\n## Acceptance Criteria\n\n* Representative eval cases include prompts where the correct behavior is a tightly scoped diff.\n* The eval output classifies unrelated edits / prompt-scope violations separately from formatting, protected-file, and normal correctness failures.\n* Launch-blocking threshold: zero severe cases where a simple user prompt produces unrelated package, shell, config, or structural changes.\n* A post-evolution validation approach is specified: either prompt-level constraint, deterministic diff-vs-prompt checks, reviewer rubric, or a combination.\n* Any implementation path keeps the distinction clear between:\n * unrelated edits outside the prompt scope\n * formatting/comment preservation regressions covered by darkmatter/nixmac#137\n * protected-file enforcement covered by [ENG-236](https://linear.app/darkmatterlabs/issue/ENG-236/enforce-protected-file-constraints-in-rust)\n\n## Source\n\nSupersedes [ENG-210](https://linear.app/darkmatterlabs/issue/ENG-210/trust-and-safety-ai-should-only-make-changes-directly-related-to-the)'s unique diff-scope requirement without keeping the old \"Trust & Safety\" framing.\n\n---\n\n## Acceptance Criteria / Gherkin Specs\n\n```gherkin\nScenario: Simple prompt results in a tightly scoped diff with no unrelated changes\n Given I submit a simple prompt (e.g., \"Install git\")\n When nixmac evolves my config\n Then the diff contains only changes directly related to installing git\n And no unrelated packages, shell settings, config options, or structural changes appear in the diff\n\nScenario: Eval suite includes tightly-scoped prompt cases\n Given the ENG-495 acceptance criteria\n When the Evolve quality eval suite is reviewed\n Then it includes representative cases where correct behavior is a scoped diff\n And unrelated-edit violations are classified separately from formatting and protected-file regressions\n\nScenario: Launch-blocking threshold — zero severe scope violations\n Given the eval results are analyzed for launch readiness\n Then zero cases of a simple prompt producing unrelated package, shell, config, or structural changes are found\n And if any such cases exist, the launch is blocked until they are resolved\n\nScenario: Post-evolution validation approach is documented\n Given this ticket is resolved\n Then a validation approach is specified for catching scope violations: one of prompt-level constraint, deterministic diff-vs-prompt checks, reviewer rubric, or a combination\n And the approach is documented in the eval suite or associated design doc\n\nScenario: Scope violations are clearly distinguished from formatting and protected-file issues\n Given an eval case produces unexpected output\n When the eval classifier categorizes the failure\n Then \"unrelated edits outside prompt scope\" is a distinct category from formatting regressions (ENG-477) and protected-file violations (ENG-236)\n And the failure category accurately describes the root cause without conflation\n```",
"reactions": {
"url": "https://api.github.com/repos/darkmatter/nixmac/issues/158/reactions",
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
},
"timeline_url": "https://api.github.com/repos/darkmatter/nixmac/issues/158/timeline",
"performed_via_github_app": null,
"state_reason": null,
"pinned_comment": null
},
"changes": {
"body": {
"from": "## Problem\n\nThe Evolve agent must not make tangential or “helpful” changes that the user did not ask for. This is broader than preserving Nix formatting: if the user asks for one change, the resulting diff should stay scoped to that prompt.\n\nThis preserves the distinct acceptance criterion from [ENG-210](https://linear.app/darkmatterlabs/issue/ENG-210/trust-and-safety-ai-should-only-make-changes-directly-related-to-the) while moving it under the broader Evolve quality eval suite.\n\n## Acceptance Criteria\n\n* Representative eval cases include prompts where the correct behavior is a tightly scoped diff.\n* The eval output classifies unrelated edits / prompt-scope violations separately from formatting, protected-file, and normal correctness failures.\n* Launch-blocking threshold: zero severe cases where a simple user prompt produces unrelated package, shell, config, or structural changes.\n* A post-evolution validation approach is specified: either prompt-level constraint, deterministic diff-vs-prompt checks, reviewer rubric, or a combination.\n* Any implementation path keeps the distinction clear between:\n * unrelated edits outside the prompt scope\n * formatting/comment preservation regressions covered by darkmatter/nixmac#137\n * protected-file enforcement covered by [ENG-236](https://linear.app/darkmatterlabs/issue/ENG-236/enforce-protected-file-constraints-in-rust)\n\n## Source\n\nSupersedes ENG-210's unique diff-scope requirement without keeping the old “Trust & Safety” framing."
}
},
"repository": {
"id": 1209959628,
"node_id": "R_kgDOSB6EzA",
"name": "nixmac",
"full_name": "darkmatter/nixmac",
"private": false,
"owner": {
"login": "darkmatter",
"id": 17834193,
"node_id": "MDEyOk9yZ2FuaXphdGlvbjE3ODM0MTkz",
"avatar_url": "https://avatars.githubusercontent.com/u/17834193?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/darkmatter",
"html_url": "https://github.com/darkmatter",
"followers_url": "https://api.github.com/users/darkmatter/followers",
"following_url": "https://api.github.com/users/darkmatter/following{/other_user}",
"gists_url": "https://api.github.com/users/darkmatter/gists{/gist_id}",
"starred_url": "https://api.github.com/users/darkmatter/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/darkmatter/subscriptions",
"organizations_url": "https://api.github.com/users/darkmatter/orgs",
"repos_url": "https://api.github.com/users/darkmatter/repos",
"events_url": "https://api.github.com/users/darkmatter/events{/privacy}",
"received_events_url": "https://api.github.com/users/darkmatter/received_events",
"type": "Organization",
"user_view_type": "public",
"site_admin": false
},
"html_url": "https://github.com/darkmatter/nixmac",
"description": "Home manager and nix-darwin that understands plain English",
"fork": false,
"url": "https://api.github.com/repos/darkmatter/nixmac",
"forks_url": "https://api.github.com/repos/darkmatter/nixmac/forks",
"keys_url": "https://api.github.com/repos/darkmatter/nixmac/keys{/key_id}",
"collaborators_url": "https://api.github.com/repos/darkmatter/nixmac/collaborators{/collaborator}",
"teams_url": "https://api.github.com/repos/darkmatter/nixmac/teams",
"hooks_url": "https://api.github.com/repos/darkmatter/nixmac/hooks",
"issue_events_url": "https://api.github.com/repos/darkmatter/nixmac/issues/events{/number}",
"events_url": "https://api.github.com/repos/darkmatter/nixmac/events",
"assignees_url": "https://api.github.com/repos/darkmatter/nixmac/assignees{/user}",
"branches_url": "https://api.github.com/repos/darkmatter/nixmac/branches{/branch}",
"tags_url": "https://api.github.com/repos/darkmatter/nixmac/tags",
"blobs_url": "https://api.github.com/repos/darkmatter/nixmac/git/blobs{/sha}",
"git_tags_url": "https://api.github.com/repos/darkmatter/nixmac/git/tags{/sha}",
"git_refs_url": "https://api.github.com/repos/darkmatter/nixmac/git/refs{/sha}",
"trees_url": "https://api.github.com/repos/darkmatter/nixmac/git/trees{/sha}",
"statuses_url": "https://api.github.com/repos/darkmatter/nixmac/statuses/{sha}",
"languages_url": "https://api.github.com/repos/darkmatter/nixmac/languages",
"stargazers_url": "https://api.github.com/repos/darkmatter/nixmac/stargazers",
"contributors_url": "https://api.github.com/repos/darkmatter/nixmac/contributors",
"subscribers_url": "https://api.github.com/repos/darkmatter/nixmac/subscribers",
"subscription_url": "https://api.github.com/repos/darkmatter/nixmac/subscription",
"commits_url": "https://api.github.com/repos/darkmatter/nixmac/commits{/sha}",
"git_commits_url": "https://api.github.com/repos/darkmatter/nixmac/git/commits{/sha}",
"comments_url": "https://api.github.com/repos/darkmatter/nixmac/comments{/number}",
"issue_comment_url": "https://api.github.com/repos/darkmatter/nixmac/issues/comments{/number}",
"contents_url": "https://api.github.com/repos/darkmatter/nixmac/contents/{+path}",
"compare_url": "https://api.github.com/repos/darkmatter/nixmac/compare/{base}...{head}",
"merges_url": "https://api.github.com/repos/darkmatter/nixmac/merges",
"archive_url": "https://api.github.com/repos/darkmatter/nixmac/{archive_format}{/ref}",
"downloads_url": "https://api.github.com/repos/darkmatter/nixmac/downloads",
"issues_url": "https://api.github.com/repos/darkmatter/nixmac/issues{/number}",
"pulls_url": "https://api.github.com/repos/darkmatter/nixmac/pulls{/number}",
"milestones_url": "https://api.github.com/repos/darkmatter/nixmac/milestones{/number}",
"notifications_url": "https://api.github.com/repos/darkmatter/nixmac/notifications{?since,all,participating}",
"labels_url": "https://api.github.com/repos/darkmatter/nixmac/labels{/name}",
"releases_url": "https://api.github.com/repos/darkmatter/nixmac/releases{/id}",
"deployments_url": "https://api.github.com/repos/darkmatter/nixmac/deployments",
"created_at": "2026-04-14T00:37:13Z",
"updated_at": "2026-06-01T06:15:50Z",
"pushed_at": "2026-06-01T06:32:03Z",
"git_url": "git://github.com/darkmatter/nixmac.git",
"ssh_url": "git@github.com:darkmatter/nixmac.git",
"clone_url": "https://github.com/darkmatter/nixmac.git",
"svn_url": "https://github.com/darkmatter/nixmac",
"homepage": "https://nixmac.com",
"size": 678800,
"stargazers_count": 5,
"watchers_count": 5,
"language": "Rust",
"has_issues": true,
"has_projects": true,
"has_downloads": true,
"has_wiki": true,
"has_pages": false,
"has_discussions": false,
"forks_count": 1,
"mirror_url": null,
"archived": false,
"disabled": false,
"open_issues_count": 77,
"license": {
"key": "mit",
"name": "MIT License",
"spdx_id": "MIT",
"url": "https://api.github.com/licenses/mit",
"node_id": "MDc6TGljZW5zZTEz"
},
"allow_forking": true,
"is_template": false,
"web_commit_signoff_required": false,
"has_pull_requests": true,
"pull_request_creation_policy": "all",
"topics": [
"home-manager",
"nix",
"nix-darwin",
"nix-flake",
"opencode"
],
"visibility": "public",
"forks": 1,
"open_issues": 77,
"watchers": 5,
"default_branch": "develop",
"custom_properties": {}
},
"organization": {
"login": "darkmatter",
"id": 17834193,
"node_id": "MDEyOk9yZ2FuaXphdGlvbjE3ODM0MTkz",
"url": "https://api.github.com/orgs/darkmatter",
"repos_url": "https://api.github.com/orgs/darkmatter/repos",
"events_url": "https://api.github.com/orgs/darkmatter/events",
"hooks_url": "https://api.github.com/orgs/darkmatter/hooks",
"issues_url": "https://api.github.com/orgs/darkmatter/issues",
"members_url": "https://api.github.com/orgs/darkmatter/members{/member}",
"public_members_url": "https://api.github.com/orgs/darkmatter/public_members{/member}",
"avatar_url": "https://avatars.githubusercontent.com/u/17834193?v=4",
"description": ""
},
"enterprise": {
"id": 469843,
"slug": "darkmatter",
"name": "darkmatter",
"node_id": "E_kgDOAAcrUw",
"avatar_url": "https://avatars.githubusercontent.com/b/469843?v=4",
"description": "",
"website_url": "darkmatter.io",
"html_url": "https://github.com/enterprises/darkmatter",
"created_at": "2025-09-07T16:01:00Z",
"updated_at": "2026-05-09T15:34:55Z"
},
"sender": {
"login": "czxtm",
"id": 1325802,
"node_id": "MDQ6VXNlcjEzMjU4MDI=",
"avatar_url": "https://avatars.githubusercontent.com/u/1325802?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/czxtm",
"html_url": "https://github.com/czxtm",
"followers_url": "https://api.github.com/users/czxtm/followers",
"following_url": "https://api.github.com/users/czxtm/following{/other_user}",
"gists_url": "https://api.github.com/users/czxtm/gists{/gist_id}",
"starred_url": "https://api.github.com/users/czxtm/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/czxtm/subscriptions",
"organizations_url": "https://api.github.com/users/czxtm/orgs",
"repos_url": "https://api.github.com/users/czxtm/repos",
"events_url": "https://api.github.com/users/czxtm/events{/privacy}",
"received_events_url": "https://api.github.com/users/czxtm/received_events",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"installation": {
"id": 131074261,
"node_id": "MDIzOkludGVncmF0aW9uSW5zdGFsbGF0aW9uMTMxMDc0MjYx"
}
}