Payload
{
"action": "edited",
"changes": {
"body": {
"from": "<!-- nixmac-storybook-preview -->\n### 🎨 Storybook preview\n\n[Open Storybook preview](https://0d703255.nixmac-storybook.pages.dev)\n\nUpdated for cc829b3882ebff260c36af916c88c5592de75b89"
}
},
"issue": {
"url": "https://api.github.com/repos/darkmatter/nixmac/issues/553",
"repository_url": "https://api.github.com/repos/darkmatter/nixmac",
"labels_url": "https://api.github.com/repos/darkmatter/nixmac/issues/553/labels{/name}",
"comments_url": "https://api.github.com/repos/darkmatter/nixmac/issues/553/comments",
"events_url": "https://api.github.com/repos/darkmatter/nixmac/issues/553/events",
"html_url": "https://github.com/darkmatter/nixmac/pull/553",
"id": 4919279827,
"node_id": "PR_kwDOSB6EzM7zX2Gr",
"number": 553,
"title": "feat(evolve): curb exploration thrash with repeat-call short-circuiting and decisive-exploration guidance",
"user": {
"login": "arximboldi",
"id": 4521138,
"node_id": "MDQ6VXNlcjQ1MjExMzg=",
"avatar_url": "https://avatars.githubusercontent.com/u/4521138?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/arximboldi",
"html_url": "https://github.com/arximboldi",
"followers_url": "https://api.github.com/users/arximboldi/followers",
"following_url": "https://api.github.com/users/arximboldi/following{/other_user}",
"gists_url": "https://api.github.com/users/arximboldi/gists{/gist_id}",
"starred_url": "https://api.github.com/users/arximboldi/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/arximboldi/subscriptions",
"organizations_url": "https://api.github.com/users/arximboldi/orgs",
"repos_url": "https://api.github.com/users/arximboldi/repos",
"events_url": "https://api.github.com/users/arximboldi/events{/privacy}",
"received_events_url": "https://api.github.com/users/arximboldi/received_events",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"labels": [],
"state": "open",
"locked": false,
"assignees": [],
"milestone": null,
"comments": 2,
"created_at": "2026-07-18T17:17:15Z",
"updated_at": "2026-07-19T19:44:31Z",
"closed_at": null,
"assignee": null,
"author_association": "CONTRIBUTOR",
"issue_field_values": [],
"type": null,
"active_lock_reason": null,
"draft": false,
"pull_request": {
"url": "https://api.github.com/repos/darkmatter/nixmac/pulls/553",
"html_url": "https://github.com/darkmatter/nixmac/pull/553",
"diff_url": "https://github.com/darkmatter/nixmac/pull/553.diff",
"patch_url": "https://github.com/darkmatter/nixmac/pull/553.patch",
"merged_at": null
},
"body": "In #550 we bring the Eval suite back in shape. This is one of a series of PR's addressing some initial findings running it.\r\n\r\nThis addresses item 4 of the evolve-loop convergence follow-ups (`apps/eval/wip/eval-suite-findings.md` on `jp/eval-fixes`). Companion to #551 and #552.\r\n\r\n## Problem\r\n\r\nBaseline case 50 spent all 18 iterations on `list_files`/`search_code`/`read_file` — including literally repeated identical calls — and made zero edits. Case 65 (intentionally vague \"make my mac look cool\") ground through 25 iterations / 453k tokens instead of using the conversational-response path.\r\n\r\n## Changes (2 commits)\r\n\r\n- **Engine:** a read-only tool call (`list_files`, `read_file`, `search_code`, `search_docs`, `search_packages`) repeated with identical arguments within a 10-iteration window is answered with a nudge to commit to a decision instead of re-executed. Call identity uses key-sorted canonical JSON so argument order doesn't matter. Any edit or `build_check` clears the window (they can change file contents / write `flake.lock`); beyond the window the call re-runs, since its earlier result may have been pruned from context by retention.\r\n- **System prompt:** two additions — vague-*intent* requests get a short conversational reply instead of open-ended exploration; and exploration is bounded: no identical repeat calls, \"no existing file for this\" is a normal outcome that means *create the config*, and conversational bail-out is only for unclear intent, never for missing files.\r\n\r\nThe \"never for missing files\" clause is not theoretical: the first draft lacked it, and in eval case 39 (\"add an `ll` alias\") the model bailed out conversationally at iteration 4 instead of creating the zsh config. The recalibrated wording fixed the vague case without that regression.\r\n\r\n## Testing\r\n\r\n- 3 unit tests for repeat-call identity (argument-order stability, distinct args, mutating/repeatable tools excluded).\r\n- Eval spot-checks (single stochastic samples, PR1+PR2+PR3 binary): case 65 now ends conversationally at iteration 2 / 18k tokens (baseline: 25 it / 453k); case 50 generated cleanly with git config implemented on repro (baseline: 18 it of pure thrash, 0 edits); case 44 generated with a verified build. gpt-oss-120b variance is large — the full 28-case sweep is the real re-measure.\r\n- Full `cargo test` green except the 2 pre-existing sandbox-only `prepare_import_target_*` failures.\n",
"reactions": {
"url": "https://api.github.com/repos/darkmatter/nixmac/issues/553/reactions",
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
},
"timeline_url": "https://api.github.com/repos/darkmatter/nixmac/issues/553/timeline",
"performed_via_github_app": null,
"state_reason": null
},
"comment": {
"url": "https://api.github.com/repos/darkmatter/nixmac/issues/comments/5012195681",
"html_url": "https://github.com/darkmatter/nixmac/pull/553#issuecomment-5012195681",
"issue_url": "https://api.github.com/repos/darkmatter/nixmac/issues/553",
"id": 5012195681,
"node_id": "IC_kwDOSB6EzM8AAAABKsAJYQ",
"user": {
"login": "darkmatter[bot]",
"id": 283346107,
"node_id": "BOT_kgDOEOOEuw",
"avatar_url": "https://avatars.githubusercontent.com/in/3663660?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/darkmatter%5Bbot%5D",
"html_url": "https://github.com/apps/darkmatter",
"followers_url": "https://api.github.com/users/darkmatter%5Bbot%5D/followers",
"following_url": "https://api.github.com/users/darkmatter%5Bbot%5D/following{/other_user}",
"gists_url": "https://api.github.com/users/darkmatter%5Bbot%5D/gists{/gist_id}",
"starred_url": "https://api.github.com/users/darkmatter%5Bbot%5D/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/darkmatter%5Bbot%5D/subscriptions",
"organizations_url": "https://api.github.com/users/darkmatter%5Bbot%5D/orgs",
"repos_url": "https://api.github.com/users/darkmatter%5Bbot%5D/repos",
"events_url": "https://api.github.com/users/darkmatter%5Bbot%5D/events{/privacy}",
"received_events_url": "https://api.github.com/users/darkmatter%5Bbot%5D/received_events",
"type": "Bot",
"user_view_type": "public",
"site_admin": false
},
"created_at": "2026-07-18T17:19:39Z",
"updated_at": "2026-07-19T19:46:55Z",
"body": "<!-- nixmac-storybook-preview -->\n### 🎨 Storybook preview\n\n[Open Storybook preview](https://0d703255.nixmac-storybook.pages.dev)\n\nUpdated for cc829b3882ebff260c36af916c88c5592de75b89\n\n---\n\n### ⚠️ Detected UI changes (5)\n\nThese stories' HTML snapshots changed. I've added screenshots + links to the changed stories below. Review them carefully then accept the changes to regenerate baselines and include them in this PR:\n\n#### [UI/InputGroup › Inline Addons](https://0d703255.nixmac-storybook.pages.dev/?path=/story/ui-inputgroup--inline-addons)\n\n\n\n#### [UI/InputGroup › Block Addons](https://0d703255.nixmac-storybook.pages.dev/?path=/story/ui-inputgroup--block-addons)\n\n\n\n#### [UI/Select › Provider](https://0d703255.nixmac-storybook.pages.dev/?path=/story/ui-select--provider)\n\n\n\n#### [Flows/Evolve › Playground](https://0d703255.nixmac-storybook.pages.dev/?path=/story/flows-evolve--playground)\n\n\n\n#### [Flows/Evolve › 1. Begin (idle)](https://0d703255.nixmac-storybook.pages.dev/?path=/story/flows-evolve--begin)\n\n\n\n---\n\n### Accept UI changes\n\n- [ ] Click here to accept these changes\n\nAlternatively, you can run `bun run test:update-snapshots` locally to re-generate the baselines and then push the changes to this PR.\n\n<details>\n<summary>What does this do?</summary>\n\nThe screenshots above show UI changes detected by the Storybook\nsnapshot tests run on this PR. Each image is the rendered output of\na Storybook story from the code in this PR branch; the snapshot\ntest compared it against the committed baseline in\n`__snapshots__/` and flagged the difference.\n\nChecking the box tells the `darkmatter[bot]` to regenerate the\nbaselines from this PR's current code and commit them directly to\nthis branch. The new baselines become the source of truth for\nfuture runs — only accept after confirming the visual changes are\nintentional.\n\nComparison baseline: the committed `__snapshots__/` files on this\nPR branch (carried forward from develop). Accept updates them in\nplace on this branch.\n\n</details>",
"author_association": "CONTRIBUTOR",
"reactions": {
"url": "https://api.github.com/repos/darkmatter/nixmac/issues/comments/5012195681/reactions",
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
},
"performed_via_github_app": {
"id": 3663660,
"client_id": "Iv23lifA2SLzhklKJM5M",
"slug": "darkmatter",
"node_id": "A_kwHOARAg0c4AN-cs",
"owner": {
"login": "darkmatter",
"id": 17834193,
"node_id": "MDEyOk9yZ2FuaXphdGlvbjE3ODM0MTkz",
"avatar_url": "https://avatars.githubusercontent.com/u/17834193?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/darkmatter",
"html_url": "https://github.com/darkmatter",
"followers_url": "https://api.github.com/users/darkmatter/followers",
"following_url": "https://api.github.com/users/darkmatter/following{/other_user}",
"gists_url": "https://api.github.com/users/darkmatter/gists{/gist_id}",
"starred_url": "https://api.github.com/users/darkmatter/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/darkmatter/subscriptions",
"organizations_url": "https://api.github.com/users/darkmatter/orgs",
"repos_url": "https://api.github.com/users/darkmatter/repos",
"events_url": "https://api.github.com/users/darkmatter/events{/privacy}",
"received_events_url": "https://api.github.com/users/darkmatter/received_events",
"type": "Organization",
"user_view_type": "public",
"site_admin": false
},
"name": "darkmatter",
"description": "This is the darkmatter agent - mention the bot to have it review PR's, fix issues, and drive the agent.",
"external_url": "https://dm.sh",
"html_url": "https://github.com/apps/darkmatter",
"created_at": "2026-05-10T08:36:27Z",
"updated_at": "2026-07-01T11:26:40Z",
"permissions": {
"actions": "write",
"actions_variables": "read",
"agent_secrets": "read",
"agent_variables": "read",
"checks": "write",
"contents": "write",
"copilot_agent_settings": "write",
"custom_properties_for_organizations": "write",
"deployments": "write",
"discussions": "write",
"enterprise_organization_installations": "write",
"enterprise_organization_installation_repositories": "write",
"environments": "write",
"issues": "write",
"issue_fields": "read",
"issue_types": "read",
"members": "read",
"metadata": "read",
"organization_actions_variables": "write",
"organization_administration": "write",
"organization_agent_secrets": "read",
"organization_agent_variables": "read",
"organization_codespaces": "write",
"organization_codespaces_secrets": "write",
"organization_copilot_agent_settings": "read",
"organization_custom_properties": "write",
"organization_dependabot_dismissal_requests": "read",
"organization_dependabot_secrets": "read",
"organization_events": "read",
"organization_hooks": "write",
"organization_personal_access_tokens": "write",
"organization_personal_access_token_requests": "write",
"organization_private_registries": "write",
"organization_secrets": "write",
"organization_self_hosted_runners": "write",
"org_copilot_content_exclusion": "read",
"packages": "write",
"pages": "write",
"pull_requests": "write",
"repository_hooks": "write",
"secrets": "read",
"statuses": "write",
"user_events": "read",
"workflows": "write"
},
"events": [
"commit_comment",
"create",
"deployment",
"deployment_review",
"deployment_status",
"discussion",
"discussion_comment",
"issues",
"issue_comment",
"merge_queue_entry",
"pull_request",
"pull_request_review",
"pull_request_review_comment",
"pull_request_review_thread",
"release",
"status",
"sub_issues",
"workflow_dispatch",
"workflow_job",
"workflow_run"
]
},
"minimized": null
},
"repository": {
"id": 1209959628,
"node_id": "R_kgDOSB6EzA",
"name": "nixmac",
"full_name": "darkmatter/nixmac",
"private": false,
"owner": {
"login": "darkmatter",
"id": 17834193,
"node_id": "MDEyOk9yZ2FuaXphdGlvbjE3ODM0MTkz",
"avatar_url": "https://avatars.githubusercontent.com/u/17834193?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/darkmatter",
"html_url": "https://github.com/darkmatter",
"followers_url": "https://api.github.com/users/darkmatter/followers",
"following_url": "https://api.github.com/users/darkmatter/following{/other_user}",
"gists_url": "https://api.github.com/users/darkmatter/gists{/gist_id}",
"starred_url": "https://api.github.com/users/darkmatter/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/darkmatter/subscriptions",
"organizations_url": "https://api.github.com/users/darkmatter/orgs",
"repos_url": "https://api.github.com/users/darkmatter/repos",
"events_url": "https://api.github.com/users/darkmatter/events{/privacy}",
"received_events_url": "https://api.github.com/users/darkmatter/received_events",
"type": "Organization",
"user_view_type": "public",
"site_admin": false
},
"html_url": "https://github.com/darkmatter/nixmac",
"description": "Home manager and nix-darwin that understands plain English",
"fork": false,
"url": "https://api.github.com/repos/darkmatter/nixmac",
"forks_url": "https://api.github.com/repos/darkmatter/nixmac/forks",
"keys_url": "https://api.github.com/repos/darkmatter/nixmac/keys{/key_id}",
"collaborators_url": "https://api.github.com/repos/darkmatter/nixmac/collaborators{/collaborator}",
"teams_url": "https://api.github.com/repos/darkmatter/nixmac/teams",
"hooks_url": "https://api.github.com/repos/darkmatter/nixmac/hooks",
"issue_events_url": "https://api.github.com/repos/darkmatter/nixmac/issues/events{/number}",
"events_url": "https://api.github.com/repos/darkmatter/nixmac/events",
"assignees_url": "https://api.github.com/repos/darkmatter/nixmac/assignees{/user}",
"branches_url": "https://api.github.com/repos/darkmatter/nixmac/branches{/branch}",
"tags_url": "https://api.github.com/repos/darkmatter/nixmac/tags",
"blobs_url": "https://api.github.com/repos/darkmatter/nixmac/git/blobs{/sha}",
"git_tags_url": "https://api.github.com/repos/darkmatter/nixmac/git/tags{/sha}",
"git_refs_url": "https://api.github.com/repos/darkmatter/nixmac/git/refs{/sha}",
"trees_url": "https://api.github.com/repos/darkmatter/nixmac/git/trees{/sha}",
"statuses_url": "https://api.github.com/repos/darkmatter/nixmac/statuses/{sha}",
"languages_url": "https://api.github.com/repos/darkmatter/nixmac/languages",
"stargazers_url": "https://api.github.com/repos/darkmatter/nixmac/stargazers",
"contributors_url": "https://api.github.com/repos/darkmatter/nixmac/contributors",
"subscribers_url": "https://api.github.com/repos/darkmatter/nixmac/subscribers",
"subscription_url": "https://api.github.com/repos/darkmatter/nixmac/subscription",
"commits_url": "https://api.github.com/repos/darkmatter/nixmac/commits{/sha}",
"git_commits_url": "https://api.github.com/repos/darkmatter/nixmac/git/commits{/sha}",
"comments_url": "https://api.github.com/repos/darkmatter/nixmac/comments{/number}",
"issue_comment_url": "https://api.github.com/repos/darkmatter/nixmac/issues/comments{/number}",
"contents_url": "https://api.github.com/repos/darkmatter/nixmac/contents/{+path}",
"compare_url": "https://api.github.com/repos/darkmatter/nixmac/compare/{base}...{head}",
"merges_url": "https://api.github.com/repos/darkmatter/nixmac/merges",
"archive_url": "https://api.github.com/repos/darkmatter/nixmac/{archive_format}{/ref}",
"downloads_url": "https://api.github.com/repos/darkmatter/nixmac/downloads",
"issues_url": "https://api.github.com/repos/darkmatter/nixmac/issues{/number}",
"pulls_url": "https://api.github.com/repos/darkmatter/nixmac/pulls{/number}",
"milestones_url": "https://api.github.com/repos/darkmatter/nixmac/milestones{/number}",
"notifications_url": "https://api.github.com/repos/darkmatter/nixmac/notifications{?since,all,participating}",
"labels_url": "https://api.github.com/repos/darkmatter/nixmac/labels{/name}",
"releases_url": "https://api.github.com/repos/darkmatter/nixmac/releases{/id}",
"deployments_url": "https://api.github.com/repos/darkmatter/nixmac/deployments",
"created_at": "2026-04-14T00:37:13Z",
"updated_at": "2026-07-19T15:23:06Z",
"pushed_at": "2026-07-19T19:41:26Z",
"git_url": "git://github.com/darkmatter/nixmac.git",
"ssh_url": "org-17834193@github.com:darkmatter/nixmac.git",
"clone_url": "https://github.com/darkmatter/nixmac.git",
"svn_url": "https://github.com/darkmatter/nixmac",
"homepage": "https://nixmac.com",
"size": 688809,
"stargazers_count": 21,
"watchers_count": 21,
"language": "Rust",
"has_issues": true,
"has_projects": false,
"has_downloads": true,
"has_wiki": false,
"has_pages": true,
"has_discussions": true,
"forks_count": 2,
"mirror_url": null,
"archived": false,
"disabled": false,
"open_issues_count": 99,
"license": {
"key": "mit",
"name": "MIT License",
"spdx_id": "MIT",
"url": "https://api.github.com/licenses/mit",
"node_id": "MDc6TGljZW5zZTEz"
},
"allow_forking": true,
"is_template": false,
"web_commit_signoff_required": false,
"has_pull_requests": true,
"pull_request_creation_policy": "all",
"topics": [
"home-manager",
"nix",
"nix-darwin",
"nix-flake",
"opencode"
],
"visibility": "public",
"forks": 2,
"open_issues": 99,
"watchers": 21,
"default_branch": "main",
"custom_properties": {}
},
"organization": {
"login": "darkmatter",
"id": 17834193,
"node_id": "MDEyOk9yZ2FuaXphdGlvbjE3ODM0MTkz",
"url": "https://api.github.com/orgs/darkmatter",
"repos_url": "https://api.github.com/orgs/darkmatter/repos",
"events_url": "https://api.github.com/orgs/darkmatter/events",
"hooks_url": "https://api.github.com/orgs/darkmatter/hooks",
"issues_url": "https://api.github.com/orgs/darkmatter/issues",
"members_url": "https://api.github.com/orgs/darkmatter/members{/member}",
"public_members_url": "https://api.github.com/orgs/darkmatter/public_members{/member}",
"avatar_url": "https://avatars.githubusercontent.com/u/17834193?v=4",
"description": ""
},
"enterprise": {
"id": 469843,
"slug": "darkmatter",
"name": "darkmatter",
"node_id": "E_kgDOAAcrUw",
"avatar_url": "https://avatars.githubusercontent.com/b/469843?v=4",
"description": "",
"website_url": "darkmatter.io",
"html_url": "https://github.com/enterprises/darkmatter",
"created_at": "2025-09-07T16:01:00Z",
"updated_at": "2026-07-07T15:59:26Z"
},
"sender": {
"login": "darkmatter[bot]",
"id": 283346107,
"node_id": "BOT_kgDOEOOEuw",
"avatar_url": "https://avatars.githubusercontent.com/in/3663660?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/darkmatter%5Bbot%5D",
"html_url": "https://github.com/apps/darkmatter",
"followers_url": "https://api.github.com/users/darkmatter%5Bbot%5D/followers",
"following_url": "https://api.github.com/users/darkmatter%5Bbot%5D/following{/other_user}",
"gists_url": "https://api.github.com/users/darkmatter%5Bbot%5D/gists{/gist_id}",
"starred_url": "https://api.github.com/users/darkmatter%5Bbot%5D/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/darkmatter%5Bbot%5D/subscriptions",
"organizations_url": "https://api.github.com/users/darkmatter%5Bbot%5D/orgs",
"repos_url": "https://api.github.com/users/darkmatter%5Bbot%5D/repos",
"events_url": "https://api.github.com/users/darkmatter%5Bbot%5D/events{/privacy}",
"received_events_url": "https://api.github.com/users/darkmatter%5Bbot%5D/received_events",
"type": "Bot",
"user_view_type": "public",
"site_admin": false
},
"installation": {
"id": 131074261,
"node_id": "MDIzOkludGVncmF0aW9uSW5zdGFsbGF0aW9uMTMxMDc0MjYx"
}
}