The new Grok 4.7 is a welcome update to the frontier model family, and a serious one for coding abilities. It performs very well with long-context, long-running tasks - while also having a very high token consumption; tanking its real-world utility value. Grok 4.7 scores a 75.3 on the AI Matic bench score, which is exactly the same score earned by Gemini 3.8 Flash. In our findings, it’s also the first to clear half the unprompted bug hunt and three quarters of the explicit task list all in the same run.
It costs about $2 input and $6 output per million tokens, unchanged from Grok 4.6.
Let’s take you through the numbers.
TLDR of Grok 4.7 LLM testing
Use Grok 4.7 when these conditions are met
- The work is internal or the client contract already permits it. Capability at this price is the best value in the current field, and the data-handling default is the only thing standing in the way.
- You need one model to both work a backlog and find what is not on it. No other model we have scored does both in a single pass.
- You are already on Grok 4.6. Same price, same API, 500K context, nothing to requalify on the interface.
- The surface is interactive. A 0.85 second time to first token is the fastest we have measured and it is felt directly by a user waiting on a response.
Do not use Grok 4.7 when these conditions are met
- The client mandates Zero Data Retention or a specific cloud. There is no ZDR story and no Azure or Bedrock route, and no amount of benchmark performance substitutes for either.
- Client data would flow through default terms. Training on conversations is the default unless enterprise terms say otherwise, and that is a contract question to settle before a pilot rather than after.
- The job is unattended and multi-hour. Wall clock of 16 to 33 minutes per task with 81k output tokens on xhigh needs a timeout and a token budget before it goes near production.
- Cross-tenant authorization review is the deliverable. SALES-001 has now survived five runs by five models, and SLA-002 was missed here too.
The one-line verdict on Grok 4.7 is that it does the most complete job from an efficiency standpoint yet it is the hardest to pitch to a client. Data handling and the absence of a zero data retention (ZDR) narrative makes one want to check the contract once again.
GPT-6 Astra still leads on total coverage at 33 of 57, and it got there entirely through bug mode. Grok reaches 31 through a mix of 25 and 6. Two runs, near-identical totals, opposite working styles.
Fix rate tracks severity downward: 77.8% of criticals, 47.1% of highs, 42.1% of mediums, one of three lows. For an audit workload - ordering is the right one, and it is not something every run has managed.
Fail-open detection is the strongest single behaviour that we noticed. Every authorization path that returned ‘True’ on an error was found and simultaneously flipped: SALES-002 on the MCP org check, ORG-003 on the platform super-admin check, WM-001 on the privilege-escalation role comparison, COMMON-002 and COMMON-004 on the shared auth backend. The whole shared-library blast radius closed, which is four issues that reach every service that imports them.
It missed the fail-quiet variant. ORG-004 drops a status check rather than adding an error branch, and it went untouched. A model that pattern-matches on return True inside an except block will not see a missing if member.status != "active" line, and that is exactly what happened.
The most expensive misses are the ones where the model was in the right function and stopped one line short. There are three, and together they are four one-line changes.
The BUGS-003 result is the interesting one. The model did not skip the file. It rewrote the guard from scratch and reproduced the same omission the benchmark had injected. That is a model reaching for a default set of terminal statuses and getting the same incomplete answer a hurried engineer would. Prompting harder will not fix it - a test that asserts CANCELLED is terminal will.
Two cold spots that Grok 4.7 missed
The ai_copilot service was never opened.
Two of its three issues are untouched, including AI_COPILOT-001, the chat-session IDOR that every previous run caught. Grok 4.7 is the only model in five runs to miss it, and it missed it by never reading the directory rather than by getting it wrong.
Sales is the other cold spot: five of eight sales issues missed despite heavy work elsewhere in the same service. Between them these two areas hold eight more issues and two more criticals. Pointing the model at them explicitly is the single highest-value follow-up, worth more than the three near-misses combined.
One result better than the reference
On task SLA-001 the model added idx_sla_commitment_due on (is_deleted, status, next_check_at) rather than restoring golden's idx_sla_eval_scan, which leads with organization_id. Reading the actual scan query, evaluate_due_commitments() at services.py:243 filters on status, next_check_at and is_deleted, and never on organization_id. The model's index matches the query that runs. The reference answer does not.
Actioning Grok 4.7 thoroughly
Correct golden/'s idx_sla_eval_scan. A benchmark whose oracle carries a worse answer than a model under test will keep scoring good work as a deviation. This is the second time LatticeBench has been beaten by a submission on its own ground, after the org-invites-public regex.
Behaviour changes to review before merging
I can tell you that none of this is a fabrication - the code is real, it parses and the reasoning behind it is coherent. They change golden/behaviour and are not benchmark fixes, so they belong in a PR review rather than in the score.
The review rule here is simply - every permission_classes change and every Kong plugin addition gets read by a person before merge. That is cheap to enforce and it catches all three of the above.
Price and cost per accepted result
At $2 and $6 per million tokens, Grok 4.7 is priced identically to Grok 4.6 and undercuts everything else in the comparison set except Gemini 3.8 Flash. Cost per token is the very easily the main issue.
DeepSWE v1.1 at 71.0 is the lowest in the set - Astra 74.1, Gemini 3.8 Flash 73.8, Opus 5 73.7. Three points below the field at a fraction of Astra's price is a defensible trade for most coding work and a bad one for the narrow band of tasks where those three points are the difference between a merge and a rework.
Cost per completed job is where the number gets worse. The xhigh setting burns roughly 81,000 output tokens on a single Artificial Analysis task. CursorBench runs about $6 per task and Vals about $11.90 per test. Cheap per token stops being cheap when a long agentic job spends tokens like that, and the AI Matic unit-economics axis holds at 7.5 for exactly this reason rather than scoring higher on list price alone.
Latency
Time to first token is 0.85 seconds, the fastest of any model we have scored (so far) and roughly fifteen times faster off the mark than Gemini 3.8 Flash. Throughput is 42 tokens per second on xhigh and 54 on high, which is modest.
However, wall clock is the problem. An Artificial Analysis task takes about 16 minutes and a Vals test about 33. The model starts instantly and seems to finish slowly. Interactive chat surfaces get the benefit; unattended multi-hour agent runs need an explicit timeout policy, because the failure mode is a job that is still going rather than a job that errored.
Deployment and data handling
Set against GPT-6 Astra, which ships with Zero Data Retention (ZDR) on Bedrock at launch, this is the clearest separation in the current field. Astra costs 8.3 times more per output token and answers the two questions procurement always asks. Grok answers neither and costs a fraction. Which one is correct depends on the client, not on the benchmark.
The AI Matic Bench Score
The vendor tables answer a research question more than anything else: is the new model better. GoML delivery teams answer a slightly different one: does putting this model into a running enterprise pipeline improve accepted outcomes per cost unit and per hour, without forcing a requalification cycle. The AI Matic Bench Score is our rubric for the second question, scored 0 to 10 per axis from published evidence, independent measurement, or our own LatticeBench runs.
Here’s the formula for the AI Matic Bench Score
AI Matic Bench Score = ( sum of ( axis score x axis weight ) ) x 10, where weights sum to 1.00 and each axis score is on a 0 to 10 band.
Scoring and derivation
Sensitivity
The total is not precise to a tenth. Moving agentic tool reliability up half a point takes it to 76.0. Moving unit economics up half a point takes it to 76.3. Both together reach 77.0. On the nine-axis v0.2 rubric, with claim reliability at 9.0 from zero fabrications across 54 findings, the score is 77.2. Treat 75.3 as accurate to plus or minus one point and use it as the headline.
What is deliberately excluded
- The roughly 23 organic findings are not folded into 31 of 57. They are a PR review queue and scoring them as benchmark points would inflate coverage.
- The self-report's severity table is not scored. Manifest severity is ground truth and the model's own labels are advisory.
- The SLAControlView permission swap is counted as a behaviour change, not a fix. If a client pipeline uses service tokens, that patch fails in the wrong direction.
What the score does not cover
- No GoML-run cost measurement. Every cost-per-task figure comes from Artificial Analysis or Vals rather than from our own workloads.
- Runs 1 and 2 were prose-scored while runs 3, 4 and 6 were diff-scored, so the five-run comparison carries a method difference wherever LatticeBench feeds an axis.
- No computer-use benchmarks published for this model, so that part of agentic reliability is unmeasured rather than measured low.
- No multilingual or Indic-language testing, which matters for a material share of our delivery work.
- Band anchors are descriptive, not operational, and no inter-rater check has been run.
Cross-model position
Grok 4.7 and Gemini 3.8 Flash both score 75.3 and share almost nothing. Gemini wins on deployment envelope, regulated-domain depth and evidence quality, all disclosure and distribution advantages. Grok wins on agentic tool reliability and long-horizon completion, both earned on our own bench. Astra leads the field at 79.5 and is the only model in the set scoring below 7.0 on unit economics.
Two identical totals, two different decisions
A tie on the AI Matic bench score means the models are equally strong on average across our eight evaluation axes. That does not mean they are interchangeable for every engagement. Reweight the criteria for a regulated client, and Gemini pulls ahead. Reweight them for an internal agentic build with no ZDR requirement, and Grok does.
The total is a starting point for that reweighting, rather than substituting.
Follow-up on this run
Four one-line changes close the three near-misses and take the run from 31 to 35 of 57: the guard at WM-005 line 388, CANCELLED in the BUGS-003 terminal set, and the get_active_org_member_role swap in SLA-004 and ENGAGEMENT-001. Pointing the model explicitly at ai_copilot/apps/ and the sales service is worth more, since those two cold spots hold eight issues including two criticals, and that is the change that would move the long-horizon axis rather than just the count. Resolve the SLAControlView permission change before treating the tree as a clean patch, and correct golden/'s SLA index while the finding is fresh.
Verification method
Basically, scoring is diff-based. Every verdict comes from comparing the model's patched file against bench/, which is the injected state, and golden/, which is the oracle, per manifest entry. The self-report by itself was used only to cross-check for fabricated claims, and all 54 findings cite file paths with real corresponding changes in the tree. Where a fix used a different mechanism from the manifest's, it was credited when it converges on golden behaviour and flagged when it does not.
The AI Matic axis scores are GoML editorial judgment applied to that evidence... and are clearly labelled as such. The derivation column states what supports each score. The sensitivity figures above were recomputed independently and the published totals reproduce exactly. Anyone reweighting the axes for their own workload will get a different total, which is the intended use.
TLDR of Grok 4.7 LLM testing
Use Grok 4.7 when these conditions are met
- The work is internal or the client contract already permits it. Capability at this price is the best value in the current field, and the data-handling default is the only thing standing in the way.
- You need one model to both work a backlog and find what is not on it. No other model we have scored does both in a single pass.
- You are already on Grok 4.6. Same price, same API, 500K context, nothing to requalify on the interface.
- The surface is interactive. A 0.85 second time to first token is the fastest we have measured and it is felt directly by a user waiting on a response.
Do not use Grok 4.7 when these conditions are met
- The client mandates Zero Data Retention or a specific cloud. There is no ZDR story and no Azure or Bedrock route, and no amount of benchmark performance substitutes for either.
- Client data would flow through default terms. Training on conversations is the default unless enterprise terms say otherwise, and that is a contract question to settle before a pilot rather than after.
- The job is unattended and multi-hour. Wall clock of 16 to 33 minutes per task with 81k output tokens on xhigh needs a timeout and a token budget before it goes near production.
- Cross-tenant authorization review is the deliverable. SALES-001 has now survived five runs by five models, and SLA-002 was missed here too.
The Final GoML Position
Grok 4.7 is a raw capability bargain that absolutely demands strict operational discipline. In pure engineering execution, it delivers what few frontier models can: simultaneous backlog burndown and unprompted vulnerability discovery, anchored by class-leading time-to-first-token.
However, benchmark performance cannot outrun the on-field reality aspect. Until xAI establishes a credible Zero Data Retention (ZDR) posture and multi-cloud hyperscaler availability, Grok 4.7 belongs squarely in internal agentic stacks, high-throughput developer tooling and contractually permissive environments. Be sure to use it where execution speed and autonomy dictate the bottom line. And pause the moment client data governance and compliance take first priority.
Stay tuned to the GoML blog for more unique LLM testing with the most practical standard for enterprises - the AI Matic bench score.
