Kimi K2.7 vs Claude for Long-Context Work: We Fed Both a 180-Page PDF

2026-09-04 · 3 min read

comparisonkimiclaudelong-context

Long-context marketing loves round numbers — "1M tokens", "2M tokens" — but a big window and a useful answer at page 140 are two different claims. We ran the same 180-page document (a bundled set of vendor contracts, roughly 95K tokens) through Kimi K2.7 and Claude on AIWITH.CHAT and asked questions that required pulling facts from specific, scattered pages rather than summarizing the whole thing.

Task 1: Find every clause with a specific dollar threshold

We asked both models to list every clause across the 180 pages that mentioned a liability cap above $50,000, with page numbers. Claude found 11 of 12 such clauses and cited pages correctly. Kimi K2.7 found 9 of 12, and one of its page citations pointed to the wrong document section — the clause existed, but two pages off from where it actually was.

Edge: Claude, for anything where the citation itself needs to be trustworthy, not just the fact.

Task 2: Summarize the whole bundle in under 300 words

For a plain top-level summary — what these contracts are, who the parties are, what the overall obligations look like — both models produced accurate, readable summaries. Kimi K2.7's was arguably tighter and less padded.

Edge: Tie, slight edge to Kimi K2.7 on concision.

Task 3: Cross-reference a definition used differently in two sections

One contract defined "Confidential Information" narrowly in section 3, then a later amendment (page 140) redefined it more broadly. We asked "does the amendment change what counts as confidential, and how." Claude caught the redefinition and explained the practical difference. Kimi K2.7 answered with the section 3 definition only, missing that page 140 superseded it.

Edge: Claude, for tasks where something buried deep in the document changes the answer.

Task 4: Raw retrieval — "what does page 156 say about termination notice"

A direct lookup question. Both models retrieved the correct passage verbatim. No meaningful difference.

Edge: Tie. For simple retrieval, the window size claims don't matter — both handled it fine.

What the window size actually buys you

Kimi K2.7's context window is real and its raw retrieval is solid, which covers a lot of everyday long-document use. Where it fell short in our test was the harder case: reconciling information that contradicts or supersedes something stated earlier in the same document. That's not a context-length problem, it's a reasoning-over-long-context problem, and it's where Claude's answers stayed more reliable as the document got denser.

If your long documents are single-pass reads (contracts, reports, transcripts) with no internal contradictions to track, Kimi K2.7's speed and cost make it the practical default. If you're auditing something where an earlier clause might be quietly overridden later — legal documents, spec revisions, anything with amendments — that's worth the extra Claude pass.

Running both without two subscriptions

The workflow that actually works here is cheap-model-first: run the long document through Kimi K2.7 for the fast pass, then re-run the specific question through Claude only when the answer needs to be trustworthy enough to act on. AIWITH.CHAT puts both in the same $9.9/month plan, so switching per question doesn't mean paying for two separate subscriptions.