#570 fix(crawl): wait out rate-limited status polls and page results only once the crawl ends

firecrawl_crawl starts a crawl and then polls GET /v2/crawl/:id until it ends. Two things in that loop made the tool fail while the crawl itself was fine:

• Each poll read every result page. getCrawlStatusWithOrigin follows next to collect all documents, and the loop called it on every tick, so a crawl with several result pages issued several status reads every couple of seconds and quickly used up the crawl-status rate limit.
• A rate-limited poll failed the tool. The 429 surfaced as Request failed with status code 429, the crawl ID was lost, and the agent typically started the same crawl again.

Now the loop reads only the first status page while the crawl runs and collects result pages once, after it reaches completed, failed or cancelled. A 429 on a status read waits for Retry-After (or retry_after_seconds, default 10s, capped at 60s) and keeps polling; other errors still fail as before.

Test: tests/mcp-crawl-polling.test.mjs serves a running crawl with a second result page, a 429 on the second status read, then completion; it fails on main with the 429 and passes here (full suite passes).

---

Summary by cubic

Fixes firecrawl_crawl polling so a healthy crawl no longer fails the tool.

• Polls only the first status page while the crawl runs and collects result pages once it reaches completed, failed or cancelled, reusing the terminal poll's response as the first result page.
• A 429 on a status read now waits for Retry-After (seconds or HTTP-date, or retry_after_seconds, default 10s, capped at 60s) and keeps polling; other errors still fail as before.