PR #356 - Unstructured-IO/unstructured-python-client

What

Jobs the new preflight check rejects no longer break the SDK. Workflows can also opt out of preflight from the SDK.

Before, after a job is rejected:

client.jobs.list_jobs(...)   → ValidationError: 'REJECTED' is not a valid JobStatus   (whole page lost)
client.jobs.get_job(...)     → ValidationError

After:

job.status   → JobStatus.REJECTED
job.status   → "SOME_FUTURE_STATUS"   # a status this SDK version doesn't know comes back as a plain str instead of raising

details.node_stats[0].failure_category → PreflightFailureCategory.AUTH_INVALID_CREDENTIAL
details.node_stats[0].failure_reason   → "The AWS Access Key Id you provided does not exist."

JobStatus and JobProcessingStatus gain REJECTED. JobInformation.status and JobDetails.processing_status also accept unknown values. Known values are still enum members, so job.status == JobStatus.COMPLETED and .value behave as before.
JobNodeDetails gains failure_category (Union[PreflightFailureCategory, str]) and failure_reason, so a caller can see why a job was rejected. The server sets them only on the node that failed preflight; before this, pydantic dropped them.
skip_preflight is now on CreateWorkflow, UpdateWorkflow (unset = unchanged) and WorkflowInformation. For a one-off create_job, the API defines the flag inside the request_data JSON. The docs now say to set it there; the SDK does not rewrite that string.
Releases as 0.47.0.

Why

The SDK models used to be regenerated from the API spec every night. That generator has been dead since January (removed in #349). Since then, new server values reach the SDK only when someone adds them by hand. REJECTED didn't get added, and a closed enum turns a new value into a hard failure.

This PR also adds a way to catch that kind of drift before a customer does (DTPL-696 acceptance criterion 4):

weekly CI → fetch live OpenAPI specs → compare with SDK models → new missing field/enum value → open/update a GitHub issue
                                                                    (a failed download fails the run but opens no issue)

make check-spec-drift runs the check locally. Gaps we already know about are recorded in scripts/spec_drift_baseline.json (20 schemas today, e.g. JobInformation.processing_status, the teradata connector type). The check fails only on new drift, or on a baseline entry that has since been covered.
CONTRIBUTING.md documents how to resolve a drift issue.

Replaces #353, which got here through a custom _missing_ enum hook and code that spliced bytes into request_data. This version uses a plain pydantic union and leaves request_data alone.

Notes for consumers

Type checkers now see these two status fields as Union[JobStatus, str] / Union[JobProcessingStatus, str]. Code that reads .value or .name on them needs to handle the str case. Runtime behavior for known statuses is unchanged.
Follow-up, not in this PR: the SDK's other "open" enums (connector types, partition Strategy, etc.) use OpenEnumMeta / validate_open_enum. On Python 3.12+ those return a raw str even for known values, and they emit pydantic serializer warnings. The long-term fix is to move them to the same union pattern used here. That changes their runtime types, so it belongs in its own PR.

Validation

make test-unit: 419 passed. make test-contract: 72 passed. The new job/workflow contract tests pass on Python 3.11, 3.12 and 3.13, and they fail without the model changes.
make lint is clean. make check-spec-drift passes against the live specs.
Two rounds of Codex review. Round 1 found that enums declared inline on a property escaped the drift check; that is fixed and has a test. Round 2 was clean.
cubic: three findings fixed (no drift issue on a fetch failure, no duplicate issues from overlapping runs, changelog wording). Re-review clean, and full CI is green, including integration on 3.11–3.13.

Maintenance validation (2026-10-19, keepalive base refresh)

Synchronized main through its 0.46.5 keepalive release while retaining the authored 0.47.0 version and user-agent. Manual conflict resolution touched only the changelog and version file; main runtime and test changes merged automatically.
Local validation: 419 unit tests and 71 contract tests passed (one existing expected unit failure). Four localhost socket tests needed bind permission and passed without source changes.
Exact maintenance-head CI: https://github.com/Unstructured-IO/unstructured-python-client/actions/runs/39329146842.

Review follow-up (2026-10-20)

Added failure_category / failure_reason to JobNodeDetails with a new PreflightFailureCategory enum, and dropped both from the drift baseline. A new contract test covers a known category, an unknown category (plain str) and a passing node (both null); it passes on 3.11, 3.12 and 3.13.
Removed the JobStatus / JobProcessingStatus entries from overlay_client.yaml. Nothing reads the overlay since the generator was removed, and x-speakeasy-unknown-values would describe the OpenEnumMeta style, not the union shipped here.
make test-unit 419 passed, make test-contract 72 passed, make lint clean, make check-spec-drift clean against the live specs.