Reject Duplicate JSON Keys in CometAPI Responses

Last reviewed: November 2, 2026 (Asia/Shanghai).

Direct answer

Reject duplicate JSON object names while decoding the response, before they collapse into a Python dictionary. Use object_pairs_hook to detect repeated names, parse_constant to reject NaN and infinity literals, and a finite-number check for numeric overflow. Then run endpoint and application validation on the resulting object. Repeat that boundary when structured assistant content or tool arguments contain another JSON document.

A default decoder can hide the defect. Python’s JSON reference documents that repeated names keep their final value and that non-finite number literals are accepted by default. Setting strict=True does not change those choices; its documented purpose concerns unescaped control characters inside strings.

The protocol distinction matters. RFC 8259 , sections 4 and 6, recommends unique object names and excludes NaN and infinity literals from its number grammar. Duplicate-name rejection is this guide’s interoperability policy. The standard’s recommendation does not mean that every conforming parser must reject duplicate-bearing objects.

Who this is for

This guide is for Python gateway maintainers accepting non-streaming CometAPI Chat Completions responses and applications consuming JSON-formatted assistant results. Use it when a successful parse currently doubles as the application’s acceptance check, or when a validator receives only the dictionary produced by an SDK.

The examples use synthetic bytes and Python’s standard library. They demonstrate a local decoder policy, not an observed provider defect. No authenticated API request is needed to reproduce the behavior.

Key takeaways

Detect duplicates during decoding; an ordinary dictionary has already lost the earlier occurrence.
Reject repeated names at every object depth, including objects nested inside arrays.
Keep JSON decoding, endpoint validation, and application rules as separate gates.
Validate each embedded JSON document separately when the feature expects structured content or tool arguments.
Classify a rejected response before considering another generation attempt.

Contract details to verify

Identify the two JSON boundaries

The CometAPI Chat Completions reference documents the non-streaming completion envelope, the generated message, structured-output options, and JSON-encoded function arguments. Its JSON Object mode requests JSON without enforcing the application’s schema, and its guide says to check refusal and length-limited outcomes before consuming a structured result.

There are two decoding points to inventory:

Boundary	Input	Required next check
HTTP response envelope	Bounded response bytes	Endpoint object, completion status, choices, and the permitted message branch
Structured application value	Content or arguments selected after envelope validation	Feature schema, value ranges, authorization, and permitted effects

Apply the decoder below to the outer bytes before a convenience method converts them into a dictionary. For an SDK that hides those bytes, inspect its documented raw-response or transport extension point and place the check there. Re-encoding an already-decoded dictionary cannot recover discarded duplicates.

For the second boundary, first establish that the selected content belongs to a JSON-producing feature. Ordinary assistant prose remains prose. A string containing the word “NaN” is not a non-finite numeric token. For a structured feature, decode the selected string under a separate content-size allowance and then apply its schema. The site’s JSON fallback validation guide covers the surrounding schema and promotion decision.

Reject ambiguity before dictionary construction

Consider the synthetic document {"answer":"first","answer":"second"}. A validator receiving Python’s default result sees only {"answer":"second"}. Even an exact schema for the answer property cannot report the occurrence the decoder discarded. Checking required fields after parsing therefore cannot substitute for checking object-name uniqueness during parsing.

Names must be compared after JSON escape decoding. The names answer and \u0061nswer identify the same decoded string when that escape occurs in the JSON document. The hook receives decoded names and can detect that collision. A text search for repeated quoted substrings cannot handle this boundary reliably.

Repeated names in separate objects are different: {"left":{"x":1},"right":{"x":2}} has no duplicate within either object. The policy rejects repetition within one object, rather than banning the same field name throughout a document.

Use an explicit decoder policy

Save this as duplicate_json_boundary.py. Call it with a positive byte limit selected for the route; the sample does not prescribe a universal production allowance.

import json
import math


class ResponseContractError(ValueError):
    pass


def unique_object(pairs):
    result = {}
    for name, value in pairs:
        if name in result:
            raise ResponseContractError("duplicate_json_name")
        result[name] = value
    return result


def reject_constant(_token):
    raise ResponseContractError("non_finite_json_constant")


def finite_float(token):
    value = float(token)
    if not math.isfinite(value):
        raise ResponseContractError("json_number_out_of_range")
    return value


def decode_json_object(raw: bytes, *, max_bytes: int) -> dict:
    if type(max_bytes) is not int or max_bytes <= 0:
        raise ValueError("invalid_json_byte_limit")
    if type(raw) is not bytes:
        raise ResponseContractError("invalid_body_type")
    if len(raw) > max_bytes:
        raise ResponseContractError("json_body_too_large")
    try:
        text = raw.decode("utf-8", errors="strict")
        value = json.loads(
            text,
            object_pairs_hook=unique_object,
            parse_constant=reject_constant,
            parse_float=finite_float,
            strict=True,
        )
    except ResponseContractError:
        raise
    except (UnicodeError, ValueError, RecursionError):
        raise ResponseContractError("invalid_json_document") from None
    if not isinstance(value, dict):
        raise ResponseContractError("json_root_not_object")
    return value

The Python reference defines the hooks and supplies an ordered list of pairs for each decoded object. The custom hook examines that list before producing the replacement dictionary. parse_constant handles the three non-standard literals; parse_float handles ordinary JSON fractional or exponent tokens. The latter check catches a token such as 1e400 when conversion to the selected binary float representation produces infinity. RFC 8259 permits implementation limits on numeric range, so this is a deliberate representation policy rather than a claim that exponent notation is invalid.

The helper requires a top-level object because these two application boundaries expect objects. JSON itself permits other top-level values. UTF-8 decoding is explicit, consistent with RFC 8259’s network interchange requirement. Syntax errors, malformed encoding, and excessive interpreter recursion become a bounded rejection code.

The size check runs after the caller has obtained raw: enforce received and decompressed body limits while reading as well. A byte cap does not provide an exact nesting-depth or CPU limit. The Python reference warns that hostile JSON can consume substantial resources; use a parser with appropriate limits or a separately bounded decoding worker where those controls are required. Decide how the application handles lone surrogate escapes and string normalization, too.

For numbers needing exact decimal arithmetic, choose a compatible decimal representation and downstream validator. This float policy rejects non-finite results but does not promise exact decimal precision or a shared integer range across languages. Very small values can also underflow to zero. Add field-specific limits after decoding.

Happy-path operator workflow

Locate the first conversion from response bytes to application objects in one adapter. Keep the HTTP status and endpoint-family checks, then route the bounded body through the decoder before accepting it as a completion envelope. Require the selected message branch and terminal outcome to satisfy the feature’s contract.

If that branch contains a structured result, convert the selected string to UTF-8 under its own limit, decode it with the same uniqueness policy, and apply the application schema. Treat encoding failure as rejection. A successful outer decode says nothing about duplicate names hidden inside a string: strings are not recursively interpreted as JSON by the outer decoder.

Complete business validation before storing an accepted result or executing a tool. Record route revision, decoder-policy revision, boundary name, response status class, accepted or rejected outcome, and a bounded reason code. Keep raw bodies, prompts, credentials, generated text, and attacker-controlled field names out of routine logs.

Error-path operator workflow

On duplicate-name, non-finite-number, encoding, or shape rejection, stop consumption of that result. Salvaging a preferred occurrence, stripping a duplicate with a regular expression, replacing a non-finite value with zero, or asking a repair model to reinterpret the bytes would change the result without establishing which interpretation was intended. Keep any separately designed repair workflow outside the normal acceptance path.

Keep the failure associated with the original operation. An HTTP response rejected by a local decoder may still follow completed remote work. A parse failure does not establish that generation was canceled, uncharged, or safe to repeat. Use the site’s request replay checks before any separately authorized replacement attempt.

When the failure rate changes after a release, compare the exact adapter and decoder revisions using synthetic reproductions. Determine whether a wrapper decoded first, a middleware layer rewrote the body, or the selected feature returned a different content branch. Preserve evidence through the approved restricted incident process when raw material is needed; a routine log line should carry a reference and reason code rather than the payload itself.

Verify the boundary offline

Save this as duplicate_json_probe.py beside the helper and run python3 duplicate_json_probe.py. The byte allowance is an illustrative fixture limit.

import json
import math
from duplicate_json_boundary import decode_json_object, ResponseContractError

assert json.loads('{"answer":"first","answer":"second"}') == {
    "answer": "second"
}
assert math.isnan(json.loads('{"score":NaN}', strict=True)["score"])

accepted = [
    (b'{"answer":"ok","score":0.5}', {"answer": "ok", "score": 0.5}),
    (b'{"left":{"x":1},"right":{"x":2}}',
     {"left": {"x": 1}, "right": {"x": 2}}),
    (b'{"items":[{"id":1},{"id":2}]}',
     {"items": [{"id": 1}, {"id": 2}]}),
    (b'{"text":"NaN is a word"}', {"text": "NaN is a word"}),
]
rejected = [
    (b'{"answer":"first","answer":"second"}', "duplicate_json_name"),
    (b'{"item":{"id":1,"id":2}}', "duplicate_json_name"),
    (b'{"items":[{"id":1,"id":2}]}', "duplicate_json_name"),
    (br'{"answer":1,"\u0061nswer":2}', "duplicate_json_name"),
    (b'{"score":NaN}', "non_finite_json_constant"),
    (b'{"score":Infinity}', "non_finite_json_constant"),
    (b'{"score":-Infinity}', "non_finite_json_constant"),
    (b'{"score":1e400}', "json_number_out_of_range"),
    (b'[]', "json_root_not_object"),
    (b'null', "json_root_not_object"),
    (b'{"x":1} {"x":2}', "invalid_json_document"),
    (b'{"text":"\xff"}', "invalid_json_document"),
    (b'{"text":"unescaped\nline"}', "invalid_json_document"),
    (b'{"x":1,}', "invalid_json_document"),
]
for raw, expected in accepted:
    assert decode_json_object(raw, max_bytes=4096) == expected
for raw, expected in rejected:
    try:
        decode_json_object(raw, max_bytes=4096)
    except ResponseContractError as exc:
        assert str(exc) == expected
    else:
        raise AssertionError("rejected fixture was accepted")
try:
    decode_json_object(b'{"answer":"ok"}', max_bytes=4)
except ResponseContractError as exc:
    assert str(exc) == "json_body_too_large"
else:
    raise AssertionError("size limit was ignored")
print("decoder fixtures passed")

A local run on November 2, 2026 with Python 3.12.3 accepted all four valid fixtures and rejected all fourteen invalid fixtures plus the byte-limit case. It also reproduced default last-value selection and default acceptance of NaN despite strict=True. These are standard-library observations with synthetic bytes, not measurements of the live API, SDK transport, or deployed middleware.

Before rollout, run the fixtures through the actual adapter. Add outer-envelope duplicates, duplicate names inside the selected structured-content string, rejected finish outcomes, and missing schema fields. Assert that every rejected case produces zero accepted-result writes and zero tool executions. Test stream event framing separately if the route uses SSE; this helper expects a complete JSON document and cannot parse an entire event stream as one envelope.

Failure modes

The schema validator runs after a permissive parse. The earlier occurrence has vanished. Move duplicate detection to the first byte-to-object conversion.

Only top-level names are checked. A nested message or tool-argument object can still collapse. Apply the pair hook to every object and decode embedded documents separately.

strict=True is treated as a universal policy. Duplicate names and non-finite constants remain accepted by the default decoder. Configure and test the specific hooks.

Only named infinity literals are rejected. A large exponent can overflow a binary float. Check the converted result or choose another reviewed numeric representation.

A decoder rejection triggers blind fallback. A second generation changes cost and may repeat work. Keep the original operation’s deadline, attempt allowance, and replay decision.

FAQ

Are duplicate object names forbidden by JSON grammar?

RFC 8259 recommends unique names and describes inconsistent receiver behavior when they repeat. This guide makes rejection an application interoperability requirement. That recommendation is not a universal syntax prohibition.

Can an object hook detect duplicates after a dictionary is built?

A normal dictionary cannot retain two values under the same key. Use the ordered-pair hook during decoding, before dictionary construction discards the earlier occurrence.

Does JSON Schema mode remove the local checks?

Retain the client acceptance boundary. Request-side structured-output settings, decoder behavior, completion outcomes, and application permissions answer different questions. Verify the selected model and endpoint contract separately.

Should the helper parse every message string as JSON?

Apply the second decode only where the feature explicitly expects a structured document. Ordinary text and JSON string values should not be reinterpreted recursively.

Reader next step

Trace one non-streaming route to its first JSON decode, install the uniqueness and numeric policy there, and run the synthetic fixtures through the adapter. Repeat the check for its structured-content or tool-argument boundary. Accept the change only when rejected documents cannot reach storage, a consumer, or a tool executor.

When that boundary is verified, Start with CometAPI and use a small authorized staging request to check the selected endpoint and feature separately.

Sources checked

RFC Editor / IETF: RFC 8259 — fetched November 2, 2026. December 2017 Internet Standard; sections 4, 6, 8, and 9 support name uniqueness guidance, numeric grammar and range limits, UTF-8 interchange, and permitted parser limits.
Python Software Foundation: Python 3.12 JSON reference — fetched November 2, 2026. Supports duplicate-name and non-finite defaults, decoder hooks, control-character strictness, encoding behavior, and resource cautions.
CometAPI: Create a chat completion — fetched November 2, 2026. Supports the completion envelope, structured-output distinctions, refusal and length checks, and JSON-encoded function arguments.

These current primary references support an evergreen decoder task. The rejection policy, integration sequence, incident decisions, and rollout criteria are engineering recommendations. The reported local results are limited to the Python version and synthetic cases stated above.