Identity and order

A writer emits records in ascending sort-key order. Records with equal sort keys keep the order they were given in.

A writer should not emit two records with one dedup key. A reader accepts such a file, and diff lists the key under duplicate_keys.

An opaque record whose kind is a typed kind at a higher v has the same dedup key as the version-1 record with the same primary id. Merge treats the two as one record.

// Identity: the canonical JSON form, and the keys that decide whether
// two records are "the same record".
use plxi_sdk::{canonical_json, Record};

fn main() -> Result<(), plxi_sdk::Error> {
    // canonical_json sorts object members by name, keeps array order
    // and removes insignificant whitespace.
    let value = serde_json::json!({
        "z": [3, 1.5, -0.0],
        "a": {"y": null, "b": true}
    });
    let canon = r#"{"a":{"b":true,"y":null},"z":[3,1.5,-0.0]}"#;
    assert_eq!(canonical_json(&value), canon);

    // A typed record's key is (kind, primary id).
    let line = r#"{"k":"ent","id":"e1","t":"Person"}"#;
    let ent = Record::from_json_line(line)?;
    let parts = (ent.kind(), ent.version(), ent.primary_id());
    assert_eq!(parts, ("ent", 1, "e1"));
    assert_eq!(ent.dedup_key(), ("ent".to_string(), "e1".to_string()));
    assert_eq!(ent.sort_key(), (3, "ent".to_string(), "e1".to_string()));

    // A kind this SDK has no schema for is kept as written ("opaque"),
    // so a newer writer's records survive a round trip through an
    // older reader.
    let line = r#"{"k":"x-future","v":7,"id":"n1","z":{"b":1,"a":2}}"#;
    let opaque = Record::from_json_line(line)?;
    assert_eq!(opaque.json_line(), line);
    let canon = r#"{"id":"n1","k":"x-future","v":7,"z":{"a":2,"b":1}}"#;
    assert_eq!(opaque.canonical_json(), canon);
    assert_eq!(opaque.sort_key().0, u32::MAX); // unknown kinds sort last

    // Two spellings of one object are one record: an opaque record
    // with no id member is keyed by its canonical form.
    let a = r#"{"k":"x-note","b":1,"a":{"y":2.0,"x":1}}"#;
    let b = r#"{"a":{"x":1,"y":2.00},"b":1,"k":"x-note"}"#;
    let a = Record::from_json_line(a)?;
    let b = Record::from_json_line(b)?;
    assert_ne!(a.json_line(), b.json_line());
    assert_eq!(a.dedup_key(), b.dedup_key());
    println!("{}", a.primary_id());
    Ok(())
}

{"a":{"x":1,"y":2.0},"b":1,"k":"x-note"}

The last line printed is a primary id. The record has none of the five id members, so its primary id is the identity form of the whole record, and two spellings of it share one key.

The identity form of a JSON value is the value written with object members sorted by name at every depth, array order unchanged, and no whitespace between tokens. Names sort by their UTF-8 bytes. The SDK calls it canonical JSON.

It is used for the fallback primary id above, to decide whether a record changed in a diff, and for the expected values of the conformance cases.

canonical_json returns it for any serde_json::Value; Record::canonical_json returns it for a record.

A number written as an integer that fits a signed or unsigned 64-bit integer is written back exactly. Every other number is a binary64 value, the 64-bit floating-point type of IEEE 754, written as the shortest decimal string that reads back to the same value.

When two shortest strings read back to the same value, the spelling is the one serde_json 1.0.149 writes. The binary64 value 0x42e87faaebb9a0d4 reads back from both 215492859907334.62 and 215492859907334.63; it is written 215492859907334.62.

A reader returns the binary64 value nearest the number it reads, with ties to even. Under these two rules, reading an identity form and writing it again gives the same bytes.

An implementation in any language writes numbers by these rules, not with its language's default formatter. JavaScript's JSON.stringify writes 1.0 as 1 and -0.0 as 0; Python's json.dumps writes 0.00001 as 1e-05. The conformance corpus has a numbers case that checks these spellings. Spec §10.2 ↗ · Spec §10.4 ↗

RFC 8785, the JSON Canonicalization Scheme, is a different canonical form. An implementation does not use a library for it in place of the identity form. The two differ in seven ways.

Strings, the literals true, false and null, and the absence of whitespace agree.