Skip to content
PLXI
On this page· 3

Write a pack

  • file format 6
  • Rust SDK 1.0 preview

Goal: turn JSON lines into a pack, in memory or on disk.

Steps

  1. 1. Parse each JSON line with Record::from_json_line. A line that is not valid JSON fails with json; one that is not an object with a string k fails with invalid_record.
  2. 2. Pass the records to write, which returns the pack as bytes, or to write_to_path, which writes a file. None means no appendix.
  3. 3. Open the result to read it back, as the quickstart does.

Needs the Rust SDK

write_and_read.rsrust
// Write a pack in memory, open it, and read the records back.
use plxi_sdk::{write, OpenOptions, Pack, Record};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // A record is one JSON object with a string member "k" (its kind).
    let lines = [
        r#"{"k":"rel","src":"e1","tgt":"e2","kind":"knows"}"#,
        r#"{"k":"ent","id":"e2","t":"Person"}"#,
        r#"{"k":"ent","id":"e1","t":"Person","name":"Ada","salience":0.5}"#,
    ];
    let records = lines
        .iter()
        .map(|l| Record::from_json_line(l))
        .collect::<Result<Vec<_>, _>>()?;

    // `write` returns the whole pack as bytes. No file is touched.
    let bytes = write(&records, None)?;

    // Opening verifies the pack by default (digest, header, record count).
    let pack = Pack::from_bytes(bytes, OpenOptions::default())?;
    assert_eq!(pack.record_count(), 3);
    assert!(!pack.has_appendix());

    // Records come back in the format's sort order (entities before
    // relations), spelled the way the writer emits them.
    let out: Vec<String> = pack
        .records()
        .map(|r| r.map(|rec| rec.json_line().to_string()))
        .collect::<Result<_, _>>()?;
    assert_eq!(
        out,
        [
            concat!(
                r#"{"k":"ent","v":1,"id":"e1","t":"Person","#,
                r#""name":"Ada","salience":0.5,"stub":false}"#
            ),
            r#"{"k":"ent","v":1,"id":"e2","t":"Person","stub":false}"#,
            r#"{"k":"rel","v":1,"src":"e1","tgt":"e2","kind":"knows"}"#,
        ]
    );
    for line in &out {
        println!("{line}");
    }
    println!("sha256 = {}", pack.sha256_hex());

    // Keep a copy on disk, for checking with standard tools.
    std::fs::write("sample.plxi", pack.to_bytes())?;
    println!("wrote sample.plxi, {} bytes", pack.to_bytes().len());
    Ok(())
}
Output
{"k":"ent","v":1,"id":"e1","t":"Person","name":"Ada","salience":0.5,"stub":false}
{"k":"ent","v":1,"id":"e2","t":"Person","stub":false}
{"k":"rel","v":1,"src":"e1","tgt":"e2","kind":"knows"}
sha256 = 068f161e337da4b2b0c319a3539b9c62f4b15715d0f3c4de8e1664e5ef66bb28
wrote sample.plxi, 544 bytes

Ran with cargo run --example 01_write_and_read · exit status 0

Things to know

  • The order given to write does not matter. Records are written in sort-key order; records with equal sort keys keep the order they were given in.
  • The writer spells each record its own way: k first, then v, then the kind's members, plus the members it always emits, such as "stub":false on an entity.
  • The writer always emits a final header line.
  • write reports a failure while writing to memory as internal. write_to_path can also fail with io.
  • write builds the whole pack in memory. To ship a .plxi.gz, compress the bytes, as example 8 does with the flate2 crate. Example 8
  • To write binary data, pass an AppendixSpec. Example 4

Reference

Sections