Write

file format 6
Rust SDK 1.0 preview

Writing a pack from records, in memory or to a file, with an optional appendix. Records are written in sort-key order with a final header line.

On this page, Result<T> is std::result::Result<T, plxi_sdk::Error>.

Every public struct and enum with a public field or variant is marked #[non_exhaustive]: code outside the crate builds one through its constructors and matches an enum with a _ arm.

Functions

write

Spec §5.3

pub fn write(records: &[Record], appendix: Option<&AppendixSpec>) -> Result<Vec<u8>>

Summary

Write a pack from records and an optional appendix, in memory.

Records are written in sort-key order (spec §5.3); records with equal keys keep their input order. An empty AppendixSpec writes no appendix.

Parameters

records: &[Record]
appendix: Option<&AppendixSpec>

Returns

Result<Vec<u8>>

Errors

internal

internal only: every input is a parsed Record, so a failure here is an SDK defect.

Example

Example 1: Write and read

Spec

Spec §5.3

write_to_path

Spec §2

pub fn write_to_path(
    path: impl AsRef<Path>,
    records: &[Record],
    appendix: Option<&AppendixSpec>,
) -> Result<()>

Summary: write to a file at path.
Parameters: path: impl AsRef<Path>
records: &[Record]
appendix: Option<&AppendixSpec>
Returns: Result<()>
Errors: io
io when the file cannot be written; otherwise as write.
Example: Example 8: Files, limits, errors
Spec: Spec §2

Structs

struct

AppendixSpec

Spec §7

#[non_exhaustive]
#[derive(Debug, Clone, Default, PartialEq)]
pub struct AppendixSpec { /* private fields */ }

Summary: What to embed as the pack's CSDT appendix.
Example: Example 4: Appendix and embeddings
Spec: Spec §7

method

AppendixSpec::compact

Spec §7

pub fn compact<R: AsRef<[u8]>>(rows: &[R]) -> Option<AppendixSpec>

Summary: An appendix with one Compact embedding section holding rows, each exactly COMPACT_RECORD_SIZE bytes. None when a row has another length (a caller mistake, not an error kind).
Parameters: rows: &[R]
Returns: Option<AppendixSpec>
Example: Example 4: Appendix and embeddings
Spec: Spec §7

method

AppendixSpec::empty

Spec §7

pub fn empty() -> AppendixSpec

Summary: An appendix with no sections yet (add tensors to it).
Parameters: None.
Returns: AppendixSpec
Example: Example 4: Appendix and embeddings
Spec: Spec §7

method

AppendixSpec::with_tensor_f32

Spec §7

pub fn with_tensor_f32(
    mut self,
    name: &str,
    shape: &[u64],
    data: &[f32],
) -> Option<AppendixSpec>

Summary: Add an F32 tensor. None when shape's element count is not data.len().
Parameters: name: &str
shape: &[u64]
data: &[f32]
Returns: Option<AppendixSpec>
Example: Example 4: Appendix and embeddings
Spec: Spec §7

Anyone may implement this format. Copying or republishing the text of this specification requires permission from Cintile Inc.