Even though the syntax looks very much like TL-B, it cannot be used in most of the TL-B tooling. Unlike in real TL-B, these schemas serialize to a bitstring with no 1023 bit length limit, and without any refs.
General scheme
Internal references, absent cells, and complete BoCs
For an arbitrary cellc in a given BoC, references to it can be either:
- internal if the cell corresponding to the reference is also represented in BoC,
- external if it’s not in BoC. Such cell
cis called absent from this BoC.
Outline of serialization process
This paragraphs provide a textual description of the BoC serialization process. The specific implementation of the serialization and TL-B schemes is left to the choice of developers.For a specific example of TL-B schema and pseudocode of related cell serialization, see TL-B schema.
B consisting of n cells can be outlined as follows.
- List the cells from B in a topological order:
c1, ..., cn(withc1, ..., ckas root cells, ifBis a forest). - Choose the smallest number of bytes
sthat can contain the binary representation ofn. Serialize each cellciin a way similar to standard representation algorithm, with exceptions:d1 = r + 8s + 16h + 32lwhereh = 1if the cell’s hashes are explicitly included into the serialization; otherwise,h = 0(whenr = 7,hmust be1);- if
h = 1, after bytesb1andb2the serialization is continued byl + 132-byte higher hashes ofc; - unsigned big-endian s-bit integer
jused instead of hashHash(cj)to represent internal references to cellcj.
- Concatenate the representations of cells
cithus obtained in the increasing order ofi. - Optionally, an index can be constructed that consists of
nt-bytes integer entries where:- is the total length in bytes of the representations of cells
cjwithj ≤ i; tis the smallest number of bytes that can contain the binary representation of .
- is the total length in bytes of the representations of cells
- An optional CRC32C may be appended to the serialization for integrity verification purposes.
ci the serialized bag of cells may be easily accessed by its index i without deserializing all other cells, or even without loading the entire serialized bag of cells in memory.
A final serialization of the bag of cells must include a magic number indicating the precise format of the serialization, followed by integers s, t, n, an optional index consisting of n * t bytes, bytes with the cell representations, and an optional CRC32C checksum. Each specific implementation of the serialization process must comply with these fields and their order but, for example, may take into account number of roots, number of absent cells, and so one.
Serialization
Beware this is not an actual TL-B schema. TL-B describes serialization to cells, i.e. bits and refs, with the limit of 1023 bits per cell. This serialization describes serialization into a bitstring of arbitrary length without any refs, even though it uses syntax similar to TL-B.
PseudoTL-B
size is s, off_bytes is t, cells is n, tot_cells_size is (the total size of the serialization of all cells in bytes), index is the optional index , cell_data is the concatenation of the cells representations, and crc32c is the optional 4-bytes CRC32C checksum.
This schema additionally includes:
- the 1-bit
has_idxflag that indicates whether the index is included in the serialization; - the 1-bit
has_crc32cflag that indicates whether the CRC32C checksum is included in the serialization; - the 1-bit
has_cache_bitsand 2-bitflagsfields that are reserved for future use (flagsmust be zero); - the
rootsfield that indicates the number of root cells in the BoC; - the
absentfield that indicates the number of absent cells in the BoC; - the
root_listfield that is an indices sequence of the root cells in the BoC.
Example: manual
Consider the following example of a tree of cells:- The first is a 24-bit cell.
- The second is a 8-bit cell that itself references a 24-bit cell.
b1 and b2 for each of the three unique cells.
So, we obtain:
b is not a multiple of eight, a binary 1 and up to six binary 0s are appended to the data bits. After that, the data is split into 8-bit groups.
0x020160000202010102fe00010200060aaaaa0000.
Now that we’ve serialized our cells into a flat 20-byte array, it’s time to pack them into a complete BoC format.
Example: TypeScript
According to the TL-B scheme above there is the SDK for serialization and parsing BoC. Only serialization of BoCs with one root and no absent cells is supported. There are two main functions:serializeBocfor serialization. It has two parameters:rootand options object with two boolean flags:idxandcrc32. They indicate whether indexes and CRC32C will be included in serialization. The output is a Buffer with serialization.deserializeBocfor parsing. It has one parameter:src, a Buffer that contains a serialized BoC. The output is a roots list of a given BoC.
Cell: