docs/guide/cpp/field-configuration.md

title: Field Configuration sidebar_position: 5 id: cpp_field_configuration license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

This page explains how to configure field-level metadata for serialization.

Overview

Apache Fory™ provides two ways to specify field-level metadata at compile time:

fory::field<> template - Inline metadata in struct definition
FORY_FIELD_TAGS macro - Non-invasive metadata added separately

These enable:

Tag IDs: Assign compact numeric IDs for schema evolution
Nullability: Mark pointer fields as nullable
Reference Tracking: Enable reference tracking for shared pointers

The fory::field Template

template <typename T, int16_t Id, typename... Options>
class field;

Template Parameters

Parameter	Description
`T`	The underlying field type
`Id`	Field tag ID (int16_t) for compact serialization
`Options`	Optional tags: `fory::nullable`, `fory::ref`

Basic Usage

#include "fory/serialization/fory.h"

using namespace fory::serialization;

struct Person {
  fory::field<std::string, 0> name;
  fory::field<int32_t, 1> age;
  fory::field<std::optional<std::string>, 2> nickname;
};
FORY_STRUCT(Person, name, age, nickname);

The fory::field<> wrapper is transparent - you can use it like the underlying type:

Person person;
person.name = "Alice";           // Direct assignment
person.age = 30;
std::string n = person.name;     // Implicit conversion
int a = person.age.get();        // Explicit get()

Tag Types

fory::nullable

Marks a smart pointer field as nullable (can be nullptr):

struct Node {
  fory::field<std::string, 0> name;
  fory::field<std::shared_ptr<Node>, 1, fory::nullable> next;  // Can be nullptr
};
FORY_STRUCT(Node, name, next);

Valid for: std::shared_ptr<T>, std::unique_ptr<T>

Note: For nullable primitives or strings, use std::optional<T> instead:

// Correct: use std::optional for nullable primitives
fory::field<std::optional<int32_t>, 0> optional_value;

// Wrong: nullable is not allowed for primitives
// fory::field<int32_t, 0, fory::nullable> value;  // Compile error!

fory::not_null

Explicitly marks a pointer field as non-nullable. This is the default for smart pointers, but can be used for documentation:

fory::field<std::shared_ptr<Data>, 0, fory::not_null> data;  // Must not be nullptr

Valid for: std::shared_ptr<T>, std::unique_ptr<T>

fory::ref

Enables reference tracking for shared pointer fields. When multiple fields reference the same object, it will be serialized once and shared:

struct Graph {
  fory::field<std::string, 0> name;
  fory::field<std::shared_ptr<Graph>, 1, fory::ref> left;    // Ref tracked
  fory::field<std::shared_ptr<Graph>, 2, fory::ref> right;   // Ref tracked
};
FORY_STRUCT(Graph, name, left, right);

Valid for: std::shared_ptr<T> only (requires shared ownership)

Combining Tags

Multiple tags can be combined for shared pointers:

// Nullable + ref tracking
fory::field<std::shared_ptr<Node>, 0, fory::nullable, fory::ref> link;

Type Rules

Type	Allowed Options	Nullability
Primitives, strings	None	Use `std::optional<T>` if nullable
`std::optional<T>`	None	Inherently nullable
`std::shared_ptr<T>`	`nullable`, `ref`	Non-null by default
`std::unique_ptr<T>`	`nullable`	Non-null by default

Complete Example

#include "fory/serialization/fory.h"

using namespace fory::serialization;

// Define a struct with various field configurations
struct Document {
  // Required fields (non-nullable)
  fory::field<std::string, 0> title;
  fory::field<int32_t, 1> version;

  // Optional primitive using std::optional
  fory::field<std::optional<std::string>, 2> description;

  // Nullable pointer
  fory::field<std::unique_ptr<std::string>, 3, fory::nullable> metadata;

  // Reference-tracked shared pointer
  fory::field<std::shared_ptr<Document>, 4, fory::ref> parent;

  // Nullable + reference-tracked
  fory::field<std::shared_ptr<Document>, 5, fory::nullable, fory::ref> related;
};
FORY_STRUCT(Document, title, version, description, metadata, parent, related);

int main() {
  auto fory = Fory::builder().xlang(true).build();
  fory.register_struct<Document>(100);

  Document doc;
  doc.title = "My Document";
  doc.version = 1;
  doc.description = "A sample document";
  doc.metadata = nullptr;  // Allowed because nullable
  doc.parent = std::make_shared<Document>();
  doc.parent->title = "Parent Doc";
  doc.related = nullptr;  // Allowed because nullable

  auto bytes = fory.serialize(doc).value();
  auto decoded = fory.deserialize<Document>(bytes).value();
}

Compile-Time Validation

Invalid configurations are caught at compile time:

// Error: nullable and not_null are mutually exclusive
fory::field<std::shared_ptr<int>, 0, fory::nullable, fory::not_null> bad1;

// Error: nullable only valid for smart pointers
fory::field<int32_t, 0, fory::nullable> bad2;

// Error: ref only valid for shared_ptr
fory::field<std::unique_ptr<int>, 0, fory::ref> bad3;

// Error: options not allowed for std::optional (inherently nullable)
fory::field<std::optional<int>, 0, fory::nullable> bad4;

Backwards Compatibility

Existing structs without fory::field<> wrappers continue to work:

// Old style - still works
struct LegacyPerson {
  std::string name;
  int32_t age;
};
FORY_STRUCT(LegacyPerson, name, age);

// New style with field metadata
struct ModernPerson {
  fory::field<std::string, 0> name;
  fory::field<int32_t, 1> age;
};
FORY_STRUCT(ModernPerson, name, age);

FORY_FIELD_TAGS Macro

The FORY_FIELD_TAGS macro provides a non-invasive way to add field metadata without modifying struct definitions. This is useful for:

Third-party types: Add metadata to types you don't own
Clean structs: Keep struct definitions as pure C++
Isolated dependencies: Confine Fory headers to serialization config files

Usage

// user_types.h - NO fory headers needed!
struct Document {
  std::string title;
  int32_t version;
  std::optional<std::string> description;
  std::shared_ptr<User> author;
  std::shared_ptr<User> reviewer;
  std::shared_ptr<Document> parent;
  std::unique_ptr<Data> data;
};

// serialization_config.cpp - fory config isolated here
#include "fory/serialization/fory.h"
#include "user_types.h"

FORY_STRUCT(Document, title, version, description, author, reviewer, parent, data)

FORY_FIELD_TAGS(Document,
  (title, 0),                      // string: non-nullable
  (version, 1),                    // int: non-nullable
  (description, 2),                // optional: inherently nullable
  (author, 3),                     // shared_ptr: non-nullable (default)
  (reviewer, 4, nullable),         // shared_ptr: nullable
  (parent, 5, ref),                // shared_ptr: non-nullable, with ref tracking
  (data, 6, nullable)              // unique_ptr: nullable
)

FORY_FIELD_TAGS Options

Field Type	Valid Combinations
Primitives, strings	`(field, id)` only
`std::optional<T>`	`(field, id)` only
`std::shared_ptr<T>`	`(field, id)`, `(field, id, nullable)`, `(field, id, ref)`, `(field, id, nullable, ref)`
`std::unique_ptr<T>`	`(field, id)`, `(field, id, nullable)`

API Comparison

Aspect	`fory::field<>` Wrapper	`FORY_FIELD_TAGS` Macro
Struct definition	Modified (wrapped types)	Unchanged (pure C++)
IDE support	Template noise	Excellent (clean types)
Third-party classes	Not supported	Supported
Header dependencies	Required everywhere	Isolated to config
Migration effort	High (change all fields)	Low (add one macro)

FORY_FIELD_CONFIG Macro

The FORY_FIELD_CONFIG macro is the most powerful and flexible way to configure field-level serialization. It provides:

Builder pattern API: Fluent, chainable configuration with F(id).option1().option2()
Encoding control: Specify how unsigned integers are encoded (varint, fixed, tagged)
Compile-time verification: Field names are verified against member pointers
Cross-language compatibility: Configure encoding to match other languages (Java, Rust, etc.)

Basic Syntax

FORY_FIELD_CONFIG(StructType,
    (field1, fory::F(0)),                           // Simple: just ID
    (field2, fory::F(1).nullable()),                // With nullable
    (field3, fory::F(2).varint()),                  // With encoding
    (field4, fory::F(3).nullable().ref()),          // Multiple options
    (field5, 4)                                     // Backward compatible: integer ID
);

The F() Builder

The fory::F(id) factory creates a FieldMeta object that supports method chaining:

fory::F(0)                    // Create with field ID 0
    .nullable()               // Mark as nullable
    .ref()                    // Enable reference tracking
    .varint()                 // Use variable-length encoding
    .fixed()                  // Use fixed-size encoding
    .tagged()                 // Use tagged encoding
    .monomorphic()            // Mark as monomorphic type
    .compress(false)          // Disable compression

Tip: To use F() without the fory:: prefix, add a using declaration:

using fory::F;

FORY_FIELD_CONFIG(MyStruct,
    (field1, F(0).varint()),      // No prefix needed
    (field2, F(1).nullable())
);

Encoding Options for Unsigned Integers

For uint32_t and uint64_t fields, you can specify the wire encoding:

Method	Type ID	Description	Use Case
`.varint()`	VAR_UINT32/64	Variable-length encoding (1-5 or 1-10 bytes)	Values typically small
`.fixed()`	UINT32/64	Fixed-size encoding (always 4 or 8 bytes)	Values uniformly distributed
`.tagged()`	TAGGED_UINT64	Tagged hybrid encoding with size hint (uint64)	Mixed small and large values (uint64)

Note: uint8_t and uint16_t always use fixed encoding (UINT8, UINT16).

Complete Example

#include "fory/serialization/fory.h"

using namespace fory::serialization;

// Define struct with unsigned integer fields
struct MetricsData {
  // Counters - often small values, use varint for space efficiency
  uint32_t requestCount;
  uint64_t bytesSent;

  // IDs - uniformly distributed, use fixed for consistent performance
  uint32_t userId;
  uint64_t sessionId;

  // Timestamps - use tagged encoding for mixed value ranges
  uint64_t createdAt;

  // Nullable fields
  std::optional<uint32_t> errorCount;
  std::optional<uint64_t> lastAccessTime;
};

FORY_STRUCT(MetricsData, requestCount, bytesSent, userId, sessionId,
            createdAt, errorCount, lastAccessTime);

// Configure field encoding
FORY_FIELD_CONFIG(MetricsData,
    // Small counters - varint saves space
    (requestCount, fory::F(0).varint()),
    (bytesSent, fory::F(1).varint()),

    // IDs - fixed for consistent performance
    (userId, fory::F(2).fixed()),
    (sessionId, fory::F(3).fixed()),

    // Timestamp - tagged encoding
    (createdAt, fory::F(4).tagged()),

    // Nullable fields
    (errorCount, fory::F(5).nullable().varint()),
    (lastAccessTime, fory::F(6).nullable().tagged())
);

int main() {
  auto fory = Fory::builder().xlang(true).build();
  fory.register_struct<MetricsData>(100);

  MetricsData data{
      .requestCount = 42,
      .bytesSent = 1024,
      .userId = 12345678,
      .sessionId = 9876543210,
      .createdAt = 1704067200000000000ULL, // 2024-01-01 in nanoseconds
      .errorCount = 3,
      .lastAccessTime = std::nullopt
  };

  auto bytes = fory.serialize(data).value();
  auto decoded = fory.deserialize<MetricsData>(bytes).value();
}

Cross-Language Compatibility

When serializing data to be read by other languages, use FORY_FIELD_CONFIG to match their encoding expectations:

Java Compatibility:

// Java uses these type IDs for unsigned integers:
// - Byte (u8): UINT8 (fixed)
// - Short (u16): UINT16 (fixed)
// - Integer (u32): VAR_UINT32 (varint) or UINT32 (fixed)
// - Long (u64): VAR_UINT64 (varint), UINT64 (fixed), or TAGGED_UINT64

struct JavaCompatible {
  uint8_t byteField;      // Maps to Java Byte
  uint16_t shortField;    // Maps to Java Short
  uint32_t intVarField;   // Maps to Java Integer with varint
  uint32_t intFixedField; // Maps to Java Integer with fixed
  uint64_t longVarField;  // Maps to Java Long with varint
  uint64_t longTagged;    // Maps to Java Long with tagged
};

FORY_STRUCT(JavaCompatible, byteField, shortField, intVarField,
            intFixedField, longVarField, longTagged);

FORY_FIELD_CONFIG(JavaCompatible,
    (byteField, fory::F(0)),                    // UINT8 (auto)
    (shortField, fory::F(1)),                   // UINT16 (auto)
    (intVarField, fory::F(2).varint()),         // VAR_UINT32
    (intFixedField, fory::F(3).fixed()),        // UINT32
    (longVarField, fory::F(4).varint()),        // VAR_UINT64
    (longTagged, fory::F(5).tagged())           // TAGGED_UINT64
);

Schema Evolution with FORY_FIELD_CONFIG

In compatible mode, fields can have different nullability between sender and receiver:

// Version 1: All fields non-nullable
struct DataV1 {
  uint32_t id;
  uint64_t timestamp;
};
FORY_STRUCT(DataV1, id, timestamp);
FORY_FIELD_CONFIG(DataV1,
    (id, fory::F(0).varint()),
    (timestamp, fory::F(1).tagged())
);

// Version 2: Added nullable fields
struct DataV2 {
  uint32_t id;
  uint64_t timestamp;
  std::optional<uint32_t> version;  // New nullable field
};
FORY_STRUCT(DataV2, id, timestamp, version);
FORY_FIELD_CONFIG(DataV2,
    (id, fory::F(0).varint()),
    (timestamp, fory::F(1).tagged()),
    (version, fory::F(2).nullable().varint())  // New field with nullable
);

FORY_FIELD_CONFIG Options Reference

Method	Description	Valid For
`.nullable()`	Mark field as nullable	Smart pointers, primitives
`.ref()`	Enable reference tracking	`std::shared_ptr` only
`.monomorphic()`	Mark pointer as always pointing to one type	Smart pointers
`.varint()`	Use variable-length encoding	`uint32_t`, `uint64_t`
`.fixed()`	Use fixed-size encoding	`uint32_t`, `uint64_t`
`.tagged()`	Use tagged hybrid encoding	`uint64_t` only
`.compress(v)`	Enable/disable field compression	All types

Comparing Field Configuration Macros

Feature	`fory::field<>`	`FORY_FIELD_TAGS`	`FORY_FIELD_CONFIG`
Struct modification	Required (wrap types)	None	None
Encoding control	No	No	Yes (varint/fixed/tagged)
Builder pattern	No	No	Yes
Compile-time verify	Yes	Limited	Yes (member pointers)
Cross-lang compat	Limited	Limited	Full
Recommended for	Simple structs	Third-party types	Complex/xlang structs

Overview

The fory::field Template

Template Parameters

Basic Usage

Tag Types

fory::nullable

fory::not_null

fory::ref

Combining Tags

Type Rules

Complete Example

Compile-Time Validation

Backwards Compatibility

FORY_FIELD_TAGS Macro

Usage

FORY_FIELD_TAGS Options

API Comparison

FORY_FIELD_CONFIG Macro

Basic Syntax

The F() Builder

Encoding Options for Unsigned Integers

Complete Example

Cross-Language Compatibility

Schema Evolution with FORY_FIELD_CONFIG

FORY_FIELD_CONFIG Options Reference

Comparing Field Configuration Macros

Related Topics