title: Java Serialization Format sidebar_position: 1 id: java_serialization_spec license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Spec overview

Apache Fory Java serialization is a dynamic binary format for Java object graphs. It supports shared references, circular references, polymorphism, and optional schema evolution. The format is stream friendly: shared type metadata is written inline when needed and there is no meta start offset.

The Java native format is an extension of the xlang wire format and reuses the same core framing and encodings; see docs/specification/xlang_serialization_spec.md for the shared baseline.

Overall layout:

| fory header | object ref meta | object type meta | object value data |

All data is encoded in little endian byte order. When running on a big endian platform, array serializers swap byte order on write/read so the on-wire layout remains little endian.

Fory header

Java native serialization writes a one byte bitmap header. The header layout mirrors the xlang bitmap and uses the same flag bits.

|     5 bits    | 1 bit | 1 bit | 1 bit |
+--------------+-------+-------+-------+
| reserved     |  oob  | xlang | null  |
  • null flag: 1 when object is null, 0 otherwise. If object is null, other bits are not set.
  • xlang flag: 1 when serialization uses xlang format, 0 when serialization uses Java native format.
  • oob flag: 1 when BufferCallback is not null, 0 otherwise.

The header is always a single byte; no language ID is written.

Reference meta

Reference tracking uses the same flags as the xlang specification.

FlagByte ValueDescription
NULL FLAG-3Object is null. No further bytes are written for this object.
REF FLAG-2Object was already serialized. Followed by unsigned varint32 reference ID.
NOT_NULL VALUE FLAG-1Object is non-null but reference tracking is disabled for this type. Object data follows immediately.
REF VALUE FLAG0Object is referencable and this is its first occurrence. Object data follows. Assigns next reference ID.

When reference tracking is disabled globally or for a specific field/type, only NULL FLAG and NOT_NULL VALUE FLAG are used.

Type system and type IDs

Java native serialization uses the unified type ID layout shared with xlang:

full_type_id = (user_type_id << 8) | internal_type_id
  • internal_type_id is the low 8 bits describing the kind (enum/struct/ext, named variants, or a built-in type).
  • user_type_id is the numeric registration ID (0-based) for user-defined enum/struct/ext types.
  • Named types use NAMED_* internal IDs and carry names in metadata rather than embedding a user ID.

Shared internal type IDs (0-63)

Java native mode shares the xlang internal IDs for all values below 64. IDs 0~56 are defined by the xlang spec, while 57~63 are reserved for future internal use. These IDs are stable across languages.

See the internal type ID table in Xlang Serialization Format. Java shares all IDs < 64, with 57~63 reserved for future internal use.

Java native built-in type IDs

Java native serialization assigns Java-specific built-ins starting at Types.BOUND + 5 (Types.BOUND is 64; 5 IDs are reserved for future use). Type IDs in 0~56 are shared with xlang; 57~63 are reserved; 64+ are only valid in Java native mode.

Type IDNameDescription
69VOID_IDjava.lang.Void
70CHAR_IDjava.lang.Character
71PRIMITIVE_VOID_IDvoid
72PRIMITIVE_BOOL_IDboolean
73PRIMITIVE_INT8_IDbyte
74PRIMITIVE_CHAR_IDchar
75PRIMITIVE_INT16_IDshort
76PRIMITIVE_INT32_IDint
77PRIMITIVE_FLOAT32_IDfloat
78PRIMITIVE_INT64_IDlong
79PRIMITIVE_FLOAT64_IDdouble
80PRIMITIVE_BOOLEAN_ARRAY_IDboolean[]
81PRIMITIVE_BYTE_ARRAY_IDbyte[]
82PRIMITIVE_CHAR_ARRAY_IDchar[]
83PRIMITIVE_SHORT_ARRAY_IDshort[]
84PRIMITIVE_INT_ARRAY_IDint[]
85PRIMITIVE_FLOAT_ARRAY_IDfloat[]
86PRIMITIVE_LONG_ARRAY_IDlong[]
87PRIMITIVE_DOUBLE_ARRAY_IDdouble[]
88STRING_ARRAY_IDString[]
89OBJECT_ARRAY_IDObject[]
90ARRAYLIST_IDjava.util.ArrayList
91HASHMAP_IDjava.util.HashMap
92HASHSET_IDjava.util.HashSet
93CLASS_IDjava.lang.Class
94EMPTY_OBJECT_IDempty object stub
95LAMBDA_STUB_IDlambda stub
96JDK_PROXY_STUB_IDJDK proxy stub
97REPLACE_STUB_IDwriteReplace/readResolve stub
98NONEXISTENT_META_SHARED_IDmeta-shared unknown class stub

Registration and named types

User-defined enum/struct/ext types can be registered by numeric ID or by name.

  • Numeric registration: full_type_id = (user_id << 8) | internal_type_id.
  • Name registration: type meta uses namespace and type name (see below).
  • Unregistered types are encoded as named types using namespace = package name and type name = simple class name.

Named type selection rules for unregistered types:

  • enum -> NAMED_ENUM
  • struct-like serializers -> NAMED_STRUCT (or NAMED_COMPATIBLE_STRUCT in compatible mode)
  • all other custom serializers -> NAMED_EXT

Type meta encoding

Every value is written with a type ID followed by optional type metadata:

  1. Write type_id using varuint32 small7 encoding.
  2. For NAMED_ENUM, NAMED_STRUCT, NAMED_EXT, NAMED_COMPATIBLE_STRUCT:
    • If meta share is enabled: write shared class meta (streaming format).
    • Otherwise: write namespace and type name as meta strings.
  3. For COMPATIBLE_STRUCT:
    • If meta share is enabled: write shared class meta (streaming format).
    • Otherwise: no extra meta (type ID is sufficient).
  4. All other types: no extra meta.

Shared class meta (streaming)

When meta share is enabled, Java uses the streaming shared meta protocol and writes TypeDef bytes inline on first use.

| varuint32: index_marker | [class def bytes if new] |

index_marker = (index << 1) | flag
flag = 1 -> reference
flag = 0 -> new type
  • If flag == 1, this is a reference to a previously written type. No class def bytes follow.
  • If flag == 0, this is a new type definition and class def bytes are written inline.

The index is assigned sequentially in the order types are first encountered.

Schema modes

Java native serialization supports two schema modes:

  • Schema consistent (compatible mode disabled): fields are serialized in a fixed order and no ClassDef is required. Type meta uses STRUCT or NAMED_STRUCT for user-defined classes.
  • Schema evolution (compatible mode enabled): fields are serialized with schema evolution metadata (ClassDef). Type meta uses COMPATIBLE_STRUCT or NAMED_COMPATIBLE_STRUCT.

ClassDef format (compatible mode)

ClassDef is the schema evolution metadata encoded for compatible structs. It is written inline when shared meta is enabled, or referenced by index when already seen.

Binary layout

| 8 bytes header | [varuint32 extra size] | class meta bytes |

Header layout (lower bits on the right):

| 50-bit hash | 4 bits reserved | 1 bit compress | 1 bit has_fields_meta | 8-bit size |
  • size: lower 8 bits. If size equals the mask (0xFF), write extra size as varuint32 and add it.
  • compress: set when payload is compressed.
  • has_fields_meta: set when field metadata is present.
  • reserved: bits 10-13 are reserved for future use and must be zero.
  • hash: 50-bit hash of the payload and flags.

Class meta bytes

Class meta encodes a linearized class hierarchy (from parent to leaf) and field metadata:

| num_classes | class_layer_0 | class_layer_1 | ... |

class_layer:
| num_fields << 1 | registered_flag | [type_id if registered] |
| namespace | type_name | field_infos |
  • num_classes stores (num_layers - 1) in a single byte.
    • If it equals 0b1111, read an extra varuint32 small7 and add it.
    • The actual number of layers is num_classes + 1.
  • registered_flag is 1 if the class is registered by numeric ID.
  • If registered by ID, the class type ID follows (varuint32 small7).
  • If registered by name or unregistered, namespace and type name are written as meta strings.

Field info

Each field uses a compact header followed by its name bytes (omitted when TAG_ID is used) and its type info:

| field_header | [field_name_bytes] | field_type |

field_header bits:

  • bit 0: trackingRef
  • bit 1: nullable
  • bits 2-3: field name encoding
  • bits 4-6: name length (len-1), or tag ID when TAG_ID is used; value 7 indicates extended length
  • bit 7: reserved (0)

Field name encoding:

  • 0: UTF8
  • 1: ALL_TO_LOWER_SPECIAL
  • 2: LOWER_UPPER_DIGIT_SPECIAL
  • 3: TAG_ID (field name omitted, tag ID stored in size field)

If length is extended (size==7), an extra varuint32 small7 storing (len-1) - 7 follows.

Field type encoding

Field types are encoded with a type tag and optional nested type info. For nested types, the header includes nullable/trackingRef flags in the low bits. Top-level field types use the tag only (no flags).

Type tags:

TagField type
0Object (ObjectFieldType)
1Map (MapFieldType)
2Collection/List/Set (CollectionFieldType)
3Array (ArrayFieldType)
4Enum (EnumFieldType)
5+Registered type (RegisteredFieldType)

Encoding rules:

  • ObjectFieldType: write tag 0.
  • MapFieldType: write tag 1, then key type, then value type.
  • CollectionFieldType: write tag 2, then element type.
  • ArrayFieldType: write tag 3, then dimensions, then component type.
  • EnumFieldType: write tag 4.
  • RegisteredFieldType: write tag 5 + type_id.

For nested types, nullable/trackingRef flags are stored in the low bits of the header as (type_tag << 2) | (nullable << 1) | tracking_ref.

Meta string encoding

Namespace, type names, and field names use the same meta string encodings as the xlang spec.

Package and type names

Header format:

| 6 bits size | 2 bits encoding |
  • size is the byte length of the encoded name.
  • if size == 63, write extra length (size - 63) as varuint32 small7.

Encodings:

  • Package name: UTF8, ALL_TO_LOWER_SPECIAL, LOWER_UPPER_DIGIT_SPECIAL
  • Type name: UTF8, LOWER_UPPER_DIGIT_SPECIAL, FIRST_TO_LOWER_SPECIAL, ALL_TO_LOWER_SPECIAL

Field names

Field name encoding is described in the ClassDef field header section. When using TAG_ID, the field name bytes are omitted and the tag ID is stored in the size field.

Encoding algorithms

See the xlang specification for encoding algorithms and tables: docs/specification/xlang_serialization_spec.md#meta-string.

Value encodings

This section describes the byte layouts for common built-in serializers used in Java native serialization. Custom serializers (EXT) may define additional formats but must still follow the reference and type meta rules described above.

Primitives

  • boolean: 1 byte (0x00 or 0x01).
  • byte: 1 byte.
  • short: 2 bytes little endian.
  • char: 2 bytes little endian (UTF-16 code unit).
  • int:
    • fixed: 4 bytes little endian.
    • varint: signed varint32 (ZigZag) when compressInt is enabled.
  • long:
    • fixed: 8 bytes little endian.
    • varint: signed varint64 (ZigZag) when longEncoding=VARINT.
    • tagged: tagged int64 when longEncoding=TAGGED.
  • float: IEEE 754 float32, little endian.
  • double: IEEE 754 float64, little endian.

Varint encodings follow the xlang spec: docs/specification/xlang_serialization_spec.md#unsigned-varint32.

String

Strings are encoded as:

| varuint36_small: (num_bytes << 2) | coder | string bytes |
  • coder: 2-bit value
    • 0: LATIN1
    • 1: UTF16
    • 2: UTF8
  • num_bytes: byte length of the encoded string payload.

UTF16 is encoded as little endian 2-byte code units.

Enum

  • If serializeEnumByName is enabled: write enum name as a meta string.
  • Otherwise: write enum ordinal as varuint32 small7.

Binary (byte[])

Primitive byte arrays are encoded as:

| varuint32: num_bytes | raw bytes |

Primitive arrays

Primitive arrays use writePrimitiveArrayWithSize unless compression is enabled:

| varuint32: byte_length | raw bytes |
  • compressIntArray: int[] encoded as | varuint32: length | varint32... |.
  • compressLongArray: long[] encoded as | varuint32: length | varint64/tagged... |.

Object arrays

Object arrays encode length and a monomorphic flag:

| varuint32_small7: (length << 1) | mono_flag |
  • If mono_flag == 1, all elements share a known component serializer. Each element uses ref flags and the component serializer writes the value.
  • If mono_flag == 0, each element uses ref flags and writes its own class info and data.

Collections (List/Set)

Collections encode length and a one-byte elements header:

| varuint32_small7: length | elements_header | [elem_class_info] | elements... |

elements_header bits (see CollectionFlags):

  • bit 0: TRACKING_REF
  • bit 1: HAS_NULL
  • bit 2: IS_DECL_ELEMENT_TYPE
  • bit 3: IS_SAME_TYPE

If IS_SAME_TYPE is set and IS_DECL_ELEMENT_TYPE is not set, the element class info is written once before the elements. Element values then follow with either ref flags (if TRACKING_REF) or per-element null flags (if HAS_NULL).

If IS_SAME_TYPE is not set, each element is written with its own class info and data (and optionally ref flags).

Maps

Maps encode entry count and then a sequence of chunks. Each chunk groups entries that share key and value types.

| varuint32_small7: size | chunk_1 | chunk_2 | ... |

chunk (non-null entries):
| header | chunk_size | [key_class_info] | [value_class_info] | entries... |

header bits (see MapFlags):

  • bit 0: TRACKING_KEY_REF
  • bit 1: KEY_HAS_NULL
  • bit 2: KEY_DECL_TYPE
  • bit 3: TRACKING_VALUE_REF
  • bit 4: VALUE_HAS_NULL
  • bit 5: VALUE_DECL_TYPE

If KEY_DECL_TYPE or VALUE_DECL_TYPE is unset, the corresponding class info is written once at the start of the chunk. chunk_size is a single byte (1..255) and MAX_CHUNK_SIZE is 255.

Null key/value entries

Entries with null key or null value are encoded as special single-entry chunks without a chunk_size byte:

  • null key, non-null value: NULL_KEY_VALUE_DECL_TYPE* flags, then value payload
  • null value, non-null key: NULL_VALUE_KEY_DECL_TYPE* flags, then key payload
  • null key and null value: KV_NULL header only

These chunks always represent exactly one entry.

Objects and structs

Object values are encoded as:

| ref meta | type meta | field data |

Field data is written by the serializer selected by the class info. For standard object serialization:

  • Fields are sorted deterministically using DescriptorGrouper order: primitives, boxed primitives, built-ins, collections, maps, then other fields, with names sorted within each category.
  • For compatible mode, MetaSharedSerializer uses ClassDef field metadata to read and skip unknown fields.
  • For each field, the serializer uses field metadata (nullable, trackingRef, polymorphic) to decide whether to write ref flags and/or type meta before the field value.

Extensions (EXT)

Extension types are encoded by their registered serializer. Type meta is still written before the value as described above. The serializer is responsible for the value layout.

Out-of-band buffers

When a BufferCallback is provided, the oob flag is set in the header and serializers may emit buffer references instead of inline bytes (for example, large primitive arrays). The out-of-band buffer protocol is specific to the callback implementation; the main stream only contains references to those buffers.