blob: e5b19b3fc57ef9f3cb06c6eb89703933b4f19e1f [file] [view]
---
title: Basic Serialization
sidebar_position: 1
id: basic-serialization
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---
This guide covers the core serialization APIs in the default xlang mode for Fory Go.
## Creating a Fory Instance
Create a Fory instance and register your types before serialization:
```go
import "github.com/apache/fory/go/fory"
f := fory.New(fory.WithXlang(true))
// Register struct with a type ID
f.RegisterStruct(User{}, 1)
f.RegisterStruct(Order{}, 2)
// Or register with a name (more flexible, less prone to ID conflicts, but higher serialization cost)
f.RegisterStructByName(User{}, "example.User")
// Register enum types
f.RegisterEnum(Color(0), 3)
```
`fory.New()` uses xlang mode with compatible schema evolution. The example sets
`fory.WithXlang(true)` explicitly so the mode choice is visible. For Go-only
payloads that need native mode, configure `fory.WithXlang(false)` explicitly in
the native-mode examples.
**Important**: The Fory instance should be reused across serialization calls. Creating a new instance involves allocating internal buffers, type caches, and resolvers, which is expensive. The default Fory instance is not thread-safe; for concurrent usage, use the thread-safe wrapper (see [Thread Safety](thread-safety.md)).
See [Type Registration](type-registration.md) for more details.
## Core API
### Serialize and Deserialize
The primary API for serialization:
```go
// Serialize any value
data, err := f.Serialize(value)
if err != nil {
// Handle error
}
// Deserialize into target
var result MyType
err = f.Deserialize(data, &result)
if err != nil {
// Handle error
}
```
### Marshal and Unmarshal
Aliases for `Serialize` and `Deserialize` (familiar to Go developers):
```go
data, err := f.Marshal(value)
err = f.Unmarshal(data, &result)
```
## Serializing Primitives
```go
// Integers
data, _ := f.Serialize(int64(42))
var i int64
f.Deserialize(data, &i) // i = 42
// Floats
data, _ = f.Serialize(float64(3.14))
var fl float64
f.Deserialize(data, &fl) // fl = 3.14
// Strings
data, _ = f.Serialize("hello")
var s string
f.Deserialize(data, &s) // s = "hello"
// Booleans
data, _ = f.Serialize(true)
var b bool
f.Deserialize(data, &b) // b = true
```
## Serializing Collections
### Slices
```go
// String slice
strs := []string{"a", "b", "c"}
data, _ := f.Serialize(strs)
var result []string
f.Deserialize(data, &result)
// result = ["a", "b", "c"]
// Integer slice
nums := []int64{1, 2, 3}
data, _ = f.Serialize(nums)
var intResult []int64
f.Deserialize(data, &intResult)
// intResult = [1, 2, 3]
```
### Maps
```go
// String to string map
m := map[string]string{"key": "value"}
data, _ := f.Serialize(m)
var result map[string]string
f.Deserialize(data, &result)
// result = {"key": "value"}
// String to int map
m2 := map[string]int64{"count": 42}
data, _ = f.Serialize(m2)
var result2 map[string]int64
f.Deserialize(data, &result2)
// result2 = {"count": 42}
```
## Serializing Structs
### Basic Struct Serialization
Only **exported fields** (starting with uppercase) are serialized:
```go
type User struct {
ID int64 // Serialized
Name string // Serialized
password string // NOT serialized (unexported)
}
f.RegisterStruct(User{}, 1)
user := &User{ID: 1, Name: "Alice", password: "secret"}
data, _ := f.Serialize(user)
var result User
f.Deserialize(data, &result)
// result.ID = 1, result.Name = "Alice", result.password = ""
```
### Nested Structs
```go
type Address struct {
City string
Country string
}
type Person struct {
Name string
Address Address
}
f.RegisterStruct(Address{}, 1)
f.RegisterStruct(Person{}, 2)
person := &Person{
Name: "Alice",
Address: Address{City: "NYC", Country: "USA"},
}
data, _ := f.Serialize(person)
var result Person
f.Deserialize(data, &result)
// result.Address.City = "NYC"
```
### Pointer Fields
```go
type Node struct {
Value int32
Child *Node
}
// Use WithTrackRef for pointer fields
f := fory.New(fory.WithXlang(true), fory.WithTrackRef(true))
f.RegisterStruct(Node{}, 1)
root := &Node{
Value: 1,
Child: &Node{Value: 2, Child: nil},
}
data, _ := f.Serialize(root)
var result Node
f.Deserialize(data, &result)
// result.Child.Value = 2
```
## Streaming API
For scenarios where you want to control the buffer:
### SerializeTo
Serialize to an existing buffer:
```go
buf := fory.NewByteBuffer(nil)
// Serialize multiple values to same buffer
f.SerializeTo(buf, value1)
f.SerializeTo(buf, value2)
// Get all serialized data
data := buf.GetByteSlice(0, buf.WriterIndex())
```
### DeserializeFrom
Deserialize from an existing buffer:
```go
buf := fory.NewByteBuffer(data)
var result1, result2 MyType
f.DeserializeFrom(buf, &result1)
f.DeserializeFrom(buf, &result2)
```
## Generic API (Type-Safe)
Fory Go provides generic functions for type-safe serialization:
```go
import "github.com/apache/fory/go/fory"
type User struct {
ID int64
Name string
}
// Type-safe serialization
user := &User{ID: 1, Name: "Alice"}
data, err := fory.Serialize(f, user)
// Type-safe deserialization
var result User
err = fory.Deserialize(f, data, &result)
```
The generic API:
- Infers type at compile time
- Provides better type safety
- May offer performance benefits
## Error Handling
Always check errors from serialization operations:
```go
data, err := f.Serialize(value)
if err != nil {
switch e := err.(type) {
case fory.Error:
fmt.Printf("Fory error: %s (kind: %d)\n", e.Error(), e.Kind())
default:
fmt.Printf("Unknown error: %v\n", err)
}
return
}
err = f.Deserialize(data, &result)
if err != nil {
// Handle deserialization error
}
```
Common error kinds:
- `ErrKindBufferOutOfBound`: Read/write beyond buffer bounds
- `ErrKindTypeMismatch`: Type ID mismatch during deserialization
- `ErrKindUnknownType`: Unknown type encountered
- `ErrKindMaxDepthExceeded`: Recursion depth limit exceeded
- `ErrKindHashMismatch`: Struct hash mismatch (schema changed)
See [Troubleshooting](troubleshooting.md) for error resolution.
## Nil Handling
### Nil Pointers
```go
var ptr *User = nil
data, _ := f.Serialize(ptr)
var result *User
f.Deserialize(data, &result)
// result = nil
```
### Empty Collections
```go
// Nil slice
var slice []string = nil
data, _ := f.Serialize(slice)
var result []string
f.Deserialize(data, &result)
// result = nil
// Empty slice (different from nil)
empty := []string{}
data, _ = f.Serialize(empty)
f.Deserialize(data, &result)
// result = [] (empty, not nil)
```
## Complete Example
```go
package main
import (
"fmt"
"github.com/apache/fory/go/fory"
)
type Order struct {
ID int64
Customer string
Items []Item
Total float64
}
type Item struct {
Name string
Quantity int32
Price float64
}
func main() {
f := fory.New(fory.WithXlang(true))
f.RegisterStruct(Order{}, 1)
f.RegisterStruct(Item{}, 2)
order := &Order{
ID: 12345,
Customer: "Alice",
Items: []Item{
{Name: "Widget", Quantity: 2, Price: 9.99},
{Name: "Gadget", Quantity: 1, Price: 24.99},
},
Total: 44.97,
}
// Serialize
data, err := f.Serialize(order)
if err != nil {
panic(err)
}
fmt.Printf("Serialized %d bytes\n", len(data))
// Deserialize
var result Order
if err := f.Deserialize(data, &result); err != nil {
panic(err)
}
fmt.Printf("Order ID: %d\n", result.ID)
fmt.Printf("Customer: %s\n", result.Customer)
fmt.Printf("Items: %d\n", len(result.Items))
fmt.Printf("Total: %.2f\n", result.Total)
}
```
## Cross-Language Interoperability
The default xlang format is shared by all Fory runtimes. The following sections cover its cross-language type mapping, type identity, and interoperability requirements.
Fory Go enables seamless data exchange with Java, Python, C++, Rust,
JavaScript/TypeScript, C#, Swift, Dart, Scala, and Kotlin. This guide covers
xlang compatibility and type mapping.
### Xlang Configuration
Go defaults to xlang mode with compatible schema evolution. Set the mode explicitly in xlang examples:
```go
f := fory.New(fory.WithXlang(true))
```
### Type Registration for Xlang
Use consistent type IDs across all languages:
#### Go
```go
type User struct {
ID int64
Name string
}
f := fory.New(fory.WithXlang(true))
f.RegisterStruct(User{}, 1)
data, _ := f.Serialize(&User{ID: 1, Name: "Alice"})
```
#### Java
```java
public class User {
public long id;
public String name;
}
Fory fory = Fory.builder().withXlang(true).build();
fory.register(User.class, 1);
User user = fory.deserialize(data, User.class);
```
#### Python
```python
from dataclasses import dataclass
import pyfory
@dataclass
class User:
id: pyfory.Int64
name: str
fory = pyfory.Fory(xlang=True)
fory.register(User, type_id=1)
user = fory.deserialize(data)
```
### Type Mapping
See [Type Mapping Specification](../../specification/xlang_type_mapping.md) for detailed type mappings across all languages.
### Field Ordering
Cross-language serialization requires consistent field ordering. Fory sorts fields by their snake_case names alphabetically.
Go field names are converted to snake_case for sorting:
```go
type Example struct {
UserID int64 // -> user_id
FirstName string // -> first_name
Age int32 // -> age
}
// Sorted order: age, first_name, user_id
```
Ensure other languages use matching field names that produce the same snake_case ordering, or use field IDs for explicit control:
```go
type Example struct {
UserID int64 `fory:"id=0"`
FirstName string `fory:"id=1"`
Age int32 `fory:"id=2"`
}
```
### Examples
#### Go to Java
**Go (Serializer)**:
```go
type Order struct {
ID int64
Customer string
Total float64
Items []string
}
f := fory.New(fory.WithXlang(true))
f.RegisterStruct(Order{}, 1)
order := &Order{
ID: 12345,
Customer: "Alice",
Total: 99.99,
Items: []string{"Widget", "Gadget"},
}
data, _ := f.Serialize(order)
// Send 'data' to Java service
```
**Java (Deserializer)**:
```java
public class Order {
public long id;
public String customer;
public double total;
public List<String> items;
}
Fory fory = Fory.builder().withXlang(true).build();
fory.register(Order.class, 1);
Order order = fory.deserialize(data, Order.class);
```
#### Python to Go
**Python (Serializer)**:
```python
from dataclasses import dataclass
import pyfory
@dataclass
class Message:
id: pyfory.Int64
content: str
timestamp: pyfory.Int64
fory = pyfory.Fory(xlang=True)
fory.register(Message, type_id=1)
msg = Message(id=1, content="Hello from Python", timestamp=1234567890)
data = fory.serialize(msg)
```
**Go (Deserializer)**:
```go
type Message struct {
ID int64
Content string
Timestamp int64
}
f := fory.New(fory.WithXlang(true))
f.RegisterStruct(Message{}, 1)
var msg Message
f.Deserialize(data, &msg)
fmt.Println(msg.Content) // "Hello from Python"
```
#### Nested Structures
Cross-language nested structures require all types to be registered:
### Lists and Dense Arrays
Go slices are ordinary `list<T>` carriers unless a field tag explicitly requests
the dense `array<T>` schema. Use `array<T>` only for one-dimensional bool or
numeric data.
| Fory schema | Go carrier and tag sketch |
| ----------------- | ------------------------------------------------------ |
| `list<int32>` | `[]int32` / `fory:"type=list(element=int32)"` |
| `array<bool>` | `[]bool` / `fory:"type=array(element=bool)"` |
| `array<int8>` | `[]int8` / `fory:"type=array(element=int8)"` |
| `array<int16>` | `[]int16` / `fory:"type=array(element=int16)"` |
| `array<int32>` | `[]int32` / `fory:"type=array(element=int32)"` |
| `array<int64>` | `[]int64` / `fory:"type=array(element=int64)"` |
| `array<uint8>` | `[]uint8` / `fory:"type=array(element=uint8)"` |
| `array<uint16>` | `[]uint16` / `fory:"type=array(element=uint16)"` |
| `array<uint32>` | `[]uint32` / `fory:"type=array(element=uint32)"` |
| `array<uint64>` | `[]uint64` / `fory:"type=array(element=uint64)"` |
| `array<float16>` | `[]float16.Float16` / `type=array(element=float16)` |
| `array<bfloat16>` | `[]bfloat16.BFloat16` / `type=array(element=bfloat16)` |
| `array<float32>` | `[]float32` / `fory:"type=array(element=float32)"` |
| `array<float64>` | `[]float64` / `fory:"type=array(element=float64)"` |
**Go**:
```go
type Address struct {
Street string
City string
Country string
}
type Company struct {
Name string
Address Address
}
f := fory.New(fory.WithXlang(true))
f.RegisterStruct(Address{}, 1)
f.RegisterStruct(Company{}, 2)
```
**Java**:
```java
public class Address {
public String street;
public String city;
public String country;
}
public class Company {
public String name;
public Address address;
}
fory.register(Address.class, 1);
fory.register(Company.class, 2);
```
### Common Issues
#### Field Name Mismatch
Go uses PascalCase, other languages may use camelCase or snake_case. Fields are matched by their snake_case conversion:
```go
// Go
type User struct {
FirstName string // -> first_name
}
// Java - field name converted to snake_case must match
public class User {
public String firstName; // -> first_name (matches)
}
```
#### Type Interpretation
Go unsigned types map to Java signed types with the same bit pattern:
```go
var value uint64 = 18446744073709551615 // Max uint64
```
Java's `long` holds the same bits but interprets as -1. Use `Long.toUnsignedString()` in Java if unsigned interpretation is needed.
#### Nil vs Null
Go nil slices/maps serialize differently based on configuration:
```go
var slice []string = nil
// In xlang mode: serializes based on nullable configuration
```
Ensure other languages handle null appropriately.
### Interoperability Best Practices
1. **Use consistent type IDs**: Same numeric ID for the same type across all languages
2. **Register all types**: Including nested struct types
3. **Match field ordering**: Use same snake_case names or explicit field IDs
4. **Test cross-language**: Run integration tests early and often
5. **Handle type differences**: Be aware of signed/unsigned interpretation differences
### Related Guides
- [Type Registration](type-registration.md)
- [Supported Types](supported-types.md)
- [Schema Evolution](schema-evolution.md)
- [Xlang Serialization Specification](../../specification/xlang_serialization_spec.md)
- [Type Mapping Specification](../../specification/xlang_type_mapping.md)
### Built-in values
```go
package main
import forygo "github.com/apache/fory/go/fory"
import "fmt"
func main() {
list := []any{true, false, "str", -1.1, 1, make([]int32, 10), make([]float64, 20)}
fory := forygo.NewFory(forygo.WithXlang(true))
bytes, err := fory.Marshal(list)
if err != nil {
panic(err)
}
var newValue any
// bytes can be deserialized by other languages
if err := fory.Unmarshal(bytes, &newValue); err != nil {
panic(err)
}
fmt.Println(newValue)
dict := map[string]any{
"k1": "v1",
"k2": list,
"k3": -1,
}
bytes, err = fory.Marshal(dict)
if err != nil {
panic(err)
}
// bytes can be deserialized by other languages
if err := fory.Unmarshal(bytes, &newValue); err != nil {
panic(err)
}
fmt.Println(newValue)
}
```
### Custom values
```go
package main
import forygo "github.com/apache/fory/go/fory"
import "fmt"
func main() {
type SomeClass1 struct {
F1 any
F2 map[int8]int32
}
type SomeClass2 struct {
F1 any
F2 string
F3 []any
F4 map[int8]int32
F5 int8
F6 int16
F7 int32
F8 int64
F9 float32
F10 float64
F11 []int16
F12 []int16
}
serializer := forygo.NewFory(forygo.WithXlang(true))
if err := serializer.RegisterStructByName(SomeClass1{}, "example.SomeClass1"); err != nil {
panic(err)
}
if err := serializer.RegisterStructByName(SomeClass2{}, "example.SomeClass2"); err != nil {
panic(err)
}
obj1 := &SomeClass1{F1: true, F2: map[int8]int32{-1: 2}}
obj := &SomeClass2{
F1: obj1,
F2: "abc",
F3: []any{"abc", "abc"},
F4: map[int8]int32{1: 2},
F5: 127,
F6: 32767,
F7: 2147483647,
F8: 9223372036854775807,
F9: 1.0 / 2,
F10: 1.0 / 3.0,
F11: []int16{1, 2},
F12: []int16{-1, 4},
}
bytes, err := serializer.Marshal(obj)
if err != nil {
panic(err)
}
var newValue any
// bytes can be deserialized by other languages
if err := serializer.Unmarshal(bytes, &newValue); err != nil {
panic(err)
}
fmt.Println(newValue)
}
```
### Shared and circular references
```go
package main
import forygo "github.com/apache/fory/go/fory"
import "fmt"
func main() {
type SomeClass struct {
F1 *SomeClass
F2 map[string]string
F3 map[string]string
}
fory := forygo.NewFory(forygo.WithXlang(true), forygo.WithTrackRef(true))
if err := fory.RegisterStruct(SomeClass{}, 65); err != nil {
panic(err)
}
value := &SomeClass{F2: map[string]string{"k1": "v1", "k2": "v2"}}
value.F3 = value.F2
value.F1 = value
bytes, err := fory.Marshal(value)
if err != nil {
panic(err)
}
var newValue any
// bytes can be deserialized by other languages
if err := fory.Unmarshal(bytes, &newValue); err != nil {
panic(err)
}
fmt.Println(newValue)
}
```
## Related Topics
- [Configuration](configuration.md)
- [Type Registration](type-registration.md)
- [Supported Types](supported-types.md)
- [References](references.md)