)]}' { "commit": "8c4407d5ebab903c0c1f4550118313fdc4e87f28", "tree": "ddaf322ce2b5e981ded10a419ab6abf2b2ed7f1d", "parents": [ "9642d7426194147ab9b9859dd40809431e8e04f4" ], "author": { "name": "songzhendong", "email": "xiaodong12315@qq.com", "time": "Mon Aug 17 08:11:01 2026 +0800" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 17 08:11:01 2026 +0800" }, "message": "feat(grpc): native multi-backend failover via grpc-js (#144)\n\nSupport a comma-separated list in SW_AGENT_COLLECTOR_BACKEND_SERVICES so the\nagent fails over between OAP backends, implemented with grpc-js\u0027s own resolver\nand load balancing rather than an application-level channel state machine.\n\nTransport\n- A single address keeps the built-in `dns:` resolver, so one hostname with\n multiple A/AAAA records is load-balanced and periodically re-resolved.\n- Multiple addresses register a small `sw-static` resolver (the built-in\n `ipv4:`/`ipv6:` schemes accept IP literals only) and use `pick_first` with\n `shuffleAddressList`, which spreads agents across backends while keeping the\n channel authority stable.\n- Reconnection uses grpc-js exponential backoff (1s to 30s); the agent no longer\n polls channel state or re-selects backends itself.\n- No client keepalive: the agent\u0027s own reporting traffic is the liveness signal,\n and a stock OAP (grpc-java defaults) answers unsolicited pings with\n GOAWAY ENHANCE_YOUR_CALM.\n- Retry is scoped to ManagementService/reportInstanceProperties, the only\n idempotent unary call. Trace and meter uploads are never retried at the gRPC\n layer, so a replayed stream cannot duplicate segments.\n\nReporting semantics\n- Best effort: a failed report discards its batch and is never re-sent, so\n statistics cannot be double-counted. Discards are logged with a running total.\n- flush() is a bounded single attempt, not a delivery guarantee. It forces out\n pending data first (which has not been attempted yet), then waits on any prior\n in-flight report with the remaining budget, so a stuck RPC cannot stall a host\n such as AWS Lambda.\n- Authentication failures keep the channel and emit a throttled error pointing at\n SW_AGENT_AUTHENTICATION; switching backends cannot fix a cluster-wide token.\n- A backend that is reachable but unresponsive is not failed over: failover\n targets unreachable backends, not functional health.\n\nBehavior changes\n- Host http_proxy/https_proxy no longer affect the OAP uplink\n (grpc.enable_http_proxy\u003d0); agent to OAP via HTTP proxy is not supported.\n- Default SW_AGENT_LOGGING_LEVEL is now `warn`, so connection and data-loss\n diagnostics are visible without extra configuration.\n- Under TLS, all backends must present certificates sharing SANs because\n verification follows the channel authority; one DNS name with multiple records\n is preferred for HA.\n\nDocs and tests\n- docs/en/setup/configuration.md and docs/en/advanced/troubleshooting.md cover the\n address forms, the diagnostics and the best-effort contract.\n- Unit tests for the resolver, channel manager, report coalescing and both senders,\n plus a remote-e2e case that stops the active mock collector and asserts reporting\n continues on the standby.\n\nNo new configuration options are introduced.", "tree_diff": [ { "type": "modify", "old_id": "4b0fe78763b4f00322bd1ce517610b48213411ad", "old_mode": 33188, "old_path": ".github/workflows/test.yaml", "new_id": "78930dd04190851f82482d053c432003aef1b3b8", "new_mode": 33188, "new_path": ".github/workflows/test.yaml" }, { "type": "modify", "old_id": "4218ee48a3c6e6ea3b83f45234c8ac893746fb5a", "old_mode": 33188, "old_path": "docs/en/advanced/troubleshooting.md", "new_id": "5636867af54578d0882fb396d5af8a849d506195", "new_mode": 33188, "new_path": "docs/en/advanced/troubleshooting.md" }, { "type": "modify", "old_id": "2660baa26a16efc391cc2a775bb62c30813e21ab", "old_mode": 33188, "old_path": "docs/en/setup/configuration.md", "new_id": "e4d77b3f45bca52d74140960b1e327bce7dd8a1a", "new_mode": 33188, "new_path": "docs/en/setup/configuration.md" }, { "type": "modify", "old_id": "e5dd18db611472910bc83a89027b611e7ee5df63", "old_mode": 33188, "old_path": "src/agent/core/boot/ServiceManager.ts", "new_id": "43b6daebd39bab9a82c195ab124ce3ba86031449", "new_mode": 33188, "new_path": "src/agent/core/boot/ServiceManager.ts" }, { "type": "modify", "old_id": "93b8d7139f320986a165f13004b9c78206f0e508", "old_mode": 33188, "old_path": "src/agent/core/meter/MeterSender.ts", "new_id": "2101b423d62246428b55abdb27fb2b9c6a643a1c", "new_mode": 33188, "new_path": "src/agent/core/meter/MeterSender.ts" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "e01eefd632d7e8d83db085d19ef9424fb7461b08", "new_mode": 33188, "new_path": "src/agent/core/remote/BackendAddressResolver.ts" }, { "type": "modify", "old_id": "4eae558880fa4ea09ca4f7dc77c7550081917293", "old_mode": 33188, "old_path": "src/agent/core/remote/GRPCChannel.ts", "new_id": "9a31510399906794c21de9ae992db53ee0029999", "new_mode": 33188, "new_path": "src/agent/core/remote/GRPCChannel.ts" }, { "type": "modify", "old_id": "5eb2fedc64a4200e192fc166e8231d4d1ed602f4", "old_mode": 33188, "old_path": "src/agent/core/remote/GRPCChannelManager.ts", "new_id": "8fbf0163973a9416215f96aa1523a539a05a4f6b", "new_mode": 33188, "new_path": "src/agent/core/remote/GRPCChannelManager.ts" }, { "type": "modify", "old_id": "a558caf17c4127fb52385c09fda0bb1ffde456f5", "old_mode": 33188, "old_path": "src/agent/core/remote/ServiceManagementClient.ts", "new_id": "53c0e19f7b867660cfbb424c921ac09e18f4f8c0", "new_mode": 33188, "new_path": "src/agent/core/remote/ServiceManagementClient.ts" }, { "type": "modify", "old_id": "3f1812d7dd4ea4e1dcc5e32b2be47642f6079f89", "old_mode": 33188, "old_path": "src/agent/core/remote/TraceSegmentServiceClient.ts", "new_id": "c85e491f471505be8a4f00f526135a8fdc2ff996", "new_mode": 33188, "new_path": "src/agent/core/remote/TraceSegmentServiceClient.ts" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "f64e67c59280a6c58aa60d656cf15138ac22a1d7", "new_mode": 33188, "new_path": "src/agent/core/remote/coalesceReport.ts" }, { "type": "modify", "old_id": "33d67a2179f047c2ccf1d5388b71a83851ea6604", "old_mode": 33188, "old_path": "src/index.ts", "new_id": "047569b0430798ada0d35eb3222a912ef5d3a3ca", "new_mode": 33188, "new_path": "src/index.ts" }, { "type": "modify", "old_id": "32db1aac691a60c9b8413e0fbb3af06064d0fb26", "old_mode": 33188, "old_path": "src/logging/index.ts", "new_id": "e8c84dbab6d3d56a8d961f8ea66867439c45d618", "new_mode": 33188, "new_path": "src/logging/index.ts" }, { "type": "modify", "old_id": "bd199264b952aaa83ee2a0a8a34aa282ccffc891", "old_mode": 33188, "old_path": "src/trace/context/SpanContext.ts", "new_id": "fba2e870b44babb5b1b7eb4dc1a24794baa92371", "new_mode": 33188, "new_path": "src/trace/context/SpanContext.ts" }, { "type": "modify", "old_id": "5e907d4f4fa0d9e5f6f32de6f53a31061ae488e4", "old_mode": 33188, "old_path": "tests/plugins/common/Dockerfile.agent", "new_id": "7db6a1c838b2bacabf64dab3c60e568782663843", "new_mode": 33188, "new_path": "tests/plugins/common/Dockerfile.agent" }, { "type": "modify", "old_id": "504dfa915c031fd90ce236d01299386bbb6a7263", "old_mode": 33188, "old_path": "tests/plugins/common/base-compose.yml", "new_id": "10eb7d5bf491287323022b5d424b19dcfaddec39", "new_mode": 33188, "new_path": "tests/plugins/common/base-compose.yml" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "1fca36d9cae3737ae36932e19964aab79b81b590", "new_mode": 33188, "new_path": "tests/remote-e2e/common/server.ts" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "aba99e3316864a694849fd057d67f0795626b611", "new_mode": 33188, "new_path": "tests/remote-e2e/static-failover/docker-compose.yml" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "5230691dfa34a0f92ebb13911a3d103acf223062", "new_mode": 33188, "new_path": "tests/remote-e2e/static-failover/test.ts" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "5646fe22e765952b819b609b667ea9afd6420207", "new_mode": 33188, "new_path": "tests/remote-e2e/support/helpers.ts" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "2cdca8d148c2e3984052ac81d53431ab02e02792", "new_mode": 33188, "new_path": "tests/remote/BackendAddressResolver.test.ts" }, { "type": "modify", "old_id": "ad6f16e339bf651d2bdf0a7397a4df351182dd59", "old_mode": 33188, "old_path": "tests/remote/GRPCChannelManager.test.ts", "new_id": "cf17f37a8c05da8f380f7aaa6e4e67d6e1de5b39", "new_mode": 33188, "new_path": "tests/remote/GRPCChannelManager.test.ts" }, { "type": "modify", "old_id": "c1a3018d06877c4ee9ed607b449390aeb88fa903", "old_mode": 33188, "old_path": "tests/remote/MeterSender.test.ts", "new_id": "8fde7aa8df5fcf4f7fcb64650933c5cda60b4518", "new_mode": 33188, "new_path": "tests/remote/MeterSender.test.ts" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "7921efc3d0d5e15a176fe744119affd093a9bc38", "new_mode": 33188, "new_path": "tests/remote/TraceSegmentServiceClient.test.ts" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "a4558c8563c4d97baf82d76eed2cbaa9994056d9", "new_mode": 33188, "new_path": "tests/remote/coalesceReport.test.ts" } ] }