| <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Dubbo – Golang</title><link>https://dubbo.apache.org/zh-cn/blog/golang/</link><description>Recent content in Golang on Apache Dubbo</description><generator>Hugo -- gohugo.io</generator><language>zh-cn</language><atom:link href="https://dubbo.apache.org/zh-cn/blog/golang/index.xml" rel="self" type="application/rss+xml"/><item><title>Blog: dubbo-go 白话文</title><link>https://dubbo.apache.org/zh-cn/blog/2021/02/20/dubbo-go-%E7%99%BD%E8%AF%9D%E6%96%87/</link><pubDate>Sat, 20 Feb 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/02/20/dubbo-go-%E7%99%BD%E8%AF%9D%E6%96%87/</guid><description> |
| <h2 id="一前言">一、前言</h2> |
| <blockquote> |
| <p>本文基于 dubbogo <a href="https://github.com/apache/dubbo-go/releases/tag/v1.5.4">1.5.4</a> 版本</p> |
| </blockquote> |
| <p>最近开始参与 dubbogo 的一些开发测试，之前都是直接拿 <a href="https://github.com/apache/dubbo-go-samples">samples</a> 的例子验证功能，而这次为了复现一个功能问题，打算从零开始搭建一个 dubbo-go 和 dubbo 调用的工程，踩到了一些新人使用 dubbogo 的坑，把这个过程记录下供大家参考。</p> |
| <p>通过本文你可以了解到：</p> |
| <ul> |
| <li>如何常规配置 dubbogo 消费方去调用 dubbo 和 dubbogo 服务提供方</li> |
| <li>通过一个实际的 BUG 介绍解决问题的思路</li> |
| </ul> |
| <h2 id="二解决问题">二、解决问题</h2> |
| <h3 id="21-准备-dubbo-服务提供者">2.1 准备 dubbo 服务提供者</h3> |
| <h4 id="211-基本定义">2.1.1 基本定义</h4> |
| <p>定义 <code>DemoService</code> 接口：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#268bd2">public</span> <span style="color:#268bd2">interface</span> <span style="color:#268bd2">DemoService</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> String <span style="color:#268bd2">sayHello</span><span style="color:#719e07">(</span>String name<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> String <span style="color:#268bd2">sayHello</span><span style="color:#719e07">(</span>User user<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> String <span style="color:#268bd2">sayHello</span><span style="color:#719e07">(</span>User user<span style="color:#719e07">,</span> String name<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span> |
| </span></span></code></pre></div><p>定义 <code>User</code> 对象：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#268bd2">public</span> <span style="color:#268bd2">class</span> <span style="color:#268bd2">User</span> <span style="color:#268bd2">implements</span> Serializable <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">private</span> String name<span style="color:#719e07">;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">private</span> <span style="color:#dc322f">int</span> age<span style="color:#719e07">;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">......</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span> |
| </span></span></code></pre></div><h4 id="212-启动-dubbo-服务提供者">2.1.2 启动 dubbo 服务提供者</h4> |
| <p>用的 <a href="https://dubbo.apache.org/zh-cn/docsv2.7/user/configuration/api/">dubbo 官方示例代码</a>:</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#268bd2">public</span> <span style="color:#268bd2">static</span> <span style="color:#dc322f">void</span> <span style="color:#268bd2">main</span><span style="color:#719e07">(</span>String<span style="color:#719e07">[]</span> args<span style="color:#719e07">)</span> <span style="color:#268bd2">throws</span> IOException <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 服务实现 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> DemoService demoService <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> DemoServiceImpl<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 当前应用配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ApplicationConfig application <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> ApplicationConfig<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> application<span style="color:#719e07">.</span>setName<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;demoProvider&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 连接注册中心配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> RegistryConfig registry <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> RegistryConfig<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setAddress<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;127.0.0.1:2181&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setProtocol<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;zookeeper&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setUsername<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setPassword<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 服务提供者协议配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ProtocolConfig protocol <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> ProtocolConfig<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> protocol<span style="color:#719e07">.</span>setName<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;dubbo&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> protocol<span style="color:#719e07">.</span>setPort<span style="color:#719e07">(</span>12345<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> protocol<span style="color:#719e07">.</span>setThreads<span style="color:#719e07">(</span>200<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 注意：ServiceConfig为重对象，内部封装了与注册中心的连接，以及开启服务端口 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 服务提供者暴露服务配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ServiceConfig<span style="color:#719e07">&lt;</span>DemoService<span style="color:#719e07">&gt;</span> service <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> ServiceConfig<span style="color:#719e07">&lt;&gt;();</span> <span style="color:#586e75">// 此实例很重，封装了与注册中心的连接，请自行缓存，否则可能造成内存和连接泄漏 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> service<span style="color:#719e07">.</span>setApplication<span style="color:#719e07">(</span>application<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> service<span style="color:#719e07">.</span>setRegistry<span style="color:#719e07">(</span>registry<span style="color:#719e07">);</span> <span style="color:#586e75">// 多个注册中心可以用setRegistries() |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> service<span style="color:#719e07">.</span>setProtocol<span style="color:#719e07">(</span>protocol<span style="color:#719e07">);</span> <span style="color:#586e75">// 多个协议可以用setProtocols() |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> service<span style="color:#719e07">.</span>setInterface<span style="color:#719e07">(</span>DemoService<span style="color:#719e07">.</span>class<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> service<span style="color:#719e07">.</span>setRef<span style="color:#719e07">(</span>demoService<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> service<span style="color:#719e07">.</span>setVersion<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;1.0.0&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> service<span style="color:#719e07">.</span>setGroup<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;tc&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> service<span style="color:#719e07">.</span>setTimeout<span style="color:#719e07">(</span>60 <span style="color:#719e07">*</span> 1000<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 暴露及注册服务 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> service<span style="color:#719e07">.</span>export<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> System<span style="color:#719e07">.</span>in<span style="color:#719e07">.</span>read<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span> |
| </span></span></code></pre></div><p>查看 zookeeper 看是否注册成功：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#268bd2">$ls</span> /dubbo/com.funnycode.DemoService/providers |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">[</span>dubbo%3A%2F%2F127.0.0.1%3A12345%2Fcom.funnycode.DemoService%3Fanyhost%3Dtrue%26application%3DdemoProvider%26deprecated%3Dfalse%26dubbo%3D2.0.2%26dynamic%3Dtrue%26generic%3Dfalse%26group%3Dtc%26interface%3Dcom.funnycode.DemoService%26methods%3DsayHello%26pid%3D18167%26release%3D2.7.7%26revision%3D1.0.0%26side%3Dprovider%26threads%3D200%26timestamp%3D1606896020691%26version%3D1.0.0<span style="color:#719e07">]</span> |
| </span></span></code></pre></div><p>如上的输出表示服务提供方已经启动。</p> |
| <h3 id="22-准备-dubbogo-服务消费者">2.2 准备 dubbogo 服务消费者</h3> |
| <h4 id="221-基本定义">2.2.1 基本定义</h4> |
| <p>定义 <code>User</code> 对象：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> User <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> Name <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Age <span style="color:#dc322f">int</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (User) <span style="color:#268bd2">JavaClassName</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;com.funnycode.User&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>定义 <code>DemoProvider</code> 接口：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> DemoProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> SayHello <span style="color:#268bd2">func</span>(ctx context.Context, name <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) <span style="color:#2aa198">`dubbo:&#34;sayHello&#34;`</span> |
| </span></span><span style="display:flex;"><span> SayHello2 <span style="color:#268bd2">func</span>(ctx context.Context, user User) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) <span style="color:#2aa198">`dubbo:&#34;sayHello&#34;`</span> |
| </span></span><span style="display:flex;"><span> SayHello3 <span style="color:#268bd2">func</span>(ctx context.Context, user User, name <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) <span style="color:#2aa198">`dubbo:&#34;sayHello&#34;`</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>DemoProvider) <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;DemoProvider&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h4 id="222-启动-dubbogo-消费者">2.2.2 启动 dubbogo 消费者</h4> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;\n\n\nstart to test dubbo&#34;</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> res, err <span style="color:#719e07">:=</span> demoProvider.<span style="color:#268bd2">SayHello</span>(context.<span style="color:#268bd2">TODO</span>(), <span style="color:#2aa198">&#34;tc&#34;</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, res) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> user <span style="color:#719e07">:=</span> User{ |
| </span></span><span style="display:flex;"><span> Name: <span style="color:#2aa198">&#34;tc&#34;</span>, |
| </span></span><span style="display:flex;"><span> Age: <span style="color:#2aa198">18</span>, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> res, err = demoProvider.<span style="color:#268bd2">SayHello2</span>(context.<span style="color:#268bd2">TODO</span>(), user) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, res) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> res, err = demoProvider.<span style="color:#268bd2">SayHello3</span>(context.<span style="color:#268bd2">TODO</span>(), user, <span style="color:#2aa198">&#34;tc&#34;</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, res) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">initSignal</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="23-请求结果分析">2.3 请求结果分析</h3> |
| <h4 id="231-直接调用">2.3.1 直接调用</h4> |
| <blockquote> |
| <p>确认问题的存在</p> |
| </blockquote> |
| <p>第一个接口的参数是字符串，可以正常返回 <code>[2020-12-03/18:59:12 main.main: client.go: 29] response result: Hello tc</code> |
| 第二、三两个接口存在 <code>User</code> 对象，无法调用成功。错误信息如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>2020-12-02T17:10:47.739+0800 INFO getty/listener.go:87 session<span style="color:#719e07">{</span>session session-closed, Read Bytes: 924, Write Bytes: 199, Read Pkgs: 0, Write Pkgs: 1<span style="color:#719e07">}</span> got error<span style="color:#719e07">{</span>java exception:Fail to decode request due to: java.lang.IllegalArgumentException: Service not found:com.funnycode.DemoService, sayHello |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode<span style="color:#719e07">(</span>DecodeableRpcInvocation.java:134<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode<span style="color:#719e07">(</span>DecodeableRpcInvocation.java:80<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.DecodeHandler.decode<span style="color:#719e07">(</span>DecodeHandler.java:57<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.DecodeHandler.received<span style="color:#719e07">(</span>DecodeHandler.java:44<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run<span style="color:#719e07">(</span>ChannelEventRunnable.java:57<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.util.concurrent.ThreadPoolExecutor.runWorker<span style="color:#719e07">(</span>ThreadPoolExecutor.java:1149<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.util.concurrent.ThreadPoolExecutor<span style="color:#268bd2">$Worker</span>.run<span style="color:#719e07">(</span>ThreadPoolExecutor.java:624<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.lang.Thread.run<span style="color:#719e07">(</span>Thread.java:748<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span>, will be closed. |
| </span></span></code></pre></div><p>错误正如 <a href="https://github.com/apache/dubbo-go/issues/900">issue</a> 中描述的一模一样，因为错误信息返回到了消费端，可以看到 Java 那边的错误堆栈信息，所以直接去看 <code>DecodeableRpcInvocation.decode#134</code>。</p> |
| <h4 id="232-断点查看">2.3.2 断点查看</h4> |
| <p>代码如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#586e75">// 反序列化 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">public</span> <span style="color:#268bd2">class</span> <span style="color:#268bd2">DecodeableRpcInvocation</span> <span style="color:#268bd2">extends</span> RpcInvocation <span style="color:#268bd2">implements</span> Codec<span style="color:#719e07">,</span> Decodeable <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">public</span> Object <span style="color:#268bd2">decode</span><span style="color:#719e07">(</span>Channel channel<span style="color:#719e07">,</span> InputStream input<span style="color:#719e07">)</span> <span style="color:#268bd2">throws</span> IOException <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">......</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#719e07">(</span>serviceDescriptor <span style="color:#719e07">!=</span> <span style="color:#cb4b16">null</span><span style="color:#719e07">)</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 方法描述里面根据方法名查找 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> MethodDescriptor methodDescriptor <span style="color:#719e07">=</span> serviceDescriptor<span style="color:#719e07">.</span>getMethod<span style="color:#719e07">(</span>getMethodName<span style="color:#719e07">(),</span> desc<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#719e07">(</span>methodDescriptor <span style="color:#719e07">!=</span> <span style="color:#cb4b16">null</span><span style="color:#719e07">)</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> pts <span style="color:#719e07">=</span> methodDescriptor<span style="color:#719e07">.</span>getParameterClasses<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">this</span><span style="color:#719e07">.</span>setReturnTypes<span style="color:#719e07">(</span>methodDescriptor<span style="color:#719e07">.</span>getReturnTypes<span style="color:#719e07">());</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">}</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">}</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 表示没有找到方法 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> <span style="color:#719e07">(</span>pts <span style="color:#719e07">==</span> DubboCodec<span style="color:#719e07">.</span>EMPTY_CLASS_ARRAY<span style="color:#719e07">)</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#719e07">(!</span>RpcUtils<span style="color:#719e07">.</span>isGenericCall<span style="color:#719e07">(</span>path<span style="color:#719e07">,</span> getMethodName<span style="color:#719e07">())</span> <span style="color:#719e07">&amp;&amp;</span> <span style="color:#719e07">!</span>RpcUtils<span style="color:#719e07">.</span>isEcho<span style="color:#719e07">(</span>path<span style="color:#719e07">,</span> getMethodName<span style="color:#719e07">()))</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">throw</span> <span style="color:#719e07">new</span> IllegalArgumentException<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;Service not found:&#34;</span> <span style="color:#719e07">+</span> path <span style="color:#719e07">+</span> <span style="color:#2aa198">&#34;, &#34;</span> <span style="color:#719e07">+</span> getMethodName<span style="color:#719e07">());</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">}</span> |
| </span></span><span style="display:flex;"><span> pts <span style="color:#719e07">=</span> ReflectUtils<span style="color:#719e07">.</span>desc2classArray<span style="color:#719e07">(</span>desc<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">}</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">......</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">}</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span> |
| </span></span></code></pre></div><ul> |
| <li>查看 <code>MethodDescriptor</code>，即找方法是否存在，存在的话就会设置好 <code>ParameterClasses</code></li> |
| <li>如果上面没找到，<code>pts == DubboCodec.EMPTY_CLASS_ARRAY</code> 就会满足条件，进而判断是否是泛化调用或者是 echo 调用，如果都不是则报服务找不到方法错误</li> |
| <li>desc 是 <code>Ljava/lang/Object</code> ，很明显并没有参数是 Object 的方法，所以必然是会报错的</li> |
| </ul> |
| <p>补充说明：</p> |
| <p><strong>方法查询</strong></p> |
| <p>代码如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#268bd2">public</span> MethodDescriptor <span style="color:#268bd2">getMethod</span><span style="color:#719e07">(</span>String methodName<span style="color:#719e07">,</span> String params<span style="color:#719e07">)</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> Map<span style="color:#719e07">&lt;</span>String<span style="color:#719e07">,</span> MethodDescriptor<span style="color:#719e07">&gt;</span> methods <span style="color:#719e07">=</span> descToMethods<span style="color:#719e07">.</span>get<span style="color:#719e07">(</span>methodName<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#719e07">(</span>CollectionUtils<span style="color:#719e07">.</span>isNotEmptyMap<span style="color:#719e07">(</span>methods<span style="color:#719e07">))</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> methods<span style="color:#719e07">.</span>get<span style="color:#719e07">(</span>params<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">}</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">null</span><span style="color:#719e07">;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span> |
| </span></span></code></pre></div><p>优点：</p> |
| <p>比之前的版本加了方法的元信息缓存起来，不使用反射可以提高效率，可以理解用空间换时间。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/from-scratch/dfsa01.jpg" alt="dfsa01.jpg"></p> |
| <h3 id="24-解决问题">2.4 解决问题</h3> |
| <blockquote> |
| <p>因为直接撸代码并 hold 不住，所以通过比较来查看问题所在。</p> |
| </blockquote> |
| <h4 id="241-启动-dubbo-服务消费者">2.4.1 启动 dubbo 服务消费者</h4> |
| <p>通过 api 模式启动，参考官方例子。启动这个是为了查看 Java 版本的传输内容。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#268bd2">public</span> <span style="color:#268bd2">static</span> <span style="color:#dc322f">void</span> <span style="color:#268bd2">main</span><span style="color:#719e07">(</span>String<span style="color:#719e07">[]</span> args<span style="color:#719e07">)</span> <span style="color:#268bd2">throws</span> InterruptedException <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 当前应用配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ApplicationConfig application <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> ApplicationConfig<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> application<span style="color:#719e07">.</span>setName<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;demoProvider2&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 连接注册中心配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> RegistryConfig registry <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> RegistryConfig<span style="color:#719e07">();</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setAddress<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;127.0.0.1:2181&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setProtocol<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;zookeeper&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setUsername<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> registry<span style="color:#719e07">.</span>setPassword<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 注意：ReferenceConfig为重对象，内部封装了与注册中心的连接，以及与服务提供方的连接 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 引用远程服务 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ReferenceConfig<span style="color:#719e07">&lt;</span>DemoService<span style="color:#719e07">&gt;</span> reference |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">=</span> <span style="color:#719e07">new</span> ReferenceConfig<span style="color:#719e07">&lt;&gt;();</span> <span style="color:#586e75">// 此实例很重，封装了与注册中心的连接以及与提供者的连接，请自行缓存，否则可能造成内存和连接泄漏 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> reference<span style="color:#719e07">.</span>setApplication<span style="color:#719e07">(</span>application<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> reference<span style="color:#719e07">.</span>setRegistry<span style="color:#719e07">(</span>registry<span style="color:#719e07">);</span> <span style="color:#586e75">// 多个注册中心可以用setRegistries() |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> reference<span style="color:#719e07">.</span>setInterface<span style="color:#719e07">(</span>DemoService<span style="color:#719e07">.</span>class<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> reference<span style="color:#719e07">.</span>setVersion<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;1.0.0&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> reference<span style="color:#719e07">.</span>setGroup<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;tc&#34;</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> reference<span style="color:#719e07">.</span>setCheck<span style="color:#719e07">(</span><span style="color:#cb4b16">true</span><span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> reference<span style="color:#719e07">.</span>setTimeout<span style="color:#719e07">(</span>1000 <span style="color:#719e07">*</span> 60<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 和本地bean一样使用xxxService |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> DemoService demoService <span style="color:#719e07">=</span> reference<span style="color:#719e07">.</span>get<span style="color:#719e07">();</span> <span style="color:#586e75">// 注意：此代理对象内部封装了所有通讯细节，对象较重，请缓存复用 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> System<span style="color:#719e07">.</span>out<span style="color:#719e07">.</span>println<span style="color:#719e07">(</span>demoService<span style="color:#719e07">.</span>sayHello<span style="color:#719e07">(</span><span style="color:#719e07">new</span> User<span style="color:#719e07">(</span><span style="color:#2aa198">&#34;tc&#34;</span><span style="color:#719e07">,</span> 18<span style="color:#719e07">)));</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> TimeUnit<span style="color:#719e07">.</span>MINUTES<span style="color:#719e07">.</span>sleep<span style="color:#719e07">(</span>10<span style="color:#719e07">);</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span> |
| </span></span></code></pre></div><p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/from-scratch/dfsa02.png" alt="dfsa02.png"></p> |
| <p>desc 肉眼可见的是 <code>Lcom/funnycode/User</code>，这个就是正确的对象了。</p> |
| <h4 id="242-查找-dubbogo-为什么不对">2.4.2 查找 dubbogo 为什么不对</h4> |
| <p>代码位置：</p> |
| <p><code>protocol/dubbo/impl/hessian.go:120#marshalRequest</code></p> |
| <p>代码实现：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">marshalRequest</span>(encoder <span style="color:#719e07">*</span>hessian.Encoder, p DubboPackage) ([]<span style="color:#dc322f">byte</span>, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> service <span style="color:#719e07">:=</span> p.Service |
| </span></span><span style="display:flex;"><span> request <span style="color:#719e07">:=</span> <span style="color:#268bd2">EnsureRequestPayload</span>(p.Body) |
| </span></span><span style="display:flex;"><span> encoder.<span style="color:#268bd2">Encode</span>(DEFAULT_DUBBO_PROTOCOL_VERSION) |
| </span></span><span style="display:flex;"><span> encoder.<span style="color:#268bd2">Encode</span>(service.Path) |
| </span></span><span style="display:flex;"><span> encoder.<span style="color:#268bd2">Encode</span>(service.Version) |
| </span></span><span style="display:flex;"><span> encoder.<span style="color:#268bd2">Encode</span>(service.Method) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> args, ok <span style="color:#719e07">:=</span> request.Params.([]<span style="color:#268bd2">interface</span>{}) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> !ok { |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;request args are: %+v&#34;</span>, request.Params) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span>, perrors.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;@params is not of type: []interface{}&#34;</span>) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> types, err <span style="color:#719e07">:=</span> <span style="color:#268bd2">getArgsTypeList</span>(args) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span>, perrors.<span style="color:#268bd2">Wrapf</span>(err, <span style="color:#2aa198">&#34; PackRequest(args:%+v)&#34;</span>, args) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> encoder.<span style="color:#268bd2">Encode</span>(types) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> _, v <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> args { |
| </span></span><span style="display:flex;"><span> encoder.<span style="color:#268bd2">Encode</span>(v) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">......</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>断点可以发现，types 返回的时候就已经是 <code>Object</code> 了，没有返回 <code>User</code>，那么继续跟进去查看代码。</p> |
| <ul> |
| <li><code>protocol/dubbo/impl/hessian.go:394#getArgsTypeList</code></li> |
| <li><code>protocol/dubbo/impl/hessian.go:418#getArgType</code></li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">getArgType</span>(v <span style="color:#268bd2">interface</span>{}) <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 常见的类型处理 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">......</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">default</span>: |
| </span></span><span style="display:flex;"><span> t <span style="color:#719e07">:=</span> reflect.<span style="color:#268bd2">TypeOf</span>(v) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> reflect.Ptr <span style="color:#719e07">==</span> t.<span style="color:#268bd2">Kind</span>() { |
| </span></span><span style="display:flex;"><span> t = reflect.<span style="color:#268bd2">TypeOf</span>(reflect.<span style="color:#268bd2">ValueOf</span>(v).<span style="color:#268bd2">Elem</span>()) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">switch</span> t.<span style="color:#268bd2">Kind</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> reflect.Struct: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;java.lang.Object&#34;</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">......</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>很明显当发现是 <code>reflect.Struct</code> 的时候就返回了 <code>java.lang.Object</code>，所以参数就变成了 <code>Object</code>，那么因为 Java 代码那边依赖这个类型所以就调用失败了。</p> |
| <h4 id="243-其它版本验证">2.4.3 其它版本验证</h4> |
| <p>因为反馈是 2.7.7 出错，所以先考虑到在之前的版本是否功能正常，于是把服务提供者切换到 dubbo 2.7.3，发现调用仍然有错误，如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>2020-12-02T21:52:25.945+0800 INFO getty/listener.go:85 session<span style="color:#719e07">{</span>session session-closed, Read Bytes: 4586, Write Bytes: 232, Read Pkgs: 0, Write Pkgs: 1<span style="color:#719e07">}</span> got error<span style="color:#719e07">{</span>java exception:org.apache.dubbo.rpc.RpcException: Failed to invoke remote proxy method sayHello to registry://127.0.0.1:2181/org.apache.dubbo.registry.RegistryService?application<span style="color:#719e07">=</span>demoProvider&amp;<span style="color:#268bd2">dubbo</span><span style="color:#719e07">=</span>2.0.2&amp;<span style="color:#268bd2">export</span><span style="color:#719e07">=</span>dubbo%3A%2F%2F192.168.0.113%3A12345%2Fcom.funnycode.DemoService%3Fanyhost%3Dtrue%26application%3DdemoProvider%26bind.ip%3D192.168.0.113%26bind.port%3D12345%26deprecated%3Dfalse%26dubbo%3D2.0.2%26dynamic%3Dtrue%26generic%3Dfalse%26group%3Dtc%26interface%3Dcom.funnycode.DemoService%26methods%3DsayHello%26pid%3D23889%26register%3Dtrue%26release%3D2.7.3%26revision%3D1.0.0%26side%3Dprovider%26threads%3D200%26timeout%3D60000%26timestamp%3D1606916702204%26version%3D1.0.0&amp;<span style="color:#268bd2">pid</span><span style="color:#719e07">=</span>23889&amp;<span style="color:#268bd2">registry</span><span style="color:#719e07">=</span>zookeeper&amp;<span style="color:#268bd2">release</span><span style="color:#719e07">=</span>2.7.3&amp;<span style="color:#268bd2">timestamp</span><span style="color:#719e07">=</span>1606916702193, cause: Not found method <span style="color:#2aa198">&#34;sayHello&#34;</span> in class com.funnycode.DemoServiceImpl. |
| </span></span><span style="display:flex;"><span>org.apache.dubbo.rpc.RpcException: Failed to invoke remote proxy method sayHello to registry://127.0.0.1:2181/org.apache.dubbo.registry.RegistryService?application<span style="color:#719e07">=</span>demoProvider&amp;<span style="color:#268bd2">dubbo</span><span style="color:#719e07">=</span>2.0.2&amp;<span style="color:#268bd2">export</span><span style="color:#719e07">=</span>dubbo%3A%2F%2F192.168.0.113%3A12345%2Fcom.funnycode.DemoService%3Fanyhost%3Dtrue%26application%3DdemoProvider%26bind.ip%3D192.168.0.113%26bind.port%3D12345%26deprecated%3Dfalse%26dubbo%3D2.0.2%26dynamic%3Dtrue%26generic%3Dfalse%26group%3Dtc%26interface%3Dcom.funnycode.DemoService%26methods%3DsayHello%26pid%3D23889%26register%3Dtrue%26release%3D2.7.3%26revision%3D1.0.0%26side%3Dprovider%26threads%3D200%26timeout%3D60000%26timestamp%3D1606916702204%26version%3D1.0.0&amp;<span style="color:#268bd2">pid</span><span style="color:#719e07">=</span>23889&amp;<span style="color:#268bd2">registry</span><span style="color:#719e07">=</span>zookeeper&amp;<span style="color:#268bd2">release</span><span style="color:#719e07">=</span>2.7.3&amp;<span style="color:#268bd2">timestamp</span><span style="color:#719e07">=</span>1606916702193, cause: Not found method <span style="color:#2aa198">&#34;sayHello&#34;</span> in class com.funnycode.DemoServiceImpl. |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke<span style="color:#719e07">(</span>AbstractProxyInvoker.java:107<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.config.invoker.DelegateProviderMetaDataInvoker.invoke<span style="color:#719e07">(</span>DelegateProviderMetaDataInvoker.java:56<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.InvokerWrapper.invoke<span style="color:#719e07">(</span>InvokerWrapper.java:56<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.filter.ExceptionFilter.invoke<span style="color:#719e07">(</span>ExceptionFilter.java:55<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.monitor.support.MonitorFilter.invoke<span style="color:#719e07">(</span>MonitorFilter.java:92<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.filter.TimeoutFilter.invoke<span style="color:#719e07">(</span>TimeoutFilter.java:48<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.dubbo.filter.TraceFilter.invoke<span style="color:#719e07">(</span>TraceFilter.java:81<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.filter.ContextFilter.invoke<span style="color:#719e07">(</span>ContextFilter.java:96<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.filter.GenericFilter.invoke<span style="color:#719e07">(</span>GenericFilter.java:148<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.filter.ClassLoaderFilter.invoke<span style="color:#719e07">(</span>ClassLoaderFilter.java:38<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.filter.EchoFilter.invoke<span style="color:#719e07">(</span>EchoFilter.java:41<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$1</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:82<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper<span style="color:#268bd2">$CallbackRegistrationInvoker</span>.invoke<span style="color:#719e07">(</span>ProtocolFilterWrapper.java:157<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol<span style="color:#268bd2">$1</span>.reply<span style="color:#719e07">(</span>DubboProtocol.java:152<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest<span style="color:#719e07">(</span>HeaderExchangeHandler.java:102<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received<span style="color:#719e07">(</span>HeaderExchangeHandler.java:193<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.DecodeHandler.received<span style="color:#719e07">(</span>DecodeHandler.java:51<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run<span style="color:#719e07">(</span>ChannelEventRunnable.java:57<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.util.concurrent.ThreadPoolExecutor.runWorker<span style="color:#719e07">(</span>ThreadPoolExecutor.java:1149<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.util.concurrent.ThreadPoolExecutor<span style="color:#268bd2">$Worker</span>.run<span style="color:#719e07">(</span>ThreadPoolExecutor.java:624<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.lang.Thread.run<span style="color:#719e07">(</span>Thread.java:748<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span>Caused by: org.apache.dubbo.common.bytecode.NoSuchMethodException: Not found method <span style="color:#2aa198">&#34;sayHello&#34;</span> in class com.funnycode.DemoServiceImpl. |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.common.bytecode.Wrapper1.invokeMethod<span style="color:#719e07">(</span>Wrapper1.java<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.proxy.javassist.JavassistProxyFactory<span style="color:#268bd2">$1</span>.doInvoke<span style="color:#719e07">(</span>JavassistProxyFactory.java:47<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke<span style="color:#719e07">(</span>AbstractProxyInvoker.java:84<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> ... <span style="color:#2aa198">27</span> more |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span>, will be closed. |
| </span></span></code></pre></div><p>虽然和 2.7.7 的代码是不一样的，但是通过错误也能看出来是在代理增强类里面方法找不到，大概率是反射找不到方法，所以归根结底也是参数的问题。</p> |
| <h4 id="244-修复问题">2.4.4 修复问题</h4> |
| <p>修复相对简单，就是拿到 <code>struct</code> 定义的 <code>JavaClassName</code>。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">case</span> reflect.Struct: |
| </span></span><span style="display:flex;"><span> v, ok <span style="color:#719e07">:=</span> v.(hessian.POJO) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> ok { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> v.<span style="color:#268bd2">JavaClassName</span>() |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;java.lang.Object&#34;</span> |
| </span></span></code></pre></div><h4 id="243-验证结果">2.4.3 验证结果</h4> |
| <p>再次执行消费者，运行（提供方 2.7.7 和 2.7.3）正常，输出如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#719e07">[</span>2020-12-03/20:04:06 main.main: client.go: 29<span style="color:#719e07">]</span> response result: Hello tc |
| </span></span><span style="display:flex;"><span>... |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">[</span>2020-12-03/20:04:09 main.main: client.go: 41<span style="color:#719e07">]</span> response result: Hello tc You are <span style="color:#2aa198">18</span> |
| </span></span><span style="display:flex;"><span>... |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">[</span>2020-12-03/20:04:09 main.main: client.go: 48<span style="color:#719e07">]</span> response result: Hello tc You are <span style="color:#2aa198">18</span> |
| </span></span></code></pre></div><h2 id="三细节叨叨">三、细节叨叨</h2> |
| <h3 id="31-如何配置-dubbgo-消费者">3.1 如何配置 dubbgo 消费者</h3> |
| <p>细心的你是否已经发现，在我 dubbogo 的消费端接口叫 <code>DemoProvider</code>，然后发现提供者叫 <code>DemoService</code>，这个又是如何正常运行的？</p> |
| <p>实际上和 <code>client.yml</code> 中配置项 <code>references</code> 有关，在配置文件详细说明了 <code>interface</code>，<code>version</code>，<code>group</code> 等，你还可以通过 methods 配置方法的超时时间等信息。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">references</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;DemoProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 可以指定多个registry，使用逗号隔开;不指定默认向所有注册中心注册</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;zk1&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface</span>: <span style="color:#2aa198">&#34;com.funnycode.DemoService&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">version</span>: <span style="color:#2aa198">&#34;1.0.0&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">group</span>: <span style="color:#2aa198">&#34;tc&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;SayHello&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">3</span> |
| </span></span><span style="display:flex;"><span> ...... |
| </span></span></code></pre></div><h3 id="32-全局的-group-和-version-怎么配置">3.2 全局的 group 和 version 怎么配置</h3> |
| <p>配置文件如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#586e75"># application config</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">application</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">organization</span>: <span style="color:#2aa198">&#34;dubbogoproxy.com&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;Demo Micro Service&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">module</span>: <span style="color:#2aa198">&#34;dubbogoproxy tc client&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">version</span>: <span style="color:#2aa198">&#34;1.0.0&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">group</span>: <span style="color:#2aa198">&#34;tc&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">owner</span>: <span style="color:#2aa198">&#34;ZX&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">environment</span>: <span style="color:#2aa198">&#34;dev&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">references</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;DemoProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 可以指定多个registry，使用逗号隔开;不指定默认向所有注册中心注册</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;zk1&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface</span>: <span style="color:#2aa198">&#34;com.funnycode.DemoService&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75"># version: &#34;1.0.0&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75"># group: &#34;tc&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;SayHello&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">3</span> |
| </span></span></code></pre></div><p>从使用的习惯来讲，肯定是 <code>application</code> 表示了全局的配置，但是我发现启动的时候在 <code>application</code> 配置的 <code>version</code> 和 <code>group</code> 并不会赋值给接口，启动会报服务提供方找不到，如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>2020-12-03T20:15:42.208+0800 DEBUG zookeeper/registry.go:237 Create a zookeeper node:/dubbo/com.funnycode.DemoService/consumers/consumer%3A%2F%2F30.11.176.107%2FDemoProvider%3Fapp.version%3D1.0.0%26application%3DDemo+Micro+Service%26async%3Dfalse%26bean.name%3DDemoProvider%26cluster%3Dfailover%26environment%3Ddev%26generic%3Dfalse%26group%3D%26interface%3Dcom.funnycode.DemoService%26ip%3D30.11.176.107%26loadbalance%3D%26methods.SayHello.loadbalance%3D%26methods.SayHello.retries%3D3%26methods.SayHello.sticky%3Dfalse%26module%3Ddubbogoproxy+tc+client%26name%3DDemo+Micro+Service%26organization%3Ddubbogoproxy.com%26owner%3DZX%26pid%3D38692%26protocol%3Ddubbo%26provided-by%3D%26reference.filter%3Dcshutdown%26registry.role%3D0%26release%3Ddubbo-golang-1.3.0%26retries%3D%26side%3Dconsumer%26sticky%3Dfalse%26timestamp%3D1606997742%26version%3D |
| </span></span></code></pre></div><p><code>version</code> 和 <code>group</code> 都是空。必须把 <code>DemoProvider</code> 下的 <code>version</code> 和 <code>group</code> 注释打开。</p> |
| <h3 id="33-怎么指定调用的方法名">3.3 怎么指定调用的方法名</h3> |
| <h4 id="331-go-调用-java">3.3.1 go 调用 java</h4> |
| <p>dubbogo 调用 dubbo，因为 go 是大写的方法名，java 里面是小写的方法名，所以会出现如下错误：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>2020-12-02T17:10:47.739+0800 INFO getty/listener.go:87 session<span style="color:#719e07">{</span>session session-closed, Read Bytes: 924, Write Bytes: 199, Read Pkgs: 0, Write Pkgs: 1<span style="color:#719e07">}</span> got error<span style="color:#719e07">{</span>java exception:Fail to decode request due to: java.lang.IllegalArgumentException: Service not found:com.funnycode.DemoService, SayHello |
| </span></span><span style="display:flex;"><span>java.lang.IllegalArgumentException: Service not found:com.funnycode.DemoService, SayHello |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode<span style="color:#719e07">(</span>DecodeableRpcInvocation.java:134<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode<span style="color:#719e07">(</span>DecodeableRpcInvocation.java:80<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.DecodeHandler.decode<span style="color:#719e07">(</span>DecodeHandler.java:57<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.DecodeHandler.received<span style="color:#719e07">(</span>DecodeHandler.java:44<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run<span style="color:#719e07">(</span>ChannelEventRunnable.java:57<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.util.concurrent.ThreadPoolExecutor.runWorker<span style="color:#719e07">(</span>ThreadPoolExecutor.java:1149<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.util.concurrent.ThreadPoolExecutor<span style="color:#268bd2">$Worker</span>.run<span style="color:#719e07">(</span>ThreadPoolExecutor.java:624<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> at java.lang.Thread.run<span style="color:#719e07">(</span>Thread.java:748<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span>, will be closed. |
| </span></span></code></pre></div><p>细心的读者可能已经注意到了，我在消费端的接口声明是有个 <code>dubbo:&quot;sayHello&quot;</code> 的，表示方法名是 sayHello，这样在服务提供方就可以得到 sayHello 这个方法名。</p> |
| <p>还有我声明的三个方法都指明它们的方法名叫 <code>dubbo:&quot;sayHello&quot;</code>，这是因为 Java 可以方法名字一样进行重载，而 go 是不能方法名重复的。</p> |
| <h4 id="332-go-调用-go">3.3.2 go 调用 go</h4> |
| <blockquote> |
| <p>直接贴能跑通的代码</p> |
| </blockquote> |
| <p>我的提供者接口：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> DemoProvider <span style="color:#268bd2">struct</span>{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>DemoProvider) <span style="color:#268bd2">SayHello</span>(ctx context.Context, name <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;Hello &#34;</span> <span style="color:#719e07">+</span> name, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>DemoProvider) <span style="color:#268bd2">SayHello4</span>(ctx context.Context, user <span style="color:#719e07">*</span>User) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;Hello &#34;</span> <span style="color:#719e07">+</span> user.Name <span style="color:#719e07">+</span> <span style="color:#2aa198">&#34; You are &#34;</span> <span style="color:#719e07">+</span> strconv.<span style="color:#268bd2">Itoa</span>(user.Age), <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>DemoProvider) <span style="color:#268bd2">SayHello5</span>(ctx context.Context, user <span style="color:#719e07">*</span>User, name <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;Hello &#34;</span> <span style="color:#719e07">+</span> name <span style="color:#719e07">+</span> <span style="color:#2aa198">&#34; You are &#34;</span> <span style="color:#719e07">+</span> strconv.<span style="color:#268bd2">Itoa</span>(user.Age), <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>DemoProvider) <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;DemoProvider&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>DemoProvider) <span style="color:#268bd2">MethodMapper</span>() <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#dc322f">string</span>{ |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;SayHello&#34;</span>: <span style="color:#2aa198">&#34;sayHello&#34;</span>, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>我的消费者接口：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> DemoProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 调用 java 和 go |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> SayHello <span style="color:#268bd2">func</span>(ctx context.Context, name <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) <span style="color:#2aa198">`dubbo:&#34;sayHello&#34;`</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 只调用 java |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> SayHello2 <span style="color:#268bd2">func</span>(ctx context.Context, user <span style="color:#719e07">*</span>User) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) <span style="color:#2aa198">`dubbo:&#34;sayHello&#34;`</span> |
| </span></span><span style="display:flex;"><span> SayHello3 <span style="color:#268bd2">func</span>(ctx context.Context, user <span style="color:#719e07">*</span>User, name <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) <span style="color:#2aa198">`dubbo:&#34;sayHello&#34;`</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 只调用 go |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> SayHello4 <span style="color:#268bd2">func</span>(ctx context.Context, user <span style="color:#719e07">*</span>User) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span> SayHello5 <span style="color:#268bd2">func</span>(ctx context.Context, user <span style="color:#719e07">*</span>User, name <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>启动服务消费者：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;\n\n\nstart to test dubbo&#34;</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> res, err <span style="color:#719e07">:=</span> demoProvider.<span style="color:#268bd2">SayHello</span>(context.<span style="color:#268bd2">TODO</span>(), <span style="color:#2aa198">&#34;tc&#34;</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, res) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> user <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>User{ |
| </span></span><span style="display:flex;"><span> Name: <span style="color:#2aa198">&#34;tc&#34;</span>, |
| </span></span><span style="display:flex;"><span> Age: <span style="color:#2aa198">18</span>, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> res, err = demoProvider.<span style="color:#268bd2">SayHello4</span>(context.<span style="color:#268bd2">TODO</span>(), user) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, res) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> res, err = demoProvider.<span style="color:#268bd2">SayHello5</span>(context.<span style="color:#268bd2">TODO</span>(), user, <span style="color:#2aa198">&#34;tc&#34;</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> gxlog.<span style="color:#268bd2">CInfo</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, res) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">initSignal</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>这里需要注意 <code>MethodMapper</code> 方法，有时候需要在这个方法中配置方法名的映射关系，否则还是会出现找不到方法的错误。</p> |
| <p>比如因为配置 <code>dubbo:&quot;sayHello&quot;</code> ，所以在 go 里面请求 <code>SayHello</code> 变成了 <code>sayHello</code>，那么服务提供方通过 <code>MethodMapper</code> 方法配置后使得提供方也是 <code>sayHello</code>，这样 go 和 java 下暴露的都是小写的 <code>sayHello</code>。</p> |
| <h3 id="34-为什么会用-hessian2">3.4 为什么会用 hessian2</h3> |
| <p>老司机都懂，在 dubbo 中 SPI 机制的默认值就是 hessian2</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-java" data-lang="java"><span style="display:flex;"><span><span style="color:#268bd2">@SPI</span><span style="color:#719e07">(</span><span style="color:#2aa198">&#34;hessian2&#34;</span><span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">public</span> <span style="color:#268bd2">interface</span> <span style="color:#268bd2">Serialization</span> <span style="color:#719e07">{</span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">}</span> |
| </span></span></code></pre></div><p>而在 dubbo-go 中：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">NewDubboCodec</span>(reader <span style="color:#719e07">*</span>bufio.Reader) <span style="color:#719e07">*</span>ProtocolCodec { |
| </span></span><span style="display:flex;"><span> s, _ <span style="color:#719e07">:=</span> <span style="color:#268bd2">GetSerializerById</span>(constant.S_Hessian2) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>ProtocolCodec{ |
| </span></span><span style="display:flex;"><span> reader: reader, |
| </span></span><span style="display:flex;"><span> pkgType: <span style="color:#2aa198">0</span>, |
| </span></span><span style="display:flex;"><span> bodyLen: <span style="color:#2aa198">0</span>, |
| </span></span><span style="display:flex;"><span> headerRead: <span style="color:#cb4b16">false</span>, |
| </span></span><span style="display:flex;"><span> serializer: s.(Serializer), |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="35-hessian序列化源码">3.5 hessian序列化源码</h3> |
| <blockquote> |
| <p>可以自行断点查看，两边基本上一样，我也是通过两边比出来的，RpcInvocation.getParameterTypesDesc() 就是方法的参数</p> |
| </blockquote> |
| <ul> |
| <li>go 代码 <code>protocol/dubbo/impl/hessian.go:120#marshalRequest</code></li> |
| <li>java 代码 <code>org.apache.dubbo.rpc.protocol.dubbo.DubboCodec#encodeRequestData(org.apache.dubbo.remoting.Channel, org.apache.dubbo.common.serialize.ObjectOutput, java.lang.Object, java.lang.String)</code></li> |
| </ul> |
| <h3 id="36-dubbogo-服务提供者的方法对象需要是指针对象">3.6 dubbogo 服务提供者的方法对象需要是指针对象</h3> |
| <p>之前的例子都是 copy 的，这次是纯手打的，才发现了这个问题。</p> |
| <p>如果你的提供类似：<code>func (p *DemoProvider) SayHello4(ctx context.Context, user User) (string, error)</code>，那么会出现如下错误：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>2020-12-03T12:42:32.834+0800 ERROR getty/listener.go:280 OnMessage panic: reflect: Call using *main.User as <span style="color:#b58900">type</span> main.User |
| </span></span><span style="display:flex;"><span>github.com/apache/dubbo-go/remoting/getty.<span style="color:#719e07">(</span>*RpcServerHandler<span style="color:#719e07">)</span>.OnMessage.func1 |
| </span></span></code></pre></div><p>参数里面的 <code>User</code> 需要改成 <code>*User</code>。</p> |
| <h3 id="37-dubbogo-服务消费者的方法对象可以是非指针对象">3.7 dubbogo 服务消费者的方法对象可以是非指针对象</h3> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span>SayHello4 <span style="color:#268bd2">func</span>(ctx context.Context, user <span style="color:#719e07">*</span>User) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// or |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>SayHello4 <span style="color:#268bd2">func</span>(ctx context.Context, user User) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) |
| </span></span></code></pre></div><p>因为在参数序列化的时候会对指针做操作：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span>t <span style="color:#719e07">:=</span> reflect.<span style="color:#268bd2">TypeOf</span>(v) |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">if</span> reflect.Ptr <span style="color:#719e07">==</span> t.<span style="color:#268bd2">Kind</span>() { |
| </span></span><span style="display:flex;"><span> t = reflect.<span style="color:#268bd2">TypeOf</span>(reflect.<span style="color:#268bd2">ValueOf</span>(v).<span style="color:#268bd2">Elem</span>()) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><a href="https://github.com/apache/dubbo-go/blob/v1.5.4/protocol/dubbo/impl/hessian.go#L486">完整代码</a></p> |
| <h3 id="38-配置文件说明">3.8 配置文件说明</h3> |
| <p>dubbogo 主要有三个配置文件：</p> |
| <ul> |
| <li>server.yaml 服务提供方的配置文件</li> |
| <li>client.yaml 服务消费方的配置文件</li> |
| <li>log.yaml 日志文件</li> |
| </ul> |
| <p>如果你什么都不配置，会出现：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>2021/01/11 15:31:41 <span style="color:#719e07">[</span>InitLog<span style="color:#719e07">]</span> warn: log configure file name is nil |
| </span></span><span style="display:flex;"><span>2021/01/11 15:31:41 <span style="color:#719e07">[</span>consumerInit<span style="color:#719e07">]</span> application configure<span style="color:#719e07">(</span>consumer<span style="color:#719e07">)</span> file name is nil |
| </span></span><span style="display:flex;"><span>2021/01/11 15:31:41 <span style="color:#719e07">[</span>providerInit<span style="color:#719e07">]</span> application configure<span style="color:#719e07">(</span>provider<span style="color:#719e07">)</span> file name is nil |
| </span></span></code></pre></div><p>这样是没法正常使用的。如果你是服务提供方，必须要配置 server.yaml 文件，如果你是服务消费方，必须要配置 client.yaml，实际我们的应用应该既是消费者又是提供者，所以往往两个文件都是需要配置的。</p> |
| <p>服务提供方正常启动是会有如下输出的：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>2021-01-11T15:36:55.003+0800 INFO protocol/protocol.go:205 The cached exporter keys is dubbo://:20000/DemoProvider?accesslog<span style="color:#719e07">=</span>&amp;app.version<span style="color:#719e07">=</span>1.0.0&amp;<span style="color:#268bd2">application</span><span style="color:#719e07">=</span>Demo+Micro+Service&amp;<span style="color:#268bd2">auth</span><span style="color:#719e07">=</span>&amp;bean.name<span style="color:#719e07">=</span>DemoProvider&amp;<span style="color:#268bd2">cluster</span><span style="color:#719e07">=</span>failover&amp;<span style="color:#268bd2">environment</span><span style="color:#719e07">=</span>dev&amp;execute.limit<span style="color:#719e07">=</span>&amp;execute.limit.rejected.handler<span style="color:#719e07">=</span>&amp;<span style="color:#268bd2">group</span><span style="color:#719e07">=</span>tc&amp;<span style="color:#268bd2">interface</span><span style="color:#719e07">=</span>com.funnycode.DemoService&amp;<span style="color:#268bd2">loadbalance</span><span style="color:#719e07">=</span>random&amp;methods.SayHello.loadbalance<span style="color:#719e07">=</span>random&amp;methods.SayHello.retries<span style="color:#719e07">=</span>3&amp;methods.SayHello.tps.limit.interval<span style="color:#719e07">=</span>&amp;methods.SayHello.tps.limit.rate<span style="color:#719e07">=</span>&amp;methods.SayHello.tps.limit.strategy<span style="color:#719e07">=</span>&amp;methods.SayHello.weight<span style="color:#719e07">=</span>0&amp;methods.SayHello4.loadbalance<span style="color:#719e07">=</span>random&amp;methods.SayHello4.retries<span style="color:#719e07">=</span>3&amp;methods.SayHello4.tps.limit.interval<span style="color:#719e07">=</span>&amp;methods.SayHello4.tps.limit.rate<span style="color:#719e07">=</span>&amp;methods.SayHello4.tps.limit.strategy<span style="color:#719e07">=</span>&amp;methods.SayHello4.weight<span style="color:#719e07">=</span>0&amp;methods.SayHello5.loadbalance<span style="color:#719e07">=</span>random&amp;methods.SayHello5.retries<span style="color:#719e07">=</span>3&amp;methods.SayHello5.tps.limit.interval<span style="color:#719e07">=</span>&amp;methods.SayHello5.tps.limit.rate<span style="color:#719e07">=</span>&amp;methods.SayHello5.tps.limit.strategy<span style="color:#719e07">=</span>&amp;methods.SayHello5.weight<span style="color:#719e07">=</span>0&amp;<span style="color:#268bd2">module</span><span style="color:#719e07">=</span>dubbogoproxy+tc+client&amp;<span style="color:#268bd2">name</span><span style="color:#719e07">=</span>Demo+Micro+Service&amp;<span style="color:#268bd2">organization</span><span style="color:#719e07">=</span>dubbogoproxy.com&amp;<span style="color:#268bd2">owner</span><span style="color:#719e07">=</span>ZX&amp;param.sign<span style="color:#719e07">=</span>&amp;registry.role<span style="color:#719e07">=</span>3&amp;<span style="color:#268bd2">release</span><span style="color:#719e07">=</span>dubbo-golang-1.3.0&amp;<span style="color:#268bd2">retries</span><span style="color:#719e07">=</span>&amp;<span style="color:#268bd2">serialization</span><span style="color:#719e07">=</span>&amp;service.filter<span style="color:#719e07">=</span>echo%2Ctoken%2Caccesslog%2Ctps%2Cgeneric_service%2Cexecute%2Cpshutdown&amp;<span style="color:#268bd2">side</span><span style="color:#719e07">=</span>provider&amp;ssl-enabled<span style="color:#719e07">=</span>false&amp;<span style="color:#268bd2">timestamp</span><span style="color:#719e07">=</span>1610350614&amp;tps.limit.interval<span style="color:#719e07">=</span>&amp;tps.limit.rate<span style="color:#719e07">=</span>&amp;tps.limit.rejected.handler<span style="color:#719e07">=</span>&amp;tps.limit.strategy<span style="color:#719e07">=</span>&amp;tps.limiter<span style="color:#719e07">=</span>&amp;<span style="color:#268bd2">version</span><span style="color:#719e07">=</span>1.0.0&amp;<span style="color:#268bd2">warmup</span><span style="color:#719e07">=</span>100! |
| </span></span><span style="display:flex;"><span>2021-01-11T15:36:55.003+0800 INFO dubbo/dubbo_protocol.go:86 Export service: dubbo://:20000/DemoProvider?accesslog<span style="color:#719e07">=</span>&amp;app.version<span style="color:#719e07">=</span>1.0.0&amp;<span style="color:#268bd2">application</span><span style="color:#719e07">=</span>Demo+Micro+Service&amp;<span style="color:#268bd2">auth</span><span style="color:#719e07">=</span>&amp;bean.name<span style="color:#719e07">=</span>DemoProvider&amp;<span style="color:#268bd2">cluster</span><span style="color:#719e07">=</span>failover&amp;<span style="color:#268bd2">environment</span><span style="color:#719e07">=</span>dev&amp;execute.limit<span style="color:#719e07">=</span>&amp;execute.limit.rejected.handler<span style="color:#719e07">=</span>&amp;<span style="color:#268bd2">group</span><span style="color:#719e07">=</span>tc&amp;<span style="color:#268bd2">interface</span><span style="color:#719e07">=</span>com.funnycode.DemoService&amp;<span style="color:#268bd2">loadbalance</span><span style="color:#719e07">=</span>random&amp;methods.SayHello.loadbalance<span style="color:#719e07">=</span>random&amp;methods.SayHello.retries<span style="color:#719e07">=</span>3&amp;methods.SayHello.tps.limit.interval<span style="color:#719e07">=</span>&amp;methods.SayHello.tps.limit.rate<span style="color:#719e07">=</span>&amp;methods.SayHello.tps.limit.strategy<span style="color:#719e07">=</span>&amp;methods.SayHello.weight<span style="color:#719e07">=</span>0&amp;methods.SayHello4.loadbalance<span style="color:#719e07">=</span>random&amp;methods.SayHello4.retries<span style="color:#719e07">=</span>3&amp;methods.SayHello4.tps.limit.interval<span style="color:#719e07">=</span>&amp;methods.SayHello4.tps.limit.rate<span style="color:#719e07">=</span>&amp;methods.SayHello4.tps.limit.strategy<span style="color:#719e07">=</span>&amp;methods.SayHello4.weight<span style="color:#719e07">=</span>0&amp;methods.SayHello5.loadbalance<span style="color:#719e07">=</span>random&amp;methods.SayHello5.retries<span style="color:#719e07">=</span>3&amp;methods.SayHello5.tps.limit.interval<span style="color:#719e07">=</span>&amp;methods.SayHello5.tps.limit.rate<span style="color:#719e07">=</span>&amp;methods.SayHello5.tps.limit.strategy<span style="color:#719e07">=</span>&amp;methods.SayHello5.weight<span style="color:#719e07">=</span>0&amp;<span style="color:#268bd2">module</span><span style="color:#719e07">=</span>dubbogoproxy+tc+client&amp;<span style="color:#268bd2">name</span><span style="color:#719e07">=</span>Demo+Micro+Service&amp;<span style="color:#268bd2">organization</span><span style="color:#719e07">=</span>dubbogoproxy.com&amp;<span style="color:#268bd2">owner</span><span style="color:#719e07">=</span>ZX&amp;param.sign<span style="color:#719e07">=</span>&amp;registry.role<span style="color:#719e07">=</span>3&amp;<span style="color:#268bd2">release</span><span style="color:#719e07">=</span>dubbo-golang-1.3.0&amp;<span style="color:#268bd2">retries</span><span style="color:#719e07">=</span>&amp;<span style="color:#268bd2">serialization</span><span style="color:#719e07">=</span>&amp;service.filter<span style="color:#719e07">=</span>echo%2Ctoken%2Caccesslog%2Ctps%2Cgeneric_service%2Cexecute%2Cpshutdown&amp;<span style="color:#268bd2">side</span><span style="color:#719e07">=</span>provider&amp;ssl-enabled<span style="color:#719e07">=</span>false&amp;<span style="color:#268bd2">timestamp</span><span style="color:#719e07">=</span>1610350614&amp;tps.limit.interval<span style="color:#719e07">=</span>&amp;tps.limit.rate<span style="color:#719e07">=</span>&amp;tps.limit.rejected.handler<span style="color:#719e07">=</span>&amp;tps.limit.strategy<span style="color:#719e07">=</span>&amp;tps.limiter<span style="color:#719e07">=</span>&amp;<span style="color:#268bd2">version</span><span style="color:#719e07">=</span>1.0.0&amp;<span style="color:#268bd2">warmup</span><span style="color:#719e07">=</span><span style="color:#2aa198">100</span> |
| </span></span></code></pre></div><h3 id="39-复现代码">3.9 复现代码</h3> |
| <ul> |
| <li><a href="https://github.com/cityiron/java_study/tree/master/dubbo2.7.7/dg-issue900">https://github.com/cityiron/java_study/tree/master/dubbo2.7.7/dg-issue900</a></li> |
| <li><a href="https://github.com/cityiron/golang_study/tree/master/dubbogo/1.5.4/arg-bug">https://github.com/cityiron/golang_study/tree/master/dubbogo/1.5.4/arg-bug</a></li> |
| </ul> |
| <h2 id="四参考">四、参考</h2> |
| <ul> |
| <li><a href="https://dubbo.apache.org/zh-cn/docsv2.7/user/configuration/api/">https://dubbo.apache.org/zh-cn/docsv2.7/user/configuration/api/</a></li> |
| <li><a href="https://github.com/apache/dubbo-go/issues/257">https://github.com/apache/dubbo-go/issues/257</a></li> |
| </ul> |
| <hr> |
| <p>篇幅有限，就介绍到这里。欢迎有兴趣的同学来参与 <a href="https://github.com/apache/dubbo-go/tree/release-3.0">dubbogo3.0</a> 的建设，感谢阅读。</p></description></item><item><title>Blog: dubbo-go源码笔记（二）客户端调用过程</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/15/dubbo-go%E6%BA%90%E7%A0%81%E7%AC%94%E8%AE%B0%E4%BA%8C%E5%AE%A2%E6%88%B7%E7%AB%AF%E8%B0%83%E7%94%A8%E8%BF%87%E7%A8%8B/</link><pubDate>Fri, 15 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/15/dubbo-go%E6%BA%90%E7%A0%81%E7%AC%94%E8%AE%B0%E4%BA%8C%E5%AE%A2%E6%88%B7%E7%AB%AF%E8%B0%83%E7%94%A8%E8%BF%87%E7%A8%8B/</guid><description> |
| <p>随着微服务架构的流行，许多高性能 rpc 框架应运而生，由阿里开源的 dubbo 框架 go 语言版本的 dubbo-go 也成为了众多开发者不错的选择。本文将介绍 dubbo-go 框架的基本使用方法，以及从 export 调用链的角度进行 server 端源码导读，希望能引导读者进一步认识这款框架。</p> |
| <h2 id="前言">前言</h2> |
| <p>有了上一篇文章<a href="https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E6%BA%90%E7%A0%81%E7%AC%94%E8%AE%B0%E4%B8%80server-%E7%AB%AF%E5%BC%80%E5%90%AF%E6%9C%8D%E5%8A%A1%E8%BF%87%E7%A8%8B/" title="">《dubbo-go 源码笔记（一）Server服务暴露过程详解》</a> 的铺垫，可以大致上类比客户端服务类似于服务端启动过程。其中最大的区别是服务端通过zk注册服务，发布自己的ivkURL并订阅事件开启监听；而服务端应该是通过zk注册组件，<strong>拿到需要调用的serviceURL</strong>，<strong>更新invoker</strong>并<strong>重写用户的RPCService</strong>，从而实现对远程过程调用细节的封装。</p> |
| <h2 id="1-配置文件和客户端源码">1. 配置文件和客户端源码</h2> |
| <h4 id="11-client配置文件">1.1 client配置文件</h4> |
| <p>helloworld提供的demo：profiles/client.yaml</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">registries </span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;demoZk&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">timeout </span>: <span style="color:#2aa198">&#34;3s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;127.0.0.1:2181&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">username</span>: <span style="color:#2aa198">&#34;&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">password</span>: <span style="color:#2aa198">&#34;&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">references</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;UserProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 可以指定多个registry，使用逗号隔开;不指定默认向所有注册中心注册</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;demoZk&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol </span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface </span>: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods </span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;GetUser&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">3</span> |
| </span></span></code></pre></div><p>可看到配置文件与之前讨论过的server端非常类似，其refrences部分字段就是对当前服务要主调的服务的配置，其中详细说明了调用协议、注册协议、接口id、调用方法、集群策略等，这些配置都会在之后与注册组件交互，重写ivk、调用的过程中使用到。</p> |
| <h4 id="12-客户端使用框架源码">1.2 客户端使用框架源码</h4> |
| <p>user.go</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetConsumerService</span>(userProvider) |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>main.go</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> time.<span style="color:#268bd2">Sleep</span>(<span style="color:#2aa198">3e9</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;\n\n\nstart to test dubbo&#34;</span>) |
| </span></span><span style="display:flex;"><span> user <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>User{} |
| </span></span><span style="display:flex;"><span> err <span style="color:#719e07">:=</span> userProvider.<span style="color:#268bd2">GetUser</span>(context.<span style="color:#268bd2">TODO</span>(), []<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;A001&#34;</span>}, user) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, user) |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">initSignal</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>官网提供的helloworld demo的源码。可看到与服务端类似，在user.go内注册了rpc-service，以及需要rpc传输的结构体user。</p> |
| <p>在main函数中，同样调用了config.Load()函数，之后就可以直接通过实现好的rpc-service：userProvider 直接调用对应的功能函数，即可实现rpc调用。</p> |
| <p>可以猜到，从hessian注册结构、SetConsumerService，到调用函数.GetUser()期间，用户定义的rpc-service也就是userProvider对应的函数被重写，重写后的GetUser函数已经包含了实现了远程调用逻辑的invoker。</p> |
| <p>接下来，就要通过阅读源码，看看dubbo-go是如何做到的。</p> |
| <h2 id="2-实现远程过程调用">2. 实现远程过程调用</h2> |
| <h4 id="21-加载配置文件">2.1 加载配置文件</h4> |
| <p>config/config_loader.go :Load()</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// Load Dubbo Init |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">Load</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// init router |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">initRouter</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// init the global event dispatcher |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> extension.<span style="color:#268bd2">SetAndInitGlobalDispatcher</span>(<span style="color:#268bd2">GetBaseConfig</span>().EventDispatcherType) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// start the metadata report if config set |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> err <span style="color:#719e07">:=</span> <span style="color:#268bd2">startMetadataReport</span>(<span style="color:#268bd2">GetApplicationConfig</span>().MetadataType, <span style="color:#268bd2">GetBaseConfig</span>().MetadataReportConfig); err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;Provider starts metadata report error, and the error is {%#v}&#34;</span>, err) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// reference config |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">loadConsumerConfig</span>() |
| </span></span></code></pre></div><p>在main函数中调用的config.Load()函数，进而调用了loadConsumerConfig，类似于之前讲到的server端配置读入函数。</p> |
| <p>在loadConsumerConfig函数中，进行了三步操作：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p1.png" alt="img"></p> |
| <ol> |
| <li>检查配置文件并将配置写入内存</li> |
| <li><strong>在for循环内部</strong>，依次引用（refer）并且实例化（implement）每个被调reference。</li> |
| <li>等待三秒钟所有invoker就绪</li> |
| </ol> |
| <p>其中重要的就是for循环里面的引用和实例化，两步操作，会在接下来展开讨论。</p> |
| <p>至此，配置已经被写入了框架。</p> |
| <h4 id="22-获取远程service-url实现可供调用的invoker">2.2 获取远程Service URL，实现可供调用的invoker</h4> |
| <p>上述的ref.Refer完成的就是这部分的操作。</p> |
| <p>图（一）</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p2.png" alt="img"></p> |
| <h5 id="221-构造注册url">2.2.1 构造注册url</h5> |
| <p>和server端类似，存在注册url和服务url，dubbo习惯将服务url作为注册url的sub。</p> |
| <p>config/reference_config.go: Refer()</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">/</span> Refer <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (c <span style="color:#719e07">*</span>ReferenceConfig) <span style="color:#268bd2">Refer</span>(_ <span style="color:#268bd2">interface</span>{}) { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//（一）配置url参数(serviceUrl)，将会作为sub |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> cfgURL <span style="color:#719e07">:=</span> common.<span style="color:#268bd2">NewURLWithOptions</span>( |
| </span></span><span style="display:flex;"><span> common.<span style="color:#268bd2">WithPath</span>(c.id), |
| </span></span><span style="display:flex;"><span> common.<span style="color:#268bd2">WithProtocol</span>(c.Protocol), |
| </span></span><span style="display:flex;"><span> common.<span style="color:#268bd2">WithParams</span>(c.<span style="color:#268bd2">getUrlMap</span>()), |
| </span></span><span style="display:flex;"><span> common.<span style="color:#268bd2">WithParamsValue</span>(constant.BEAN_NAME_KEY, c.id), |
| </span></span><span style="display:flex;"><span> ) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// （二）注册地址可以通过url格式给定，也可以通过配置格式给定 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 这一步的意义就是配置-&gt;提取信息生成URL |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> c.Url <span style="color:#719e07">!=</span> <span style="color:#2aa198">&#34;&#34;</span> {<span style="color:#586e75">// 用户给定url信息，可以是点对点的地址，也可以是注册中心的地址 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 1. user specified URL, could be peer-to-peer address, or register center&#39;s address. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> urlStrings <span style="color:#719e07">:=</span> gxstrings.<span style="color:#268bd2">RegSplit</span>(c.Url, <span style="color:#2aa198">&#34;\\s*[;]+\\s*&#34;</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> _, urlStr <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> urlStrings { |
| </span></span><span style="display:flex;"><span> serviceUrl, err <span style="color:#719e07">:=</span> common.<span style="color:#268bd2">NewURL</span>(urlStr) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> {<span style="color:#586e75">// 配置读入注册中心的信息 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// assemble SubURL from register center&#39;s configuration mode |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 这是注册url，protocol = registry,包含了zk的用户名、密码、ip等等 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> c.urls = <span style="color:#268bd2">loadRegistries</span>(c.Registry, consumerConfig.Registries, common.CONSUMER) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// set url to regUrls |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">for</span> _, regUrl <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> c.urls { |
| </span></span><span style="display:flex;"><span> regUrl.SubURL = cfgURL<span style="color:#586e75">// regUrl的subURl存当前配置url |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//至此，无论通过什么形式，已经拿到了全部的regURL |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// （三）获取registryProtocol实例，调用其Refer方法，传入新构建好的regURL |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> <span style="color:#b58900">len</span>(c.urls) <span style="color:#719e07">==</span> <span style="color:#2aa198">1</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 这一步访问到registry/protocol/protocol.go registryProtocol.Refer |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 这里是registry |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> c.invoker = extension.<span style="color:#268bd2">GetProtocol</span>(c.urls[<span style="color:#2aa198">0</span>].Protocol).<span style="color:#268bd2">Refer</span>(<span style="color:#719e07">*</span>c.urls[<span style="color:#2aa198">0</span>]) |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 如果有多个注册中心，即有多个invoker,则采取集群策略 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> invokers <span style="color:#719e07">:=</span> <span style="color:#b58900">make</span>([]protocol.Invoker, <span style="color:#2aa198">0</span>, <span style="color:#b58900">len</span>(c.urls)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span></code></pre></div><p>这个函数中，已经处理完从Register配置到RegisterURL的转换,即图（一）中部分：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p3.png" alt="img"></p> |
| <p>接下来，已经拿到的url将被传递给RegistryProtocol，进一步refer。</p> |
| <h5 id="222-registryprotocol获取到zkregistry实例进一步refer">2.2.2 registryProtocol获取到zkRegistry实例，进一步Refer</h5> |
| <p>registry/protocol/protocol.go: Refer</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// Refer provider service from registry center |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// 拿到的是配置文件registries的url，他能够生成一个invoker = 指向目的addr，以供客户端直接调用。 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (proto <span style="color:#719e07">*</span>registryProtocol) <span style="color:#268bd2">Refer</span>(url common.URL) protocol.Invoker { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> registryUrl = url |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 这里拿到的是referenceConfig，serviceUrl里面包含了Reference的所有信息，包含interfaceName、method等等 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">var</span> serviceUrl = registryUrl.SubURL |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> registryUrl.Protocol <span style="color:#719e07">==</span> constant.REGISTRY_PROTOCOL {<span style="color:#586e75">// registryUrl.Proto = &#34;registry&#34; |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> protocol <span style="color:#719e07">:=</span> registryUrl.<span style="color:#268bd2">GetParam</span>(constant.REGISTRY_KEY, <span style="color:#2aa198">&#34;&#34;</span>) |
| </span></span><span style="display:flex;"><span> registryUrl.Protocol = protocol<span style="color:#586e75">//替换成了具体的值，比如&#34;zookeeper&#34; |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 接口对象 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">var</span> reg registry.Registry |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// （一）实例化接口对象，缓存策略 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> regI, loaded <span style="color:#719e07">:=</span> proto.registries.<span style="color:#268bd2">Load</span>(registryUrl.<span style="color:#268bd2">Key</span>()); !loaded { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 缓存中不存在当前registry，新建一个reg |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> reg = <span style="color:#268bd2">getRegistry</span>(<span style="color:#719e07">&amp;</span>registryUrl) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 缓存起来 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> proto.registries.<span style="color:#268bd2">Store</span>(registryUrl.<span style="color:#268bd2">Key</span>(), reg) |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> reg = regI.(registry.Registry) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 到这里，获取到了reg实例 zookeeper的registry |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">//（二）根据Register的实例zkRegistry和传入的regURL新建一个directory |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 这一步存在复杂的异步逻辑，从注册中心拿到了目的service的真实addr，获取了invoker并放入directory， |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 这一步将在下面详细给出步骤 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// new registry directory for store service url from registry |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> directory, err <span style="color:#719e07">:=</span> extension.<span style="color:#268bd2">GetDefaultRegistryDirectory</span>(<span style="color:#719e07">&amp;</span>registryUrl, reg) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;consumer service %v create registry directory error, error message is %s, and will return nil invoker!&#34;</span>, |
| </span></span><span style="display:flex;"><span> serviceUrl.<span style="color:#268bd2">String</span>(), err.<span style="color:#268bd2">Error</span>()) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// （三）DoRegister 在zk上注册当前client service |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> err = reg.<span style="color:#268bd2">Register</span>(<span style="color:#719e07">*</span>serviceUrl) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;consumer service %v register registry %v error, error message is %s&#34;</span>, |
| </span></span><span style="display:flex;"><span> serviceUrl.<span style="color:#268bd2">String</span>(), registryUrl.<span style="color:#268bd2">String</span>(), err.<span style="color:#268bd2">Error</span>()) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// （四）new cluster invoker，将directory写入集群，获得具有集群策略的invoker |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> cluster <span style="color:#719e07">:=</span> extension.<span style="color:#268bd2">GetCluster</span>(serviceUrl.<span style="color:#268bd2">GetParam</span>(constant.CLUSTER_KEY, constant.DEFAULT_CLUSTER)) |
| </span></span><span style="display:flex;"><span> invoker <span style="color:#719e07">:=</span> cluster.<span style="color:#268bd2">Join</span>(directory) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// invoker保存 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> proto.invokers = <span style="color:#b58900">append</span>(proto.invokers, invoker) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> invoker |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>可详细阅读上述注释，这个函数完成了从url到invoker的全部过程</p> |
| <p>（一）首先获得Registry对象，默认是之前实例化的zkRegistry，和之前server获取Registry的处理很类似。 |
| （二）通过构造一个新的directory，异步拿到之前在zk上注册的server端信息，生成invoker |
| （三）在zk上注册当前service |
| （四）集群策略，获得最终invoker</p> |
| <p>这一步完成了图（一）中所有余下的绝大多数操作，接下来就需要详细的查看directory的构造过程：</p> |
| <h5 id="223-构造directory包含较复杂的异步操作">2.2.3 构造directory（包含较复杂的异步操作）</h5> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p4.png" alt="img"></p> |
| <p>图（二）</p> |
| <p>上述的 <code>extension.GetDefaultRegistryDirectory(&amp;registryUrl, reg)</code>函数，本质上调用了已经注册好的<code>NewRegistryDirectory</code>函数:</p> |
| <p>registry/directory/directory.go: NewRegistryDirectory()</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// NewRegistryDirectory will create a new RegistryDirectory |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// 这个函数作为default注册在extension上面 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// url为注册url，reg为zookeeper registry |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">NewRegistryDirectory</span>(url <span style="color:#719e07">*</span>common.URL, registry registry.Registry) (cluster.Directory, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> url.SubURL <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span>, perrors.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;url is invalid, suburl can not be nil&#34;</span>) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> dir <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>RegistryDirectory{ |
| </span></span><span style="display:flex;"><span> BaseDirectory: directory.<span style="color:#268bd2">NewBaseDirectory</span>(url), |
| </span></span><span style="display:flex;"><span> cacheInvokers: []protocol.Invoker{}, |
| </span></span><span style="display:flex;"><span> cacheInvokersMap: <span style="color:#719e07">&amp;</span>sync.Map{}, |
| </span></span><span style="display:flex;"><span> serviceType: url.SubURL.<span style="color:#268bd2">Service</span>(), |
| </span></span><span style="display:flex;"><span> registry: registry, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> dir.consumerConfigurationListener = <span style="color:#268bd2">newConsumerConfigurationListener</span>(dir) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">go</span> dir.<span style="color:#268bd2">subscribe</span>(url.SubURL) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> dir, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>首先构造了一个注册directory，开启携程调用其subscribe函数，传入serviceURL。</p> |
| <p>这个directory目前包含了对应的zkRegistry，以及传入的URL，他cacheInvokers的部分是空的。</p> |
| <p>进入dir.subscribe(url.SubURL)这个异步函数：</p> |
| <p>registry/directory/directory.go: subscribe()</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// subscribe from registry |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (dir <span style="color:#719e07">*</span>RegistryDirectory) <span style="color:#268bd2">subscribe</span>(url <span style="color:#719e07">*</span>common.URL) { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 增加两个监听， |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> dir.consumerConfigurationListener.<span style="color:#268bd2">addNotifyListener</span>(dir) |
| </span></span><span style="display:flex;"><span> dir.referenceConfigurationListener = <span style="color:#268bd2">newReferenceConfigurationListener</span>(dir, url) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// subscribe调用 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> dir.registry.<span style="color:#268bd2">Subscribe</span>(url, dir) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>重点来了，他调用了zkRegistry的Subscribe方法,与此同时将自己作为ConfigListener传入</p> |
| <blockquote> |
| <p>我认为这种传入listener的设计模式非常值得学习，而且很有java的味道。</p> |
| <p>针对等待zk返回订阅信息这样的异步操作，需要传入一个Listener，这个Listener需要实现Notify方法，进而在作为参数传入内部之后，可以被异步地调用Notify，将内部触发的异步事件“传递出来”，再进一步处理加工。</p> |
| <p>层层的Listener事件链，能将传入的原始serviceURL通过zkConn发送给zk服务，获取到服务端注册好的url对应的二进制信息。</p> |
| <p>而Notify回调链，则将这串byte[]一步一步解析、加工；以事件的形式向外传递，最终落到directory上的时候，已经是成型的newInvokers了。</p> |
| <p>具体细节不再以源码形式展示，可参照上图查阅源码。</p> |
| </blockquote> |
| <p>至此已经拿到了server端注册好的真实invoker。</p> |
| <p>完成了图（一）中的部分：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p5.png" alt="img"></p> |
| <h5 id="224-构造带有集群策略的clusterinvoker">2.2.4 构造带有集群策略的clusterinvoker</h5> |
| <p>经过上述操作，已经拿到了server端Invokers，放入了directory的cacheinvokers数组里面缓存。</p> |
| <p>后续的操作对应本文2.2.2的第四步，由directory生成带有特性集群策略的invoker</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// （四）new cluster invoker，将directory写入集群，获得具有集群策略的invoker |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> cluster <span style="color:#719e07">:=</span> extension.<span style="color:#268bd2">GetCluster</span>(serviceUrl.<span style="color:#268bd2">GetParam</span>(constant.CLUSTER_KEY, constant.DEFAULT_CLUSTER)) |
| </span></span><span style="display:flex;"><span> invoker <span style="color:#719e07">:=</span> cluster.<span style="color:#268bd2">Join</span>(directory) |
| </span></span><span style="display:flex;"><span><span style="color:#2aa198">123</span> |
| </span></span></code></pre></div><p>Join函数的实现就是如下函数：</p> |
| <p>cluster/cluster_impl/failover_cluster_invokers.go: newFailoverClusterInvoker()</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">newFailoverClusterInvoker</span>(directory cluster.Directory) protocol.Invoker { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>failoverClusterInvoker{ |
| </span></span><span style="display:flex;"><span> baseClusterInvoker: <span style="color:#268bd2">newBaseClusterInvoker</span>(directory), |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#2aa198">12345</span> |
| </span></span></code></pre></div><p>dubbo-go框架默认选择failover策略，既然返回了一个invoker，我们查看一下failoverClusterInvoker的Invoker方法，看他是如何将集群策略封装到Invoker函数内部的：</p> |
| <p>cluster/cluster_impl/failover_cluster_invokers.go: Invoker()</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// Invoker 函数 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (invoker <span style="color:#719e07">*</span>failoverClusterInvoker) <span style="color:#268bd2">Invoke</span>(ctx context.Context, invocation protocol.Invocation) protocol.Result { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//调用List方法拿到directory缓存的所有invokers |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> invokers <span style="color:#719e07">:=</span> invoker.directory.<span style="color:#268bd2">List</span>(invocation) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">:=</span> invoker.<span style="color:#268bd2">checkInvokers</span>(invokers, invocation); err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> {<span style="color:#586e75">// 检查是否可以实现调用 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>protocol.RPCResult{Err: err} |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 获取来自用户方向传入的 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> methodName <span style="color:#719e07">:=</span> invocation.<span style="color:#268bd2">MethodName</span>() |
| </span></span><span style="display:flex;"><span> retries <span style="color:#719e07">:=</span> <span style="color:#268bd2">getRetries</span>(invokers, methodName) |
| </span></span><span style="display:flex;"><span> loadBalance <span style="color:#719e07">:=</span> <span style="color:#268bd2">getLoadBalance</span>(invokers[<span style="color:#2aa198">0</span>], invocation) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; i <span style="color:#719e07">&lt;=</span> retries; i<span style="color:#719e07">++</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 重要！这里是集群策略的体现，失败后重试！ |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">//Reselect before retry to avoid a change of candidate `invokers`. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">//NOTE: if `invokers` changed, then `invoked` also lose accuracy. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> i &gt; <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">:=</span> invoker.<span style="color:#268bd2">checkWhetherDestroyed</span>(); err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>protocol.RPCResult{Err: err} |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> invokers = invoker.directory.<span style="color:#268bd2">List</span>(invocation) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">:=</span> invoker.<span style="color:#268bd2">checkInvokers</span>(invokers, invocation); err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>protocol.RPCResult{Err: err} |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 这里是负载均衡策略的体现！选择特定ivk进行调用。 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ivk <span style="color:#719e07">:=</span> invoker.<span style="color:#268bd2">doSelect</span>(loadBalance, invocation, invokers, invoked) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> ivk <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">continue</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> invoked = <span style="color:#b58900">append</span>(invoked, ivk) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//DO INVOKE |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> result = ivk.<span style="color:#268bd2">Invoke</span>(ctx, invocation) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> result.<span style="color:#268bd2">Error</span>() <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> providers = <span style="color:#b58900">append</span>(providers, ivk.<span style="color:#268bd2">GetUrl</span>().<span style="color:#268bd2">Key</span>()) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">continue</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> result |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><blockquote> |
| <p>看了很多Invoke函数的实现，所有类似的Invoker函数都包含两个方向，一个是用户方向的invcation，一个是函数方向的底层invokers。</p> |
| <p>而集群策略的invoke函数本身作为接线员，把invocation一步步解析，根据调用需求和集群策略，选择特定的invoker来执行</p> |
| <p>proxy函数也是这样，一个是用户方向的ins[] reflect.Type, 一个是函数方向的invoker。</p> |
| <p>proxy函数负责将ins转换为invocation，调用对应invoker的invoker函数，实现连通。</p> |
| <p>而出于这样的设计，可以在一步步Invoker封装的过程中，每个Invoker只关心自己负责操作的部分，从而使整个调用栈解耦。</p> |
| <p>妙啊！！！</p> |
| </blockquote> |
| <p>至此，我们理解了failoverClusterInvoker 的Invoke函数实现，也正是和这个集群策略Invoker被返回，接受来自上方的调用。</p> |
| <p>已完成图（一）中的：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p6.png" alt="img"></p> |
| <h5 id="225-在zookeeper上注册当前client">2.2.5 在zookeeper上注册当前client</h5> |
| <p>拿到invokers后，可以回到：</p> |
| <p>config/refrence_config.go: Refer()函数了。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#b58900">len</span>(c.urls) <span style="color:#719e07">==</span> <span style="color:#2aa198">1</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 这一步访问到registry/protocol/protocol.go registryProtocol.Refer |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> c.invoker = extension.<span style="color:#268bd2">GetProtocol</span>(c.urls[<span style="color:#2aa198">0</span>].Protocol).<span style="color:#268bd2">Refer</span>(<span style="color:#719e07">*</span>c.urls[<span style="color:#2aa198">0</span>]) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// （一）拿到了真实的invokers |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 如果有多个注册中心，即有多个invoker,则采取集群策略 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> invokers <span style="color:#719e07">:=</span> <span style="color:#b58900">make</span>([]protocol.Invoker, <span style="color:#2aa198">0</span>, <span style="color:#b58900">len</span>(c.urls)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> cluster <span style="color:#719e07">:=</span> extension.<span style="color:#268bd2">GetCluster</span>(hitClu) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// If &#39;zone-aware&#39; policy select, the invoker wrap sequence would be: |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// ZoneAwareClusterInvoker(StaticDirectory) -&gt; |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// FailoverClusterInvoker(RegistryDirectory, routing happens here) -&gt; Invoker |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> c.invoker = cluster.<span style="color:#268bd2">Join</span>(directory.<span style="color:#268bd2">NewStaticDirectory</span>(invokers)) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// （二）create proxy，为函数配置代理 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> c.Async { |
| </span></span><span style="display:flex;"><span> callback <span style="color:#719e07">:=</span> <span style="color:#268bd2">GetCallback</span>(c.id) |
| </span></span><span style="display:flex;"><span> c.pxy = extension.<span style="color:#268bd2">GetProxyFactory</span>(consumerConfig.ProxyFactory).<span style="color:#268bd2">GetAsyncProxy</span>(c.invoker, callback, cfgURL) |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 这里c.invoker已经是目的addr了 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> c.pxy = extension.<span style="color:#268bd2">GetProxyFactory</span>(consumerConfig.ProxyFactory).<span style="color:#268bd2">GetProxy</span>(c.invoker, cfgURL) |
| </span></span><span style="display:flex;"><span> } |
| </span></span></code></pre></div><p>我们有了可以打通的invokers，但还不能直接调用，因为invoker的入参是invocation，而调用函数使用的是具体的参数列表。需要通过一层proxy来规范入参和出参。</p> |
| <p>接下来新建一个默认proxy，放置在c.proxy内，以供后续使用</p> |
| <p>至此，完成了图（一）中最后的操作</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p7.png" alt="img"></p> |
| <h3 id="23-将调用逻辑以代理函数的形式写入rpc-service">2.3 将调用逻辑以代理函数的形式写入rpc-service</h3> |
| <p>上面完成了config.Refer操作</p> |
| <p>回到config/config_loader.go: loadConsumerConfig()</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p8.png" alt="img"></p> |
| <p>下一个重要的函数是Implement，他完的操作较为简单：旨在使用上面生成的c.proxy代理，链接用户自己定义的rpcService到clusterInvoker的信息传输。</p> |
| <p>函数较长，只选取了重要的部分:</p> |
| <p>common/proxy/proxy.go: Implement()</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// Implement |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// proxy implement |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// In consumer, RPCService like: |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// type XxxProvider struct { |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// Yyy func(ctx context.Context, args []interface{}, rsp *Zzz) error |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// } |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// Implement 实现的过程，就是proxy根据函数名和返回值，通过调用invoker 构造出拥有远程调用逻辑的代理函数 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// 将当前rpc所有可供调用的函数注册到proxy.rpc内 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>Proxy) <span style="color:#268bd2">Implement</span>(v common.RPCService) { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// makeDubboCallProxy 这是一个构造代理函数，这个函数的返回值是func(in []reflect.Value) []reflect.Value 这样一个函数 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 这个被返回的函数是请求实现的载体，由他来发起调用获取结果 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> makeDubboCallProxy <span style="color:#719e07">:=</span> <span style="color:#268bd2">func</span>(methodName <span style="color:#dc322f">string</span>, outs []reflect.Type) <span style="color:#268bd2">func</span>(in []reflect.Value) []reflect.Value { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#268bd2">func</span>(in []reflect.Value) []reflect.Value { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 根据methodName和outs的类型，构造这样一个函数，这个函数能将in 输入的value转换为输出的value |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 这个函数具体的实现如下： |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 目前拿到了 methodName、所有入参的interface和value，出参数reply |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// （一）根据这些生成一个 rpcinvocation |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> inv = invocation_impl.<span style="color:#268bd2">NewRPCInvocationWithOptions</span>( |
| </span></span><span style="display:flex;"><span> invocation_impl.<span style="color:#268bd2">WithMethodName</span>(methodName), |
| </span></span><span style="display:flex;"><span> invocation_impl.<span style="color:#268bd2">WithArguments</span>(inIArr), |
| </span></span><span style="display:flex;"><span> invocation_impl.<span style="color:#268bd2">WithReply</span>(reply.<span style="color:#268bd2">Interface</span>()), |
| </span></span><span style="display:flex;"><span> invocation_impl.<span style="color:#268bd2">WithCallBack</span>(p.callBack), |
| </span></span><span style="display:flex;"><span> invocation_impl.<span style="color:#268bd2">WithParameterValues</span>(inVArr)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> k, value <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> p.attachments { |
| </span></span><span style="display:flex;"><span> inv.<span style="color:#268bd2">SetAttachments</span>(k, value) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// add user setAttachment |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> atm <span style="color:#719e07">:=</span> invCtx.<span style="color:#268bd2">Value</span>(constant.AttachmentKey) <span style="color:#586e75">// 如果传入的ctx里面有attachment，也要写入inv |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> m, ok <span style="color:#719e07">:=</span> atm.(<span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#dc322f">string</span>); ok { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> k, value <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> m { |
| </span></span><span style="display:flex;"><span> inv.<span style="color:#268bd2">SetAttachments</span>(k, value) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 至此构造inv完毕 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// (二）触发Invoker 之前已经将cluster_invoker放入proxy，使用Invoke方法，通过getty远程过程调用 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> result <span style="color:#719e07">:=</span> p.invoke.<span style="color:#268bd2">Invoke</span>(invCtx, inv) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 如果有attachment，则加入 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> <span style="color:#b58900">len</span>(result.<span style="color:#268bd2">Attachments</span>()) &gt; <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> invCtx = context.<span style="color:#268bd2">WithValue</span>(invCtx, constant.AttachmentKey, result.<span style="color:#268bd2">Attachments</span>()) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> numField <span style="color:#719e07">:=</span> valueOfElem.<span style="color:#268bd2">NumField</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; i &lt; numField; i<span style="color:#719e07">++</span> { |
| </span></span><span style="display:flex;"><span> t <span style="color:#719e07">:=</span> typeOf.<span style="color:#268bd2">Field</span>(i) |
| </span></span><span style="display:flex;"><span> methodName <span style="color:#719e07">:=</span> t.Tag.<span style="color:#268bd2">Get</span>(<span style="color:#2aa198">&#34;dubbo&#34;</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> methodName <span style="color:#719e07">==</span> <span style="color:#2aa198">&#34;&#34;</span> { |
| </span></span><span style="display:flex;"><span> methodName = t.Name |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> f <span style="color:#719e07">:=</span> valueOfElem.<span style="color:#268bd2">Field</span>(i) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> f.<span style="color:#268bd2">Kind</span>() <span style="color:#719e07">==</span> reflect.Func <span style="color:#719e07">&amp;&amp;</span> f.<span style="color:#268bd2">IsValid</span>() <span style="color:#719e07">&amp;&amp;</span> f.<span style="color:#268bd2">CanSet</span>() { <span style="color:#586e75">// 针对于每个函数 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> outNum <span style="color:#719e07">:=</span> t.Type.<span style="color:#268bd2">NumOut</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 规定函数输出只能有1/2个 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> outNum <span style="color:#719e07">!=</span> <span style="color:#2aa198">1</span> <span style="color:#719e07">&amp;&amp;</span> outNum <span style="color:#719e07">!=</span> <span style="color:#2aa198">2</span> { |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Warnf</span>(<span style="color:#2aa198">&#34;method %s of mtype %v has wrong number of in out parameters %d; needs exactly 1/2&#34;</span>, |
| </span></span><span style="display:flex;"><span> t.Name, t.Type.<span style="color:#268bd2">String</span>(), outNum) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">continue</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// The latest return type of the method must be error. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 规定最后一个返回值一定是error |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> returnType <span style="color:#719e07">:=</span> t.Type.<span style="color:#268bd2">Out</span>(outNum <span style="color:#719e07">-</span> <span style="color:#2aa198">1</span>); returnType <span style="color:#719e07">!=</span> typError { |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Warnf</span>(<span style="color:#2aa198">&#34;the latest return type %s of method %q is not error&#34;</span>, returnType, t.Name) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">continue</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 获取到所有的出参类型，放到数组里 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">var</span> funcOuts = <span style="color:#b58900">make</span>([]reflect.Type, outNum) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; i &lt; outNum; i<span style="color:#719e07">++</span> { |
| </span></span><span style="display:flex;"><span> funcOuts[i] = t.Type.<span style="color:#268bd2">Out</span>(i) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// do method proxy here: |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// （三）调用make函数，传入函数名和返回值，获得能调用远程的proxy，将这个proxy替换掉原来的函数位置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> f.<span style="color:#268bd2">Set</span>(reflect.<span style="color:#268bd2">MakeFunc</span>(f.<span style="color:#268bd2">Type</span>(), <span style="color:#268bd2">makeDubboCallProxy</span>(methodName, funcOuts))) |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Debugf</span>(<span style="color:#2aa198">&#34;set method [%s]&#34;</span>, methodName) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>正如之前所说，proxy的作用是将用户定义的函数参数列表，转化为抽象的invocation传入Invoker，进行调用。</p> |
| <p>其中已标明有三处较为重要的地方：</p> |
| <ol> |
| <li>在代理函数中实现由参数列表生成Invocation的逻辑</li> |
| <li>在代理函数实现调用Invoker的逻辑</li> |
| <li>将代理函数替换为原始rpc-service对应函数 |
| 至此，也就解决了一开始的问题： |
| client.go: main()</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> user <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>User{} |
| </span></span><span style="display:flex;"><span> err <span style="color:#719e07">:=</span> userProvider.<span style="color:#268bd2">GetUser</span>(context.<span style="color:#268bd2">TODO</span>(), []<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;A001&#34;</span>}, user) |
| </span></span></code></pre></div><p>这里直接调用用户定义的rpcService的函数GetUser，这里实际调用的是经过重写入的函数代理，所以就能实现远程调用了。</p> |
| <h3 id="3-从client到server的invoker嵌套链--小结">3. 从client到server的invoker嵌套链- 小结</h3> |
| <p>在阅读dubbo-go源码的过程中，我能发现一条清晰的invoker-proxy嵌套链，我希望通过图的形式来展现：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code2/p9.png" alt="img"></p> |
| <blockquote> |
| <p>作者简介 李志信 (GitHubID LaurenceLiZhixin)，中山大学软件工程专业在校学生，擅长使用 Java/Go 语言，专注于云原生和微服务等技术方向。</p> |
| </blockquote></description></item><item><title>Blog: dubbogo 3.0：牵手 gRPC 走向云原生时代</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/15/dubbogo-3.0%E7%89%B5%E6%89%8B-grpc-%E8%B5%B0%E5%90%91%E4%BA%91%E5%8E%9F%E7%94%9F%E6%97%B6%E4%BB%A3/</link><pubDate>Fri, 15 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/15/dubbogo-3.0%E7%89%B5%E6%89%8B-grpc-%E8%B5%B0%E5%90%91%E4%BA%91%E5%8E%9F%E7%94%9F%E6%97%B6%E4%BB%A3/</guid><description> |
| <p>自从 2011 年 Dubbo 开源之后，被大量中小公司采用，一直是国内最受欢迎的 RPC 框架。2014 年，由于阿里内部组织架构调整，Dubbo 暂停维护了一段时间，之后随着 Spring Cloud 的面世，两个体系在融合中一起助推了微服务的火热。</p> |
| <p>不过这世界变化快，自从以 docker 为代表的的容器技术和以 K8s 为代表的容器编排技术登上舞台之后，云原生时代到来了。在云原生时代，不可变的基础设施给原有的中间件带来了不可变的中间件基础设施：gRPC 统一了底层通信层；protobuf 统一了序列化协议；以 envoy + istio 为代表的 service mesh 逐渐统一了服务的控制面与数据面。</p> |
| <p>dubbogo 的天然使命是：Bridging the gap between Java and Go。保持 Go 应用与 Java 应用互联互通的同时，借助 Go 语言（事实上的第一云原生语言）的优势拥抱云原生时代。dubbogo 社区 2020 年勠力打造三支箭：</p> |
| <ul> |
| <li>已经发布的对齐 dubbo 2.7 的 dubbogo v1.5 版本；</li> |
| <li>近期将要发布的 sidecar 形态的 dubbo-go-proxy 项目；</li> |
| <li>以及处于进行时的 dubbogo 3.0。</li> |
| </ul> |
| <p>用一句话概括 dubbogo 3.0 即是：新通信协议、新序列化协议、新应用注册模型以及新的服务治理能力！本文主要着重讨论 dubbogo 3.0 的新通信协议和应用级服务注册发现模型。</p> |
| <h2 id="dubbogo-30-vs-grpc">dubbogo 3.0 vs gRPC</h2> |
| <p>知己知彼，方能进步。dubbogo 3.0 的通信层改进主要借鉴了 gRPC。</p> |
| <p>gRPC 协议，简单来说就是 http2 协议的基础之上，增加了特定的协议 header：“grpc-” 开头的 header 字段，采用特定的打解包工具（protobuf）对数据进行序列化，从而实现 RPC 调用。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/3.0-plan/p1.webp" alt="img"></p> |
| <p>众所周知，gRPC 几乎没有服务治理能力，而阿里云现有 dubbo 框架兼具 RPC 和服务治理能力，整体实力不逊于 gRPC。但在“大家都用 gRPC” 这样的背景之下，dubbogo 3.0 的新通信协议就必须<strong>完美兼容 gRPC</strong>，对开发者已部署的服务完全兼容，并在此基础之上延续已有 dubbo 协议和服务治理能力，进而推出一系列新策略：比如 mesh 支持、应用级服务注册等。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/3.0-plan/p2.webp" alt="img"></p> |
| <h2 id="dubbogo-30-vs-dubbogo-15">dubbogo 3.0 vs dubbogo 1.5</h2> |
| <p>目前已有的 dubbo 2.7 协议已经尽可能实现了 gRPC 的支持。开发者可以通过 protoc-gen-dubbo 工具将 pb IDL 协议转换为框架支持的 stub，再借助底层 gRPC conn 的 RPC 过程，将已有的服务治理能力自上而下传递给 gRPC，因此实现了 gRPC 服务的支持。</p> |
| <p>dubbo-go v1.5.x 也支持 gRPC 的 Stream 调用。和 unary RPC 类似，通过产生框架支持的 stub，在底层 gRPC stream 调用的基础之上，将流式 RPC 的能力和并入框架。但由于 dubbo v2.7.x / dubbo-go v1.5.x 本身并不支持流式调用，所以没有对 gRPC stream 调用的进行上层服务治理支持。</p> |
| <p>开发者所面临的问题就是：我们在使用 dubbo-go2.7 进行 grpc 协议传输的时候，或多或少不是那么放心。</p> |
| <p>而即将推出的 dubbo-go 3.0 协议将从根源解决这个问题。</p> |
| <h2 id="协议兼容的三种层次">协议兼容的三种层次</h2> |
| <p>笔者认为，一款服务框架对于第三方协议的支持可分为三个程度：应用层次、协议层次、传输层次。</p> |
| <p>一款框架如果在一个协议的 sdk 之上封装接口，可以认为它处于应用层次支持，这样的框架需要遵循下层 sdk 的接口，可扩展性较差。</p> |
| <p>处于协议层次的框架，从配置层到服务治理层均由本框架提供，而在此之下的协议层到网络传输层均使用某个固定的通信协议，这样的框架可以解决服务治理的问题，但框架本身无法与第三方协议完全适配，如果不适配就会出现对第三方协议支持的削弱，比如上面说到的 dubbo-go 1.5 对 stream rpc 支持的缺陷。</p> |
| <p>如果想进一步支持更多的第三方协议，需要从传输层下手，真正了解第三方协议的具体字段、所依赖的底层协议（比如 HTTP2）的帧模型和数据流，再开发出与第三方协议完全一致的数据交互模块，作为本框架的底层。这样做的好处是最大程度赋予了协议的可扩展性，可以在兼容已有协议的基础之上，可选地增加开发者需要的字段，从而实现已有协议无法实现的功能，就比如 dubbogo 3.0 将支持的反压策略。</p> |
| <h2 id="基于-http2-的通信流程">基于 HTTP2 的通信流程</h2> |
| <p>gRPC 一次基于 HTTP2 的 unary rpc 调用传输主要流程如下：</p> |
| <ul> |
| <li>client 发送 Magic 信息： |
| PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n；</li> |
| <li>server 收到并检查是否正确；</li> |
| <li>client 和 server 互相发送 setting 帧，收到后发送 ACK 确认；</li> |
| <li>client 发送 Header 帧，包含 gRPC 协议字段，以 End Headers 作为 Header 结束标志；</li> |
| <li>client 紧接着发送 Data 帧，包含 RPC 调用的 request 信息，以 End Stream 作为 Data 结束标志；</li> |
| <li>server 调用函数获得结果；</li> |
| <li>server 发送 Header 帧，包含 gRPC 协议字段，以 End Headers 作为 Header 结束标志；</li> |
| <li>server 紧接着发送 Data 帧，包含 RPC 调用回传的 response 信息；</li> |
| <li>server 紧接着再次发送 Header 帧，包含 RPC 状态和 message 信息，以 End Stream 作为本次 RPC 调用结束标志。</li> |
| </ul> |
| <p>其中包含 gRPC 调用信息的 HTTP2 Header 帧如下图：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/3.0-plan/p3.webp" alt="img"></p> |
| <p>另外，在 gRPC 的 stream 调用中，可在 server 端回传的过程中发送多次 Data，调用结束后再发送 Header 终止 RPC 过程，并汇报状态信息。</p> |
| <p>dubbogo 3.0 的通信层将在 HTTP2 通信协议之上采用同样的通信流程，以保证与 gRPC 的底层通信沟通能力。</p> |
| <h2 id="dubbogo-30-预期通信架构">dubbogo 3.0 预期通信架构</h2> |
| <p>除了通信协议采用 HTTP2 外，dubbogo 3.0 将采用基于 google protobuf 的 triple 协议【下面称为 dubbo3 协议】作为 dubbogo 3.0 的序列化协议，为 dubbo 将来支持更多的编程语言打下通信协议层面的基础。</p> |
| <p>目前设计的 dubbogo 3.0 传输模型如下：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/3.0-plan/p4.webp" alt="img"></p> |
| <ul> |
| <li>为保证同时支持 unary RPC 和 stream RPC，在 server 端和 client 端增加数据流结构，以异步调用的形式完成数据传递；</li> |
| <li>继续支持原有的 TCP 通信能力；</li> |
| <li>在 HTTP2 的通信协议之上支持 dubbo3 协议，decode 过程兼容 gRPC 使用的 protobuf，保证与 gRPC 服务打通。</li> |
| </ul> |
| <h2 id="应用级服务注册发现">应用级服务注册发现</h2> |
| <h4 id="1-应用级服务注册发现介绍">1. 应用级服务注册发现介绍</h4> |
| <p>dubbogo 3.0 使用的新一代服务注册发现体系，将摒弃旧版的“接口级注册发现”，使用“应用级别注册发现”。</p> |
| <p>简单地说，接口级别注册发现，在注册中心中以 RPC 服务为 key，以实例列表作为 value 来组织数据的，而我们新引入的“应用粒度的服务发现”，它以应用名（Application）作为 key，以这个应用部署的一组实例（Instance）列表作为 value。这带来两点不同：</p> |
| <ul> |
| <li>数据映射关系变了，从 RPC Service -&gt; Instance 变为 Application -&gt; Instance；</li> |
| <li>数据变少了，注册中心没有了 RPC Service 及其相关配置信息。</li> |
| </ul> |
| <p>可以认为，基于应用粒度的模型所存储和推送的数据量是和应用、实例数成正比的，只有当我们的应用数增多或应用的实例数增长时，地址推送压力才会上涨。</p> |
| <p>而对于基于接口粒度的模型，数据量是和接口数量正相关的，鉴于一个应用通常发布多个接口的现状，其数量级一般是比应用粒度的数十倍。另外一个关键点在于，接口的定义更多的是业务侧的内部行为，接口粒度导致的集群规模评估的不透明，而实例、应用增长都通常是在运维侧的规划之中，可控性较好。</p> |
| <p>工商银行曾经对这两个模型进行生产测算：应用级服务注册模型可以让注册中心上的数据量变成原来的 1.68%，新模型可以让 zookeeper 轻松支撑 10 万级别的服务量和 10 万级别的节点量。</p> |
| <h4 id="2-元数据中心同步机制的引入">2. 元数据中心同步机制的引入</h4> |
| <p>数据中心的数据量变少所造成的结果，是 RPC 服务相关的数据在注册中心消失了，只有 application - instance 这两个层级的数据。为了保证这部分缺少的 RPC 服务数据仍然能被 Consumer 端正确的感知，我们在 Consumer 和 Provider 间建立了一条单独的通信通道，目前针对元数据同步有两种具体的可选方案，分别是：</p> |
| <ul> |
| <li>内建 MetadataService；</li> |
| <li>独立的元数据中心，通过中细化的元数据集群协调数据。</li> |
| </ul> |
| <h4 id="3-兼容旧版本-dubbo-go">3. 兼容旧版本 dubbo-go</h4> |
| <p>为了使整个开发流程对老的 dubbo-go 用户更透明，同时避免指定 provider 对可扩展性带来的影响），我们设计了一套 RPC服务到应用名的映射关系，以尝试在 consumer 自动完成 RPC 服务到 provider 应用名的转换。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/3.0-plan/p5.webp" alt="img"></p> |
| <p>这套设计可以让 dubbogo 3.0 中同时保持对 dubbo v2.6.x、dubbo v2.7.x 和 dubbo v3.0.x 三个大版的互联互通。</p> |
| <h2 id="统一路由的支持">统一路由的支持</h2> |
| <p>路由在概念上可以理解为从已有的所有 IP 地址列表中，根据特定的路由规则，挑选出需要的 ip 地址子集。路由的过程需要根据配置好的路由规则进行筛选，最终取所有路由规则的交集获得结果。多个路由如同流水线一样，形成一条路由链，从所有的地址表中筛选出最终目的地址集合，再通过负载均衡策略选择访问的地址。</p> |
| <h4 id="1-路由链">1. 路由链</h4> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/3.0-plan/p6.webp" alt="img"></p> |
| <p>可以把路由链的逻辑简单理解为 target = rn(&hellip;r3(r2(r1(src))))。对于每一个 router 内部的逻辑，可以抽象为输入地址 addrs-in 与 router 中按全量地址 addrs-all 实现切分好的 n 个<strong>互不相交</strong>的地址池 addrs-pool-1 &hellip; addrs-pool-n 按实现定义好的规则取交集作为输出地址。以此类推，完成整个路由链的计算。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/3.0-plan/p7.webp" alt="img"></p> |
| <h4 id="2-failover">2. failover</h4> |
| <p>在路由规则配置文件中可以配置 failover 字段。在寻找地址失败时可以 failover， 选择其他 subset，并且顺序执行下来，直到找到地址，否则最后报地址找不到异常。</p> |
| <h4 id="3-兜底路由">3. 兜底路由</h4> |
| <p>在的路由规则配置中，可以配置一个没有任何条件的 match, 最终的结果是至少会有一个 subset 被选到，以达到地址空保护的作用。</p> |
| <p>作为 2020 年 dubbogo 社区打造并将在 2021 年初亮出的的三支箭之一，dubbogo 3.0 将带来不同平常且焕然一新的开发体验，敬请广大开发者期待！</p> |
| <p>如果你有任何疑问，欢迎钉钉扫码加入交流群【钉钉群号 31363295】：</p> |
| <p>dubbogo 3.0 目前是社区和 dubbo 官方团队&ndash; 阿里中间件团队共同合作开发。</p> |
| <p>阿里云-中间件团队招募对 dubbo3 (java &amp; go)、dapr、arthas 有兴趣的开发者。可以钉钉联系 northlatitude，或者发送邮件至 <a href="mailto:beiwei.ly@alibaba-inc.com">beiwei.ly@alibaba-inc.com</a>。</p> |
| <blockquote> |
| <p>作者简介</p> |
| <p><strong>李志信</strong> (GitHubID LaurenceLiZhixin)，阿里云云原生中间件团队开发工程师，dubbogo 社区开发者，中山大学软件工程专业在校学生，擅长使用 Go 语言，专注于云原生和微服务等技术方向。</p> |
| <p><strong>于雨</strong>(github @AlexStocks)，dubbo-go 项目和社区负责人，一个有十多年服务端做着基础架构研发一线工作经验的程序员，陆续参与改进过 Muduo/Pika/Dubbo/Sentinel-go 等知名项目，目前在蚂蚁金服可信原生部从事容器编排和 service mesh 工作。</p> |
| </blockquote></description></item><item><title>Blog: 分布式事务框架 seata-golang 通信模型详解</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/15/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1%E6%A1%86%E6%9E%B6-seata-golang-%E9%80%9A%E4%BF%A1%E6%A8%A1%E5%9E%8B%E8%AF%A6%E8%A7%A3/</link><pubDate>Fri, 15 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/15/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1%E6%A1%86%E6%9E%B6-seata-golang-%E9%80%9A%E4%BF%A1%E6%A8%A1%E5%9E%8B%E8%AF%A6%E8%A7%A3/</guid><description> |
| <h2 id="简介">简介</h2> |
| <p>Java 的世界里，大家广泛使用一个高性能网络通信框架 —— netty，很多 RPC 框架都是基于 netty 来实现的。在 golang 的世界里，getty 也是一个类似 netty 的高性能网络通信库。getty 最初由 dubbo-go 项目负责人于雨开发，作为底层通信库在 dubbo-go 中使用。随着 dubbo-go 捐献给 apache 基金会，在社区小伙伴的共同努力下，getty 也最终进入到 apache 这个大家庭，并改名 dubbo-getty。</p> |
| <p>18 年的时候，我在公司里实践微服务，当时遇到最大的问题就是分布式事务问题。同年，阿里在社区开源他们的分布式事务解决方案，我也很快关注到这个项目，起初还叫 fescar，后来更名 seata。由于我对开源技术很感兴趣，加了很多社区群，当时也很关注 dubbo-go 这个项目，在里面默默潜水。随着对 seata 的了解，逐渐萌生了做一个 go 版本的分布式事务框架的想法。</p> |
| <p>要做一个 golang 版的分布式事务框架，首先需要解决的一个问题就是如何实现 RPC 通信。dubbo-go 就是摆在眼前很好的一个例子，遂开始研究 dubbo-go 的底层 getty。</p> |
| <h2 id="如何基于-getty-实现-rpc-通信">如何基于 getty 实现 RPC 通信</h2> |
| <p>getty 框架的整体模型图如下：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/seata/p1.webp" alt="img"></p> |
| <p>下面结合相关代码，详述 seata-golang 的 RPC 通信过程。</p> |
| <h3 id="1-建立连接">1. 建立连接</h3> |
| <p>实现 RPC 通信，首先要建立网络连接，这里先从 client.go 开始看起。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (c <span style="color:#719e07">*</span>client) <span style="color:#268bd2">connect</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> ( |
| </span></span><span style="display:flex;"><span> err <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> ss Session |
| </span></span><span style="display:flex;"><span> ) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 建立一个 session 连接 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ss = c.<span style="color:#268bd2">dial</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> ss <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// client has been closed |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">break</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> err = c.<span style="color:#268bd2">newSession</span>(ss) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 收发报文 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ss.(<span style="color:#719e07">*</span>session).<span style="color:#268bd2">run</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 此处省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">break</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// don&#39;t distinguish between tcp connection and websocket connection. Because |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// gorilla/websocket/conn.go:(Conn)Close also invoke net.Conn.Close() |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ss.<span style="color:#268bd2">Conn</span>().<span style="color:#268bd2">Close</span>() |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><code>connect()</code> 方法通过 <code>dial()</code> 方法得到了一个 session 连接，进入 <code>dial()</code> 方法：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (c <span style="color:#719e07">*</span>client) <span style="color:#268bd2">dial</span>() Session { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">switch</span> c.endPointType { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> TCP_CLIENT: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> c.<span style="color:#268bd2">dialTCP</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> UDP_CLIENT: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> c.<span style="color:#268bd2">dialUDP</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> WS_CLIENT: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> c.<span style="color:#268bd2">dialWS</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> WSS_CLIENT: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> c.<span style="color:#268bd2">dialWSS</span>() |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>我们关注的是 TCP 连接，所以继续进入 <code>c.dialTCP()</code> 方法：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (c <span style="color:#719e07">*</span>client) <span style="color:#268bd2">dialTCP</span>() Session { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> ( |
| </span></span><span style="display:flex;"><span> err <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> conn net.Conn |
| </span></span><span style="display:flex;"><span> ) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> c.<span style="color:#268bd2">IsClosed</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> c.sslEnabled { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> sslConfig, err <span style="color:#719e07">:=</span> c.tlsConfigBuilder.<span style="color:#268bd2">BuildTlsConfig</span>(); err <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> <span style="color:#719e07">&amp;&amp;</span> sslConfig <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> d <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>net.Dialer{Timeout: connectTimeout} |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 建立加密连接 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> conn, err = tls.<span style="color:#268bd2">DialWithDialer</span>(d, <span style="color:#2aa198">&#34;tcp&#34;</span>, c.addr, sslConfig) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 建立 tcp 连接 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> conn, err = net.<span style="color:#268bd2">DialTimeout</span>(<span style="color:#2aa198">&#34;tcp&#34;</span>, c.addr, connectTimeout) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> <span style="color:#719e07">&amp;&amp;</span> gxnet.<span style="color:#268bd2">IsSameAddr</span>(conn.<span style="color:#268bd2">RemoteAddr</span>(), conn.<span style="color:#268bd2">LocalAddr</span>()) { |
| </span></span><span style="display:flex;"><span> conn.<span style="color:#268bd2">Close</span>() |
| </span></span><span style="display:flex;"><span> err = errSelfConnect |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 返回一个 TCPSession |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">return</span> <span style="color:#268bd2">newTCPSession</span>(conn, c) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;net.DialTimeout(addr:%s, timeout:%v) = error:%+v&#34;</span>, c.addr, connectTimeout, perrors.<span style="color:#268bd2">WithStack</span>(err)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">&lt;-</span>wheel.<span style="color:#268bd2">After</span>(connectInterval) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>至此，我们知道了 getty 如何建立 TCP 连接，并返回 TCPSession。</p> |
| <h3 id="2-收发报文">2. 收发报文</h3> |
| <p>那它是怎么收发报文的呢，我们回到 connection 方法接着往下看，有这样一行 <code>ss.(*session).run()</code>，在这行代码之后，代码都是很简单的操作，我们猜测这行代码运行的逻辑里面一定包含收发报文的逻辑，接着进入 <code>run()</code> 方法：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (s <span style="color:#719e07">*</span>session) <span style="color:#268bd2">run</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">go</span> s.<span style="color:#268bd2">handleLoop</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">go</span> s.<span style="color:#268bd2">handlePackage</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>这里起了两个 goroutine：<code>handleLoop</code> 和 <code>handlePackage</code>，看字面意思符合我们的猜想，进入 <code>handleLoop()</code> 方法：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (s <span style="color:#719e07">*</span>session) <span style="color:#268bd2">handleLoop</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// A select blocks until one of its cases is ready to run. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// It choose one at random if multiple are ready. Otherwise it choose default branch if none is ready. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">select</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> outPkg, ok = <span style="color:#719e07">&lt;-</span>s.wQ: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> iovec = iovec[:<span style="color:#2aa198">0</span>] |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> idx <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; idx &lt; maxIovecNum; idx<span style="color:#719e07">++</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 通过 s.writer 将 interface{} 类型的 outPkg 编码成二进制的比特 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> pkgBytes, err = s.writer.<span style="color:#268bd2">Write</span>(s, outPkg) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> iovec = <span style="color:#b58900">append</span>(iovec, pkgBytes) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 将这些二进制比特发送出去 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> err = s.<span style="color:#268bd2">WriteBytesArray</span>(iovec[:]<span style="color:#719e07">...</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;%s, [session.handleLoop]s.WriteBytesArray(iovec len:%d) = error:%+v&#34;</span>, |
| </span></span><span style="display:flex;"><span> s.<span style="color:#268bd2">sessionToken</span>(), <span style="color:#b58900">len</span>(iovec), perrors.<span style="color:#268bd2">WithStack</span>(err)) |
| </span></span><span style="display:flex;"><span> s.<span style="color:#268bd2">stop</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// break LOOP |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> flag = <span style="color:#cb4b16">false</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> <span style="color:#719e07">&lt;-</span>wheel.<span style="color:#268bd2">After</span>(s.period): |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> flag { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> wsFlag { |
| </span></span><span style="display:flex;"><span> err <span style="color:#719e07">:=</span> wsConn.<span style="color:#268bd2">writePing</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Warnf</span>(<span style="color:#2aa198">&#34;wsConn.writePing() = error:%+v&#34;</span>, perrors.<span style="color:#268bd2">WithStack</span>(err)) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 定时执行的逻辑，心跳等 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> s.listener.<span style="color:#268bd2">OnCron</span>(s) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>通过上面的代码，我们不难发现，<code>handleLoop()</code> 方法处理的是发送报文的逻辑，RPC 需要发送的消息首先由 <code>s.writer</code> 编码成二进制比特，然后通过建立的 TCP 连接发送出去。这个 <code>s.writer</code> 对应的 Writer 接口是 RPC 框架必须要实现的一个接口。</p> |
| <p>继续看 <code>handlePackage()</code> 方法：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (s <span style="color:#719e07">*</span>session) <span style="color:#268bd2">handlePackage</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> _, ok <span style="color:#719e07">:=</span> s.Connection.(<span style="color:#719e07">*</span>gettyTCPConn); ok { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> s.reader <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> errStr <span style="color:#719e07">:=</span> fmt.<span style="color:#268bd2">Sprintf</span>(<span style="color:#2aa198">&#34;session{name:%s, conn:%#v, reader:%#v}&#34;</span>, s.name, s.Connection, s.reader) |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Error</span>(errStr) |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(errStr) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> err = s.<span style="color:#268bd2">handleTCPPackage</span>() |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> <span style="color:#719e07">if</span> _, ok <span style="color:#719e07">:=</span> s.Connection.(<span style="color:#719e07">*</span>gettyWSConn); ok { |
| </span></span><span style="display:flex;"><span> err = s.<span style="color:#268bd2">handleWSPackage</span>() |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> <span style="color:#719e07">if</span> _, ok <span style="color:#719e07">:=</span> s.Connection.(<span style="color:#719e07">*</span>gettyUDPConn); ok { |
| </span></span><span style="display:flex;"><span> err = s.<span style="color:#268bd2">handleUDPPackage</span>() |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(fmt.<span style="color:#268bd2">Sprintf</span>(<span style="color:#2aa198">&#34;unknown type session{%#v}&#34;</span>, s)) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>进入 <code>handleTCPPackage()</code> 方法：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (s <span style="color:#719e07">*</span>session) <span style="color:#268bd2">handleTCPPackage</span>() <span style="color:#dc322f">error</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> conn = s.Connection.(<span style="color:#719e07">*</span>gettyTCPConn) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> bufLen = <span style="color:#2aa198">0</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// for clause for the network timeout condition check |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// s.conn.SetReadTimeout(time.Now().Add(s.rTimeout)) |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 从 TCP 连接中收到报文 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> bufLen, err = conn.<span style="color:#268bd2">recv</span>(buf) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">break</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 将收到的报文二进制比特写入 pkgBuf |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> pktBuf.<span style="color:#268bd2">Write</span>(buf[:bufLen]) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> pktBuf.<span style="color:#268bd2">Len</span>() <span style="color:#719e07">&lt;=</span> <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">break</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 通过 s.reader 将收到的报文解码成 RPC 消息 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> pkg, pkgLen, err = s.reader.<span style="color:#268bd2">Read</span>(s, pktBuf.<span style="color:#268bd2">Bytes</span>()) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 省略部分代码 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> s.<span style="color:#268bd2">UpdateActive</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 将收到的消息放入 TaskQueue 供 RPC 消费端消费 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> s.<span style="color:#268bd2">addTask</span>(pkg) |
| </span></span><span style="display:flex;"><span> pktBuf.<span style="color:#268bd2">Next</span>(pkgLen) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// continue to handle case 5 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> exit { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">break</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> perrors.<span style="color:#268bd2">WithStack</span>(err) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>从上面的代码逻辑我们分析出，RPC 消费端需要将从 TCP 连接收到的二进制比特报文解码成 RPC 能消费的消息，这个工作由 s.reader 实现，所以，我们要构建 RPC 通信层也需要实现 s.reader 对应的 Reader 接口。</p> |
| <h3 id="3-底层处理网络报文的逻辑如何与业务逻辑解耦">3. 底层处理网络报文的逻辑如何与业务逻辑解耦</h3> |
| <p>我们都知道，netty 通过 boss 线程和 worker 线程实现了底层网络逻辑和业务逻辑的解耦。那么，getty 是如何实现的呢？</p> |
| <p>在 <code>handlePackage()</code> 方法最后，我们看到，收到的消息被放入了 <code>s.addTask(pkg)</code> 这个方法，接着往下分析：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (s <span style="color:#719e07">*</span>session) <span style="color:#268bd2">addTask</span>(pkg <span style="color:#268bd2">interface</span>{}) { |
| </span></span><span style="display:flex;"><span> f <span style="color:#719e07">:=</span> <span style="color:#268bd2">func</span>() { |
| </span></span><span style="display:flex;"><span> s.listener.<span style="color:#268bd2">OnMessage</span>(s, pkg) |
| </span></span><span style="display:flex;"><span> s.<span style="color:#268bd2">incReadPkgNum</span>() |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> taskPool <span style="color:#719e07">:=</span> s.<span style="color:#268bd2">EndPoint</span>().<span style="color:#268bd2">GetTaskPool</span>(); taskPool <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> taskPool.<span style="color:#268bd2">AddTaskAlways</span>(f) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">f</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><code>pkg</code> 参数传递到了一个匿名方法，这个方法最终放入了 <code>taskPool</code>。这个方法很关键，在我后来写 seata-golang 代码的时候，就遇到了一个坑，这个坑后面分析。</p> |
| <p>接着我们看一下 taskPool 的定义：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// NewTaskPoolSimple build a simple task pool |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">NewTaskPoolSimple</span>(size <span style="color:#dc322f">int</span>) GenericTaskPool { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> size &lt; <span style="color:#2aa198">1</span> { |
| </span></span><span style="display:flex;"><span> size = runtime.<span style="color:#268bd2">NumCPU</span>() <span style="color:#719e07">*</span> <span style="color:#2aa198">100</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>taskPoolSimple{ |
| </span></span><span style="display:flex;"><span> work: <span style="color:#b58900">make</span>(<span style="color:#268bd2">chan</span> task), |
| </span></span><span style="display:flex;"><span> sem: <span style="color:#b58900">make</span>(<span style="color:#268bd2">chan</span> <span style="color:#268bd2">struct</span>{}, size), |
| </span></span><span style="display:flex;"><span> done: <span style="color:#b58900">make</span>(<span style="color:#268bd2">chan</span> <span style="color:#268bd2">struct</span>{}), |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>构建了一个缓冲大小为 size （默认为 <code>runtime.NumCPU() * 100</code>） 的 channel <code>sem</code>。再看方法 <code>AddTaskAlways(t task)</code>：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>taskPoolSimple) <span style="color:#268bd2">AddTaskAlways</span>(t task) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">select</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> <span style="color:#719e07">&lt;-</span>p.done: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">default</span>: |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">select</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> p.work <span style="color:#719e07">&lt;-</span> t: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">default</span>: |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">select</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> p.work <span style="color:#719e07">&lt;-</span> t: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> p.sem <span style="color:#719e07">&lt;-</span> <span style="color:#268bd2">struct</span>{}{}: |
| </span></span><span style="display:flex;"><span> p.wg.<span style="color:#268bd2">Add</span>(<span style="color:#2aa198">1</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">go</span> p.<span style="color:#268bd2">worker</span>(t) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">default</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">goSafely</span>(t) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>加入的任务，会先由 len(p.sem) 个 goroutine 去消费，如果没有 goroutine 空闲，则会启动一个临时的 goroutine 去运行 t()。相当于有 len(p.sem) 个 goroutine 组成了 goroutine pool，pool 中的 goroutine 去处理业务逻辑，而不是由处理网络报文的 goroutine 去运行业务逻辑，从而实现了解耦。写 seata-golang 时遇到的一个坑，就是忘记设置 taskPool 造成了处理业务逻辑和处理底层网络报文逻辑的 goroutine 是同一个，我在业务逻辑中阻塞等待一个任务完成时，阻塞了整个 goroutine，使得阻塞期间收不到任何报文。</p> |
| <h3 id="4-具体实现">4. 具体实现</h3> |
| <p>下面的代码见getty.go ：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// Reader is used to unmarshal a complete pkg from buffer |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> Reader <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Read</span>(Session, []<span style="color:#dc322f">byte</span>) (<span style="color:#268bd2">interface</span>{}, <span style="color:#dc322f">int</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// Writer is used to marshal pkg and write to session |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> Writer <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// if @Session is udpGettySession, the second parameter is UDPContext. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Write</span>(Session, <span style="color:#268bd2">interface</span>{}) ([]<span style="color:#dc322f">byte</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// ReadWriter interface use for handle application packages |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> ReadWriter <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> Reader |
| </span></span><span style="display:flex;"><span> Writer |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// EventListener is used to process pkg that received from remote session |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> EventListener <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// invoked when session opened |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// If the return error is not nil, @Session will be closed. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">OnOpen</span>(Session) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// invoked when session closed. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">OnClose</span>(Session) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// invoked when got error. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">OnError</span>(Session, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// invoked periodically, its period can be set by (Session)SetCronPeriod |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">OnCron</span>(Session) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// invoked when getty received a package. Pls attention that do not handle long time |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// logic processing in this func. You&#39;d better set the package&#39;s maximum length. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// If the message&#39;s length is greater than it, u should should return err in |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// Reader{Read} and getty will close this connection soon. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// If ur logic processing in this func will take a long time, u should start a goroutine |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// pool(like working thread pool in cpp) to handle the processing asynchronously. Or u |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// can do the logic processing in other asynchronous way. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// !!!In short, ur OnMessage callback func should return asap. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// If this is a udp event listener, the second parameter type is UDPContext. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">OnMessage</span>(Session, <span style="color:#268bd2">interface</span>{}) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>通过对整个 getty 代码的分析，我们只要实现 <code>ReadWriter</code> 来对 RPC 消息编解码，再实现 <code>EventListener</code> 来处理 RPC 消息的对应的具体逻辑，将 <code>ReadWriter</code> 实现和 <code>EventLister</code> 实现注入到 RPC 的 Client 和 Server 端，则可实现 RPC 通信。</p> |
| <h3 id="1编解码协议实现">1）编解码协议实现</h3> |
| <p>下面是 seata 协议的定义：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/seata/p2.webp" alt="img"></p> |
| <p>在 ReadWriter 接口的实现 <code>RpcPackageHandler</code> 中，调用 Codec 方法对消息体按照上面的格式编解码：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>// 消息编码为二进制比特 |
| </span></span><span style="display:flex;"><span>func MessageEncoder(codecType byte, in interface{}) []byte { |
| </span></span><span style="display:flex;"><span> switch codecType { |
| </span></span><span style="display:flex;"><span> case SEATA: |
| </span></span><span style="display:flex;"><span> return SeataEncoder(in) |
| </span></span><span style="display:flex;"><span> default: |
| </span></span><span style="display:flex;"><span> log.Errorf(&#34;not support codecType, %s&#34;, codecType) |
| </span></span><span style="display:flex;"><span> return nil |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>// 二进制比特解码为消息体 |
| </span></span><span style="display:flex;"><span>func MessageDecoder(codecType byte, in []byte) (interface{}, int) { |
| </span></span><span style="display:flex;"><span> switch codecType { |
| </span></span><span style="display:flex;"><span> case SEATA: |
| </span></span><span style="display:flex;"><span> return SeataDecoder(in) |
| </span></span><span style="display:flex;"><span> default: |
| </span></span><span style="display:flex;"><span> log.Errorf(&#34;not support codecType, %s&#34;, codecType) |
| </span></span><span style="display:flex;"><span> return nil, 0 |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="2client-端实现">2）Client 端实现</h3> |
| <p>再来看 client 端 <code>EventListener</code> 的实现 <code>RpcRemotingClient</code>：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (client <span style="color:#719e07">*</span>RpcRemoteClient) <span style="color:#268bd2">OnOpen</span>(session getty.Session) <span style="color:#dc322f">error</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">go</span> <span style="color:#268bd2">func</span>() |
| </span></span><span style="display:flex;"><span> request <span style="color:#719e07">:=</span> protocal.RegisterTMRequest{AbstractIdentifyRequest: protocal.AbstractIdentifyRequest{ |
| </span></span><span style="display:flex;"><span> ApplicationId: client.conf.ApplicationId, |
| </span></span><span style="display:flex;"><span> TransactionServiceGroup: client.conf.TransactionServiceGroup, |
| </span></span><span style="display:flex;"><span> }} |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 建立连接后向 Transaction Coordinator 发起注册 TransactionManager 的请求 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> _, err <span style="color:#719e07">:=</span> client.<span style="color:#268bd2">sendAsyncRequestWithResponse</span>(session, request, RPC_REQUEST_TIMEOUT) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 将与 Transaction Coordinator 建立的连接保存在连接池供后续使用 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> clientSessionManager.<span style="color:#268bd2">RegisterGettySession</span>(session) |
| </span></span><span style="display:flex;"><span> client.GettySessionOnOpenChannel <span style="color:#719e07">&lt;-</span> session.<span style="color:#268bd2">RemoteAddr</span>() |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> }() |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// OnError ... |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (client <span style="color:#719e07">*</span>RpcRemoteClient) <span style="color:#268bd2">OnError</span>(session getty.Session, err <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> clientSessionManager.<span style="color:#268bd2">ReleaseGettySession</span>(session) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// OnClose ... |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (client <span style="color:#719e07">*</span>RpcRemoteClient) <span style="color:#268bd2">OnClose</span>(session getty.Session) { |
| </span></span><span style="display:flex;"><span> clientSessionManager.<span style="color:#268bd2">ReleaseGettySession</span>(session) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// OnMessage ... |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (client <span style="color:#719e07">*</span>RpcRemoteClient) <span style="color:#268bd2">OnMessage</span>(session getty.Session, pkg <span style="color:#268bd2">interface</span>{}) { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Info</span>(<span style="color:#2aa198">&#34;received message:{%v}&#34;</span>, pkg) |
| </span></span><span style="display:flex;"><span> rpcMessage, ok <span style="color:#719e07">:=</span> pkg.(protocal.RpcMessage) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> ok { |
| </span></span><span style="display:flex;"><span> heartBeat, isHeartBeat <span style="color:#719e07">:=</span> rpcMessage.Body.(protocal.HeartBeatMessage) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> isHeartBeat <span style="color:#719e07">&amp;&amp;</span> heartBeat <span style="color:#719e07">==</span> protocal.HeartBeatMessagePong { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Debugf</span>(<span style="color:#2aa198">&#34;received PONG from %s&#34;</span>, session.<span style="color:#268bd2">RemoteAddr</span>()) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> rpcMessage.MessageType <span style="color:#719e07">==</span> protocal.MSGTYPE_RESQUEST <span style="color:#719e07">||</span> |
| </span></span><span style="display:flex;"><span> rpcMessage.MessageType <span style="color:#719e07">==</span> protocal.MSGTYPE_RESQUEST_ONEWAY { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Debugf</span>(<span style="color:#2aa198">&#34;msgId:%s, body:%v&#34;</span>, rpcMessage.Id, rpcMessage.Body) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 处理事务消息，提交 or 回滚 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> client.<span style="color:#268bd2">onMessage</span>(rpcMessage, session.<span style="color:#268bd2">RemoteAddr</span>()) |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> resp, loaded <span style="color:#719e07">:=</span> client.futures.<span style="color:#268bd2">Load</span>(rpcMessage.Id) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> loaded { |
| </span></span><span style="display:flex;"><span> response <span style="color:#719e07">:=</span> resp.(<span style="color:#719e07">*</span>getty2.MessageFuture) |
| </span></span><span style="display:flex;"><span> response.Response = rpcMessage.Body |
| </span></span><span style="display:flex;"><span> response.Done <span style="color:#719e07">&lt;-</span> <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> client.futures.<span style="color:#268bd2">Delete</span>(rpcMessage.Id) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// OnCron ... |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (client <span style="color:#719e07">*</span>RpcRemoteClient) <span style="color:#268bd2">OnCron</span>(session getty.Session) { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 发送心跳 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> client.<span style="color:#268bd2">defaultSendRequest</span>(session, protocal.HeartBeatMessagePing) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><code>clientSessionManager.RegisterGettySession(session)</code> 的逻辑将在下文中分析。</p> |
| <h3 id="3server-端-transaction-coordinator-实现">3）Server 端 Transaction Coordinator 实现</h3> |
| <p>代码见 <code>DefaultCoordinator</code>：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (coordinator <span style="color:#719e07">*</span>DefaultCoordinator) <span style="color:#268bd2">OnOpen</span>(session getty.Session) <span style="color:#dc322f">error</span> { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;got getty_session:%s&#34;</span>, session.<span style="color:#268bd2">Stat</span>()) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (coordinator <span style="color:#719e07">*</span>DefaultCoordinator) <span style="color:#268bd2">OnError</span>(session getty.Session, err <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 释放 TCP 连接 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> SessionManager.<span style="color:#268bd2">ReleaseGettySession</span>(session) |
| </span></span><span style="display:flex;"><span> session.<span style="color:#268bd2">Close</span>() |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;getty_session{%s} got error{%v}, will be closed.&#34;</span>, session.<span style="color:#268bd2">Stat</span>(), err) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (coordinator <span style="color:#719e07">*</span>DefaultCoordinator) <span style="color:#268bd2">OnClose</span>(session getty.Session) { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Info</span>(<span style="color:#2aa198">&#34;getty_session{%s} is closing......&#34;</span>, session.<span style="color:#268bd2">Stat</span>()) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (coordinator <span style="color:#719e07">*</span>DefaultCoordinator) <span style="color:#268bd2">OnMessage</span>(session getty.Session, pkg <span style="color:#268bd2">interface</span>{}) { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Debugf</span>(<span style="color:#2aa198">&#34;received message:{%v}&#34;</span>, pkg) |
| </span></span><span style="display:flex;"><span> rpcMessage, ok <span style="color:#719e07">:=</span> pkg.(protocal.RpcMessage) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> ok { |
| </span></span><span style="display:flex;"><span> _, isRegTM <span style="color:#719e07">:=</span> rpcMessage.Body.(protocal.RegisterTMRequest) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> isRegTM { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 将 TransactionManager 信息和 TCP 连接建立映射关系 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> coordinator.<span style="color:#268bd2">OnRegTmMessage</span>(rpcMessage, session) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> heartBeat, isHeartBeat <span style="color:#719e07">:=</span> rpcMessage.Body.(protocal.HeartBeatMessage) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> isHeartBeat <span style="color:#719e07">&amp;&amp;</span> heartBeat <span style="color:#719e07">==</span> protocal.HeartBeatMessagePing { |
| </span></span><span style="display:flex;"><span> coordinator.<span style="color:#268bd2">OnCheckMessage</span>(rpcMessage, session) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> rpcMessage.MessageType <span style="color:#719e07">==</span> protocal.MSGTYPE_RESQUEST <span style="color:#719e07">||</span> |
| </span></span><span style="display:flex;"><span> rpcMessage.MessageType <span style="color:#719e07">==</span> protocal.MSGTYPE_RESQUEST_ONEWAY { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Debugf</span>(<span style="color:#2aa198">&#34;msgId:%s, body:%v&#34;</span>, rpcMessage.Id, rpcMessage.Body) |
| </span></span><span style="display:flex;"><span> _, isRegRM <span style="color:#719e07">:=</span> rpcMessage.Body.(protocal.RegisterRMRequest) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> isRegRM { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 将 ResourceManager 信息和 TCP 连接建立映射关系 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> coordinator.<span style="color:#268bd2">OnRegRmMessage</span>(rpcMessage, session) |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> SessionManager.<span style="color:#268bd2">IsRegistered</span>(session) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">defer</span> <span style="color:#268bd2">func</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">:=</span> <span style="color:#b58900">recover</span>(); err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;Catch Exception while do RPC, request: %v,err: %w&#34;</span>, rpcMessage, err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> }() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 处理事务消息，全局事务注册、分支事务注册、分支事务提交、全局事务回滚等 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> coordinator.<span style="color:#268bd2">OnTrxMessage</span>(rpcMessage, session) |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> session.<span style="color:#268bd2">Close</span>() |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;close a unhandled connection! [%v]&#34;</span>, session) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> { |
| </span></span><span style="display:flex;"><span> resp, loaded <span style="color:#719e07">:=</span> coordinator.futures.<span style="color:#268bd2">Load</span>(rpcMessage.Id) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> loaded { |
| </span></span><span style="display:flex;"><span> response <span style="color:#719e07">:=</span> resp.(<span style="color:#719e07">*</span>getty2.MessageFuture) |
| </span></span><span style="display:flex;"><span> response.Response = rpcMessage.Body |
| </span></span><span style="display:flex;"><span> response.Done <span style="color:#719e07">&lt;-</span> <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> coordinator.futures.<span style="color:#268bd2">Delete</span>(rpcMessage.Id) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (coordinator <span style="color:#719e07">*</span>DefaultCoordinator) <span style="color:#268bd2">OnCron</span>(session getty.Session) { |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><code>coordinator.OnRegTmMessage(rpcMessage, session)</code> 注册 Transaction Manager，<code>coordinator.OnRegRmMessage(rpcMessage, session)</code> 注册 Resource Manager。具体逻辑分析见下文。</p> |
| <p>消息进入 <code>coordinator.OnTrxMessage(rpcMessage, session)</code> 方法，将按照消息的类型码路由到具体的逻辑当中：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">switch</span> msg.<span style="color:#268bd2">GetTypeCode</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> protocal.TypeGlobalBegin: |
| </span></span><span style="display:flex;"><span> req <span style="color:#719e07">:=</span> msg.(protocal.GlobalBeginRequest) |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> coordinator.<span style="color:#268bd2">doGlobalBegin</span>(req, ctx) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> protocal.TypeGlobalStatus: |
| </span></span><span style="display:flex;"><span> req <span style="color:#719e07">:=</span> msg.(protocal.GlobalStatusRequest) |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> coordinator.<span style="color:#268bd2">doGlobalStatus</span>(req, ctx) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> protocal.TypeGlobalReport: |
| </span></span><span style="display:flex;"><span> req <span style="color:#719e07">:=</span> msg.(protocal.GlobalReportRequest) |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> coordinator.<span style="color:#268bd2">doGlobalReport</span>(req, ctx) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> protocal.TypeGlobalCommit: |
| </span></span><span style="display:flex;"><span> req <span style="color:#719e07">:=</span> msg.(protocal.GlobalCommitRequest) |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> coordinator.<span style="color:#268bd2">doGlobalCommit</span>(req, ctx) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> protocal.TypeGlobalRollback: |
| </span></span><span style="display:flex;"><span> req <span style="color:#719e07">:=</span> msg.(protocal.GlobalRollbackRequest) |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> coordinator.<span style="color:#268bd2">doGlobalRollback</span>(req, ctx) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> protocal.TypeBranchRegister: |
| </span></span><span style="display:flex;"><span> req <span style="color:#719e07">:=</span> msg.(protocal.BranchRegisterRequest) |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> coordinator.<span style="color:#268bd2">doBranchRegister</span>(req, ctx) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> protocal.TypeBranchStatusReport: |
| </span></span><span style="display:flex;"><span> req <span style="color:#719e07">:=</span> msg.(protocal.BranchReportRequest) |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> coordinator.<span style="color:#268bd2">doBranchReport</span>(req, ctx) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">default</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span></code></pre></div><h3 id="4session-manager-分析">4）session manager 分析</h3> |
| <p>Client 端同 Transaction Coordinator 建立连接起连接后，通过 <code>clientSessionManager.RegisterGettySession(session)</code> 将连接保存在 <code>serverSessions = sync.Map{}</code> 这个 map 中。map 的 key 为从 session 中获取的 RemoteAddress 即 Transaction Coordinator 的地址，value 为 session。这样，Client 端就可以通过 map 中的一个 session 来向 Transaction Coordinator 注册 Transaction Manager 和 Resource Manager 了。具体代码见 <code>getty_client_session_manager.go</code>。</p> |
| <p>Transaction Manager 和 Resource Manager 注册到 Transaction Coordinator 后，一个连接既有可能用来发送 TM 消息也有可能用来发送 RM 消息。我们通过 RpcContext 来标识一个连接信息：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> RpcContext <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> Version <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> TransactionServiceGroup <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> ClientRole meta.TransactionRole |
| </span></span><span style="display:flex;"><span> ApplicationId <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> ClientId <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> ResourceSets <span style="color:#719e07">*</span>model.Set |
| </span></span><span style="display:flex;"><span> Session getty.Session |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>当收到事务消息时，我们需要构造这样一个 RpcContext 供后续事务处理逻辑使用。所以，我们会构造下列 map 来缓存映射关系：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">var</span> ( |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// session -&gt; transactionRole |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// TM will register before RM, if a session is not the TM registered, |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// it will be the RM registered |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> session_transactionroles = sync.Map{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// session -&gt; applicationId |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> identified_sessions = sync.Map{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// applicationId -&gt; ip -&gt; port -&gt; session |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> client_sessions = sync.Map{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// applicationId -&gt; resourceIds |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> client_resources = sync.Map{} |
| </span></span><span style="display:flex;"><span>) |
| </span></span></code></pre></div><p>这样，Transaction Manager 和 Resource Manager 分别通过 <code>coordinator.OnRegTmMessage(rpcMessage, session)</code> 和 <code>coordinator.OnRegRmMessage(rpcMessage, session)</code> 注册到 Transaction Coordinator 时，会在上述 client_sessions map 中缓存 applicationId、ip、port 与 session 的关系，在 client_resources map 中缓存 applicationId 与 resourceIds（一个应用可能存在多个 Resource Manager） 的关系。</p> |
| <p>在需要时，我们就可以通过上述映射关系构造一个 RpcContext。这部分的实现和 java 版 seata 有很大的不同，感兴趣的可以深入了解一下。具体代码见 <code>getty_session_manager.go</code>。</p> |
| <p>至此，我们就分析完了 seata-golang 整个 RPC 通信模型的机制。</p> |
| <h3 id="seata-golang-的未来">seata-golang 的未来</h3> |
| <p>seata-golang 从今年 4 月份开始开发，到 8 月份基本实现和 java 版 seata 1.2 协议的互通，对 mysql 数据库实现了 AT 模式（自动协调分布式事务的提交回滚），实现了 TCC 模式，TC 端使用 mysql 存储数据，使 TC 变成一个无状态应用支持高可用部署。下图展示了 AT 模式的原理：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/seata/p3.webp" alt="img"></p> |
| <p>后续，还有许多工作可以做，比如：对注册中心的支持、对配置中心的支持、和 java 版 seata 1.4 的协议互通、其他数据库的支持、raft transaction coordinator 的实现等，希望对分布式事务问题感兴趣的开发者可以加入进来一起来打造一个完善的 golang 的分布式事务框架。如果你有任何疑问，欢迎钉钉扫码加入交流群【钉钉群号 33069364】：</p> |
| <p>另外，欢迎对 dubbogo 感兴趣的朋友到 dubbogo 社区钉钉群（钉钉群号 31363295）沟通 dubbogo 技术问题。</p> |
| <h3 id="参考资料">参考资料</h3> |
| <ul> |
| <li>seata 官方: <a href="https://seata.io">https://seata.io</a></li> |
| <li>java 版 seata: <a href="https://github.com/seata/seata">https://github.com/seata/seata</a></li> |
| <li>seata-golang 项目地址: <a href="https://github.com/opentrx/seata-golang">https://github.com/opentrx/seata-golang</a></li> |
| <li>seata-golang go 夜读 b 站分享: <a href="https://www.bilibili.com/video/BV1oz411e72T">https://www.bilibili.com/video/BV1oz411e72T</a></li> |
| </ul> |
| <blockquote> |
| <p>作者简介</p> |
| <p><strong>刘晓敏</strong> (GitHubID dk-lockdown)，目前就职于 h3c 成都分公司，擅长使用 Java/Go 语言，在云原生和微服务相关技术方向均有涉猎，目前专攻分布式事务。 |
| <strong>于雨</strong>(（github @AlexStocks），dubbo-go 项目和社区负责人，一个有十多年服务端基础架构研发一线工作经验的程序员，陆续参与改进过 Muduo/Pika/Dubbo/Sentinel-go 等知名项目，目前在蚂蚁金服可信原生部从事容器编排和 service mesh 工作。</p> |
| </blockquote></description></item><item><title>Blog: dubbo-go 可信调用实现</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E5%8F%AF%E4%BF%A1%E8%B0%83%E7%94%A8%E5%AE%9E%E7%8E%B0/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E5%8F%AF%E4%BF%A1%E8%B0%83%E7%94%A8%E5%AE%9E%E7%8E%B0/</guid><description> |
| <p>Apache Dubbo/Dubbo-Go 作为阿里巴巴开源的一款服务治理框架，因其适应 Java/Go 开发者面向接口的编程习惯、完全透明的调用方式、优越的性能以及强大的扩展性等优点，在国内使用非常广泛。除此之外，Dubbo 开源版本原生集成了很多开箱即用的服务治理功能，包括链路追踪，路由、负载均衡、服务注册发现、监控、认证等。</p> |
| <p>本文将讲解如何在 Dubbo/Dubbo-Go 中实现灵活，安全和高效的身份验证和授权方案。</p> |
| <h2 id="可信的目的">可信的目的</h2> |
| <p>为什么需要鉴权认证？实际生产中类似支付之类的安全性敏感的业务会有限制匿名系统调用的需求，其他业务在接入该类敏感业务之前，需要通过审批方可正常调用，这就需要对这类敏感服务进行权限管控。尽管 Dubbo 开源版本中支持 Token 方式的鉴权实现，但是该实现方式总体来说安全性并不高，并且无法满足我们需要动态下发以及变更的灵活性需求。</p> |
| <p>针对于此，我们内部着重从巩固安全性和拓展灵活性层面重新设计了一套 Dubbo/Dubbo-Go 的服务间调用的鉴权认证功能。本文我们将主要从实现层面讲解其大致实现思路。</p> |
| <h2 id="可信方案">可信方案</h2> |
| <p>抽象来看鉴权认证主要围绕以下两个问题，</p> |
| <ul> |
| <li>身份认证：指验证应用的身份，每个应用在其生命周期内只有唯一身份，无法变更和伪造。</li> |
| <li>权限鉴定：根据身份信息鉴定权限是否满足调用。权限粒度可以进行控制。</li> |
| </ul> |
| <p>我们通过 Access Key ID/Secret Access Key (后文简称为 AK/SK) 信息标识应用和应用之间的身份关系。例如上游 应用A 依赖下游 服务B 和 C，则 A 对 B 和 C 分别有一套 AK/SK。其相互独立没有任何关系。就算 A服务 的 AK/SK 信息泄漏，也无法通过该 AK/SK 信息调用其他的服务。</p> |
| <p>在权限鉴定方面也借鉴了公有云开放 API 常用的 AK/SK 签名机制。 在请求过程中使用 SK 签名生成 SigningKey，并通过 Dubbo 的 attachment 机制将额外的元数据信息以及 SigningKey 传输到服务端，交由服务端计算和验签，验签通过方能正常处理和响应。</p> |
| <p>签名过程主要通过如下三个方式进行加强 SigningKey 的可靠性和安全性。</p> |
| <ul> |
| <li> |
| <p>验证请求者的身份</p> |
| <p>签名会通过对应应用的SK作为加密密钥对请求元数据(以及参数)进行加密，保证签名的唯一性和不可伪造性。</p> |
| </li> |
| <li> |
| <p>支持对参数进行计算签名，防止非法篡改</p> |
| <p>若请求参数在传输过程中遭到非法篡改，则收到请求后服务端验签匹配将失败，身份校验无法通过，从而防止请求参数被篡改。考虑到签名以及验签过程中加入请求参数的计算可能会影响性能，所以这个过程是可选的。</p> |
| </li> |
| <li> |
| <p>防止重放攻击</p> |
| <p>每一次请求生成的SigningKey都具有指定的有效时间。如请求被截获，该请求无法在有效时间外进行调用。一定程度避免了重放攻击。</p> |
| </li> |
| </ul> |
| <p>同时为了去掉明文配置，防止AK/SK信息泄漏，我们通过鉴权系统分发和管理所有AK/SK信息。并且通过对接内部审批流程，达到流程化和规范化。需要鉴权的应用会通过启动获取的方式拉当前应用分发出去或者是已被授权的AK/SK信息。这种方式也带来了另一种好处，新增、吊销以及更新权限信息也无需重启应用。</p> |
| <h2 id="可信流程">可信流程</h2> |
| <p>结合上面的这些需求和方案，整个接入和鉴权流程图如下所示：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/trusted-call/process.png" alt="img"></p> |
| <p>整体流程如下：</p> |
| <ol> |
| <li>使用该功能的应用需要提前申请对应的证书，并向提供服务的应用提交申请访问工单，由双方负责人审批通过后，请求鉴权服务中心自动生成键值对。</li> |
| <li>除此之外，开启鉴权认证的服务在应用启动之后，会运行一个后台线程，长轮询远鉴权服务中心，查询是否有新增权限变动信息，如果有则进行全量/增量的拉取。</li> |
| <li>上游应用在请求需要鉴权的服务时，会通过SK作为签名算法的 key，对本次请求的元数据信息甚至是参数信息进行计算得到签名，通过 Dubbo协议 Attachment字段 传送到对端，除此之外还有请求时间戳、AK信息等信息。</li> |
| <li>下游应用在处理鉴权服务时会对请求验签，验签通过则继续处理请求，否则直接返回异常。</li> |
| </ol> |
| <p>其中需要说明的是第三步，使用鉴权服务的应用和鉴权服务中心的交互需通过 HTTPS 的双向认证，并在 TLS 信道上进行数据交互，保证 AK/SK 信息传输的安全性。</p> |
| <p>该方案目前已经有 Java/Go 实现，均已合并到 dubbo/dubbo-go。除了默认的 Hmac 签名算法实现之外，我们将签名和认证方法进行抽象，以Dubbo Go中的实现为例。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// Authenticator |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> Authenticator <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Sign |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// give a sign to request |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Sign</span>(protocol.Invocation, <span style="color:#719e07">*</span>common.URL) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Authenticate |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// verify the signature of the request is valid or not |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Authenticate</span>(protocol.Invocation, <span style="color:#719e07">*</span>common.URL) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>使用者可通过 SPI 机制定制签名和认证方式，以及适配公司内部基础设施的密钥服务下发 AK/SK。</p> |
| <h2 id="示例">示例</h2> |
| <p>以 <a href="https://github.com/apache/dubbo-go-samples/tree/master/helloworld/">Helloworld 示例</a> 中的代码接入当前社区版本中的默认鉴权认证功能实现为例：</p> |
| <p>在无需改变代码的情况下，只需要在配置上增加额外的相关鉴权配置即可，dubbo-go 服务端配置示例如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">services</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;UserProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 可以指定多个registry，使用逗号隔开;不指定默认向所有注册中心注册</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;hangzhouzk&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol </span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 相当于dubbo.xml中的interface</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface </span>: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">loadbalance</span>: <span style="color:#2aa198">&#34;random&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 本服务开启auth</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">auth</span>: <span style="color:#2aa198">&#34;true&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 启用auth filter，对请求进行验签</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">filter</span>: <span style="color:#2aa198">&#34;auth&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 默认实现通过配置文件配置AK、SK</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">params</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">.accessKeyId</span>: <span style="color:#2aa198">&#34;SYD8-23DF&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">.secretAccessKey</span>: <span style="color:#2aa198">&#34;BSDY-FDF1&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">warmup</span>: <span style="color:#2aa198">&#34;100&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;GetUser&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">1</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">loadbalance</span>: <span style="color:#2aa198">&#34;random&#34;</span> |
| </span></span></code></pre></div><p>dubbo-go 客户端配置示例如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">references</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;UserProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 可以指定多个registry，使用逗号隔开;不指定默认向所有注册中心注册</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;hangzhouzk&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface</span>: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 本服务开启sign filter，需要签名</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">filter</span>: <span style="color:#2aa198">&#34;sign&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 默认实现通过配置文件配置AK、SK</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">params</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">.accessKeyId</span>: <span style="color:#2aa198">&#34;SYD8-23DF&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">.secretAccessKey</span>: <span style="color:#2aa198">&#34;BSDY-FDF1&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;GetUser&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">3</span> |
| </span></span></code></pre></div><p>可以看到，dubbo-go 接入鉴权认证的功能也十分简单。需要补充说明的是，配置文件文件中 ak/sk 都加了特殊前缀 &ldquo;.&quot;，是为了说明该字段是敏感信息，不能在发起网络请求时传输出去，相关代码可参阅 <a href="https://github.com/apache/dubbo-go/pull/509">dubbo-go-pr-509</a>。</p> |
| <h2 id="总结">总结</h2> |
| <p>Apache Dubbo 作为一款老而弥新的服务治理框架，无论是其自身还是其生态都还在飞速进化中。本文描述的最新实现的可信服务调用，是为了避免敏感接口被匿名用户调用而在 SDK 层面提供的额外保障，在 RPC 层面保障安全性。</p> |
| <p>Dubbo-Go 作为 Dubbo 生态中发展最快的成员，目前基本上保持与 Dubbo 齐头并进的态势。Dubbo-Go 社区钉钉群号为 23331795， 欢迎你的加入。</p> |
| <blockquote> |
| <p>作者信息： 郑泽超，Apache Dubbo/Dubbo-Go committer，GithubID: CodingSinger，目前就职于上海爱奇艺科技有限公司，Java/Golang 开发工程师。</p> |
| </blockquote></description></item><item><title>Blog: Dubbo-go 源码笔记（一）Server 端开启服务过程</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E6%BA%90%E7%A0%81%E7%AC%94%E8%AE%B0%E4%B8%80server-%E7%AB%AF%E5%BC%80%E5%90%AF%E6%9C%8D%E5%8A%A1%E8%BF%87%E7%A8%8B/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E6%BA%90%E7%A0%81%E7%AC%94%E8%AE%B0%E4%B8%80server-%E7%AB%AF%E5%BC%80%E5%90%AF%E6%9C%8D%E5%8A%A1%E8%BF%87%E7%A8%8B/</guid><description> |
| <p>随着微服务架构的流行，许多高性能 rpc 框架应运而生，由阿里开源的 dubbo 框架 go 语言版本的 dubbo-go 也成为了众多开发者不错的选择。本文将介绍 dubbo-go 框架的基本使用方法，以及从 export 调用链的角度进行 server 端源码导读，希望能引导读者进一步认识这款框架。</p> |
| <p>当拿到一款框架之后，一种不错的源码阅读方式大致如下：从运行最基础的 helloworld demo 源码开始 —&gt; 再查看配置文件 —&gt; 开启各种依赖服务（比如zk、consul） —&gt; 开启服务端 —&gt; 再到通过 client 调用服务端 —&gt; 打印完整请求日志和回包。调用成功之后，再根据框架的设计模型，从配置文件解析开始，自顶向下递阅读整个框架的调用栈。</p> |
| <p>对于 C/S 模式的 rpc 请求来说，整个调用栈被拆成了 client 和 server 两部分，所以可以分别从 server 端的配置文件解析阅读到 server 端的监听启动，从 client 端的配置文件解析阅读到一次 invoker Call 调用。这样一次完整请求就明晰了起来。</p> |
| <h2 id="运行官网提供的-helloworld-demo">运行官网提供的 helloworld-demo</h2> |
| <p><strong>官方 demo 相关链接</strong>：https://github.com/dubbogo/dubbo-samples/tree/master/golang/helloworld/dubbo</p> |
| <h3 id="1-dubbo-go-27-版本-quickstart">1. dubbo-go 2.7 版本 QuickStart</h3> |
| <h4 id="1开启一个-go-server-服务">1）开启一个 go-server 服务</h4> |
| <ul> |
| <li>将仓库 clone 到本地</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ git clone https://github.com/dubbogo/dubbo-samples.git |
| </span></span></code></pre></div><ul> |
| <li>进入 dubbo 目录</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ <span style="color:#b58900">cd</span> dubbo-samples/golang/helloworld/dubbo |
| </span></span></code></pre></div><p>进入目录后可看到四个文件夹，分别支持 go 和 java 的 client 以及 server，我们尝试运行一个 go 的 server。进入 app 子文件夹内，可以看到里面保存了 go 文件。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ <span style="color:#b58900">cd</span> go-server/app |
| </span></span></code></pre></div><ul> |
| <li>sample 文件结构</li> |
| </ul> |
| <p>可以在 go-server 里面看到三个文件夹：app、assembly、profiles。</p> |
| <p>其中 app 文件夹下保存 go 源码，assembly 文件夹下保存可选的针对特定环境的 build 脚本，profiles 下保存配置文件。对于 dubbo-go 框架，配置文件非常重要，没有文件将导致服务无法启动。</p> |
| <ul> |
| <li>设置指向配置文件的环境变量</li> |
| </ul> |
| <p>由于 dubbo-go 框架依赖配置文件启动，让框架定位到配置文件的方式就是通过环境变量来找。对于 server 端需要两个必须配置的环境变量：CONF_PROVIDER_FILE_PATH、APP_LOG_CONF_FILE，分别应该指向服务端配置文件、日志配置文件。</p> |
| <p>在 sample 里面，我们可以使用 dev 环境，即 profiles/dev/log.yml 和 profiles/dev/server.yml 两个文件。在 app/ 下，通过命令行中指定好这两个文件：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ <span style="color:#b58900">export</span> <span style="color:#268bd2">CONF_PROVIDER_FILE_PATH</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;../profiles/dev/server.yml&#34;</span> |
| </span></span><span style="display:flex;"><span>$ <span style="color:#b58900">export</span> <span style="color:#268bd2">APP_LOG_CONF_FILE</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;../profiles/dev/log.yml&#34;</span> |
| </span></span></code></pre></div><ul> |
| <li>设置 go 代理并运行服务</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ go run . |
| </span></span></code></pre></div><p>如果提示 timeout，则需要设置 goproxy 代理。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ <span style="color:#b58900">export</span> <span style="color:#268bd2">GOPROXY</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;http://goproxy.io&#34;</span> |
| </span></span></code></pre></div><p>再运行 go run 即可开启服务。</p> |
| <h4 id="2运行-zookeeper">2）运行 zookeeper</h4> |
| <p>安装 zookeeper，并运行 zkServer, 默认为 2181 端口。</p> |
| <h4 id="3运行-go-client-调用-server-服务">3）运行 go-client 调用 server 服务</h4> |
| <ul> |
| <li>进入 go-client 的源码目录</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ <span style="color:#b58900">cd</span> go-client/app |
| </span></span></code></pre></div><ul> |
| <li>同理，在 /app 下配置环境变量</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ <span style="color:#b58900">export</span> <span style="color:#268bd2">CONF_CONSUMER_FILE_PATH</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;../profiles/dev/client.yml&#34;</span> |
| </span></span><span style="display:flex;"><span>$ <span style="color:#b58900">export</span> <span style="color:#268bd2">APP_LOG_CONF_FILE</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;../profiles/dev/log.yml&#34;</span> |
| </span></span></code></pre></div><p>配置 go 代理：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ <span style="color:#b58900">export</span> <span style="color:#268bd2">GOPROXY</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;http://goproxy.io&#34;</span> |
| </span></span></code></pre></div><ul> |
| <li>运行程序</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ go run . |
| </span></span></code></pre></div><p>即可在日志中找到打印出的请求结果：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>response result: &amp;<span style="color:#719e07">{</span>A001 Alex Stocks <span style="color:#2aa198">18</span> 2020-10-28 14:52:49.131 +0800 CST<span style="color:#719e07">}</span> |
| </span></span></code></pre></div><p>同样，在运行的 server 中，也可以在日志中找到打印出的请求：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>req:<span style="color:#719e07">[]</span>interface <span style="color:#719e07">{}{</span><span style="color:#2aa198">&#34;A001&#34;</span><span style="color:#719e07">}</span> |
| </span></span><span style="display:flex;"><span>rsp:main.User<span style="color:#719e07">{</span>Id:<span style="color:#2aa198">&#34;A001&#34;</span>, Name:<span style="color:#2aa198">&#34;Alex Stocks&#34;</span>, Age:18, Time:time.Time<span style="color:#719e07">{</span>...<span style="color:#719e07">}</span> |
| </span></span></code></pre></div><p>恭喜！一次基于 dubbo-go 的 rpc 调用成功。</p> |
| <h4 id="4常见问题">4）常见问题</h4> |
| <ul> |
| <li>当日志开始部分出现 profiderInit 和 ConsumerInit 均失败的日志，检查环境变量中配置路径是否正确，配置文件是否正确。</li> |
| <li>当日志中出现 register 失败的情况，一般为向注册中心注册失败，检查注册中心是否开启，检查配置文件中关于 register 的端口是否正确。</li> |
| <li>sample 的默认开启端口为 20000，确保启动前无占用。</li> |
| </ul> |
| <h3 id="2-配置环境变量">2. 配置环境变量</h3> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">APP_LOG_CONF_FILE</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;../profiles/dev/log.yml&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">CONF_CONSUMER_FILE_PATH</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;../profiles/dev/client.yml&#34;</span> |
| </span></span></code></pre></div><h3 id="3-服务端源码">3. 服务端源码</h3> |
| <h4 id="1目录结构">1）目录结构</h4> |
| <p>dubbo-go 框架的 example 提供的目录如下：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p1.png" alt="img"></p> |
| <ul> |
| <li>app/ 文件夹下存放源码，可以自己编写环境变量配置脚本 buliddev.sh</li> |
| <li>assembly/ 文件夹下存放不同平台的构建脚本</li> |
| <li>profiles/ 文件夹下存放不同环境的配置文件</li> |
| <li>target/ 文件夹下存放可执行文件</li> |
| </ul> |
| <h3 id="2关键源码">2）关键源码</h3> |
| <p>源码放置在 app/ 文件夹下，主要包含 server.go 和 user.go 两个文件，顾名思义，server.go 用于使用框架开启服务以及注册传输协议；user.go 则定义了 rpc-service 结构体，以及传输协议的结构。</p> |
| <ul> |
| <li><strong>user.go</strong></li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetProviderService</span>(<span style="color:#b58900">new</span>(UserProvider)) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// ------for hessian2------ |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">type</span> User <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> Id <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Name <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Age <span style="color:#dc322f">int32</span> |
| </span></span><span style="display:flex;"><span> Time time.Time |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">type</span> UserProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u <span style="color:#719e07">*</span>UserProvider) <span style="color:#268bd2">GetUser</span>(ctx context.Context, req []<span style="color:#268bd2">interface</span>{}) (<span style="color:#719e07">*</span>User, <span style="color:#dc322f">error</span>) { |
| </span></span></code></pre></div><p>可以看到，user.go 中存在 init 函数，是服务端代码中最先被执行的部分。User 为用户自定义的传输结构体，UserProvider 为用户自定义的 rpc_service；包含一个 rpc 函数，GetUser。当然，用户可以自定义其他的 rpc 功能函数。</p> |
| <p>在 init 函数中，调用 config 的 SetProviderService 函数，将当前 rpc_service 注册在框架 config 上。</p> |
| <p><strong>可以查看 dubbo 官方文档提供的设计图：</strong></p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p2.png" alt="img"></p> |
| <p>service 层下面就是 config 层，用户服务会逐层向下注册，最终实现服务端的暴露。</p> |
| <p>rpc-service 注册完毕之后，调用 hessian 接口注册传输结构体 User。</p> |
| <p>至此，init 函数执行完毕。</p> |
| <ul> |
| <li><strong>server.go</strong></li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// they are necessary: |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// export CONF_PROVIDER_FILE_PATH=&#34;xxx&#34; |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// export APP_LOG_CONF_FILE=&#34;xxx&#34; |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">initSignal</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">initSignal</span>() { |
| </span></span><span style="display:flex;"><span> signals <span style="color:#719e07">:=</span> <span style="color:#b58900">make</span>(<span style="color:#268bd2">chan</span> os.Signal, <span style="color:#2aa198">1</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">...</span> |
| </span></span></code></pre></div><p>之后执行 main 函数。</p> |
| <p>main 函数中只进行了两个操作，首先使用 hessian 注册组件将 User 结构体注册（与之前略有重复），从而可以在接下来使用 getty 打解包。</p> |
| <p>之后调用 config.Load 函数，该函数位于框架 config/config_loader.go 内，这个函数是整个框架服务的启动点，<strong>下面会详细讲这个函数内重要的配置处理过程</strong>。执行完 Load() 函数之后，配置文件会读入框架，之后根据配置文件的内容，将注册的 service 实现到配置结构里，再调用 Export 暴露给特定的 registry，进而开启特定的 service 进行对应端口的 tcp 监听，成功启动并且暴露服务。</p> |
| <p>最终开启信号监听 initSignal() 优雅地结束一个服务的启动过程。</p> |
| <h3 id="4-客户端源码">4. 客户端源码</h3> |
| <p>客户端包含 client.go 和 user.go 两个文件，其中 user.go 与服务端完全一致，不再赘述。</p> |
| <ul> |
| <li><strong>client.go</strong></li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// they are necessary: |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// export CONF_CONSUMER_FILE_PATH=&#34;xxx&#34; |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// export APP_LOG_CONF_FILE=&#34;xxx&#34; |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> time.<span style="color:#268bd2">Sleep</span>(<span style="color:#2aa198">3e9</span>) |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;\n\n\nstart to test dubbo&#34;</span>) |
| </span></span><span style="display:flex;"><span> user <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>User{} |
| </span></span><span style="display:flex;"><span> err <span style="color:#719e07">:=</span> userProvider.<span style="color:#268bd2">GetUser</span>(context.<span style="color:#268bd2">TODO</span>(), []<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;A001&#34;</span>}, user) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, user) |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">initSignal</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>main 函数和服务端也类似，首先将传输结构注册到 hessian 上，再调用 config.Load() 函数。在下文会介绍，客户端和服务端会根据配置类型执行 config.Load() 中特定的函数 loadConsumerConfig() 和 loadProviderConfig()，从而达到“开启服务”、“调用服务”的目的。</p> |
| <p>加载完配置之后，还是通过实现服务、增加函数 proxy、申请 registry 和 reloadInvoker 指向服务端 ip 等操作，重写了客户端实例 userProvider 的对应函数，这时再通过调用 GetUser 函数，可以直接通过 invoker，调用到已经开启的服务端，实现 rpc 过程。</p> |
| <p>下面会从 server 端和 client 端两个角度，详细讲解服务启动、registry 注册和调用过程。</p> |
| <h3 id="5-自定义配置文件非环境变量方法">5. 自定义配置文件（非环境变量）方法</h3> |
| <h4 id="1服务端自定义配置文件">1）服务端自定义配置文件</h4> |
| <ul> |
| <li>var providerConfigStr = <code>xxxxx</code>// 配置文件内容，可以参考 log 和 client。在这里你可以定义配置文件的获取方式，比如配置中心，本地文件读取。</li> |
| </ul> |
| <blockquote> |
| <p><strong>log 地址</strong>：https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/profiles/release/log.yml</p> |
| <p><strong>client 地址</strong>：https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/profiles/release/client.yml</p> |
| </blockquote> |
| <ul> |
| <li>在 <code>config.Load()</code> 之前设置配置，例如：</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span> providerConfig <span style="color:#719e07">:=</span> config.ProviderConfig{} |
| </span></span><span style="display:flex;"><span> yaml.<span style="color:#268bd2">Unmarshal</span>([]<span style="color:#b58900">byte</span>(providerConfigStr), <span style="color:#719e07">&amp;</span>providerConfig) |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetProviderConfig</span>(providerConfig) |
| </span></span><span style="display:flex;"><span> defaultServerConfig <span style="color:#719e07">:=</span> dubbo.<span style="color:#268bd2">GetDefaultServerConfig</span>() |
| </span></span><span style="display:flex;"><span> dubbo.<span style="color:#268bd2">SetServerConfig</span>(defaultServerConfig) |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">SetLoggerLevel</span>(<span style="color:#2aa198">&#34;warn&#34;</span>) <span style="color:#586e75">// info,warn |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">select</span> { |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h4 id="2客户端自定义配置文件">2）客户端自定义配置文件</h4> |
| <ul> |
| <li>var consumerConfigStr = <code>xxxxx</code>// 配置文件内容，可以参考 log 和 clien。在这里你可以定义配置文件的获取方式，比如配置中心，本地文件读取。</li> |
| <li>在 <code>config.Load()</code> 之前设置配置，例如：</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> p <span style="color:#719e07">:=</span> config.ConsumerConfig{} |
| </span></span><span style="display:flex;"><span> yaml.<span style="color:#268bd2">Unmarshal</span>([]<span style="color:#b58900">byte</span>(consumerConfigStr), <span style="color:#719e07">&amp;</span>p) |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetConsumerConfig</span>(p) |
| </span></span><span style="display:flex;"><span> defaultClientConfig <span style="color:#719e07">:=</span> dubbo.<span style="color:#268bd2">GetDefaultClientConfig</span>() |
| </span></span><span style="display:flex;"><span> dubbo.<span style="color:#268bd2">SetClientConf</span>(defaultClientConfig) |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">SetLoggerLevel</span>(<span style="color:#2aa198">&#34;warn&#34;</span>) <span style="color:#586e75">// info,warn |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> user <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>User{} |
| </span></span><span style="display:flex;"><span> err <span style="color:#719e07">:=</span> userProvider.<span style="color:#268bd2">GetUser</span>(context.<span style="color:#268bd2">TODO</span>(), []<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;A001&#34;</span>}, user) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Print</span>(err) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Print</span>(user) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h2 id="server-端">Server 端</h2> |
| <p>服务暴露过程涉及到多次原始 rpcService 的封装、暴露，网上其他文章的图感觉太过笼统，在此，简要地绘制了一个用户定义服务的数据流图：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p3.png" alt="img"></p> |
| <h3 id="1-加载配置">1. 加载配置</h3> |
| <h4 id="1框架初始化">1）框架初始化</h4> |
| <p>在加载配置之前，框架提供了很多已定义好的协议、工厂等组件，都会在对应模块 init 函数内注册到 extension 模块上，以供接下来配置文件中进行选用。</p> |
| <p>其中重要的有：</p> |
| <ul> |
| <li><strong>默认函数代理工厂</strong>：common/proxy/proxy_factory/default.go</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetProxyFactory</span>(<span style="color:#2aa198">&#34;default&#34;</span>, NewDefaultProxyFactory) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>它的作用是将原始 rpc-service 进行封装，形成 proxy_invoker，更易于实现远程 call 调用，详情可见其 invoke 函数。</p> |
| <ul> |
| <li><strong>注册中心注册协议</strong>： |
| registry/protocol/protocol.go</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetProtocol</span>(<span style="color:#2aa198">&#34;registry&#34;</span>, GetProtocol) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>它负责将 invoker 暴露给对应注册中心，比如 zk 注册中心。</p> |
| <ul> |
| <li><strong>zookeeper 注册协议</strong>：registry/zookeeper/zookeeper.go</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetRegistry</span>(<span style="color:#2aa198">&#34;zookeeper&#34;</span>, newZkRegistry) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>它合并了 base_resiger，负责在服务暴露过程中，将服务注册在 zookeeper 注册器上，从而为调用者提供调用方法。</p> |
| <ul> |
| <li><strong>dubbo 传输协议</strong>：protocol/dubbo/dubbo.go</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetProtocol</span>(DUBBO, GetProtocol) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>它负责监听对应端口，将具体的服务暴露，并启动对应的事件 handler，将远程调用的 event 事件传递到 invoker 内部，调用本地 invoker 并获得执行结果返回。</p> |
| <ul> |
| <li><strong>filter 包装调用链协议</strong>：protocol/protocolwrapper/protocol_filter_wrapper.go</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetProtocol</span>(FILTER, GetProtocol) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>它负责在服务暴露过程中，将代理 invoker 打包，通过配置好的 filter 形成调用链，并交付给 dubbo 协议进行暴露。</p> |
| <p>上述提前注册好的框架已实现的组件，在整个服务暴露调用链中都会用到，会根据配置取其所需。</p> |
| <h4 id="2配置文件">2）配置文件</h4> |
| <p>服务端需要的重要配置有三个字段：services、protocols、registries。</p> |
| <p>profiles/dev/server.yml:</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">registries </span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;demoZk&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">timeout </span>: <span style="color:#2aa198">&#34;3s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;127.0.0.1:2181&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">services</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;UserProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 可以指定多个registry，使用逗号隔开;不指定默认向所有注册中心注册</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;demoZk&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol </span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 相当于dubbo.xml中的interface</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface </span>: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">loadbalance</span>: <span style="color:#2aa198">&#34;random&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">warmup</span>: <span style="color:#2aa198">&#34;100&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;GetUser&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">1</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">loadbalance</span>: <span style="color:#2aa198">&#34;random&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">protocols</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;dubbo&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">port</span>: <span style="color:#2aa198">20000</span> |
| </span></span></code></pre></div><p>其中 service 指定了要暴露的 rpc-service 名（&ldquo;UserProvider）、暴露的协议名（&ldquo;dubbo&rdquo;）、注册的协议名(&ldquo;demoZk&rdquo;)、暴露的服务所处的 interface、负载均衡策略、集群失败策略及调用的方法等等。</p> |
| <p>其中，中间服务的协议名需要和 registries 下的 mapkey 对应，暴露的协议名需要和 protocols 下的 mapkey 对应。</p> |
| <p>可以看到上述例子中，使用了 dubbo 作为暴露协议，使用了 zookeeper 作为中间注册协议，并且给定了端口。如果 zk 需要设置用户名和密码，也可以在配置中写好。</p> |
| <h4 id="3配置文件的读入和检查">3）配置文件的读入和检查</h4> |
| <blockquote> |
| <p>config/config_loader.go:: Load()</p> |
| </blockquote> |
| <p>在上述 example 的 main 函数中，有 config.Load() 函数的直接调用，该函数执行细节如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// Load Dubbo Init |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">Load</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// init router |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">initRouter</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// init the global event dispatcher |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> extension.<span style="color:#268bd2">SetAndInitGlobalDispatcher</span>(<span style="color:#268bd2">GetBaseConfig</span>().EventDispatcherType) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// start the metadata report if config set |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> err <span style="color:#719e07">:=</span> <span style="color:#268bd2">startMetadataReport</span>(<span style="color:#268bd2">GetApplicationConfig</span>().MetadataType, <span style="color:#268bd2">GetBaseConfig</span>().MetadataReportConfig); err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;Provider starts metadata report error, and the error is {%#v}&#34;</span>, err) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// reference config |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">loadConsumerConfig</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// service config |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">loadProviderConfig</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// init the shutdown callback |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">GracefulShutdownInit</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>在本文中，我们重点关心 loadConsumerConfig() 和 loadProviderConfig() 两个函数。</p> |
| <p>对于 provider 端，可以看到 loadProviderConfig() 函数代码如下：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p4.png" alt="img"></p> |
| <p>前半部分是配置的读入和检查，进入 for 循环后，是单个 service 的暴露起始点。</p> |
| <p>前面提到，在配置文件中已经写好了要暴露的 service 的种种信息，比如服务名、interface 名、method 名等等。在图中 for 循环内，会将所有 service 的服务依次实现。</p> |
| <p>for 循环的第一行，根据 key 调用 GetProviderService 函数，拿到注册的 rpcService 实例，这里对应上述提到的 init 函数中，用户手动注册的自己实现的 rpc-service 实例：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p5.png" alt="img"></p> |
| <p>这个对象也就成为了 for 循环中的 rpcService 变量，将这个对象注册通过 Implement 函数写到 sys（ServiceConfig 类型）上，设置好 sys 的 key 和协议组，最终调用了 sys 的 Export 方法。</p> |
| <p>此处对应流程图的部分：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p6.png" alt="img"></p> |
| <p>至此，框架配置结构体已经拿到了所有 service 有关的配置，以及用户定义好的 rpc-service 实例，它触发了 Export 方法，旨在将自己的实例暴露出去。这是 Export 调用链的起始点。</p> |
| <h3 id="2-原始-service-封装入-proxy_invoker">2. 原始 service 封装入 proxy_invoker</h3> |
| <blockquote> |
| <p>config/service_config.go :: Export()</p> |
| </blockquote> |
| <p>接下来进入 ServiceConfig.Export() 函数.</p> |
| <p>这个函数进行了一些细碎的操作，比如为不同的协议分配随机端口，如果指定了多个中心注册协议，则会将服务通过多个中心注册协议的 registryProtocol 暴露出去，我们只关心对于一个注册协议是如何操作的。还有一些操作比如生成调用 url 和注册 url，用于为暴露做准备。</p> |
| <h4 id="1首先通过配置生成对应-registryurl-和-serviceurl">1）首先通过配置生成对应 registryUrl 和 serviceUrl</h4> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p7.png" alt="img"></p> |
| <p>registryUrl 是用来向中心注册组件发起注册请求的，对于 zookeeper 的话，会传入其 ip 和端口号，以及附加的用户名密码等信息。</p> |
| <p>这个 regUrl 目前只存有注册（zk）相关信息，后续会补写入 ServiceIvk，即服务调用相关信息，里面包含了方法名，参数等&hellip;</p> |
| <h4 id="2对于一个注册协议将传入的-rpc-service-实例注册在-commonservicemap">2）对于一个注册协议，将传入的 rpc-service 实例注册在 common.ServiceMap</h4> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p8.png" alt="img"></p> |
| <p>这个 Register 函数将服务实例注册了两次，一次是以 Interface 为 key 写入接口服务组内，一次是以 interface 和 proto 为 key 写入特定的一个唯一的服务。</p> |
| <p>后续会从 common.Map 里面取出来这个实例。</p> |
| <h4 id="3获取默认代理工厂将实例封装入代理-invoker">3）获取默认代理工厂，将实例封装入代理 invoker</h4> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// 拿到一个proxyInvoker，这个invoker的url是传入的regUrl，这个地方将上面注册的service实例封装成了invoker |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// 这个GetProxyFactory返回的默认是common/proxy/proxy_factory/default.go |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// 这个默认工厂调用GetInvoker获得默认的proxyInvoker，保存了当前注册url |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>invoker <span style="color:#719e07">:=</span> extension.<span style="color:#268bd2">GetProxyFactory</span>(providerConfig.ProxyFactory).<span style="color:#268bd2">GetInvoker</span>(<span style="color:#719e07">*</span>regUrl) |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// 暴露出来 生成exporter,开启tcp监听 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// 这里就该跳到registry/protocol/protocol.go registryProtocol 调用的Export，将当前proxyInvoker导出 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>exporter = c.cacheProtocol.<span style="color:#268bd2">Export</span>(invoker) |
| </span></span></code></pre></div><p>这一步的 GetProxyFactory(&ldquo;default&rdquo;) 方法获取默认代理工厂，通过传入上述构造的 regUrl，将 url 封装入代理 invoker。</p> |
| <p>可以进入 common/proxy/proxy_factory/default.go::ProxyInvoker.Invoke() 函数里，看到对于 common.Map 取用为 svc 的部分，以及关于 svc 对应 Method 的实际调用 Call 的函数如下：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p9.png" alt="img"></p> |
| <p>到这里，上面 GetInvoker(*regUrl) 返回的 invoker 即为 proxy_invoker，它封装好了用户定义的 rpc_service，并将具体的调用逻辑封装入了 Invoke 函数内。</p> |
| <blockquote> |
| <p>为什么使用 Proxy_invoker 来调用？</p> |
| <p>通过这个 proxy_invoke 调用用户的功能函数，调用方式将更加抽象化，可以在代码中看到，通过 ins 和 outs 来定义入参和出参，将整个调用逻辑抽象化为 invocation 结构体，而将具体的函数名的选择、参数向下传递和 reflect 反射过程封装在 invoke 函数内，这样的设计更有利于之后远程调用。个人认为这是 dubbo Invoke 调用链的设计思想。</p> |
| <p>至此，实现了图中对应的部分：</p> |
| </blockquote> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p10.png" alt="img"></p> |
| <h3 id="3-registry-协议在-zkregistry-上暴露上面的-proxy_invoker">3. registry 协议在 zkRegistry 上暴露上面的 proxy_invoker</h3> |
| <p>上面，我们执行到了 exporter = c.cacheProtocol.Export(invoker)。</p> |
| <p>这里的 cacheProtocol 为一层缓存设计，对应到原始的 demo 上，这里是默认实现好的 registryProtocol。</p> |
| <blockquote> |
| <p>registry/protocol/protocol.go:: Export()</p> |
| </blockquote> |
| <p>这个函数内构造了多个 EventListener，非常有 java 的设计感。</p> |
| <p>我们只关心服务暴露的过程，先忽略这些监听器。</p> |
| <h4 id="1获取注册-url-和服务-url">1）获取注册 url 和服务 url</h4> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p11.png" alt="img"></p> |
| <h4 id="2获取注册中心实例-zkregistry">2）获取注册中心实例 zkRegistry</h4> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p12.png" alt="img"></p> |
| <p>一层缓存操作，如果 cache 没有需要从 common 里面重新拿 zkRegistry。</p> |
| <h4 id="3zkregistry-调用-registry-方法在-zookeeper-上注册-dubbopath">3）zkRegistry 调用 Registry 方法，在 zookeeper 上注册 dubboPath</h4> |
| <p>上述拿到了具体的 zkRegistry 实例，该实例的定义在：registry/zookeeper/registry.go。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p13.png" alt="img"></p> |
| <p>该结构体组合了 registry.BaseRegistry 结构，base 结构定义了注册器基础的功能函数，比如 Registry、Subscribe 等，但在这些默认定义的函数内部，还是会调用 facade 层（zkRegistry 层）的具体实现函数，这一设计模型能在保证已有功能函数不需要重复定义的同时，引入外层函数的实现，类似于结构体继承却又复用了代码。这一设计模式值得学习。</p> |
| <p>我们查看上述 registry/protocol/protocol.go:: Export() 函数，直接调用了:</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// 1. 通过zk注册器，调用Register()函数，将已有@root@rawurl注册到zk上 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> err <span style="color:#719e07">:=</span> reg.<span style="color:#268bd2">Register</span>(<span style="color:#719e07">*</span>registeredProviderUrl) |
| </span></span></code></pre></div><p>将已有 RegistryUrl 注册到了 zkRegistry 上。</p> |
| <p>这一步调用了 baseRegistry 的 Register 函数，进而调用 zkRegister 的 DoRegister 函数，进而调用：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p14.png" alt="img"></p> |
| <p>在这个函数里，将对应 root 创造一个新的节点。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p15.png" alt="img"></p> |
| <p>并且写入具体 node 信息，node 为 url 经过 encode 的结果，<strong>包含了服务端的调用方式。</strong></p> |
| <p>这部分的代码较为复杂，具体可以看 baseRegistry 的 processURL() 函数：http://t.tb.cn/6Xje4bijnsIDNaSmyPc4Ot。</p> |
| <p>至此，将服务端调用 url 注册到了 zookeeper 上，而客户端如果想获取到这个 url，只需要传入特定的 dubboPath，向 zk 请求即可。目前 client 是可以获取到访问方式了，但服务端的特定服务还没有启动，还没有开启特定协议端口的监听，这也是 registry/protocol/protocol.go:: Export() 函数接下来要做的事情。</p> |
| <h4 id="4proxy_invoker-封装入-wrapped_invoker得到-filter-调用链">4）proxy_invoker 封装入 wrapped_invoker，得到 filter 调用链</h4> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span> <span style="color:#586e75">// invoker封装入warppedInvoker |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> wrappedInvoker <span style="color:#719e07">:=</span> <span style="color:#268bd2">newWrappedInvoker</span>(invoker, providerUrl) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 经过为invoker增加filter调用链，再使用dubbo协议Export，开启service并且返回了Exporter 。 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// export_1 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> cachedExporter = extension.<span style="color:#268bd2">GetProtocol</span>(protocolwrapper.FILTER).<span style="color:#268bd2">Export</span>(wrappedInvoker) |
| </span></span></code></pre></div><p>新建一个 WrappedInvoker，用于之后链式调用。</p> |
| <p>拿到提前实现并注册好的 ProtocolFilterWrapper，调用 Export 方法，进一步暴露。</p> |
| <blockquote> |
| <p>protocol/protocolwrapped/protocol_filter_wrapper.go:Export()</p> |
| </blockquote> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p16.png" alt="img"></p> |
| <blockquote> |
| <p>protocol/protocolwrapped/protocol_filter_wrapper.go:buildInvokerChain</p> |
| </blockquote> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p17.png" alt="img"></p> |
| <p>可见，根据配置的内容，通过链式调用的构造，将 proxy_invoker 层层包裹在调用链的最底部，最终返回一个调用链 invoker。</p> |
| <p>对应图中部分：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p18.png" alt="img"></p> |
| <p>至此，我们已经拿到 filter 调用链，期待将这个 chain 暴露到特定端口，用于相应请求事件。</p> |
| <h4 id="5通过-dubbo-协议暴露-wrapped_invoker">5）通过 dubbo 协议暴露 wrapped_invoker</h4> |
| <blockquote> |
| <p>protocol/protocolwrapped/protocol_filter_wrapper.go:Export()</p> |
| </blockquote> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// 通过dubbo协议Export dubbo_protocol调用的 export_2 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">return</span> pfw.protocol.<span style="color:#268bd2">Export</span>(invoker) |
| </span></span></code></pre></div><p>回到上述 Export 函数的最后一行，调用了 dubboProtocol 的 Export 方法，将上述 chain 真正暴露。</p> |
| <p>该 Export 方法的具体实现在：protocol/dubbo/dubbo_protocol.go: Export()。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p19.png" alt="img"></p> |
| <p>这一函数做了两个事情：构造触发器、启动服务。</p> |
| <ul> |
| <li>将传入的 Invoker 调用 chain 进一步封装，封装成一个 exporter，再将这个 export 放入 map 保存。<strong>注意！这里把 exporter 放入了 SetExporterMap中，在下面服务启动的时候，会以注册事件监听器的形式将这个 exporter 取出！</strong></li> |
| <li>调用 dubboProtocol 的 openServer 方法，开启一个针对特定端口的监听。</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p20.png" alt="img"></p> |
| <p>如上图所示，一个 Session 被传入，开启对应端口的事件监听。</p> |
| <p>至此构造出了 exporter，完成图中部分：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p21.png" alt="img"></p> |
| <h3 id="4-注册触发动作">4. 注册触发动作</h3> |
| <p>上述只是启动了服务，但还没有看到触发事件的细节，点进上面的 s.newSession 可以看到，dubbo 协议为一个 getty 的 session 默认使用了如下配置：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p22.png" alt="img"></p> |
| <p>其中很重要的一个配置是 EventListener，传入的是 dubboServer 的默认 rpcHandler。</p> |
| <blockquote> |
| <p>protocol/dubbo/listener.go:OnMessage()</p> |
| </blockquote> |
| <p>rpcHandler 有一个实现好的 OnMessage 函数，根据 getty 的 API，当 client 调用该端口时，会触发 OnMessage。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// OnMessage notified when RPC server session got any message in connection |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (h <span style="color:#719e07">*</span>RpcServerHandler) <span style="color:#268bd2">OnMessage</span>(session getty.Session, pkg <span style="color:#268bd2">interface</span>{}) { |
| </span></span></code></pre></div><p>这一函数实现了在 getty session 接收到 rpc 调用后的一系列处理：</p> |
| <ul> |
| <li>传入包的解析</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p23.png" alt="img"></p> |
| <ul> |
| <li>根据请求包构造请求 url</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p24.png" alt="img"></p> |
| <ul> |
| <li>拿到对应请求 key，找到要被调用的 exporter</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p25.png" alt="img"></p> |
| <ul> |
| <li>拿到对应的 Invoker</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p26.png" alt="img"></p> |
| <ul> |
| <li>构造 invocation</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p27.png" alt="img"></p> |
| <ul> |
| <li>调用</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p28.png" alt="img"></p> |
| <ul> |
| <li>返回</li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/code1/p29.png" alt="img"></p> |
| <p>整个被调过程一气呵成。实现了从 getty.Session 的调用事件，到经过层层封装的 invoker 的调用。</p> |
| <p>至此，一次 rpc 调用得以正确返回。</p> |
| <h2 id="小结">小结</h2> |
| <ul> |
| <li><strong>关于 Invoker 的层层封装</strong></li> |
| </ul> |
| <p>能把一次调用抽象成一次 invoke；能把一个协议抽象成针对 invoke 的封装；能把针对一次 invoke 所做出的特定改变封装到 invoke 函数内部，可以降低模块之间的耦合性。层层封装逻辑更加清晰。</p> |
| <ul> |
| <li><strong>关于 URL 的抽象</strong></li> |
| </ul> |
| <p>关于 dubbo 的统一化请求对象 URL 的极度抽象是之前没有见过的&hellip; 个人认为这样封装能保证请求参数列表的简化和一致。但在开发的过程中，滥用极度抽象的接口可能造成&hellip; debug 的困难？以及不知道哪些字段是当前已经封装好的，哪些字段是无用的。</p> |
| <ul> |
| <li><strong>关于协议的理解</strong></li> |
| </ul> |
| <p>之前理解的协议还是太过具体化了，而关于 dubbo-go 对于 dubboProtocol 的协议，我认为是基于 getty 的进一步封装，它定义了客户端和服务端，对于 getty 的 session 应该有哪些特定的操作，从而保证主调和被调的协议一致性，而这种保证也是一种协议的体现，是由 dubbo 协议来规范的。</p> |
| <p>如果你有任何疑问，欢迎钉钉扫码加入交流群：钉钉群号 23331795！</p> |
| <blockquote> |
| <p>作者简介 <strong>李志信</strong> (GitHubID LaurenceLiZhixin)，中山大学软件工程专业在校学生，擅长使用 Java/Go 语言，专注于云原生和微服务等技术方向</p> |
| </blockquote></description></item><item><title>Blog: dubbo-go 中 REST 协议实现</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E4%B8%AD-rest-%E5%8D%8F%E8%AE%AE%E5%AE%9E%E7%8E%B0/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E4%B8%AD-rest-%E5%8D%8F%E8%AE%AE%E5%AE%9E%E7%8E%B0/</guid><description> |
| <h2 id="什么是-rest-协议">什么是 REST 协议</h2> |
| <p>REST 是 <strong>RE</strong>presentational <strong>S</strong>tate <strong>T</strong>ransfer（表述性状态转移）的简写，是一种软件架构风格。虽然 REST 架构风格不是一定要基于 HTTP 协议进行传输，但是因为 HTTP |
| 协议的通用性和易用性，现在越来越多的 web 服务采用基于 HTTP 协议实现 RESTful 架构。</p> |
| <p>在 dubbo-go 中的 REST 协议指的是一种基于 HTTP 协议的远程调用方式。简单的来讲，REST 协议就是把dubbo 服务发布成 RESTful 风格的 HTTP 接口并且能够能像调用 dubbo 接口一样的方式调用 HTTP |
| 接口。</p> |
| <h2 id="为什么要支持-rest-协议">为什么要支持 REST 协议</h2> |
| <p>在没有 REST 协议之前，小伙伴们是否一直在苦恼这样几个问题：</p> |
| <ol> |
| <li>传统的 web 服务不能直接调用我们发布的 dubbo 服务</li> |
| <li>前端不能直接调用 dubbo 服务</li> |
| <li>dubbo 服务不能发布 Open API</li> |
| </ol> |
| <p>上述问题，就是 REST 协议解决的核心问题。现在我们很多应用场景都是需要与异构的系统进行交互，而 REST 采用的 HTTP 通信协议非常适合用来打通异构系统，如图：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/rest/rest-call.webp" alt="img"></p> |
| <h2 id="rest-协议没那么简单">REST 协议没那么简单</h2> |
| <p>REST 协议核心要解决一个问题：<strong>Go 方法到 HTTP 接口的双向映射</strong>。普通 HTTP 调用 dubbo-go 服务，考虑的是HTTP 到 <strong>Go</strong> 方法的映射；而 dubbo-go 服务调用 HTTP 服务，则是 ** |
| Go** 方法到 HTTP 接口的映射。</p> |
| <p>下面是我们要与 <strong>Go</strong> 方法要做映射的 HTTP 请求协议内容：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>POST /path/{pathParam}?queryParam=1 HTTP/1.1 |
| </span></span><span style="display:flex;"><span>Accept: application/json |
| </span></span><span style="display:flex;"><span>Content-Type: application/json |
| </span></span><span style="display:flex;"><span>Host: http://localhost:8080 |
| </span></span><span style="display:flex;"><span>{&#34;id&#34;:1111} |
| </span></span></code></pre></div><p>在服务提供方，当上面这种请求发送到服务器时，我们要把它路由到下面这个 <strong>Go</strong> 方法中，在服务消费方，我们也可以通过调用下面的 <strong>Go</strong> 方法把方法参数转化为上面的HTTP请求：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> Provider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 该方法应该对应上面的http请求 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> GetResult <span style="color:#268bd2">func</span>(pathParam <span style="color:#dc322f">string</span>, queryParam <span style="color:#dc322f">string</span>, body <span style="color:#268bd2">interface</span>{}, host <span style="color:#dc322f">string</span>) (<span style="color:#719e07">*</span>Result, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>在消费方调用 <code>GetResult</code> 时，传入各个参数的值：</p> |
| <ul> |
| <li>变量 pathParam 的内容应该是字符串 &ldquo;pathParam&rdquo;；</li> |
| <li>变量 queryParam 的内容应该是字符串 &ldquo;1&rdquo; ；</li> |
| <li>变量 body 应该是有以字符串 &ldquo;id&rdquo; 为 key ，1111 为 value 的一个map；</li> |
| <li>当然 host 变量的内容应该是字符串 &ldquo;http://localhost:8080&rdquo; 。</li> |
| </ul> |
| <p>在服务端执行 <code>GetResult</code> 方法时，得到的参数会与消费方调用时传入的值相同。</p> |
| <p>总结下来，我们要建立以下这些映射关系</p> |
| <ol> |
| <li>路径映射</li> |
| <li>Header 处理（固定 Header 和 Header 值也是参数两种情况）</li> |
| <li>POST or GET or &hellip;（HTTP 方法映射）</li> |
| <li>参数映射</li> |
| </ol> |
| <p>要完成这种映射，我们首先要解决的是，如何知道这种映射关系？</p> |
| <p>答案只有一个，通过用户配置。而用户配置所面临的困难是，复杂且琐碎。（解决思路是提供大量默认配置减轻配置的负担，自定义配置方式允许用户使用自己熟悉的配置形式）</p> |
| <p>另外一个难点在于，使用何种 web 框架的问题。有些公司内部使用的是自研的 web 框架，他们有成熟的技术基础和运维能力。于是就会考虑说，能不能让 dubbo-go 在支持 REST 协议的时候，能够让他们将 REST 协议使用的web |
| 框架替换成他们所期望的呢？</p> |
| <h2 id="如何建立http接口与方法的映射关系">如何建立HTTP接口与方法的映射关系</h2> |
| <p>下面我举一个HTTP接口与方法映射的具体例子：</p> |
| <p><strong>Go</strong> 结构体定义如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> UserProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> GetUser <span style="color:#268bd2">func</span>(id <span style="color:#dc322f">string</span>, name <span style="color:#dc322f">string</span>, age <span style="color:#dc322f">int</span>) (<span style="color:#719e07">*</span>User, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>要发布的HTTP接口形式是： http://127.0.0.1/UserProvider/GetUser/{id}?name=test&amp;age=1</p> |
| <p>服务端配置如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">services</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;UserProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> //注册中心 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> //启用REST协议 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;rest&#34;</span> |
| </span></span><span style="display:flex;"><span> //DUBBO的接口名 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface</span>: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> // 服务接口路径 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_path</span>: <span style="color:#2aa198">&#34;/UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;GetUser&#34;</span> |
| </span></span><span style="display:flex;"><span> // 方法接口路径 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_path</span>: <span style="color:#2aa198">&#34;/GetUser/{id}&#34;</span> |
| </span></span><span style="display:flex;"><span> // HTTP方法 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_method</span>: <span style="color:#2aa198">&#34;GET&#34;</span> |
| </span></span><span style="display:flex;"><span> // HTTP查询参数 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_query_params</span>: <span style="color:#2aa198">&#34;1:name,2:age&#34;</span> |
| </span></span><span style="display:flex;"><span> // HTTP路径参数 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_path_params</span>: <span style="color:#2aa198">&#34;0:id&#34;</span> |
| </span></span><span style="display:flex;"><span> // 可以提供的内容类型 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_produces</span>: <span style="color:#2aa198">&#34;application/json;application/xml&#34;</span> |
| </span></span><span style="display:flex;"><span> // 可以接受的客户端参数类型 |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_consumes</span>: <span style="color:#2aa198">&#34;application/json;charset=utf-8,*/*&#34;</span> |
| </span></span><span style="display:flex;"><span> // HTTP Body |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">rest_body</span>: -<span style="color:#2aa198">1</span> |
| </span></span></code></pre></div><p>在配置文件中我们定义了方法的路径，HTTP方法等接口参数，这里需要注意的是路径参数和查询参数的配置方式，0:name 的意思是查询参数 name 对应 <code>GetUser</code> 方法的第一个参数，还有 rest_body |
| 配置的数字也是对应这方法的参数，这里没有 body 参数所有就配置了 <code>-1</code>。</p> |
| <h2 id="rest协议的调用过程">REST协议的调用过程</h2> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/rest/rest-call-process.webp" alt="img"></p> |
| <p>上图展示了用户在 Consumer 端调用 <code>GetUser</code> 方法到 Provdier 端 <code>GetUser</code> 方法被执行的整个过程，在 <code>RestClient</code> 和 <code>RestServer</code> 中分别**实现了 Go 方法参数到 HTTP |
| 请求的转换和 HTTP 请求到 Go 方法的转换，这是最为核心和复杂的部分。**换言之，我们在这里实现了前面提到的 Go 方法和 HTTP 请求的双向映射。</p> |
| <p>这里我们可以注意到 <code>RestClient</code> 和 <code>RestServer</code> 是可以用户自行扩展的，下面我将具体介绍一下在REST协议中有哪些扩展点设计。</p> |
| <h2 id="rest协议的扩展点设计">REST协议的扩展点设计</h2> |
| <p>基于 dubbo-go 良好的 extension 扩展设计，我们定义了多个扩展点，用户可以自定义功能实现。</p> |
| <h3 id="自定义http服务器">自定义HTTP服务器</h3> |
| <p>RestServer的扩展接口：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> RestServer <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// sever启动函数 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Start</span>(url common.URL) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 发布接口 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Deploy</span>(restMethodConfig <span style="color:#719e07">*</span>rest_config.RestMethodConfig, routeFunc <span style="color:#268bd2">func</span>(request RestServerRequest, response RestServerResponse)) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 删除接口 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">UnDeploy</span>(restMethodConfig <span style="color:#719e07">*</span>rest_config.RestMethodConfig) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// server关闭 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Destroy</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>在 dubbo-go 的 v1.4.0 中默认使用 go-restful 作为 HTTP 服务器，如果用户想用其他 HTTP 容器可以实现上面的接口，并在配置文件中配置使用自己自定义的服务器。</p> |
| <p>这个接口中，最核心的方法是 Deploy，在 restMethodConfig 方法参数中有用户配置的接口路径等一系列参数，routeFunc 是 HTTP 接口需要被路由执行的函数。不同的http服务器会有不同的 request 和 |
| response ，所以我们定义了 <code>RestServerRequest</code> 接口和 <code>RestServerResponse</code> 接口让用户进行适配。</p> |
| <h3 id="自定义http客户端">自定义HTTP客户端</h3> |
| <p>RestClient 的扩展接口：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// RestOptions |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> RestOptions <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> RequestTimeout time.Duration |
| </span></span><span style="display:flex;"><span> ConnectTimeout time.Duration |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// RestClientRequest |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> RestClientRequest <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> Header http.Header |
| </span></span><span style="display:flex;"><span> Location <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Path <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Method <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> PathParams <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> QueryParams <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Body <span style="color:#268bd2">interface</span>{} |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// RestClient user can implement this client interface to send request |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> RestClient <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Do</span>(request <span style="color:#719e07">*</span>RestClientRequest, res <span style="color:#268bd2">interface</span>{}) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>最后的请求到客户端时，都会被封装为 <code>RestRequest</code>，用户可以非常简单快速的扩展自己的 Client 端。<code>RestOptions</code> 中有一些客户端的超时配置，在创建自己的客户端时需要根据这些配置初始化客户端。</p> |
| <h3 id="自定义-rest-配置形式">自定义 REST 配置形式</h3> |
| <p>前面提到，REST 协议一个很麻烦的地方在于，配置很繁琐很琐碎。Go 不同于 Java，可以通过注解的形式来简化配置。</p> |
| <p>所以我们考虑到用户不同的使用习惯和公司的配置风格，提供了这个扩展点。</p> |
| <p>ConfigReader 的扩展接口：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> ConfigReader <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Consumer配置读取 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">ReadConsumerConfig</span>(reader <span style="color:#719e07">*</span>bytes.Buffer) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Provider配置读取 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">ReadProviderConfig</span>(reader <span style="color:#719e07">*</span>bytes.Buffer) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><code>ReadConsumerConfig</code> 和 <code>ReadProviderConfig</code> 方法的参数是配置文件的文件流，在实现方法中可以再次解析，也可以使用二次编译或者硬编码方式等其他方式读取配置。这是一个通用的配置读取接口，以后可以用来扩展 |
| REST 配置之外的其他配置，所以需要在方法中调用方法设置配置，如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// 设置Rest的消费者配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>config.<span style="color:#268bd2">SetRestConsumerServiceConfigMap</span>(restConsumerServiceConfigMap) |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// 设置Rest的提供者配置 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>config.<span style="color:#268bd2">SetRestProviderServiceConfigMap</span>(restProviderServiceConfigMap) |
| </span></span></code></pre></div><h2 id="如何添加-http-过滤器">如何添加 HTTP 过滤器</h2> |
| <p>因为不同 HTTP 服务器的过滤器，拦截器或者是 middleware 添加方式都不同，所以我们很难定义一个接口满足所有服务器。因此我们单独为 go-restful 定义了一个添加 filter 的方法，这里我们需要注意的一点是必须在 REST 接口发布前添加filter。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span>server_impl.<span style="color:#268bd2">AddGoRestfulServerFilter</span>(<span style="color:#268bd2">func</span>(request <span style="color:#719e07">*</span>restful.Request, response <span style="color:#719e07">*</span>restful.Response, chain <span style="color:#719e07">*</span>restful.FilterChain) { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 鉴权等功能 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> chain.<span style="color:#268bd2">ProcessFilter</span>(request, response) |
| </span></span><span style="display:flex;"><span>}) |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// 启动dubbo服务，发布rest等接口 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>config.<span style="color:#268bd2">Load</span>() |
| </span></span></code></pre></div><h2 id="展望">展望</h2> |
| <p>以上是关于 REST 协议的一些介绍，具体的实现我就不详细讲了，大家可以去参阅源码。</p> |
| <p>如果想看具体的Example，请参考：</p> |
| <p><a href="https://github.com/dubbogo/dubbo-samples/tree/master/golang/general/rest">https://github.com/dubbogo/dubbo-samples/tree/master/golang/general/rest</a></p> |
| <p>REST 未来需要支持 HTTPS 协议和基于 open tracing 标准 api 的链路追踪。REST 的配置信息未来也不是 REST协议独有的，这些配置信息未来可以作为每个 dubbo 接口的元数据，存储到元数据中心，为网关提供 |
| HTTP 协议与 dubbo 协议之间的映射关系。</p> |
| <blockquote> |
| <p>作者：蒋超，github id Patrick0308，在 杭州贝安云科技有限公司 任职服务开发工程师。</p> |
| </blockquote></description></item><item><title>Blog: dubbo-go 中将 Kubernets 原⽣作为注册中⼼的设计和实现</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E4%B8%AD%E5%B0%86-kubernets-%E5%8E%9F%E4%BD%9C%E4%B8%BA%E6%B3%A8%E5%86%8C%E4%B8%AD%E7%9A%84%E8%AE%BE%E8%AE%A1%E5%92%8C%E5%AE%9E%E7%8E%B0/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go-%E4%B8%AD%E5%B0%86-kubernets-%E5%8E%9F%E4%BD%9C%E4%B8%BA%E6%B3%A8%E5%86%8C%E4%B8%AD%E7%9A%84%E8%AE%BE%E8%AE%A1%E5%92%8C%E5%AE%9E%E7%8E%B0/</guid><description> |
| <p>今天这篇⽂章将会介绍 dubbo-go 将 Kubernetes 作为注册中⼼的服务注册的初衷、设计⽅案，以及具体实现。</p> |
| <p>到⽬前为⽌该⽅案的实现已经被合并到 dubbo-go 的 master 分⽀。具体实现为关于 Kubernetes 的 <a href="https://github.com/apache/dubbo-go/pull/400">PullRequest</a> 。</p> |
| <h2 id="kubernetes管理资源的哲学">Kubernetes管理资源的哲学</h2> |
| <p>Kubernetes 作为容器集群化管理⽅案管理资源的维度可主观的分为服务进程管理和服务接⼊管理。</p> |
| <ul> |
| <li>服务实例管理，主要体现⽅式为 Pod 设计模式加控制器模式，控制器保证具有特定标签 （ Kubernetes-Label ）的 Pod 保持在恒定的数量（多删，少补）。</li> |
| <li>服务管理，主要为 Kubernetes-Service ，该 Service 默认为具有特定标签（ Kubernetes-Label ）的 Pod 统⼀提供⼀个 VIP（ Kubernetes-ClusterIP ）所有需要请求该组 Pod 的请求都默认会按照 round-robin 的负载策略转发到真正提供服务的 Pod 。并且 CoreDNS 为该 Kubernetes-Service 提供集群内唯⼀的域名。</li> |
| </ul> |
| <h2 id="kubernetes的服务发现模型">Kubernetes的服务发现模型</h2> |
| <p>为了明确 K8s 在服务接入管理提供的解决方案，我们以 kube-apiserver 提供的 API(HTTPS) 服务为例。K8s 集群为该服务分配了一个集群内有效的 ClusterIP ，并通过 CoreDNS 为其分配了唯一的域名 kubernetes 。如果集群内的 Pod 需要访问该服务时直接通过 https://kubernetes:443 即可完成。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/k8s/k8s-service-discovery.png" alt="img"></p> |
| <p>具体流程如上图所示 ( 红⾊为客户端，绿⾊为 kube-apiserver )：</p> |
| <ol> |
| <li>⾸先客户端通过 CoreDNS 解析域名为 <strong>kubernetes</strong> 的服务获得对应的 Cluster IP 为 10.96.0.1。</li> |
| <li>客户端向 10.96.0.1 发起 HTTP 请求。</li> |
| <li>HTTP 请求 kube-proxy 所创建的 IP tables 拦截随机 DNAT 为 10.0.2.16 或者 10.0.2.15 。</li> |
| <li>Client 与最终提供服务的 Pod 建⽴连接并交互。</li> |
| </ol> |
| <p>由此可⻅，Kubernetes 提供的服务发现为域名解析级别。</p> |
| <h2 id="dubbo-的服务发现模型">dubbo 的服务发现模型</h2> |
| <p>同样为了明确 dubbo 服务发现的模型，以⼀个简单的 dubbo-consumer 发现并访问 Provider 的具体流程为例。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/k8s/dubbo-service-discovery.png" alt="img"></p> |
| <p>具体流程如上图所示：</p> |
| <ol> |
| <li>Provider 将本进程的元数据注册到 Registry 中，包括 IP，Port，以及服务名称等。</li> |
| <li>Consumer 通过 Registry 获取 Provider 的接⼊信息，直接发起请求</li> |
| </ol> |
| <p>由此可⻅，dubbo 当前的服务发现模型是针对 Endpoint 级别的，并且注册的信息不只 IP 和端⼝包括其他的⼀些元数据。</p> |
| <h2 id="无法直接使用-kubernetes-服务发现模型的原因">无法直接使用 Kubernetes 服务发现模型的原因</h2> |
| <p>通过上述两个⼩节，答案基本已经⽐较清晰了。总结⼀下，⽆法直接使⽤ Kubernetes 作为注册中⼼的原因主要为以下⼏点:</p> |
| <ol> |
| <li>Kubernetes-Service 标准的资源对象具有的服务描述字段 中并未提供完整的 dubbo 进程元数据字段因此，⽆法直接使⽤Kubernetes-Service 进⾏服务注册与发现。</li> |
| <li>dubbo-go 的服务注册是基于每个进程的，每个 dubbo 进程均需进⾏独⽴的注册。</li> |
| <li>Kubernetes-Service 默认为服务创建 VIP，提供 round-robin 的负载策略也与 dubbo-go⾃有的 Cluster 模块的负载策略形成了冲突。</li> |
| </ol> |
| <h2 id="dubbo-go-所采的注册发现案">Dubbo-go 所采⽤的注册/发现⽅案</h2> |
| <h3 id="服务注册">服务注册</h3> |
| <p>Kubernetes 基于 Service 对象实现服务注册／发现。可是 dubbo 现有⽅案为每个 dubbo-go 进程独⽴注册，因此 dubbo-go选择将该进程具有的独有的元数据写⼊运⾏该 <strong>dubbo-go</strong> 进程的 <strong>Pod</strong> 在 <strong>Kubernetes</strong>中的 <strong>Pod</strong> 资源对象的描述信息中。每个运⾏ dubbo 进程的 Pod 将本进程的元数据写⼊ Kubernetes-Pod Annotations 字段。为了避免与其他使⽤Annotations 字段的 Operator 或者其他类型的控制器（ Istio ） 的字段冲突。dubbo-go 使⽤ Key 为 <strong>dubbo.io/annotation</strong> value 为具体存储的 K/V 对的数组的 json 编码后的 base64 编码。</p> |
| <p>样例为：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">apiVersion</span>: v1 |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">kind</span>: Pod |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">metadata</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">annotations</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">dubbo.io/annotation</span>: |
| </span></span><span style="display:flex;"><span>W3siayI6Ii9kdWJibyIsInYiOiIifSx7ImsiOiIvZHViYm8vY29tLmlrdXJlbnRvLnVzZXIuVXNlcl |
| </span></span><span style="display:flex;"><span>Byb3ZpZGVyIiwidiI6IiJ9LHsiayI6Ii9kdWJiby9jb20uaWt1cmVudG8udXNlci5Vc2VyUHJvdmlk |
| </span></span><span style="display:flex;"><span>ZXIvY29uc3VtZXJzIiwidiI6IiJ9LHsiayI6Ii9kdWJibyIsInYiOiIifSx7ImsiOiIvZHViYm8vY2 |
| </span></span><span style="display:flex;"><span>9tLmlrdXJlbnRvLnVzZXIuVXNlclByb3ZpZGVyIiwidiI6IiJ9LHsiayI6Ii9kdWJiby9jb20uaWt1 |
| </span></span><span style="display:flex;"><span>cmVudG8udXNlci5Vc2VyUHJvdmlkZXIvcHJvdmlkZXJzIiwidiI6IiJ9LHsiayI6Ii9kdWJiby9jb2 |
| </span></span><span style="display:flex;"><span>0uaWt1cmVudG8udXNlci5Vc2VyUHJvdmlkZXIvY29uc3VtZXJzL2NvbnN1bWVyJTNBJTJGJTJGMTcy |
| </span></span><span style="display:flex;"><span>LjE3LjAuOCUyRlVzZXJQcm92aWRlciUzRmNhdGVnb3J5JTNEY29uc3VtZXJzJTI2ZHViYm8lM0RkdW |
| </span></span><span style="display:flex;"><span>Jib2dvLWNvbnN1bWVyLTIuNi4wJTI2cHJvdG9jb2wlM0RkdWJibyIsInYiOiIifV0= |
| </span></span></code></pre></div><p>由于每个 dubbo-go 的 Pod 均只负责注册本进程的元数据，因此 Annotations 字段⻓度也不会因为运⾏ dubbo-go 进程的 Pod 数量增加⽽增加。</p> |
| <h3 id="服务发现">服务发现</h3> |
| <p>依赖kubernetes Api-Server 提供了Watch的功能。可以观察特定namespace内各Pod对象的变化。 dubbo-go为了避免dubbo-go进程watch到与dubbo-go进程⽆关的Pod的变化，dubbo-go将watch的条件限制在当前Pod所在的namespace，以及 watch 具有 Key为<strong>dubbo.io/label</strong> Value为 <strong>dubbo.io-value</strong> 的Pod。在Watch到对应Pod的变化后实时更新本地Cache，并通过Registry提供的Subscribe通</p> |
| <p>知建⽴在注册中⼼之上的服务集群管理，或者其他功能。</p> |
| <h3 id="总体设计图">总体设计图</h3> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/k8s/design.png" alt="img"></p> |
| <p>具体流程如上图所示：</p> |
| <ol> |
| <li>启动 dubbo-go 的 Deployment 或其他类型控制器使⽤ Kubernetes Downward-Api 将本 Pod 所在 namespace 通过环境变量的形式注⼊ dubbo-go 进程。</li> |
| <li>dubbo-go 进程的 Pod 启动后通过环境变量获得当前的 namespace 以及该 Pod 名称，调⽤ Kubernetes-Apiserver PATCH 功能为本 Pod 添加 Key 为 <strong>dubbo.io/label</strong> Value为 <strong>dubbo.io-value</strong>的label。</li> |
| <li>dubbo-go 进程调⽤ Kubernetes-Apiserver 将本进程的元数据通过 PATCH 接⼝写⼊当前 Pod 的 Annotations 字段。</li> |
| <li>dubbo-go 进程 LIST 当前 namespace 下其他具有同样标签的 Pod，并解码对应的 Annotations 字段 获取其他 Pod 的信息。</li> |
| <li>dubbo-go 进程 WATCH 当前 namespace 下其他具有同样标签的 Pod 的 Annotations 的字段变化。</li> |
| </ol> |
| <h2 id="总结">总结</h2> |
| <p>K8s 已经为其承载的服务提供了一套服务发现，服务注册，以及服务集群管理机制。而 dubbo-go 的同时也拥有自成体系的服务集群管理。这两个功能点形成了冲突，在无法调谐两者的情况， dubbo-go 团队决定保持 dubbo 自有的服务集群管理系，而选择性的放弃了 Service 功能，将元数据直接写入到 Pod 对象的 Annotations 中。</p> |
| <p>当然这只是 dubbo-go 在将 K8s 作为服务注册中心的方案之一，后续社区会以更加“云原生”的形式对接 K8s ，让我们拭目以待吧。</p> |
| <p>dubbo-go 社区钉钉群 :23331795 ,欢迎你的加入。</p> |
| <blockquote> |
| <p>作者信息： 王翔，GithubID: sxllwx，就职于成都达闼科技有限公司，golang开发工程师。</p> |
| </blockquote></description></item><item><title>Blog: Dubbo-go应用维度注册模型</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go%E5%BA%94%E7%94%A8%E7%BB%B4%E5%BA%A6%E6%B3%A8%E5%86%8C%E6%A8%A1%E5%9E%8B/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/dubbo-go%E5%BA%94%E7%94%A8%E7%BB%B4%E5%BA%A6%E6%B3%A8%E5%86%8C%E6%A8%A1%E5%9E%8B/</guid><description> |
| <p>Dubbo 3.0 将至。其最重要的一点就是服务自省，其基础即是应用维度的注册模型，作为目前与 Dubbo 在功能上完全对齐的 Dubbo-go，已于 本年【2020 年】7 月份发布了其 v1.5.0 版本，实现了该模型，为年底实现与 Dubbo 3.0 对齐的新版本奠定了基础。</p> |
| <p>Dubbo-go 作为 Dubbo 的 Go 语言版本，因跨语言之故，二者针对同一模型的实现必然有较大差异，故本文注重讨论 Dubbo-go 社区自身对该模型的理解和实现，以及其与 Dubbo 之间的差异。</p> |
| <h2 id="1-引语">1 引语</h2> |
| <p>在 v1.5 以前，Dubbo-go 注册模型都是以服务为维度的，直观的理解可认为其是接口维度。譬如注册信息，按照服务维度模型其示例如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-json" data-lang="json"><span style="display:flex;"><span><span style="color:#2aa198">&#34;com.xxx.User&#34;</span>:[ |
| </span></span><span style="display:flex;"><span> {<span style="color:#268bd2">&#34;name&#34;</span>:<span style="color:#2aa198">&#34;instance1&#34;</span>, <span style="color:#268bd2">&#34;ip&#34;</span>:<span style="color:#2aa198">&#34;127.0.0.1&#34;</span>, <span style="color:#268bd2">&#34;metadata&#34;</span>:{<span style="color:#268bd2">&#34;timeout&#34;</span>:<span style="color:#2aa198">1000</span>}}, |
| </span></span><span style="display:flex;"><span> {<span style="color:#268bd2">&#34;name&#34;</span>:<span style="color:#2aa198">&#34;instance2&#34;</span>, <span style="color:#268bd2">&#34;ip&#34;</span>:<span style="color:#2aa198">&#34;127.0.0.2&#34;</span>, <span style="color:#268bd2">&#34;metadata&#34;</span>:{<span style="color:#268bd2">&#34;timeout&#34;</span>:<span style="color:#2aa198">2000</span>}}, |
| </span></span><span style="display:flex;"><span> {<span style="color:#268bd2">&#34;name&#34;</span>:<span style="color:#2aa198">&#34;instance3&#34;</span>, <span style="color:#268bd2">&#34;ip&#34;</span>:<span style="color:#2aa198">&#34;127.0.0.3&#34;</span>, <span style="color:#268bd2">&#34;metadata&#34;</span>:{<span style="color:#268bd2">&#34;timeout&#34;</span>:<span style="color:#2aa198">3000</span>}}, |
| </span></span><span style="display:flex;"><span>] |
| </span></span></code></pre></div><p>这种模式的好处是不言而喻的，简单直观，提供了细粒度的服务控制手段。</p> |
| <p>而近两年，随着云时代的到来，这种模式就暴露了不足：</p> |
| <ol> |
| <li>主流的注册模型都是应用维度的；</li> |
| <li>以服务维度来注册，那么规模与服务数量成正比，大规模集群之下【工行软件中心的接口注册规模达到万级】，注册中心压力非常大；</li> |
| </ol> |
| <h2 id="2-dubbo-go-v150-的新注册模型">2 Dubbo-go v1.5.0 的新注册模型</h2> |
| <p>这次 Dubbo-go 支持了新的注册模型，也就是应用维度的注册模型。简单而言，在应用维度注册下，其注册信息类似：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-json" data-lang="json"><span style="display:flex;"><span><span style="color:#2aa198">&#34;application1&#34;</span>: [ |
| </span></span><span style="display:flex;"><span> {<span style="color:#268bd2">&#34;name&#34;</span>:<span style="color:#2aa198">&#34;instance1&#34;</span>, <span style="color:#268bd2">&#34;ip&#34;</span>:<span style="color:#2aa198">&#34;127.0.0.1&#34;</span>, <span style="color:#268bd2">&#34;metadata&#34;</span>:{}}, |
| </span></span><span style="display:flex;"><span> {<span style="color:#268bd2">&#34;name&#34;</span>:<span style="color:#2aa198">&#34;instance2&#34;</span>, <span style="color:#268bd2">&#34;ip&#34;</span>:<span style="color:#2aa198">&#34;127.0.0.2&#34;</span>, <span style="color:#268bd2">&#34;metadata&#34;</span>:{}}, |
| </span></span><span style="display:flex;"><span> {<span style="color:#268bd2">&#34;name&#34;</span>:<span style="color:#2aa198">&#34;instanceN&#34;</span>, <span style="color:#268bd2">&#34;ip&#34;</span>:<span style="color:#2aa198">&#34;127.0.0.3&#34;</span>, <span style="color:#268bd2">&#34;metadata&#34;</span>:{}} |
| </span></span><span style="display:flex;"><span>] |
| </span></span></code></pre></div><p>在此模式之下，可以看到注册信息将会大幅度减少，集群规模只与实例数量相关。</p> |
| <p>与此同时，在实现这一个功能的时候，Dubbo-go 还希望保持两个目标：</p> |
| <ol> |
| <li>对用户完全兼容，用户迁移无感知；</li> |
| <li>保持住原本服务粒度上精细控制的能力——即保留现有的服务维度的元数据；</li> |
| </ol> |
| <p>因此 Dubbo-go 要着力解决以下几点：</p> |
| <ol> |
| <li>目前 Consumer 的配置是以接口为准的，如何根据接口找到该接口对应的应用？例如，用户配置了 <code>com.xxx.User</code> 服务，那么，Dubbo-go 怎么知道这个服务是由哪个应用来提供的呢？</li> |
| <li>在知道了是哪个应用之后，可以从注册中心拿到应用的注册信息，如实例信息等；那怎么知道 <code>com.xxx.User</code> 服务自身的元数据呢？</li> |
| </ol> |
| <p>为了解决这两个问题，在已有的注册模型的基础上，Dubbo-go 引入两个额外的组件：ServiceNameMapping 和 MetadataService。</p> |
| <p>前者用于解决服务-应用之间的映射，后者用于获取服务的元数据。</p> |
| <p>由此，Dubbo-go 的应用维度注册模型就变为：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/app-registry/app-registry-model.png" alt="img"></p> |
| <h3 id="21-servicenamemapping">2.1 ServiceNameMapping</h3> |
| <p>ServiceNameMapping 并不复杂。考虑到一般人在 Consumer 侧想要调用一个服务，其十有八九是知道这个服务是哪个应用提供的，于是 Dubbo-go 引入了新的配置项 <code>provideBy</code></p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/app-registry/provideby.png" alt="img"></p> |
| <p>当然，所谓 “十有八九”就是说有些时候确实不知道是服务是谁提供的，所以 Dubbo-go 还支持了基于配置中心的 ServiceNameMapping 实现。Dubbo-go 会用服务名作为 Key 从配置中心里面读出对应的应用名。这意味着, Provider 启动的时候，也会在配置中心将自身的 服务-应用名映射 写入配置中心。</p> |
| <h3 id="22-metadataservice">2.2 MetadataService</h3> |
| <p>MetadataService 稍微要复杂一点，有 <code>remote</code> 和 <code>local</code> 两种模式。</p> |
| <p>类似于前面的 ServiceNameMapping，Dubbo-go 提供了基于配置中心的 MetadataService 的实现，即 <code>remote</code> 模式。Provider 启动的时候，就会将服务的元数据写进去。</p> |
| <p>另外一种模式是 <code>local</code> 模式。Dubbo-go 可以直接将 MetadataService 看做是一个普通的微服务，而后由 <code>Provider</code> 所提供。类似于：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/app-registry/local-metadata-service.png" alt="img"></p> |
| <p>由此带来一个问题：</p> |
| <p>既然 Dubbo-go 将 MetadataService 看做是一个普通的服务，那么 MetadataService 的元数据，Consumer 该怎么获得呢？这是一个典型的鸡生蛋蛋生鸡的问题。</p> |
| <p>Dubbo-go 的方案非常简单粗暴，Provider 启动的时候，不仅仅往注册中心里面写入应用本身的信息，还要把它的 MetadataService 信息写入。</p> |
| <p>这是一个应用的注册信息：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/app-registry/registry-info.png" alt="img"></p> |
| <p>本质上来说，应用维度注册信息 + 服务元数据 = 服务维度注册信息。或者说，应用维度注册，只是一种重新组织这些信息的方式。</p> |
| <h2 id="3-差异与改进">3 差异与改进</h2> |
| <p>Dubbo-go v1.5.x 对标 Dubbo 2.7.5，可以认为是参照 Dubbo 2.7.5 直接实现其 Go 源码，但是考虑到 Java 和 Go 之间的语言差异，导致二者之间的实现不可能完全对等。</p> |
| <h3 id="31-修订版本号revision比对">3.1 修订版本号revision比对</h3> |
| <p>Dubbo v2.7.x 在 MetadataService 注册时，会对其 provider 应用的所有服务接口的 hash 值做为修订版本号写入元数据中心，此 revision 是对所有接口的方法以及其参数总体的计算结果。其目的是减少 consumer 端到注册中心的拉取次数。</p> |
| <p>在Go中用的计算 revision 的 hash 算法与 Java 是不一致的，而且 Go 与 Java 的方法签名信息是不相同的，所以计算出来的 hash 值一定是不一样的。</p> |
| <p>此不一致会导致如果Go应用和Java应用同时发布同一个服务的时候，Go服务和Java服务的修订版本号必定是不相同的，Consumer需要分别缓存这两个修订版本的元数据。</p> |
| <h3 id="32-应用注册时机">3.2 应用注册时机</h3> |
| <p>Dubbo-go v1.5.0 实现时，其中一个考量是全面向后兼容 v1.4.x。Dubbo-go v1.5.x 应用 consumer 既可以调用 Dubbo-go v1.4.x 应用的服务，也可以调用 Dubbo v2.6.x 应用的服务，当然也可以调用其对标的 v2.7.x 应用的服务。</p> |
| <p>为了达到兼容性，Dubbo-go v1.5.x 实现时面临一个问题：Dubbo-go provider 应用启动时有一个服务启动成功，把应用信息注册到元数据中心之后，就会把实例注册到注册中心，而 Dubbo 2.7.x 的 provider 应用则是在其所有服务接口的信息注册到元数据中心后才会注册实例！</p> |
| <p>这个问题的后果就是：Dubbo-go v1.5.0 的 provider 每次发布接口到元数据中心的同时，都会触发Dubbo-go v1.5.0 / Dubbo v2.7.x 的 consumer 应用拉取 Dubbo-go v1.5.0 应用信息，当provider 发布的服务过多时 consumer 侧性能损耗非常明显！</p> |
| <p>Dubbo-go 在 v1.5.1 中已经修复了这个问题，provider 在启动时先将其全部服务接口发布到元数据中心，然后注册实例到注册中心，减少了 consumer 拉取元数据的次数。</p> |
| <blockquote> |
| <p>本文作者： 白泽（蒋超），Github ID <a href="https://github.com/Patrick0308">@Patrick0308</a>，开源爱好者。</p> |
| </blockquote></description></item><item><title>Blog: Go 版本入 Dubbo 生态一周年</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/go-%E7%89%88%E6%9C%AC%E5%85%A5-dubbo-%E7%94%9F%E6%80%81%E4%B8%80%E5%91%A8%E5%B9%B4/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/go-%E7%89%88%E6%9C%AC%E5%85%A5-dubbo-%E7%94%9F%E6%80%81%E4%B8%80%E5%91%A8%E5%B9%B4/</guid><description> |
| <p>去年 5 月，阿里开源的高性能 RPC 框架 Dubbo 从 ASF 毕业并晋升顶级项目，同时，还宣布 Go 语言版本的 Dubbo-go 正式加入 Dubbo 官方生态。</p> |
| <p>经过一年的发展， Dubbo-go 在技术和社区运营方面都已经有了不错的成绩。Dubbo-go 是 Dubbo 的完整 Go 语言实现，在功能实现和技术路径上与 Dubbo 有不同程度的对标，项目团队预计很快便可以追平 Java 版的功能。当然，也是因为基于 Go 语言开发，Dubbo-go 更易上手，未来或将反哺 Dubbo 的云原生化。</p> |
| <p>Dubbo-go 近期还实现了 REST 协议以及 gRPC 的支持，打通了 Spring Cloud 和 gRPC 生态，再加上与 Java Dubbo 的互通，应用场景广泛。因此，它被其开发者叫做“all-in-one”的 RPC 框架。</p> |
| <p>目前 Dubbo 官方已经投入人力参与 Dubbo-go 的开发，阿里集团今年完成 HSF 和 Dubbo 的融合后，会在集团内逐步推广使用 Dubbo-go。</p> |
| <p>开源中国采访了当前正在开发中的 v1.5 版本的主要推进者邓明，回顾 Dubbo-go 的过往，尤其是最近一年的发展情况，并展望项目未来的发展。</p> |
| <h2 id="dubbo-go-过去发展回顾">Dubbo-go 过去发展回顾</h2> |
| <p><strong>OSCHINA：</strong> 作为项目主要推动者之一，可以简单回顾下 Dubbo-go 的发展历程吗？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>首先，个人代表社区，借助贵方平台，感谢 Dubbo-go 的使用者、曾经合作过的各个媒体平台以及 Dubbo 官方过去一年来对我们项目的关注，Dubbo-go 目前的发展离不开各方力量的帮助。</p> |
| <p>实际上，在 Dubbo-go 加入 Dubbo 官方生态之前，已经发展了两年。它最早由其创始人于雨在 2016 年 5 月构建，同年 9 月发布并开源的。如下时间轴图清晰记录了 Dubbo-go 的前世今生。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/1year/dubbo-go-events.png" alt="img"></p> |
| <p><strong>OSCHINA：</strong> 在去年项目刚加入 Dubbo 官方生态的时候，有开发团队成员说，Dubbo-go 当时还没能发挥出 Go 语言的优势，功能完整性还要完善。作为一个为解决 Go 项目与 Java &amp; Dubbo 项目互通的项目，经过一年的发展，项目现在能发挥出 Go 语言的优势了吗，为什么？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>和去年比起来，在发挥 Go 语言自身优势上，有了很大的提高。</p> |
| <p>Go 语言协程的个数上限比 Java 线程数目多。Go 语言的协程只运行在用户态，初始堆栈小且可伸缩，而 Java 线程启动因用户态系统态之间切换带来的额外成本被线程池抹平，所以只有在较大并发需求的场景下（核数限制的情况下，Java 线程池中最大线程数被限制），才会发挥优势。Dubbo 中类似的场景：异步处理网络和协议化的处理。我们在网络库 Getty 中加入了协程池，实现了网络收发和逻辑处理的解耦。</p> |
| <p>另外，Go 语言上手速度确实比 Java 快好几个数量级，只要搭好具有良好扩展性的架子，社区 contributor 培养的成本比 Java 低很多。得益于此，Dubbo-go 的功能和性能将很快追平 Java。</p> |
| <p>**<strong>OSCHINA：</strong> **关于 Dubbo-go 在 Java 和 Go 运行时的兼容互通和功能一致目标，目前进展如何？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>目前，Dubbo-go 已经完全对齐 Dubbo v2.6.x，正在全力开发 v1.5.0 版本可以全面对齐 v2.7.x。</p> |
| <p>Dubbo v2.7.5 之后开始支持应用维度的服务注册，这也是 v1.5.0 计划支持的核心特性。</p> |
| <p>可以剧透一下，目前 v1.5.0 版本的 Dubbo-go 开发工作已经进入了尾声，正处于测试阶段。等 v1.5.0 发布之后，我们会陆续发布几个小版本，用于对齐 Dubbo v2.7.5 之后的版本。可以说，v1.5.x 主要是为了配合 dubbo 的云原生化。</p> |
| <p>**<strong>OSCHINA：</strong> **Dubbo-go 近期实现了 REST 协议支持，可以和 Spring Cloud 生态互联；年初实现了和 gRPC 的互联，这对 Dubbo-go 有什么意义？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>Dubbo-go 在支持了 REST 协议之后，已经可以做到跟绝大部分基于 HTTP 协议的微服务框架进行通信。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/1year/dubbo-go-rest.png" alt="img"></p> |
| <p>另外一个突出优点是，支持了 gRPC 和 REST 之后，Dubbo-go 就可以考虑和一些公司内部自研的框架进行通信了。通常一些比较大的公司会自研框架，或者深度定制某些开源框架。而只要它们支持 gRPC 或者 HTTP 协议，Dubbo-go 就可以保证与这些框架的无缝衔接。</p> |
| <p>还有一个优势，REST 协议对前端更友好，可以直接把 Dubbo-go 开发的服务给前端用，而不用加一层协议转换，也避免了前端直接发起 RPC 请求。因此，Dubbo-go 也就可以成为它们在 Go 微服务框架的一个比较优秀的选择。</p> |
| <p><strong>OSCHINA：</strong> 1.4 版本中，Dubbo-go 在可观测性方面采用了 tracing 和 metric，metric 的实现参考了 Dubbo 的做法，也做了一些调整，具体是怎么样？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>可观测性是衡量一个微服务框架的重要方面。一般可观测性分成 tracing、metric 和 log 三个部分。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/1year/dubbo-go-observe.png" alt="img"></p> |
| <p>在 v1.4 Dubbo-go 之前，tracing 和 metric 是 Dubbo-go 的薄弱环节。为了支持这两个，我们考察了比较多的开源框架的做法。我们发现，因为要考虑对接非常多诸如 zipkin/cat 等监控框架，所以它们往往会设计一整套监控和度量的 API。</p> |
| <p>Dubbo 也是如此。Dubbo 的 metric 比较依赖阿里内部一个开源的 metric 的项目。这个项目也不是只能给 Dubbo 应用，而是 Java 项目都可以考虑。本质上来说，它定义了一套 API，而后提供了对不同开源框架的适配实现。把握住这一核心之后，我们就要考虑一个问题：要不要自己设计一套 API？我们的答案是 NO，并且选择了 opentracing API 作为我们监控使用的 API。</p> |
| <p>首先，我们回顾那些自己设计了 API 的开源项目，它们的 API 和 opentracing API 还比较相像。我觉得我设计不出来一个明显比 opentracing API 更加优雅的 API 了。</p> |
| <p>另外从实现效率上来说，如果我们使用 opentracing API 作为 Dubbo-go 接入 metric 和 tracing 的 API，那么，任何支持 opentracing API 的框架，都可以做到开箱即用。</p> |
| <p>目前我们正在向社区用户征集监控意见，看社区希望我们在框架内什么地方进一步埋点。我们也得到了很多反馈，下一步就要考虑进一步优化采集的数据。</p> |
| <p><strong>OSCHINA：</strong> Dubbo-go 的开发团队之前介绍，Dubbo-go 首要目的就是解决 Go 项目与 Java &amp; Dubbo 项目的互通问题，同时也为 Go 项目提供了一种 RPC 与微服务开发框架的选择。但从之前的用户使用列表来看，直接把它作为 Go 的一个 RPC 框架来使用的好像并不是特别多，现在情况是这样吗？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>这个情况已经有了很大的改善了。最开始的时候，我们的用户大部分都是 Java Dubbo 那里过来的。但是到今年，据我们了解，已经有一些用户已经是直接把 Dubbo-go 作为 RPC 框架。在经过一年的发展以后，即便不考虑和 Dubbo 保持兼容这一特点，Dubbo-go 也可以说一个比较优秀的 Go 语言 RPC 框架。</p> |
| <p>尤其是在异构系统通信和服务治理方面，我们提供了非常多样化的支持。这是很多别的 RPC 框架所不具备，或者不如我们的。</p> |
| <p><strong>OSCHINA：</strong> 总结一下这一年里，Dubbo-go 技术方面值得了解的进展吧？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>Dubbo-go 这一年的进步很大，实现了非常多非常重要的特性。</p> |
| <p>首先要提及的就是支持了很多协议，比如说基于 protobuf 的 gRPC 和 REST。这些协议保证了我们能够与市面上大多数的 RPC 框架进行通信，而且我们在 1.5.0 里面，还扩展支持支持基于 Json 的 gPRC 和 基于 protobuf 的 TCP 通信。</p> |
| <p>第二则是支持了配置中心。这个功能可以提供给用户极大的配置上的灵活性。</p> |
| <p>第三则是可观测性上改进，也就是前面提到的 metric 和 tracing。</p> |
| <p>第四则是现在正在进行的应用注册模型，它能让我们更好地拥抱 k8s 和 servise mesh。为了支持应用注册模型，我们还实现了一个元数据中心，这个元数据中心非常有利于做网关。此外还实现了很多功能，如新的限流算法，负载均衡算法和路由策略等。具体内容，欢迎大家去看我们的 release log。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/1year/dubbo-go-arch.png" alt="img"></p> |
| <p><strong>OSCHINA：</strong> 上个月，Go 官方公布的最新调查报告显示，Go 语言的主要用途包括编写 RPC 服务，其次库和框架方面增量巨大。“竞争对手”变多会影响到 Dubbo-go 原本的计划实施吗，Dubbo-go 和其他同类项目比有什么不同？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>我们对 Go 社区的进步感同身受。实际上，Dubbo-go 这一年很多功能的实现，都离不开合作社区的支持。比如说我们提供的基于 Nacos 的配置中心支持，以及现在正在测试基于 Nacos 的应用维度服务注册与发现，都十分依赖 Nacos 的 Go 语言 SDK 支持。</p> |
| <p>而且我们也注意到，别的 Go 语言的微服务框架在这一年也取得了不错的进步，这是一种很好的鞭策。在 RPC 框架上，一直都是百家齐放百花争鸣局面，迫使我们朝着“人无我有，人有我精”的方向前进。到目前来说，我们感觉我们的竞争优势还是比较明显的：</p> |
| <p>第一点就是保持了和 Dubbo 的兼容，那么原本的 Dubbo 用户在考虑 Go 语言框架的时候，我们就会是首选；</p> |
| <p>第二个竞争优势则是支持多协议。这几年一个很明显的趋势就是，一个公司的技术栈难以保持单一，因为不同框架、不同语言会有不同优点。所以 Dubbo-go 也会是那些考虑连接异构系统用户的首选；</p> |
| <p>第三则是软实力，也就是我们社区自身的优点。我们社区非常有活力，用户有什么问题都能够得到很快的响应；而我们的迭代速度，一直比较快。如果用户觉得自己能够很快获得帮助，那么他们也会倾向选择我们。</p> |
| <p><strong>OSCHINA：</strong> 我们了解到，Dubbo-go/getty 是 Dubbo-go 中比较能体现 Go 语言优势的部分，目前已经被解耦出来，可以直接用。Dubbo-go 的其他组成部分会考虑同样解耦吗？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>这可以说是一个非常长远和理想化的计划了。我们现在正在做的一件事，是把项目里面用的公共方法、和框架无关的代码抽取出来，做成一个工具类库，也就是 dubbogo-gost 这个项目。</p> |
| <p>我们注意到，不管是在 Dubbo-go，还是别的框架，这些代码都很类似，比如说对不同类型的数据排序。之前我们找过开源的 lib，但是都不尽如人意，所以我们打算把自己的拿出来，做成类似瑞士军刀一样小巧高效的工具。</p> |
| <p>另外还要提到 dubbo-go-hessian2 开源仓库。我们可以自豪地说，这个库是 Go 里面对 hessian v2 协议支持最好的开源库。不仅仅是我们在用，阿里和蚂蚁金服也在用。我们也希望吸引更加多用户来使用，帮我们改进。</p> |
| <p><strong>OSCHINA：</strong> Dubbo-go 今年 3 月 25 日的新版本 1.4.0 中“拿出了使用 k8s 作为注册中心的解决方案”，选择性放弃 Service 功能，将元数据直接写入到 Pod 对象的 Annotations 中。为什么做出这个决策，后续有什么落地计划？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>在使用 k8s 作为注册中心这个点上，讨论就花了很长的时间。</p> |
| <p>其实最初考虑的是直接使用 k8s 服务发现模型，后来发现 k8s service 和 Dubbo-go Interface 之间存在一些难以调和的矛盾。比如说 Kubernetes 已经为其承载的服务提供了⼀套服务发现，服务注册，以及服务集群管理机制。⽽ Dubbo-go 也拥有⾃成体系的服务集群管理。</p> |
| <p>这两个功能点形成了冲突，在无法调和两者的情况下，我们放弃了这个计划，并且提出了现在这个随 1.4.0 版本发布使用的模型。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/1year/dubbo-go-k8s.png" alt="img"></p> |
| <p>后续，我们将主要考虑 k8s 本身提供的 CRD + Operator 的方案，毕竟越来越多的 k8s 周边的项目都在以 Operator 作为切入点。Dubbo-go 社区后续的方案将会以 CRD 的形式在 k8s 内注册和发现服务。这样做的原因有很多，首先是为了减少 Dubbo-go 对 kube-apiserver 的直接依赖。其次是为了标准化注册模型，当服务模型以 CRD 的形式存在在 k8s 集群中之后，其他围绕 k8s 的项目可以直接使用这些资源二次开发和拓展。而这种方式更加 CloudNative。</p> |
| <p><strong>OSCHINA：</strong> Dubbo-go 现在在云原生应用上的布局是怎样的？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>社区的主要人力正与蚂蚁金服的 mosn 社区展开合作。目前有 5 个人力与 mosn 社区一起在 mosn 中实现 Dubbo 的服务发现、服务注册和基本的 RPC 通信等数据平面的能力，在 istio 层面支持通过 XDS 实现配置下发，以实现 mosn + Dubbo-go 【嵌入 mosn】 + istio 这种 sidecar 形式的云原生方案。已完成的工作已经在多点科技展开测试，近期 mosn 社区同学会在 A2M 大会上公布具体进展。</p> |
| <p>除了 sidecar 这种 proxy 形式的云原生方案，社区还计划实现 Dubbo-go【应用 sdk】 + istio 这种 proxyless 方式的云原生方案。Java 应用或者 Go 应用通过 istio 的 xDS 协议完成服务注册和发现以及路由分发。或者说，我们力求微服务和云原生共存，可以称之为“双模微服务”。这种“双模微服务”允许标准的 Dubbo-go + sidecar 和 Dubbo-go【应用 sdk】 + istio 两种模式部署的应用共存。这将是 Dubbo-go v1.6 的核心工作。</p> |
| <p><strong>OSCHINA：</strong> Dubbo-go 几乎是刚一诞生就转移到 Apache，并且很快发布了 Apache Dubbo Go v1.1.0，这对社区运营的帮助是什么，可以分享下 Dubbo-go 的运营情况和经验吗？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>可以说，Apache 基金会对我们的帮助是很大的。</p> |
| <p>一方面，Apache 自身的光环十分有助于吸引开发关注和参与；另外一方面，Apache 的一些要求，也让社区运营更加规范。</p> |
| <p>社区运营需要进一步规范化，透明化，以吸引更加多的人参与。我注意到很多优秀的社区运营做得很好，他们对 issue 的管理很细致，打上了各种标签，做到了对 issue 的轻重缓急的管理。这种标签能够很容易吸引一些打算尝试开源的新人，给社区带来新的血液。</p> |
| <p>我们尝试使用 milestone 的方式来管理 Dubbo-go 的整体进度。现在也在尝试定期召开社区会议，讨论社区发展方向，重大特性的设计，以及解决争端会议会面向整个社区，想参与的人都可以参与。</p> |
| <p>Dubbo-go 应用及规划 <strong>OSCHINA：</strong> Dubbo-go 适合什么样的企业和场景？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>我们认为，如果用户需要一款 Go 语言方面 gRPC 框架，可以考虑 Dubbo-go；如果公司有和异构系统通信的需求，Dubbo-go 也是一个比较好的选择。特别是，公司内部还有 Java Dubbo 或者 Spring Clound 之类的应用，那么 Dubbo-go 优势就更加大了。</p> |
| <p>Dubbo-go 可以说是 &quot; all-in-one &quot; 性质的 RPC 框架，自身包含服务治理等功能，非常省时省力，而且能够降低使用微服务的门槛。</p> |
| <p><strong>OSCHINA：</strong> GitHub 的用户列表中已经有来自 14 家企业的使用记录，Dubbo-go 一般会提供哪些后续支持？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>我们一直都快速响应用户的问题，而且积极鼓励用户参与到 Dubbo-go 的开发中来。目前涂鸦智能、携程等几家用户已经成为了社区贡献的主要力量。</p> |
| <p>有时候用户来做调研，进来社区咨询问题的时候，我们都会笑称他“如果选择了 Dubbo-go，就选择了一个强大的售后团队”。</p> |
| <p>社区一位很活跃的 Contributor 潘总【github id: pantianying】对我们的热情服务应该深有体会。比如他会提 issue，然后我们也会很快解决像 router、优雅退出功能就是在潘总提出之后，我们很快实现的， 还有早期一次重构之后，也是潘总尝鲜试用。尝鲜版通常有很多 BUG，所以我们都是上班打工，下班给潘总修 BUG，也算是服务周到热情用心了。</p> |
| <p>额外说下，目前 Dubbo 官方也已经投入人力参与 Dubbo-go 的开发，阿里集团今年完成 HSF 和 Dubbo 的融合后，会在集团内逐步推广使用 Dubbo-go。</p> |
| <p><strong>OSCHINA：</strong> Dubbo-go 下一版本预计什么时候发布，有没有一些长远的发展计划？</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>当前正在测试的是 v1.5 版本，希望六月份能发出来。v1.6 版本正在设计和规划中，v1.6 是和 Dubbo 3 对齐的，所以也会是一个比较大的版本。</p> |
| <p>今年我们社区主要集中在两件事上。第一件是云和云原生的支持，现在进行中的 v1.5 和 v1.6 都是围绕这一点的。今年几乎所有大的 feature 都是这方面的。</p> |
| <p>第二件事，则是进一步提高 Dubbo-go 的文档、注释和代码质量。坦白来说，Dubbo-go 的文档并不是特别好，尤其是用户文档。我们也收到了很多用户的批评正在加强 CI 和自动化测试这块。总而言之，文档与质量这件事是今年的头等大事。</p> |
| <p><strong>OSCHINA：</strong> 最后，请介绍一下自己吧。</p> |
| <p><strong>Dubbo-go 邓明：</strong></p> |
| <p>说起来挺有意思的，就是我本身是业务开发，并不是传统意义上的中间件开发或者基础架构开发。我目前在 eBay 做支付业务的开发。eBay 是一个 WLB 的公司，也就是在 eBay 我才有了足够的业余时间，开始投入到了开源社区中。</p> |
| <p>Dubbo-go 是我第一个深入参与的开源项目，也是我第一次尝试将个人对分布式系统和微服务治理的理解付诸实践的项目。它是我的第一个“孩子”。</p> |
| <p>因为工作的关系，我算是 Dubbo-go 社区投入时间最多的人之一，为 Dubbo-go 开发了一些很有意思的特性，也因此成了 Apache committer。另外我还扮演了编辑的角色，帮社区小伙伴写的博客文章把把关，润润色。我也算是 Dubbo-go 的主要管理人员了，比如说 v1.5 这个版本就是主要由我推进的——这大概还是因为我时间比较多。</p></description></item><item><title>Blog: 记一次在 mosn 对 dubbo、dubbo-go-hessian2 的性能优化</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E8%AE%B0%E4%B8%80%E6%AC%A1%E5%9C%A8-mosn-%E5%AF%B9-dubbodubbo-go-hessian2-%E7%9A%84%E6%80%A7%E8%83%BD%E4%BC%98%E5%8C%96/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E8%AE%B0%E4%B8%80%E6%AC%A1%E5%9C%A8-mosn-%E5%AF%B9-dubbodubbo-go-hessian2-%E7%9A%84%E6%80%A7%E8%83%BD%E4%BC%98%E5%8C%96/</guid><description> |
| <h3 id="背景">背景</h3> |
| <p>蚂蚁内部对 Service Mesh 的稳定性和性能要求是比较高的，内部 mosn 广泛用于生产环境。在云上和开源社区，RPC 领域 dubbo 和 spring cloud 同样广泛用于生产环境，我们在 mosn 基础上，支持了 dubbo 和 spring cloud 流量代理。我们发现在支持 dubbo 协议过程中，经过 Mesh 流量代理后，性能有非常大的性能损耗，在大商户落地 Mesh 中也对性能有较高要求，因此本文会重点描述在基于 Go 语言库 <a href="https://github.com/apache/dubbo-go-hessian2">dubbo-go-hessian2</a> 、dubbo 协议中对 <a href="https://github.com/mosn/mosn">mosn</a> 所做的性能优化。</p> |
| <h3 id="性能优化概述">性能优化概述</h3> |
| <p>根据实际业务部署场景，并没有选用高性能机器，使用普通 linux 机器，配置和压测参数如下：</p> |
| <ul> |
| <li>Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz 4 核 16G 。</li> |
| <li>pod 配置 <code>2c、1g</code>，JVM 参数 <code>-server -Xms1024m -Xmx1024m</code>。</li> |
| <li>网络延迟 0.23 ms, 2 台 linux 机器，分别部署 server + mosn, 压测程序 <a href="https://github.com/zonghaishang/rpc-performance">rpc-perfomance</a>。</li> |
| </ul> |
| <p>经过 3 轮性能优化后，使用优化版本 mosn 将会获得以下性能收益（框架随机 512 和 1k 字节压测）：</p> |
| <ul> |
| <li>512 字节数据：mosn + dubbo 服务调用 TPS 整体提升 55-82.8%，RT 降低 45% 左右，内存占用 40M，</li> |
| <li>1k 数据：mosn + dubbo 服务调用 TPS 整体提升 51.1-69.3%，RT 降低 41% 左右, 内存占用 41M。</li> |
| </ul> |
| <h3 id="性能优化工具-pprof">性能优化工具 pprof</h3> |
| <p>磨刀不误砍柴工，在性能优化前首先要找到性能卡点，找到性能卡点后，另一个难点就是如何用高效代码优化替代 slow code。因为蚂蚁 Service Mesh 是基于 go 语言实现的，我们首选 go 自带的 pprof 性能工具，我们简要介绍这个工具如何使用。如果我们 go 库自带 http.Server 时并且在 main 头部导入<code>import _ &quot;net/http/pprof&quot;</code>，go 会帮我们挂载对应的 handler , 详细可以参考 <a href="https://pkg.go.dev/net/http/pprof?tab=doc">godoc</a> 。</p> |
| <p>因为 mosn 默认会在<code>34902</code>端口暴露 http 服务，通过以下命令轻松获取 mosn 的性能诊断文件：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>go tool pprof -seconds <span style="color:#2aa198">60</span> http://benchmark-server-ip:34902/debug/pprof/profile |
| </span></span><span style="display:flex;"><span><span style="color:#586e75"># 会生成类似以下文件，该命令采样cpu 60秒</span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75"># pprof.mosn.samples.cpu.001.pb.gz</span> |
| </span></span></code></pre></div><p>然后继续用 pprof 打开诊断文件，方便在浏览器查看，在图 1-1 给出压测后 profiler 火焰图：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#586e75"># http=:8000代表pprof打开8000端口然后用于web浏览器分析</span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75"># mosnd代表mosn的二进制可执行文件，用于分析代码符号</span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75"># pprof.mosn.samples.cpu.001.pb.gz是cpu诊断文件</span> |
| </span></span><span style="display:flex;"><span>go tool pprof -http<span style="color:#719e07">=</span>:8000 mosnd pprof.mosn.samples.cpu.001.pb.gz |
| </span></span></code></pre></div><p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/moson-optimize/p1.png" alt="img"></p> |
| <p>图 1-1 mosn 性能压测火焰图</p> |
| <p>在获得诊断数据后，可以切到浏览器 Flame Graph（火焰图，go 1.11 以上版本自带），火焰图的 x 轴坐标代表 CPU 消耗情况， y 轴代码方法调用堆栈。在优化开始之前，我们借助 go 工具 pprof 可以诊断出大致的性能卡点在以下几个方面（直接压 server 端 mosn）：</p> |
| <ul> |
| <li>mosn 在接收 dubbo 请求，CPU 卡点在 streamConnection.Dispatch</li> |
| <li>mosn 在转发 dubbo 请求，CPU 卡点在 downStream.Receive</li> |
| </ul> |
| <p>可以点击火焰图任意横条，进去查看长方块耗时和堆栈明细（请参考图 1-2 和 1-3 所示）：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/moson-optimize/p2.png" alt="img"></p> |
| <p>图 1-2 Dispatch 火焰图明细</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/moson-optimize/p3.png" alt="img"></p> |
| <p>图 1-3 Receive 火焰图明细</p> |
| <h3 id="性能优化思路">性能优化思路</h3> |
| <p>本文重点记录优化了哪些 case 才能提升 50% 以上的吞吐量和降低 RT，因此后面直接分析当前优化了哪些 case。在此之前，我们以 Dispatch 为例，看下它为甚么那么吃性能 。在 terminal 中通过以下命令可以查看代码行耗费 CPU 数据（代码有删减）：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">go</span> tool pprof mosnd pprof.mosn.samples.cpu<span style="color:#2aa198">.001</span>.pb.<span style="color:#268bd2">gz</span> |
| </span></span><span style="display:flex;"><span>(pprof) list Dispatch |
| </span></span><span style="display:flex;"><span>Total: <span style="color:#2aa198">1.75</span>mins |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">370</span>ms <span style="color:#2aa198">37.15</span><span style="color:#268bd2">s</span> (flat, cum) <span style="color:#2aa198">35.46</span><span style="color:#719e07">%</span> of Total |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">123</span>:<span style="color:#268bd2">func</span> (conn <span style="color:#719e07">*</span>streamConnection) <span style="color:#268bd2">Dispatch</span>(buffer types.IoBuffer) { |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">40</span>ms <span style="color:#2aa198">630</span>ms <span style="color:#2aa198">125</span>: log.DefaultLogger.<span style="color:#268bd2">Tracef</span>(<span style="color:#2aa198">&#34;stream connection dispatch data string = %v&#34;</span>, buffer.<span style="color:#268bd2">String</span>()) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">126</span>: |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">127</span>: <span style="color:#586e75">// get sub protocol codec |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> . <span style="color:#2aa198">250</span>ms <span style="color:#2aa198">128</span>: requestList <span style="color:#719e07">:=</span> conn.codec.<span style="color:#268bd2">SplitFrame</span>(buffer.<span style="color:#268bd2">Bytes</span>()) |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">20</span>ms <span style="color:#2aa198">20</span>ms <span style="color:#2aa198">129</span>: <span style="color:#719e07">for</span> _, request <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> requestList { |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">160</span>ms <span style="color:#2aa198">134</span>: headers <span style="color:#719e07">:=</span> <span style="color:#b58900">make</span>(<span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#dc322f">string</span>) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">135</span>: <span style="color:#586e75">// support dynamic route |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">50</span>ms <span style="color:#2aa198">920</span>ms <span style="color:#2aa198">136</span>: headers[strings.<span style="color:#268bd2">ToLower</span>(protocol.MosnHeaderHostKey)] = conn.connection.<span style="color:#268bd2">RemoteAddr</span>().<span style="color:#268bd2">String</span>() |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">149</span>: |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">150</span>: <span style="color:#586e75">// get stream id |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">440</span>ms <span style="color:#2aa198">151</span>: streamID <span style="color:#719e07">:=</span> conn.codec.<span style="color:#268bd2">GetStreamID</span>(request) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">156</span>: <span style="color:#586e75">// request route |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> . <span style="color:#2aa198">50</span>ms <span style="color:#2aa198">157</span>: requestRouteCodec, ok <span style="color:#719e07">:=</span> conn.codec.(xprotocol.RequestRouting) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">158</span>: <span style="color:#719e07">if</span> ok { |
| </span></span><span style="display:flex;"><span> . <span style="color:#2aa198">20.11</span>s <span style="color:#2aa198">159</span>: routeHeaders <span style="color:#719e07">:=</span> requestRouteCodec.<span style="color:#268bd2">GetMetas</span>(request) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">165</span>: } |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">166</span>: |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">167</span>: <span style="color:#586e75">// tracing |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">80</span>ms <span style="color:#2aa198">168</span>: tracingCodec, ok <span style="color:#719e07">:=</span> conn.codec.(xprotocol.Tracing) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">169</span>: <span style="color:#268bd2">var</span> span types.Span |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">170</span>: <span style="color:#719e07">if</span> ok { |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">1.91</span>s <span style="color:#2aa198">171</span>: serviceName <span style="color:#719e07">:=</span> tracingCodec.<span style="color:#268bd2">GetServiceName</span>(request) |
| </span></span><span style="display:flex;"><span> . <span style="color:#2aa198">2.17</span>s <span style="color:#2aa198">172</span>: methodName <span style="color:#719e07">:=</span> tracingCodec.<span style="color:#268bd2">GetMethodName</span>(request) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">176</span>: |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">177</span>: <span style="color:#719e07">if</span> trace.<span style="color:#268bd2">IsEnabled</span>() { |
| </span></span><span style="display:flex;"><span> . <span style="color:#2aa198">50</span>ms <span style="color:#2aa198">179</span>: tracer <span style="color:#719e07">:=</span> trace.<span style="color:#268bd2">Tracer</span>(protocol.Xprotocol) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">180</span>: <span style="color:#719e07">if</span> tracer <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">20</span>ms <span style="color:#2aa198">1.66</span>s <span style="color:#2aa198">181</span>: span = tracer.<span style="color:#268bd2">Start</span>(conn.context, headers, time.<span style="color:#268bd2">Now</span>()) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">182</span>: } |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">183</span>: } |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">184</span>: } |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">185</span>: |
| </span></span><span style="display:flex;"><span> . <span style="color:#2aa198">110</span>ms <span style="color:#2aa198">186</span>: reqBuf <span style="color:#719e07">:=</span> networkbuffer.<span style="color:#268bd2">NewIoBufferBytes</span>(request) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">188</span>: <span style="color:#586e75">// append sub protocol header |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">950</span>ms <span style="color:#2aa198">189</span>: headers[types.HeaderXprotocolSubProtocol] = <span style="color:#b58900">string</span>(conn.subProtocol) |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">4.96</span>s <span style="color:#2aa198">190</span>: conn.<span style="color:#268bd2">OnReceive</span>(ctx, streamID, protocol.<span style="color:#268bd2">CommonHeader</span>(headers), reqBuf, span, isHearbeat) |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">30</span>ms <span style="color:#2aa198">60</span>ms <span style="color:#2aa198">191</span>: buffer.<span style="color:#268bd2">Drain</span>(requestLen) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">192</span>: } |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">193</span>:} |
| </span></span></code></pre></div><p>通过上面 <code>list Dispatch</code> 命令，性能卡点主要分布在 <code>159</code> 、 <code>171</code> 、<code>172</code> 、 <code>181</code> 、和 <code>190</code> 等行，主要卡点在解码 dubbo 参数、重复解参数、tracer、发序列化和 log 等。</p> |
| <h4 id="1-优化-dubbo-解码-getmetas">1. 优化 dubbo 解码 GetMetas</h4> |
| <p>我们通过解码 dubbo 的 body 可以获得以下信息，调用的目标接口（ interface ）和调用方法的服务分组（ group ）等信息，但是需要跳过所有业务方法参数，目前使用开源的 <a href="https://github.com/apache/dubbo-go-hessian2">dubbo-go-hessian2</a> 库，解析 string 和 map 性能较差, 提升 hessian 库解码性能，会在本文后面讲解。</p> |
| <p><strong>优化思路：</strong></p> |
| <p>在 mosn 的 ingress 端（ mosn 直接转发请求给本地 java server 进程）, 我们根据请求的 path 和 version 窥探用户使用的 interface 和 group , 构建正确的 dataID 可以进行无脑转发，无需解码 body，榨取性能提升。</p> |
| <p>我们可以在服务注册时，构建服务发布的 path 、version 和 group 到 interface 、group 映射。在 mosn 转发 dubbo 请求时可以通过读锁查 cache + 跳过解码 body，加速 mosn 性能。</p> |
| <p>因此我们构建以下 cache 实现（数组 + 链表数据结构）, 可参见 <a href="https://github.com/mosn/mosn/pull/1174/commits/9020ee9995cd15a7a4321a375a9506cf94dc70a8#diff-f5ff30debd68b8318c8236a0b5ccde07R6">优化代码 diff</a> ：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// metadata.go |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// DubboPubMetadata dubbo pub cache metadata |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">var</span> DubboPubMetadata = <span style="color:#719e07">&amp;</span>Metadata{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// DubboSubMetadata dubbo sub cache metadata |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">var</span> DubboSubMetadata = <span style="color:#719e07">&amp;</span>Metadata{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// Metadata cache service pub or sub metadata. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// speed up for decode or encode dubbo peformance. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// please do not use outside of the dubbo framwork. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> Metadata <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> data <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#719e07">*</span>Node |
| </span></span><span style="display:flex;"><span> mu sync.RWMutex <span style="color:#586e75">// protect data internal |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// Find cached pub or sub metatada. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// caller should be check match is true |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (m <span style="color:#719e07">*</span>Metadata) <span style="color:#268bd2">Find</span>(path, version <span style="color:#dc322f">string</span>) (node <span style="color:#719e07">*</span>Node, matched <span style="color:#dc322f">bool</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// we found nothing |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> m.data <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span>, <span style="color:#cb4b16">false</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> m.mu.<span style="color:#268bd2">RLocker</span>().<span style="color:#268bd2">Lock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// for performance |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// m.mu.RLocker().Unlock() should be called. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// we check head node first |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> head <span style="color:#719e07">:=</span> m.data[path] |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> head <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> <span style="color:#719e07">||</span> head.count <span style="color:#719e07">&lt;=</span> <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> m.mu.<span style="color:#268bd2">RLocker</span>().<span style="color:#268bd2">Unlock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span>, <span style="color:#cb4b16">false</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> node = head.Next |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// just only once, just return |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// for dubbo framwork, that&#39;s what we&#39;re expected. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> head.count <span style="color:#719e07">==</span> <span style="color:#2aa198">1</span> { |
| </span></span><span style="display:flex;"><span> m.mu.<span style="color:#268bd2">RLocker</span>().<span style="color:#268bd2">Unlock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> node, <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> count <span style="color:#dc322f">int</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> found <span style="color:#719e07">*</span>Node |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> ; node <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span>; node = node.Next { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> node.Version <span style="color:#719e07">==</span> version { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> found <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> found = node |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> count<span style="color:#719e07">++</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> m.mu.<span style="color:#268bd2">RLocker</span>().<span style="color:#268bd2">Unlock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> found, count <span style="color:#719e07">==</span> <span style="color:#2aa198">1</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// Register pub or sub metadata |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (m <span style="color:#719e07">*</span>Metadata) <span style="color:#268bd2">Register</span>(path <span style="color:#dc322f">string</span>, node <span style="color:#719e07">*</span>Node) { |
| </span></span><span style="display:flex;"><span> m.mu.<span style="color:#268bd2">Lock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// for performance |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// m.mu.Unlock() should be called. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> m.data <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> m.data = <span style="color:#b58900">make</span>(<span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#719e07">*</span>Node, <span style="color:#2aa198">4</span>) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// we check head node first |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> head <span style="color:#719e07">:=</span> m.data[path] |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> head <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> head = <span style="color:#719e07">&amp;</span>Node{ |
| </span></span><span style="display:flex;"><span> count: <span style="color:#2aa198">1</span>, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// update head |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> m.data[path] = head |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> insert <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>Node{ |
| </span></span><span style="display:flex;"><span> Service: node.Service, |
| </span></span><span style="display:flex;"><span> Version: node.Version, |
| </span></span><span style="display:flex;"><span> Group: node.Group, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> next <span style="color:#719e07">:=</span> head.Next |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> next <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// fist insert, just insert to head |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> head.Next = insert |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// record last element |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> head.last = insert |
| </span></span><span style="display:flex;"><span> m.mu.<span style="color:#268bd2">Unlock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// we check already exist first |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">for</span> ; next <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span>; next = next.Next { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// we found it |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> next.Version <span style="color:#719e07">==</span> node.Version <span style="color:#719e07">&amp;&amp;</span> next.Group <span style="color:#719e07">==</span> node.Group { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// release lock and no nothing |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> m.mu.<span style="color:#268bd2">Unlock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> head.count<span style="color:#719e07">++</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// append node to the end of the list |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> head.last.Next = insert |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// update last element |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> head.last = insert |
| </span></span><span style="display:flex;"><span> m.mu.<span style="color:#268bd2">Unlock</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>通过服务注册时构建好的 cache，可以在 mosn 的 stream 做解码时命中 cache , 无需解码参数获取接口和 group 信息，可参见<a href="https://github.com/mosn/mosn/pull/1174/commits/9020ee9995cd15a7a4321a375a9506cf94dc70a8#diff-73d1153005841c788c91116915f460a5R188">优化代码 diff</a> :</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// decoder.go |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// for better performance. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// If the ingress scenario is not using group, |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// we can skip parsing attachment to improve performance |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#719e07">if</span> listener <span style="color:#719e07">==</span> IngressDubbo { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> node, matched = DubboPubMetadata.<span style="color:#268bd2">Find</span>(path, version); matched { |
| </span></span><span style="display:flex;"><span> meta[ServiceNameHeader] = node.Service |
| </span></span><span style="display:flex;"><span> meta[GroupNameHeader] = node.Group |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} <span style="color:#719e07">else</span> <span style="color:#719e07">if</span> listener <span style="color:#719e07">==</span> EgressDubbo { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// for better performance. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// If the egress scenario is not using group, |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// we can skip parsing attachment to improve performance |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> node, matched = DubboSubMetadata.<span style="color:#268bd2">Find</span>(path, version); matched { |
| </span></span><span style="display:flex;"><span> meta[ServiceNameHeader] = node.Service |
| </span></span><span style="display:flex;"><span> meta[GroupNameHeader] = node.Group |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>在 mosn 的 egress 端（ mosn 直接转发请求给本地 java client 进程）, 我们采用类似的思路, 我们根据请求的 path 和 version 去窥探用户使用的 interface 和 group , 构建正确的 dataID 可以进行无脑转发，无需解码 body，榨取性能提升。</p> |
| <h4 id="2-优化-dubbo-解码参数">2. 优化 dubbo 解码参数</h4> |
| <p>在 dubbo 解码参数值的时候 ，mosn 采用的是 hessian 的正则表达式查找，非常耗费性能。我们先看下优化前后 benchmark 对比, 性能提升 50 倍。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>go <span style="color:#b58900">test</span> -bench<span style="color:#719e07">=</span>BenchmarkCountArgCount -run<span style="color:#719e07">=</span>^$ -benchmem |
| </span></span><span style="display:flex;"><span>BenchmarkCountArgCountByRegex-12 <span style="color:#2aa198">200000</span> <span style="color:#2aa198">6236</span> ns/op <span style="color:#2aa198">1472</span> B/op <span style="color:#2aa198">24</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkCountArgCountOptimized-12 <span style="color:#2aa198">10000000</span> <span style="color:#2aa198">124</span> ns/op <span style="color:#2aa198">0</span> B/op <span style="color:#2aa198">0</span> allocs/op |
| </span></span></code></pre></div><p><strong>优化思路：</strong></p> |
| <p>可以消除正则表达式，采用简单字符串解析识别参数类型个数， <a href="https://github.com/zonghaishang/dubbo/blob/e0fd702825a274379fb609229bdb06ca0586122e/dubbo-common/src/main/java/org/apache/dubbo/common/utils/ReflectUtils.java#L370">dubbo 编码参数个数字符串实现</a> 并不复杂, 主要给对象加 L 前缀、数组加 [、primitive 类型有单字符代替。采用 go 可以实现同等解析, 可以参考<a href="https://github.com/mosn/mosn/pull/1174/commits/9020ee9995cd15a7a4321a375a9506cf94dc70a8#diff-73d1153005841c788c91116915f460a5R245">优化代码 diff</a> ：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">getArgumentCount</span>(desc <span style="color:#dc322f">string</span>) <span style="color:#dc322f">int</span> { |
| </span></span><span style="display:flex;"><span> len <span style="color:#719e07">:=</span> <span style="color:#b58900">len</span>(desc) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> len <span style="color:#719e07">==</span> <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">0</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> args, next = <span style="color:#2aa198">0</span>, <span style="color:#cb4b16">false</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> _, ch <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> desc { |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// is array ? |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> ch <span style="color:#719e07">==</span> <span style="color:#2aa198">&#39;[&#39;</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">continue</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// is object ? |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> next <span style="color:#719e07">&amp;&amp;</span> ch <span style="color:#719e07">!=</span> <span style="color:#2aa198">&#39;;&#39;</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">continue</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">switch</span> ch { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">case</span> <span style="color:#2aa198">&#39;V&#39;</span>, <span style="color:#586e75">// void |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;Z&#39;</span>, <span style="color:#586e75">// boolean |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;B&#39;</span>, <span style="color:#586e75">// byte |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;C&#39;</span>, <span style="color:#586e75">// char |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;D&#39;</span>, <span style="color:#586e75">// double |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;F&#39;</span>, <span style="color:#586e75">// float |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;I&#39;</span>, <span style="color:#586e75">// int |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;J&#39;</span>, <span style="color:#586e75">// long |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#2aa198">&#39;S&#39;</span>: <span style="color:#586e75">// short |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> args<span style="color:#719e07">++</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">default</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// we found object |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">if</span> ch <span style="color:#719e07">==</span> <span style="color:#2aa198">&#39;L&#39;</span> { |
| </span></span><span style="display:flex;"><span> args<span style="color:#719e07">++</span> |
| </span></span><span style="display:flex;"><span> next = <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// end of object ? |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> } <span style="color:#719e07">else</span> <span style="color:#719e07">if</span> ch <span style="color:#719e07">==</span> <span style="color:#2aa198">&#39;;&#39;</span> { |
| </span></span><span style="display:flex;"><span> next = <span style="color:#cb4b16">false</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> args |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h4 id="3-优化-dubbo-hessian-go-解码-string-性能">3. 优化 dubbo hessian go 解码 string 性能</h4> |
| <p>在图 1-2 中可以看到 dubbo hessian go 在解码 string 占比 CPU 采样较高，我们在解码 dubbo 请求时，会解析 dubbo 框架版本、调用 path 、接口版本和方法名，这些都是 string 类型，dubbo hessian go 解析 string 会影响 RPC 性能。</p> |
| <p>我们首先跑一下 benchmar k 前后解码 string 性能对比，性能提升 56.11%， 对应到 RPC 中有 5% 左右提升。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>BenchmarkDecodeStringOriginal-12 <span style="color:#2aa198">1967202</span> <span style="color:#2aa198">613</span> ns/op <span style="color:#2aa198">272</span> B/op <span style="color:#2aa198">6</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkDecodeStringOptimized-12 <span style="color:#2aa198">4477216</span> <span style="color:#2aa198">269</span> ns/op <span style="color:#2aa198">224</span> B/op <span style="color:#2aa198">5</span> allocs/op |
| </span></span></code></pre></div><p><strong>优化思路：</strong></p> |
| <p>直接使用 UTF-8 byte 解码，性能最高，之前先解码 byte 成 rune , 对 rune 解码成 string ，及其耗费性能。增加批量 string chunk copy ，降低 read 调用，并且使用 unsafe 转换 string（避免一些校验），因为代码优化 diff 较多，这里给出<a href="https://github.com/apache/dubbo-go-hessian2/pull/188">优化代码 PR</a> 。</p> |
| <p>go SDK 代码<code>runtime/string.go#slicerunetostring</code>（ rune 转换成 string ）， 同样是把 rune 转成 byte 数组，这里给了我优化思路启发。</p> |
| <h4 id="4-优化-hessian-库编解码对象">4. 优化 hessian 库编解码对象</h4> |
| <p>虽然消除了 dubbo 的 body 解码部分，但是 mosn 在处理 dubbo 请求时，必须要借助 hessian 去 decode 请求头部的框架版本、请求 path 和接口版本值。但是每次在解码的时候都会创建序列化对象，开销非常高，因为 hessian 每次在创建 reader 的时候会 allocate 4k 数据并 reset。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">75</span>:<span style="color:#268bd2">func</span> <span style="color:#268bd2">unSerialize</span>(serializeId <span style="color:#dc322f">int</span>, data []<span style="color:#dc322f">byte</span>, parseCtl unserializeCtl) <span style="color:#719e07">*</span>dubboAttr { |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">10</span>ms <span style="color:#2aa198">140</span>ms <span style="color:#2aa198">82</span>: attr <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>dubboAttr{} |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">80</span>ms <span style="color:#2aa198">2.56</span>s <span style="color:#2aa198">83</span>: decoder <span style="color:#719e07">:=</span> hessian.<span style="color:#268bd2">NewDecoderWithSkip</span>(data[:]) |
| </span></span><span style="display:flex;"><span>ROUTINE <span style="color:#719e07">========================</span> bufio.NewReaderSize in <span style="color:#719e07">/</span>usr<span style="color:#719e07">/</span>local<span style="color:#719e07">/</span><span style="color:#719e07">go</span><span style="color:#719e07">/</span>src<span style="color:#719e07">/</span>bufio<span style="color:#719e07">/</span>bufio.<span style="color:#719e07">go</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">50</span>ms <span style="color:#2aa198">2.44</span><span style="color:#268bd2">s</span> (flat, cum) <span style="color:#2aa198">2.33</span><span style="color:#719e07">%</span> of Total |
| </span></span><span style="display:flex;"><span> . <span style="color:#2aa198">220</span>ms <span style="color:#2aa198">55</span>: r <span style="color:#719e07">:=</span> <span style="color:#b58900">new</span>(Reader) |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">50</span>ms <span style="color:#2aa198">2.22</span>s <span style="color:#2aa198">56</span>: r.<span style="color:#268bd2">reset</span>(<span style="color:#b58900">make</span>([]<span style="color:#dc322f">byte</span>, size), rd) |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">57</span>: <span style="color:#719e07">return</span> r |
| </span></span><span style="display:flex;"><span> . . <span style="color:#2aa198">58</span>:} |
| </span></span></code></pre></div><p>我们可以写个池化内存前后性能对比, 性能提升 85.4% , <a href="https://github.com/apache/dubbo-go-hessian2/blob/9b418c4e2700964f244e6b982855b4e89b45990d/string_test.go#L161">benchmark 用例</a> ：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>BenchmarkNewDecoder-12 <span style="color:#2aa198">1487685</span> <span style="color:#2aa198">803</span> ns/op <span style="color:#2aa198">4528</span> B/op <span style="color:#2aa198">9</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkNewDecoderOptimized-12 <span style="color:#2aa198">10564024</span> <span style="color:#2aa198">117</span> ns/op <span style="color:#2aa198">128</span> B/op <span style="color:#2aa198">3</span> allocs/op |
| </span></span></code></pre></div><p><strong>优化思路：</strong></p> |
| <p>在每次编解码时，池化 hessian 的 decoder 对象，新增 NewCheapDecoderWithSkip 并支持 reset 复用 decoder 。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">var</span> decodePool = <span style="color:#719e07">&amp;</span>sync.Pool{ |
| </span></span><span style="display:flex;"><span> New: <span style="color:#268bd2">func</span>() <span style="color:#268bd2">interface</span>{} { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> hessian.<span style="color:#268bd2">NewCheapDecoderWithSkip</span>([]<span style="color:#dc322f">byte</span>{}) |
| </span></span><span style="display:flex;"><span> }, |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// 在解码时按照如下方法调用 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>decoder <span style="color:#719e07">:=</span> decodePool.<span style="color:#268bd2">Get</span>().(<span style="color:#719e07">*</span>hessian.Decoder) |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// fill decode data |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>decoder.<span style="color:#268bd2">Reset</span>(data[:]) |
| </span></span><span style="display:flex;"><span>hessianPool.<span style="color:#268bd2">Put</span>(decoder) |
| </span></span></code></pre></div><h4 id="5-优化重复解码-service-和-methodname-值">5. 优化重复解码 service 和 methodName 值</h4> |
| <p>xprotocol 在实现 xprotocol.Tracing 获取服务名称和方法时，会触发调用并解析 2 次，调用开销比较大。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>10ms 1.91s 171: serviceName :<span style="color:#719e07">=</span> tracingCodec.GetServiceName<span style="color:#719e07">(</span>request<span style="color:#719e07">)</span> |
| </span></span><span style="display:flex;"><span> . 2.17s 172: methodName :<span style="color:#719e07">=</span> tracingCodec.GetMethodName<span style="color:#719e07">(</span>request<span style="color:#719e07">)</span> |
| </span></span></code></pre></div><p><strong>优化思路：</strong></p> |
| <p>因为在 GetMetas 里面已经解析过一次了，可以把解析过的 headers 传进去，如果 headers 有了就不用再去解析了，并且重构接口名称为一个，返回值为二元组，消除一次调用。</p> |
| <h4 id="6-优化-streamid-类型转换">6. 优化 streamID 类型转换</h4> |
| <p>在 go 中将 byte 数组和 streamID 进行互转的时候，比较费性能。</p> |
| <p><strong>优化思路：</strong></p> |
| <p>生产代码中, 尽量不要使用 fmt.Sprintf 和 fmt.Printf 去做类型转换和打印信息。可以使用 strconv 去转换。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span> . <span style="color:#2aa198">430</span>ms <span style="color:#2aa198">147</span>: reqIDStr <span style="color:#719e07">:=</span> fmt.<span style="color:#268bd2">Sprintf</span>(<span style="color:#2aa198">&#34;%d&#34;</span>, reqID) |
| </span></span><span style="display:flex;"><span><span style="color:#2aa198">60</span>ms <span style="color:#2aa198">4.10</span>s <span style="color:#2aa198">168</span>: fmt.<span style="color:#268bd2">Printf</span>(<span style="color:#2aa198">&#34;src=%s, len=%d, reqid:%v\n&#34;</span>, streamID, reqIDStrLen, reqIDStr) |
| </span></span></code></pre></div><h4 id="7-优化昂贵的系统调用">7. 优化昂贵的系统调用</h4> |
| <p>mosn 在解码 dubbo 的请求时，会在 header 中塞一份远程 host 的地址，并且在 for 循环中获取 remote IP，系统调用开销比较高。</p> |
| <p><strong>优化思路：</strong></p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#2aa198">50</span>ms <span style="color:#2aa198">920</span>ms <span style="color:#2aa198">136</span>: headers[strings.<span style="color:#268bd2">ToLower</span>(protocol.MosnHeaderHostKey)] = conn.connection.<span style="color:#268bd2">RemoteAddr</span>().<span style="color:#268bd2">String</span>() |
| </span></span></code></pre></div><p>在获取远程地址时，尽可能在 streamConnection 中 cache 远程 IP 值，不要每次都去调用 RemoteAddr。</p> |
| <h4 id="8-优化-slice-和-map-触发扩容和-rehash">8. 优化 slice 和 map 触发扩容和 rehash</h4> |
| <p>在 mosn 处理 dubbo 请求时，会根据接口、版本和分组去构建 dataID ，然后匹配 cluster , 会创建默认 slice 和 map 对象，经过性能诊断，导致不断 allocate slice 和 grow map 容量比较费性能。</p> |
| <p><strong>优化思路：</strong></p> |
| <p>使用 slice 和 map 时，尽可能预估容量大小，使用 make(type, capacity) 去指定初始大小。</p> |
| <h4 id="9-优化-trace-日志级别输出">9. 优化 trace 日志级别输出</h4> |
| <p>mosn 中不少代码在处理逻辑时，会打很多 trace 级别的日志，并且会传递不少参数值。</p> |
| <p><strong>优化思路：</strong></p> |
| <p>调用 trace 输出前，尽量判断一下日志级别，如果有多个 trace 调用，尽可能把所有字符串写到 buf 中，然后把 buf 内容写到日志中，并且尽可能少的调用 trace 日志方法。</p> |
| <h4 id="10-优化-tracerlog-和-metrics">10. 优化 tracer、log 和 metrics</h4> |
| <p>在大促期间，对机器的性能要求较高，经过性能诊断，tracer、mosn log 和 cloud metrics 写日志（ IO 操作）非常耗费性能。</p> |
| <p><strong>优化思路：</strong></p> |
| <p>通过配置中心下发配置或者增加大促开关，允许 API 调用这些 feature 的开关。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>/api/v1/downgrade/on |
| </span></span><span style="display:flex;"><span>/api/v1/downgrade/off |
| </span></span></code></pre></div><h4 id="11-优化-route-header-解析">11. 优化 route header 解析</h4> |
| <p>mosn 中在做路由前，需要做大量的 header 的 map 访问，比如 IDC、antvip 等逻辑判断，商业版或者开源 mosn 不需要这些逻辑，这些也会占用一些开销。</p> |
| <p><strong>优化思路：</strong></p> |
| <p>如果是云上逻辑，主站的逻辑都不走。</p> |
| <h4 id="12-优化-featuregate-调用">12. 优化 featuregate 调用</h4> |
| <p>在 mosn 中处理请求时，为了区分主站和商业版路由逻辑，会通过 featuregate 判断逻辑走哪部分。通过 featuregate 调用开销较大，需要频繁的做类型转换和多层 map 去获取。</p> |
| <p><strong>优化思路：</strong></p> |
| <p>通过一个 bool 变量记录 featuregate 对应开关，如果没有初始化过，就主动调用一下 featuregate。</p> |
| <h3 id="未来性能优化思考">未来性能优化思考</h3> |
| <p>经过几轮性能优化 ，目前看火焰图，卡点都在 connection 的 read 和 write ，可以优化的空间比较小了。但是可能从以下场景中获得收益：</p> |
| <ul> |
| <li>减少 connection 的 read 和 write 次数 (syscall) 。</li> |
| <li>优化 IO 线程模型，减少携程和上下文切换等。</li> |
| </ul> |
| <p>作为结束，给出了最终优化后的火焰图 ，大部分卡点都在系统调用和网络读写, 请参考图 1-4。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/moson-optimize/p4.png" alt="img"></p> |
| <p>图 1-4 优化版本 mosn + dubbo 火线图</p> |
| <h3 id="其他">其他</h3> |
| <p>pprof 工具异常强大，可以诊断 CPU、memory、go 协程、tracer 和死锁等，该工具可以参考 <a href="https://blog.golang.org/pprof">godoc</a>，性能优化参考：</p> |
| <ul> |
| <li><a href="https://blog.golang.org/pprof">https://blog.golang.org/pprof</a></li> |
| <li><a href="https://www.cnblogs.com/Dr-wei/p/11742414.html">https://www.cnblogs.com/Dr-wei/p/11742414.html</a></li> |
| </ul> |
| <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> |
| <iframe src="https://www.youtube.com/embed/N3PWzBeLX2M" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title="Profiling and Optimizing Go"></iframe> |
| </div> |
| <h2 id="欢迎加入-dubbo-go-社区">欢迎加入 dubbo-go 社区</h2> |
| <p>Dubbo-go 的社区钉钉群： 23331795，欢迎感兴趣的小伙伴们加入！</p> |
| <blockquote> |
| <p>关于作者 诣极，github ID zonghaishang，Apache Dubbo PMC，目前就职于蚂蚁金服中间件团队，主攻 RPC 和 Service Mesh 方向。《深入理解 Apache Dubbo 与实战》一书作者。</p> |
| </blockquote></description></item><item><title>Blog: 解构 Dubbo-go 的核心注册引擎 Nacos</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E8%A7%A3%E6%9E%84-dubbo-go-%E7%9A%84%E6%A0%B8%E5%BF%83%E6%B3%A8%E5%86%8C%E5%BC%95%E6%93%8E-nacos/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E8%A7%A3%E6%9E%84-dubbo-go-%E7%9A%84%E6%A0%B8%E5%BF%83%E6%B3%A8%E5%86%8C%E5%BC%95%E6%93%8E-nacos/</guid><description> |
| <p>近几年，随着Go语言社区逐渐发展和壮大，越来越多的公司开始尝试采用Go搭建微服务体系，也涌现了一批Go的微服务框架，如go-micro、go-kit、Dubbo-go等，跟微服务治理相关的组件也逐渐开始在Go生态发力，如Sentinel、Hystrix等都推出了Go语言版本，而作为微服务框架的核心引擎&ndash;注册中心，也是必不可缺少的组件，市面已经有多款注册中心支持Go语言，应该如何选择呢？我们可以对目前主流的支持Go语言的注册中心做个对比。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p1.png" alt="img"></p> |
| <p>根据上表的对比我们可以从以下几个维度得出结论：</p> |
| <ul> |
| <li>生态:各注册中心对Go语言都有支持，但是Nacos、 Consul、Etcd 社区活跃，zookeeper和Eureka社区活跃度较低；</li> |
| <li>易用性：Nacos、Eureka、Consul都有现成的管控平台，Etcd、zookeeper本身作为kv存储，没有相应的管控平台，Nacos支持中文界面，比较符合国人使用习惯；</li> |
| <li>场景支持：CP模型主要针对强一致场景，如金融类，AP模型适用于高可用场景，Nacos可以同时满足两种场景，Eureka主要满足高可用场景，Consul、Zookeepr、Etcd主要满足强一致场景，此外Nacos支持从其它注册中心同步数据，方便用户注册中心迁移；</li> |
| <li>功能完整性：所有注册中心都支持健康检查，Nacos、Consul支持的检查方式较多，满足不同应用场景，Zookeeper通过keep alive方式，能实时感知实例变化；Nacos、Consul和Eureka都支持负载均衡策略，Nacos通过Metadata selector支持更灵活的策略；此外，Nacos、Eureka都支持雪崩保护，避免因为过多的实例不健康对健康的实例造成雪崩效应。</li> |
| </ul> |
| <p>综合上面各维度的对比，可以了解到Nacos作为注册中心有一定的优势，那么它对Go微服务生态的集成做得如何？接下来我们首先探索下Nacos是如何与Dubbo-go集成。</p> |
| <h2 id="引言">引言</h2> |
| <p>Dubbo-go目前是Dubbo多语言生态中最火热的一个项目，从2016年发布至今，已经走过5个年头。最近，Dubbo-go发布了v1.5版本，全面兼容Dubbo 2.7.x版本，支持了应用维度的服务注册与发现，和主流的注册模型保持一致，标志着Dubbo-go向云原生迈出了关键的一步。作为驱动服务运转的核心引擎&ndash;注册中心，在切换到应用维度的注册模型后，也需要做相应的适配，本文将解析如何以Nacos为核心引擎实现应用维度的服务注册与发现，并且给出相应的实践案例。此外，本文代码基于Dubbo-go v1.5.1，Nacos-SDK-go v1.0.0和Nacos v1.3.2。</p> |
| <h2 id="服务注册与发现架构">服务注册与发现架构</h2> |
| <p>从架构中，我们可以看到，与接口级别的服务注册发现不同的是，Dubbo-go的provider启动后会调用Nacos-go-sdk的RegisterInstance接口向Nacos注册服务实例，注册的服务名即为应用名称，而不是接口名称。Conusmer启动后则会调用Subscribe接口订阅该应用的服务实例变化，并对的实例发起服务调用。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p2.png" alt="img"></p> |
| <h2 id="服务模型">服务模型</h2> |
| <p>图3是我们Dubbo-go的应用维度服务发现模型，主要有服务和实例两个层级关系，服务实例的属性主要包含实例Id、主机地址、服务端口、激活状态和元数据。图4为Nacos的服务分级存储模型，包含服务、集群和实例三个层次。两者对比，多了一个集群维度的层级，而且实例属性信息能够完全匹配。所以在Dubbo-go将应用服务实例注册到Nacos时，我们只需要将集群设置为默认集群，再填充服务和实例的相关属性，即可完成服务模型上的匹配。此外Nacos可以将服务注册到不同的Namespace下，实现多租户的隔离。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p3.png" alt="img"></p> |
| <p>! |
| <img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p4.png" alt="img"></p> |
| <h2 id="服务实例心跳维持">服务实例心跳维持</h2> |
| <p>Dubbo-go的Provider在向Nacos注册应用服务实例信息后，需要主动上报心跳，让Nacos服务端感知实例的存活与否，以判断是否将该节点从实例列表中移除。维护心跳的工作是在Nacos-SDK-go完成的，从图5代码中可以看到，当Dubbo-go调用RegisterInstance注册一个服务实例时，SDK除了调用Nacos的Register API之外，还会调用AddBeatInfo，将服务实例信息添加到本地缓存，通过后台协程定期向Nacos发送服务实例信息，保持心跳。当服务下线时，可以通过调用DeRegisterInstance执行反注册，并移除本地的心跳保持任务，Nacos实例列表中也会将该实例移除。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p5.png" alt="img"></p> |
| <h1 id="订阅服务实例变化">订阅服务实例变化</h1> |
| <p>Dubbo-go的Consumer在启动的时候会调用Nacos-SDK-go的Subscribe接口，该接口入参如图6，订阅的时候只需要传递ServiceName即应用名和回调函数SubscribeCallback，Nacos在服务实例发生变化的时候即可通过回调函数通知Dubbo-go。Nacos-SDK-go是如何感知Nacos的服务实例变化的呢？主要有两种方式：</p> |
| <ul> |
| <li> |
| <p>Nacos服务端主动推送，Nacos-SDK-go在启动的时候会监听一个UDP端口，该端口在调用Nacos Register API的时候作为参数传递，Nacos会记录Ip和端口，当服务实例发生变化时，Nacos会对所有监听该服务的Ip和端口发送UDP请求，推送变化后的服务实例信息。</p> |
| </li> |
| <li> |
| <p>Nacos-SDK-go定期查询，SDK会对订阅的服务实例定时调用查询接口，如果查询有变化则通过回调接口通知Dubbo-go。作为兜底策略保证Nacos服务端推送失败后，仍能感知到变化。</p> |
| </li> |
| </ul> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p6.png" alt="img"></p> |
| <p>此外Nacos-SDK-go还支持推空保护，当Nacos推送的实例列表为空时，不更新本地缓存，也不通知Dubbo-go变更，避免Consumer无可用实例调用，造成故障。同时，SDK还支持服务实例信息本地持久化存储，可以保证在Nacos服务故障过程中，Consumer重启也能获取到可用实例，具备容灾效果。</p> |
| <h1 id="范例实践">范例实践</h1> |
| <h2 id="环境准备">环境准备</h2> |
| <p>dubbo-go samples代码下载：https://github.com/apache/dubbo-go-samples/tree/master，基于Nacos注册中心的应用级服务发现的hello world代码目录在 registry/servicediscovery/nacos。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p7.png" alt="img"></p> |
| <p>Nacos服务端搭建，参考官方文档：https://nacos.io/zh-cn/docs/quick-start.html，或者使用官方提供的公共Nacos服务：http://console.nacos.io/nacos(账号密码:nacos，仅供测试)，或者购买阿里云服务：https://help.aliyun.com/document_detail/139460.html?spm=a2c4g.11186623.6.559.d7e264b7bLpZIs</p> |
| <h2 id="server端搭建">Server端搭建</h2> |
| <p>进入registry/servicediscovery/nacos/go-server/profiles文件，可以看到有dev、release和test三个文件夹，分别对应开发、测试和生产配置。我们使用dev配置来搭建开发环境，dev文件下有log.yml和server.yml文件，下面对server.yml配置进行修改。</p> |
| <p>remote配置，这里使用公共的Nacos服务，address支持配置多个地址，用逗号分割。params参数配置nacos-sdk的日志目录。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-Yaml" data-lang="Yaml"><span style="display:flex;"><span><span style="color:#268bd2">remote</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">nacos</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;console.nacos.io:80&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">timeout</span>: <span style="color:#2aa198">&#34;5s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">params</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">logDir</span>: <span style="color:#2aa198">&#34;/data/nacos-sdk/log&#34;</span> |
| </span></span><span style="display:flex;"><span>configCenter配置 |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">config_center</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;nacos&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;console.nacos.io:80&#34;</span> |
| </span></span></code></pre></div><p>配置server端环境变量</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-Bash" data-lang="Bash"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">CONF_PROVIDER_FILE_PATH</span><span style="color:#719e07">=</span>server端的server.yml文件路径 |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">APP_LOG_CONF_FILE</span><span style="color:#719e07">=</span>server端的log.yml文件路径 |
| </span></span></code></pre></div><p>进入registry/servicediscovery/nacos/go-server/app，运行server.go的main方法，可以从Nacos的控制台（http://console.nacos.io/nacos/#/serviceManagement?dataId=&amp;group=&amp;appName=&amp;namespace=）</p> |
| <p>看到，应用user-info-server已经注册成功。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p8.png" alt="img"></p> |
| <h2 id="client端搭建">Client端搭建</h2> |
| <p>client的配置文件在registry/servicediscovery/nacos/go-server/profiles目录下，需要修改的地方跟server端一样，这里不赘述。</p> |
| <p>配置client端环境变量</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-Bash" data-lang="Bash"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">CONF_CONSUMER_FILE_PATH</span><span style="color:#719e07">=</span>client端的server.yml文件路径 |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">APP_LOG_CONF_FILE</span><span style="color:#719e07">=</span>client端的log.yml文件路径 |
| </span></span></code></pre></div><p>进入registry/servicediscovery/nacos/go-client/app，运行client.go的main方法，看到如下日志输出，表示调用server端成功。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/nacos/p9.png" alt="img"></p> |
| <blockquote> |
| <p>作者：李志鹏 Github账号：Lzp0412，Nacos-SDK-go作者，Apache/Dubbo-go Contributor。现就职于阿里云云原生应用平台，主要参与服务发现、CoreDNS、ServiceMesh相关工作，负责推动Nacos Go微服务生态建设。</p> |
| </blockquote></description></item><item><title>Blog: 快速上手 dubbo-go</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B-dubbo-go/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B-dubbo-go/</guid><description> |
| <h2 id="前言">前言</h2> |
| <p>每次技术调研总会发现自己学不动了怎么办？用已有的知识来拓展要学习的新知识就好了~ by LinkinStar</p> |
| <p>最近需要调研使用 dubbo，之前完全是 0 基础，对于 dubbo 只存在于听说，今天上手实战一把，告诉你如何快速用 go 上手 dubbo</p> |
| <p>PS：以下的学习方式适用于很多新技术</p> |
| <h2 id="基本概念">基本概念</h2> |
| <p><strong>首先学习一个技术首先要看看它的整体架构和基本概念</strong>，每个技术都有着自己的名词解释和实现方式，如果文档齐全就简单很多。</p> |
| <p><a href="https://dubbo.apache.org/zh-cn/docs/languages/golang/dubbo-go-3.0/preface/architecture/">基本概念</a></p> |
| <p>大致浏览了背景、需求、架构之后基本上有一个大致概念</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-architecture.png" alt="img"></p> |
| <p>其实整体架构和很多微服务的架构都是类似的，就是有一个注册中心管理所有的服务列表，服务提供方先向注册中心注册，而消费方向注册中心请求服务列表，通过服务列表调用最终的服务。总的来说 dubbo 将整个过程封装在了里面，而作为使用者的我们来说更加关心业务实现，它帮我们做好了治理的工作。</p> |
| <p>然后我抓住了几个我想要知道的重点：</p> |
| <ul> |
| <li>注册中心可替换，官方推荐的是 zk</li> |
| <li>如果有变更，注册中心将基于长连接推送变更数据给消费者，注册中心，服务提供者，服务消费者三者之间均为长连接</li> |
| <li>基于软负载均衡算法，选一台提供者进行调用，如果调用失败，再选另一台调用</li> |
| <li>消费者在本地缓存了提供者列表</li> |
| </ul> |
| <h2 id="实际上手">实际上手</h2> |
| <p>官网文档中也给出如果使用 golang 开发，那么有 <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fapache%2Fdubbo-go">https://github.com/apache/dubbo-go</a> 可以用，那么废话不多说，上手实践一把先。因为你看再多，都比不上实践一把来学的快。</p> |
| <h3 id="环境搭建">环境搭建</h3> |
| <p>大多数教程就会跟你说，一个 helloWorld 需要 zookeeper 环境，但是不告诉你如何搭建，因为这对于他们来说太简单了，而我不一样，我是 0 基础，那如何快速搭建一个需要调研项目的环境呢？最好的方式就是 docker。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">version</span>: <span style="color:#2aa198">&#39;3&#39;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">services</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">zookeeper</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">image</span>: zookeeper |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">ports</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#2aa198">2181</span>:<span style="color:#2aa198">2181</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">admin</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">image</span>: apache/dubbo-admin |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">depends_on</span>: |
| </span></span><span style="display:flex;"><span> - zookeeper |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">ports</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#2aa198">8080</span>:<span style="color:#2aa198">8080</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">environment</span>: |
| </span></span><span style="display:flex;"><span> - admin.registry.address=zookeeper://zookeeper:2181 |
| </span></span><span style="display:flex;"><span> - admin.config-center=zookeeper://zookeeper:2181 |
| </span></span><span style="display:flex;"><span> - admin.metadata-report.address=zookeeper://zookeeper:2181 |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">version</span>: <span style="color:#2aa198">&#39;3&#39;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">services</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">zookeeper</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">image</span>: zookeeper |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">ports</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#2aa198">2181</span>:<span style="color:#2aa198">2181</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">admin</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">image</span>: chenchuxin/dubbo-admin |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">depends_on</span>: |
| </span></span><span style="display:flex;"><span> - zookeeper |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">ports</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#2aa198">8080</span>:<span style="color:#2aa198">8080</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">environment</span>: |
| </span></span><span style="display:flex;"><span> - dubbo.registry.address=zookeeper://zookeeper:2181 |
| </span></span><span style="display:flex;"><span> - dubbo.admin.root.password=root |
| </span></span><span style="display:flex;"><span> - dubbo.admin.guest.password=guest |
| </span></span></code></pre></div><p>上面两个 docker-compose 文件一个是官方提供的管理工具，一个包含的是个人修改之后的管理工具，记住这里有个用户名密码是 root-root，看你喜欢</p> |
| <p>废话不多说，直接创建 docker-compose.yaml 然后 <code>docker-compose up</code> 你就得到了一个环境，棒????</p> |
| <h3 id="样例下载">样例下载</h3> |
| <p>有的技术会给出官方的实验样例，dubbo-go 也不例外</p> |
| <p><a href="https://github.com/apache/dubbo-go-samples">https://github.com/apache/dubbo-go-samples</a></p> |
| <p>里面包含了 go 和 java 的样例，有了它你就能快速上手了，把它下载到本地</p> |
| <h3 id="样例运行">样例运行</h3> |
| <p>首先要做的第一步就是把 helloworld 给跑起来，进入 golang 目录，里面有个 README.md 看一下。然后开搞。</p> |
| <p>打开一个终端运行服务方</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">ARCH</span><span style="color:#719e07">=</span>mac |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">ENV</span><span style="color:#719e07">=</span>dev |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">cd</span> helloworld/dubbo/go-server |
| </span></span><span style="display:flex;"><span>sh ./assembly/<span style="color:#268bd2">$ARCH</span>/<span style="color:#268bd2">$ENV</span>.sh |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">cd</span> ./target/linux/user_info_server-0.3.1-20190517-0930-release |
| </span></span><span style="display:flex;"><span>sh ./bin/load.sh start |
| </span></span></code></pre></div><p>打开另一个终端运行客户端</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">ARCH</span><span style="color:#719e07">=</span>mac |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">ENV</span><span style="color:#719e07">=</span>dev |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">cd</span> helloworld/dubbo/go-client |
| </span></span><span style="display:flex;"><span>sh ./assembly/<span style="color:#268bd2">$ARCH</span>/<span style="color:#268bd2">$ENV</span>.sh |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">cd</span> ./target/linux/user_info_client-0.3.1-20190517-0921-release |
| </span></span><span style="display:flex;"><span>sh ./bin/load_user_info_client.sh start |
| </span></span></code></pre></div><p>启动过程中会出现一些警告，问题不大，如果成功，那么客户端会有一个调用服务端的请求，并在控制台中以白色底色进行打印</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/quickstart/console.png" alt="img"></p> |
| <p>java 的服务也有相对应的启动方式，按照 README 中所说明的也可以进行注册和调用，并且 java 和 go 之间是可以互相调用的</p> |
| <h3 id="查看服务">查看服务</h3> |
| <p>因为我们部署的时候有一个 dubbo-admin 用于管理 zk 上注册的服务，我们可以访问本地的 8080 端口看到对应的服务情况</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/quickstart/admin.png" alt="img"></p> |
| <p>image.png</p> |
| <p>至此你应该已经对于整体的链路调用有一个大致的认识，结合之前官网文档的中的架构图，应该也清晰了。</p> |
| <h2 id="如何使用">如何使用</h2> |
| <p>那么现在你已经运行起来了，那么势必就要来看看具体是如何进行使用的了。</p> |
| <h3 id="服务端">服务端</h3> |
| <p>服务端，也就是服务提供者；</p> |
| <p>位置在：<code>dubbo-samples/golang/helloworld/dubbo/go-server/app</code></p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// 将服务进行注册 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>config.<span style="color:#268bd2">SetProviderService</span>(<span style="color:#b58900">new</span>(UserProvider)) |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// 注册对象的hessian序列化 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span></code></pre></div><p>是不是看起来其实很简单，其实重点就是上面两句代码了</p> |
| <p>对于服务本身来说</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> UserProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u <span style="color:#719e07">*</span>UserProvider) <span style="color:#268bd2">GetUser</span>(ctx context.Context, req []<span style="color:#268bd2">interface</span>{}) (<span style="color:#719e07">*</span>User, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;req:%#v&#34;</span>, req) |
| </span></span><span style="display:flex;"><span> rsp <span style="color:#719e07">:=</span> User{<span style="color:#2aa198">&#34;A001&#34;</span>, <span style="color:#2aa198">&#34;Alex Stocks&#34;</span>, <span style="color:#2aa198">18</span>, time.<span style="color:#268bd2">Now</span>()} |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;rsp:%#v&#34;</span>, rsp) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>rsp, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u <span style="color:#719e07">*</span>UserProvider) <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>其实就是需要实现下面那个接口就可以了</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// rpc service interface |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> RPCService <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> <span style="color:#586e75">// rpc service id or reference id |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>} |
| </span></span></code></pre></div><p>其中有一步骤不要忘记了是config.Load()，会加载配置文件的相关配置，不然你以为注册中心的地址等等是在哪里配置的呢？</p> |
| <h3 id="客户端">客户端</h3> |
| <p>看了服务端，其实客户端也就很简单了</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span>config.<span style="color:#268bd2">SetConsumerService</span>(userProvider) |
| </span></span><span style="display:flex;"><span>hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span></code></pre></div><p>其实也是差不多的，也需要注册一个消费服务，并将序列化方式给注册上去</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span>user <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>User{} |
| </span></span><span style="display:flex;"><span>err <span style="color:#719e07">:=</span> userProvider.<span style="color:#268bd2">GetUser</span>(context.<span style="color:#268bd2">TODO</span>(), []<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;A001&#34;</span>}, user) |
| </span></span></code></pre></div><p>使用也就很简单了，同样的也需要实现 Reference 接口</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> UserProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> GetUser <span style="color:#268bd2">func</span>(ctx context.Context, req []<span style="color:#268bd2">interface</span>{}, rsp <span style="color:#719e07">*</span>User) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u <span style="color:#719e07">*</span>UserProvider) <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="回头想想">回头想想</h3> |
| <p>当我们看完了代码的上的使用，再回头想想 dubbo 的作用，你就会发现其实 dubbo 帮你做的事情真的屏蔽了很多。</p> |
| <ul> |
| <li>你不需要关心服务是怎么注册的</li> |
| <li>你不需要关心服务是怎么获取的</li> |
| <li>你不需要关系服务是怎么调用的</li> |
| <li>甚至序列化的过程都是基本透明的</li> |
| </ul> |
| <p>想对比来说，如果让你自己去是实现微服务，那是不是说，你需要实现服务的拉取，服务的负载均衡，服务的发现，序列化&hellip;..</p> |
| <p>这下你应该明白了 dubbo 是做什么的也就明白了它为什么会出现了</p> |
| <h2 id="其他能力">其他能力</h2> |
| <p>当你学习了一个技术的基本使用之后，要学习其他的能力，以便在使用的过程中不会出现自己重复造轮子或者说有轮子没用到的情况，<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fapache%2Fdubbo-samples">https://github.com/apache/dubbo-samples</a> 不止有 helloworld 还要很多别的案例供你参考，你可以继续看看并进行试验。</p> |
| <h3 id="支持扩展">支持扩展</h3> |
| <p><a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fapache%2Fdubbo-go">https://github.com/apache/dubbo-go</a></p> |
| <p>在 Feature list 中列举了 dubbo-go 所支持的相关功能，比如序列化，比如注册中心，在比如过滤器。</p> |
| <p>也就是说，在使用 dubbo-go 的时候相关功能都是插件化的，想用什么就看你自己了，比如注册中心我想用 etcd，比如调用的协议我想换成 grpc 都可以。</p> |
| <h2 id="相关问题">相关问题</h2> |
| <p><strong>对于一个技术调研，那么肯定会有相关问题等着你去发现去考虑。</strong></p> |
| <p>下面是我能想到的一些问题：</p> |
| <ul> |
| <li>当前 dubbo-go 的版本最高在 1.4，所对应的 dubbo 版本应该是 2.6.x，如果调用更高版本的服务是否会有问题</li> |
| <li>java 和 go 之间互相调用，各种类型转换之间是否存在问题，是否容易出现无法正确反序列化的问题</li> |
| <li>&hellip;.</li> |
| </ul> |
| <h2 id="后续学习">后续学习</h2> |
| <p>那么上面只是说能让你上手 dubbo-go，但是实际使用可能存在距离。为什么这么说呢？如果你在不动里面任何的原理情况下，那么如果遇到问题，你很可能就束手无策了。比如如果线上服务无法正常发现，你应该如何排查？调用过程中出现问题如何定位？</p> |
| <p>所以后续你需要做的是：</p> |
| <ul> |
| <li>看看整体设计架构和思路，明白整条链路调用过程和规则</li> |
| <li>学习它的接口设计，为什么别人设计的接口能兼容那么多的注册中心？如果让你来设计你怎么设计呢？</li> |
| <li>性能也很重要，网上说性能很不错，为什么呢？什么地方做了优化，负载均衡的算法是怎么样的，你能自定义吗？</li> |
| </ul> |
| <h2 id="总结">总结</h2> |
| <p>总的来说，对于 dubbo-go 整体的使用上还是非常好上手的，自己想了一下，如果当前项目想要接入的话，主要是服务的暴露、序列化方式、鉴权调整等存在开发工作。</p> |
| <p>上面砖头也抛的差不多了，对于你快速上手应该没有问题了，剩下的就要靠你自己了</p></description></item><item><title>Blog: 涂鸦智能 dubbo-go 亿级流量的实践与探索</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E6%B6%82%E9%B8%A6%E6%99%BA%E8%83%BD-dubbo-go-%E4%BA%BF%E7%BA%A7%E6%B5%81%E9%87%8F%E7%9A%84%E5%AE%9E%E8%B7%B5%E4%B8%8E%E6%8E%A2%E7%B4%A2/</link><pubDate>Thu, 14 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/14/%E6%B6%82%E9%B8%A6%E6%99%BA%E8%83%BD-dubbo-go-%E4%BA%BF%E7%BA%A7%E6%B5%81%E9%87%8F%E7%9A%84%E5%AE%9E%E8%B7%B5%E4%B8%8E%E6%8E%A2%E7%B4%A2/</guid><description> |
| <p>dubbo 是一个基于 Java 开发的高性能的轻量级 RPC 框架，dubbo 提供了丰富的服务治理功能和优秀的扩展能力。而 dubbo-go 在 java 与 golang 之间提供统一的服务化能力与标准，是涂鸦智能目前最需要解决的主要问题。本文分为实践和快速接入两部分，分享在涂鸦智能的 <a href="http://github.com/apache/dubbo-go">dubbo-go</a> 实战经验，意在帮助用户快速接入 dubbo-go RPC 框架，希望能让大家少走些弯路。</p> |
| <p>另外，文中的测试代码基于 dubbo-go版本 <a href="https://github.com/apache/dubbo-go/releases/tag/v1.4.0">v1.4.0</a>。</p> |
| <h2 id="dubbo-go-网关实践">dubbo-go 网关实践</h2> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/tuya/p1.png" alt="img"></p> |
| <p>dubbo-go 在涂鸦智能的使用情况如上图，接下来会为大家详细介绍落地细节，希望这些在生产环境中总结的经验能够帮助到大家。</p> |
| <h3 id="背景">背景</h3> |
| <p>在涂鸦智能，dubbo-go 已经作为了 golang 服务与原有 dubbo 集群打通的首选 RPC 框架。其中比较有代表性的 open-gateway 网关系统（下文统一称 gateway，开源版本见 <a href="https://github.com/dubbogo/dubbo-go-proxy">https://github.com/dubbogo/dubbo-go-proxy</a>）。该 gateway 动态加载内部 dubbo 接口信息，以HTTP API 的形式对外暴露。该网关意在解决上一代网关的以下痛点。</p> |
| <ul> |
| <li><code>通过页面配置 dubbo 接口开放规则，步骤繁琐，权限难以把控。</code></li> |
| <li><code>接口非 RESTful 风格，对外部开发者不友好。</code></li> |
| <li><code>依赖繁重，升级风险大。</code></li> |
| <li><code>并发性能问题。</code></li> |
| </ul> |
| <h3 id="架构设计">架构设计</h3> |
| <p>针对如上痛点，随即着手准备设计新的 gateway 架构。首先就是语言选型，golang 的协程调用模型使得 golang 非常适合构建 IO 密集型的应用，且应用部署上也较 java 简单。经过调研后我们敲定使用 golang 作为 proxy 的编码语言，并使用 dubbo-go 用于连接 dubbo provider 集群。provider 端的业务应用通过使用 java 的插件，以注解形式配置 API 配置信息，该插件会将配置信息和 dubbo 接口元数据更新到元数据注册中心（下图中的 redis ）。这样一来，配置从管理后台页面转移到了程序代码中。开发人员在编码时，非常方便地看到 dubbo 接口对外的 API 描述，无需从另外一个管理后台配置 API 的使用方式。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/tuya/p2.png" alt="img"></p> |
| <h3 id="实践">实践</h3> |
| <p>从上图可以看到，网关能动态加载 dubbo 接口信息，调用 dubbo 接口是基于 dubbo 泛化调用。泛化调用使 client 不需要构建 provider 的 interface 代码，在 dubbo-go 中表现为无需调用 config.SetConsumerService 和 hessian.RegisterPOJO 方法，而是将请求模型纯参数完成，这使得 client 动态新增、修改接口成为可能。在 <a href="https://github.com/apache/dubbo-go-samples/tree/master/generic/default/go-client">apache/dubbo-sample/golang/generic/default/go-client</a> 中的有泛化调用的演示代码。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">test</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> appName = <span style="color:#2aa198">&#34;UserProviderGer&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> referenceConfig = config.ReferenceConfig{ |
| </span></span><span style="display:flex;"><span> InterfaceName: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span>, |
| </span></span><span style="display:flex;"><span> Cluster: <span style="color:#2aa198">&#34;failover&#34;</span>, |
| </span></span><span style="display:flex;"><span> Registry: <span style="color:#2aa198">&#34;hangzhouzk&#34;</span>, |
| </span></span><span style="display:flex;"><span> Protocol: dubbo.DUBBO, |
| </span></span><span style="display:flex;"><span> Generic: <span style="color:#cb4b16">true</span>, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> referenceConfig.<span style="color:#268bd2">GenericLoad</span>(appName) <span style="color:#586e75">// appName is the unique identification of RPCService |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> |
| </span></span><span style="display:flex;"><span> time.<span style="color:#268bd2">Sleep</span>(<span style="color:#2aa198">3</span> <span style="color:#719e07">*</span> time.Second) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> resp, err <span style="color:#719e07">:=</span> referenceConfig.<span style="color:#268bd2">GetRPCService</span>().(<span style="color:#719e07">*</span>config.GenericService). |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Invoke</span>([]<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;GetUser&#34;</span>, []<span style="color:#dc322f">string</span>{<span style="color:#2aa198">&#34;java.lang.String&#34;</span>}, []<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;A003&#34;</span>}}) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>泛化调用的实现其实相当简单。其功能作用在 dubbo 的 Filter 层中。<a href="https://github.com/apache/dubbo-go/blob/master/filter/generic/filter.go">Generic Filter</a> 已经作为默认开启的 Filter 加入到 dubbo Filter 链中。其核心逻辑如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (ef <span style="color:#719e07">*</span>GenericFilter) <span style="color:#268bd2">Invoke</span>(ctx context.Context, invoker protocol.Invoker, invocation protocol.Invocation) protocol.Result { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> invocation.<span style="color:#268bd2">MethodName</span>() <span style="color:#719e07">==</span> constant.GENERIC <span style="color:#719e07">&amp;&amp;</span> <span style="color:#b58900">len</span>(invocation.<span style="color:#268bd2">Arguments</span>()) <span style="color:#719e07">==</span> <span style="color:#2aa198">3</span> { |
| </span></span><span style="display:flex;"><span> oldArguments <span style="color:#719e07">:=</span> invocation.<span style="color:#268bd2">Arguments</span>() |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> oldParams, ok <span style="color:#719e07">:=</span> oldArguments[<span style="color:#2aa198">2</span>].([]<span style="color:#268bd2">interface</span>{}); ok { |
| </span></span><span style="display:flex;"><span> newParams <span style="color:#719e07">:=</span> <span style="color:#b58900">make</span>([]hessian.Object, <span style="color:#2aa198">0</span>, <span style="color:#b58900">len</span>(oldParams)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> oldParams { |
| </span></span><span style="display:flex;"><span> newParams = <span style="color:#b58900">append</span>(newParams, hessian.<span style="color:#268bd2">Object</span>(<span style="color:#268bd2">struct2MapAll</span>(oldParams[i]))) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> newArguments <span style="color:#719e07">:=</span> []<span style="color:#268bd2">interface</span>{}{ |
| </span></span><span style="display:flex;"><span> oldArguments[<span style="color:#2aa198">0</span>], |
| </span></span><span style="display:flex;"><span> oldArguments[<span style="color:#2aa198">1</span>], |
| </span></span><span style="display:flex;"><span> newParams, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> newInvocation <span style="color:#719e07">:=</span> invocation2.<span style="color:#268bd2">NewRPCInvocation</span>(invocation.<span style="color:#268bd2">MethodName</span>(), newArguments, invocation.<span style="color:#268bd2">Attachments</span>()) |
| </span></span><span style="display:flex;"><span> newInvocation.<span style="color:#268bd2">SetReply</span>(invocation.<span style="color:#268bd2">Reply</span>()) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> invoker.<span style="color:#268bd2">Invoke</span>(ctx, newInvocation) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> invoker.<span style="color:#268bd2">Invoke</span>(ctx, invocation) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>Generic Filter 将用户请求的结构体参数转化为统一格式的 map（代码中的 struct2MapAll ），将类（ golang 中为 struct ）的正反序列化操作变成 map 的正反序列化操作。这使得无需 POJO 描述通过硬编码注入 hessain 库。</p> |
| <p>从上面代码可以看到，泛化调用实际需要动态构建的内容有 4 个，ReferenceConfig 中需要的 InterfaceName 、参数中的 method 、ParameterTypes、实际入参 requestParams。</p> |
| <p><strong>那么这些参数是如何从 HTTP API 匹配获取到的呢？</strong></p> |
| <p>这里就会用到上文提到的 provider 用于收集元数据的插件。引入插件后，应用在启动时会扫描需要暴露的 dubbo 接口，将 dubbo 元数据和 HTTP API 关联。插件使用方法大致如下，这里调了几个简单的配置作为示例，实际生产时注解内容会更多。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/tuya/p3.png" alt="img"></p> |
| <p>最终获得的 dubbo 元数据如下:</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-json" data-lang="json"><span style="display:flex;"><span>{ |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;key&#34;</span>: <span style="color:#2aa198">&#34;POST:/hello/{uid}/add&#34;</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;interfaceName&#34;</span>: <span style="color:#2aa198">&#34;com.tuya.hello.service.template.IUserServer&#34;</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;methodName&#34;</span>: <span style="color:#2aa198">&#34;addUser&#34;</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;parameterTypes&#34;</span>: [<span style="color:#2aa198">&#34;com.tuya.gateway.Context&#34;</span>, <span style="color:#2aa198">&#34;java.lang.String&#34;</span>, <span style="color:#2aa198">&#34;com.tuya.hello.User&#34;</span>], |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;parameterNames&#34;</span>: [<span style="color:#2aa198">&#34;context&#34;</span>, <span style="color:#2aa198">&#34;uid&#34;</span>, <span style="color:#2aa198">&#34;userInfo&#34;</span>], |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;updateTimestamp&#34;</span>: <span style="color:#2aa198">&#34;1234567890&#34;</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;permissionDO&#34;</span>:{}, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;voMap&#34;</span>: { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;userInfo&#34;</span>: { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;name&#34;</span>: <span style="color:#2aa198">&#34;java.lang.String&#34;</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;sex&#34;</span>: <span style="color:#2aa198">&#34;java.lang.String&#34;</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;age&#34;</span>: <span style="color:#2aa198">&#34;java.lang.Integer&#34;</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> }, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;parameterNameHumpToLine&#34;</span>: <span style="color:#cb4b16">true</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;resultFiledHumpToLine&#34;</span>: <span style="color:#cb4b16">false</span>, |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">&#34;protocolName&#34;</span>: <span style="color:#2aa198">&#34;dubbo&#34;</span>, |
| </span></span><span style="display:flex;"><span> ....... |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>Gateway 从元数据配置中心订阅到以上信息，就能把一个 API 请求匹配到一个 dubbo 接口。再从 API 请求中抓取参数作为入参。这样功能就完成了流量闭环。</p> |
| <p>以上内容，大家应该对此 gateway 的项目拓扑结构有了清晰的认知。我接着分享项目在使用 dubbo-go 过程中遇到的问题和调优经验。19 年初，当时的 dubbo-go 项目还只是构建初期，没有什么用户落地的经验。我也是一边参与社区开发，一边编码公司内部网关项目。在解决了一堆 hessain 序列化和 zookeeper 注册中心的问题后，项目最终跑通了闭环。但是，作为一个核心应用，跑通闭环离上生产环境还有很长的路要走，特别是使用了当时稳定性待测试的新框架。整个测试加上功能补全，整整花费了一个季度的时间，直到项目趋于稳定，压测效果也良好。<strong>单台网关机器（ 2C 8G ）全链路模拟真实环境压测达到 2000 QPS。由于引入了比较重的业务逻辑（单个请求平均调用 3 个 dubbo 接口），对于这个压测结果，是符合甚至超出预期的。</strong></p> |
| <p>总结了一些 dubbo-go 参数配置调优的经验，主要是一些网络相关配置。大家在跑 demo 时，应该会看到配置文件最后有一堆配置，但如果对 dubbo-go 底层网络模型不熟悉，就很难理解这些配置的含义。目前 dubbo-go 网络层以 <a href="https://github.com/AlexStocks/getty">getty</a> 为底层框架，实现读写分离和协程池管理。getty 对外暴露 session 的概念，session 提供一系列网络层方法注入的实现，因为本文不是源码解析文档，在这里不过多论述。**读者可以简单的认为 dubbo-go 维护了一个 getty session池，session 又维护了一个 TCP 连接池。对于每个连接，getty 会有读协程和写协程伴生，做到读写分离。**这里我尽量用通俗的注释帮大家梳理下对性能影响较大的几个配置含义：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">protocol_conf</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 这里是协议独立的配置，在dubbo协议下，大多数配置即为getty session相关的配置。</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">dubbo</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 一个session会始终保证connection_number个tcp连接个数，默认是16，</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 但这里建议大家配置相对小的值，一般系统不需要如此多的连接个数。</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 每隔reconnect_interval时间，检查连接个数，如果小于connection_number，</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 就建立连接。填0或不填都为默认值300ms</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">reconnect_interval</span>: <span style="color:#2aa198">0</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">connection_number</span>: <span style="color:#2aa198">2</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 客户端发送心跳的间隔</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">heartbeat_period</span>: <span style="color:#2aa198">&#34;30s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># OnCron时session的超时时间，超过session_timeout无返回就关闭session</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">session_timeout</span>: <span style="color:#2aa198">&#34;30s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 每一个dubbo interface的客户端，会维护一个最大值为pool_size大小的session池。</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 每次请求从session池中select一个。所以真实的tcp数量是session数量*connection_number，</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 而pool_size是session数量的最大值。测试总结下来一般程序4个tcp连接足以。</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">pool_size</span>: <span style="color:#2aa198">4</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># session保活超时时间，也就是超过session_timeout时间没有使用该session，就会关闭该session</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">pool_ttl</span>: <span style="color:#2aa198">600</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 处理返回值的协程池大小</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">gr_pool_size</span>: <span style="color:#2aa198">1200</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># 读数据和协程池中的缓冲队列长度，目前已经废弃。不使用缓冲队列</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">queue_len</span>: <span style="color:#2aa198">64</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">queue_number</span>: <span style="color:#2aa198">60</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">getty_session_param</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">compress_encoding</span>: <span style="color:#cb4b16">false</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">tcp_no_delay</span>: <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">tcp_keep_alive</span>: <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">keep_alive_period</span>: <span style="color:#2aa198">&#34;120s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">tcp_r_buf_size</span>: <span style="color:#2aa198">262144</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">tcp_w_buf_size</span>: <span style="color:#2aa198">65536</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">pkg_wq_size</span>: <span style="color:#2aa198">512</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">tcp_read_timeout</span>: <span style="color:#2aa198">&#34;1s&#34;</span> <span style="color:#586e75"># 每次读包的超时时间</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">tcp_write_timeout</span>: <span style="color:#2aa198">&#34;5s&#34;</span> <span style="color:#586e75"># 每次写包的超时时间</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">wait_timeout</span>: <span style="color:#2aa198">&#34;1s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">max_msg_len</span>: <span style="color:#2aa198">102400</span> <span style="color:#586e75"># 最大数据传输长度</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">session_name</span>: <span style="color:#2aa198">&#34;client&#34;</span> |
| </span></span></code></pre></div><h2 id="dubbo-go-快速接入">dubbo-go 快速接入</h2> |
| <p>前文已经展示过 dubbo-go 在涂鸦智能的实践成果，接下来介绍快速接入 dubbo-go 的方式。</p> |
| <h3 id="第一步hello-world">第一步：hello world</h3> |
| <p>dubbo-go 使用范例目前和 dubbo 一致，放置在 <a href="https://github.com/apache/dubbo-samples">apache/dubbo-samples</a> 项目中。在 dubbo-sample/golang 目录下，用户可以选择自己感兴趣的 feature 目录，快速测试代码效果。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>tree dubbo-samples/golang -L <span style="color:#2aa198">1</span> |
| </span></span><span style="display:flex;"><span>dubbo-samples/golang |
| </span></span><span style="display:flex;"><span>├── README.md |
| </span></span><span style="display:flex;"><span>├── async |
| </span></span><span style="display:flex;"><span>├── ci.sh |
| </span></span><span style="display:flex;"><span>├── configcenter |
| </span></span><span style="display:flex;"><span>├── direct |
| </span></span><span style="display:flex;"><span>├── filter |
| </span></span><span style="display:flex;"><span>├── general |
| </span></span><span style="display:flex;"><span>├── generic |
| </span></span><span style="display:flex;"><span>├── go.mod |
| </span></span><span style="display:flex;"><span>├── go.sum |
| </span></span><span style="display:flex;"><span>├── helloworld |
| </span></span><span style="display:flex;"><span>├── multi_registry |
| </span></span><span style="display:flex;"><span>└── registry |
| </span></span></code></pre></div><p>我们以 hello world 为例，按照 dubbo-samples/golang/README.md 中的步骤，分别启动 server 和 client 。可以尝试 golang 调用 java 、 java 调用 golang 、golang 调用 golang 、java 调用 java。dubbo-go 在协议上支持和 dubbo 互通。</p> |
| <p>我们以启动 go-server 为例，注册中心默认使用 zookeeper 。首先确认本地的 zookeeper 是否运行正常。然后执行以下命令，紧接着你就可以看到你的服务正常启动的日志了。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">ARCH</span><span style="color:#719e07">=</span>mac |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">ENV</span><span style="color:#719e07">=</span>dev |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">cd</span> dubbo-samples/golang/helloworld/dubbo/go-server |
| </span></span><span style="display:flex;"><span>sh ./assembly/<span style="color:#268bd2">$ARCH</span>/<span style="color:#268bd2">$ENV</span>.sh |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">cd</span> ./target/darwin/user_info_server-2.6.0-20200608-1056-dev/ |
| </span></span><span style="display:flex;"><span>sh ./bin/load.sh start |
| </span></span></code></pre></div><h3 id="第二步在项目中使用-dubbo-go">第二步：在项目中使用 dubbo-go</h3> |
| <p>上面，我们通过社区维护的测试代码和启动脚本将用例跑了起来。接下来，我们需要在自己的代码中嵌入 dubbo-go 框架。很多朋友往往是在这一步遇到问题，这里我整理的一些常见问题，希望能帮到大家。</p> |
| <h5 id="1-环境变量">1. 环境变量</h5> |
| <p>目前 dubbo-go 有 3 个环境变量需要配置。</p> |
| <ul> |
| <li><code>CONF_CONSUMER_FILE_PATH</code> : Consumer 端配置文件路径，使用 consumer 时必需。</li> |
| <li><code>CONF_PROVIDER_FILE_PATH</code>：Provider 端配置文件路径，使用 provider 时必需。</li> |
| <li><code>APP_LOG_CONF_FILE</code> ：Log 日志文件路径，必需。</li> |
| <li><code>CONF_ROUTER_FILE_PATH</code>：File Router 规则配置文件路径，使用 File Router 时需要。</li> |
| </ul> |
| <h5 id="2-代码注意点">2. 代码注意点</h5> |
| <ul> |
| <li> |
| <p>注入服务: 检查是否执行以下代码</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span># 客户端 |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetConsumerService</span>(userProvider) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span># 服务端 |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetProviderService</span>(<span style="color:#b58900">new</span>(UserProvider)) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div></li> |
| <li> |
| <p>注入序列化描述：检查是否执行以下代码</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterJavaEnum</span>(<span style="color:#268bd2">Gender</span>(MAN)) |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterJavaEnum</span>(<span style="color:#268bd2">Gender</span>(WOMAN)) |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span></code></pre></div></li> |
| </ul> |
| <h5 id="3-正确理解配置文件">3. 正确理解配置文件</h5> |
| <ul> |
| <li> |
| <p>references/services 下的 key ，如下面例子的 &ldquo;UserProvider&rdquo; 需要和服务 Reference() 返回值保持一致，此为标识改接口的 key。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">references</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;UserProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;hangzhouzk&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol </span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface </span>: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods </span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;GetUser&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">3</span> |
| </span></span></code></pre></div></li> |
| <li> |
| <p>注册中心如果只有一个注册中心集群，只需配置一个。多个 IP 用逗号隔开，如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">registries </span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;hangzhouzk&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">timeout </span>: <span style="color:#2aa198">&#34;3s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;172.16.120.181:2181,172.16.120.182:2181&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">username</span>: <span style="color:#2aa198">&#34;&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">password</span>: <span style="color:#2aa198">&#34;&#34;</span> |
| </span></span></code></pre></div></li> |
| </ul> |
| <h5 id="4-java-和-go-的问题">4. java 和 go 的问题</h5> |
| <ul> |
| <li><code>go 和 java 交互的大小写</code> ：golang 为了适配 java 的驼峰格式，在调用 java 服务时，会自动将 method 和属性首字母变成小写。很多同学故意将 java 代码写成适配 golang 的参数定义，将首字母大写，最后反而无法序列化匹配。</li> |
| </ul> |
| <h3 id="第三步拓展功能">第三步：拓展功能</h3> |
| <p>dubbo-go 和 dubbo 都提供了非常丰富的拓展机制。可以实现自定义模块代替 dubbo-go 默认模块，或者新增某些功能。比如实现 Cluster、Filter 、Router 等来适配业务的需求。这些注入方法暴露在 dubbo-go/common/extension 中，允许用户调用及配置。</p> |
| <h2 id="欢迎加入-dubbo-go-社区">欢迎加入 dubbo-go 社区</h2> |
| <p>有任何 dubbo-go 相关的问题，可以加我们的钉钉群 23331795 询问探讨，我们一定第一时间给出反馈。</p> |
| <blockquote> |
| <p>本文作者 潘天颖，Github ID @pantianying，开源爱好者，就职于涂鸦智能。</p> |
| </blockquote></description></item><item><title>Blog: dubbo-go 中如何实现路由规则功能</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/12/dubbo-go-%E4%B8%AD%E5%A6%82%E4%BD%95%E5%AE%9E%E7%8E%B0%E8%B7%AF%E7%94%B1%E8%A7%84%E5%88%99%E5%8A%9F%E8%83%BD/</link><pubDate>Tue, 12 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/12/dubbo-go-%E4%B8%AD%E5%A6%82%E4%BD%95%E5%AE%9E%E7%8E%B0%E8%B7%AF%E7%94%B1%E8%A7%84%E5%88%99%E5%8A%9F%E8%83%BD/</guid><description> |
| <h2 id="lets-go">Let‘s Go!</h2> |
| <p>最近在 Apache/dubbo-go（以下简称 dubbo-go ）社区中，路由规则突然成了呼声最高的功能之一。那到底为什么需要路由规则？</p> |
| <p>先路由规则需要实现的功能：</p> |
| <p>路由规则（ routing rule ）是为了改变网络流量所经过的途径而修改路由信息的技术，主要通过改变路由属性（包括可达性）来实现。在发起一次 RPC 调用前起到过滤目标服务器地址的作用，过滤后的地址列表，将作为消费端最终发起 RPC 调用的备选地址。</p> |
| <p>试想该下场景：使用 dubbo-go 在生产环境上，排除预发布机。使用路由规则实现不是很合适吗？</p> |
| <p>虽然知道了路由规则需要实现什么功能，但还不足以实现一个完整的路由规则功能。除此之外，还需要知道如何方便的管理路由规则。</p> |
| <h2 id="目标">目标</h2> |
| <p>综上所述，可以总结出以下 <strong>目标</strong></p> |
| <ul> |
| <li>支持方便扩展路由规则的配置；</li> |
| <li>可以方便的管理路由规则配置，如支持本地与远程配置中心管理；</li> |
| <li>与 Dubbo 现有的配置中心内的路由规则配置文件兼容，降低在新增语言栈的学习及使用成本；</li> |
| </ul> |
| <h2 id="路由规则设计">路由规则设计</h2> |
| <p>在设计之初，首先要考虑的是路由规则应该放在整个服务治理周期的哪个阶段呢？</p> |
| <p>有些读者可能会有点困惑，我连架构图都不知道，如何考虑在哪个阶段？不怕，下图马上给你解惑。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-arch-2.png" alt="img"></p> |
| <p>可以看到图中的 Router 就是路由规则插入的位置，目前路由规则主要用于控制 Consumer 到 Provider 之间的网络流量的路由路径。</p> |
| <p>除此之外，还有几个问题是需要优先考虑：</p> |
| <p>1.需要什么功能？</p> |
| <ul> |
| <li>通过配置信息生成路由规则，包括：读取并解析本地配置文件，读取并解析配置中心的配置。以责任链模式串联起来。</li> |
| <li>通过路由规则，匹配本地信息与远端服务信息，过滤出可以调用的远端节点，再进行负载均衡。</li> |
| </ul> |
| <p>2.如何设计接口？</p> |
| <p>通过第一点，我们能设计出以下接口来实现所需的功能。</p> |
| <ul> |
| <li>路由规则接口：用于路由规则过滤出可以调用的远端节点。</li> |
| <li>路由规则责任链接口：允许执行多个路由规则。</li> |
| <li>配置信息生成路由规则接口：解析内部配置信息（common.URL）生成对应的路由规则。</li> |
| <li>配置文件生成路由规则接口：解析配置文件生成对应的路由规则。</li> |
| </ul> |
| <p>3.如何实现本地与远程路由规则配置加载？</p> |
| <ul> |
| <li>本地路由规则配置：在原配置加载阶段，新增读取路由配置文件。使用 <code>FIleRouterFactory</code> 解析后，生成对应路由规则，放置到内存中备用。</li> |
| <li>远程路由规则配置：在 zookeeper 注册并监听静态资源目录后。读取静态资源，筛选符合路由规则配置信息，通过 <code>RouterFactory</code> 生成对应路由规则，放置到内存中备用。</li> |
| </ul> |
| <h3 id="router">Router</h3> |
| <p>匹配及过滤远程实例的路由规则。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// Router |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> Router <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Route determine the target invoker list. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Route</span>([]protocol.Invoker, <span style="color:#719e07">*</span>common.URL, protocol.Invocation) []protocol.Invoker |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Priority return priority in router |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// 0 to ^int(0) is better |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Priority</span>() <span style="color:#dc322f">int64</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// URL return URL in router |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">URL</span>() common.URL |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>目前已有实现类包括：</p> |
| <ul> |
| <li>listenableRouter</li> |
| <li>AppRouter</li> |
| <li>ConditionRouter</li> |
| <li>HealthCheckRouter</li> |
| <li>FileConditionRouter</li> |
| </ul> |
| <h3 id="routerchain">RouterChain</h3> |
| <p>执行多个路由规则的责任链。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// Chain |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> Chain <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Route determine the target invokers list with chain. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Route</span>([]protocol.Invoker, <span style="color:#719e07">*</span>common.URL, protocol.Invocation) []protocol.Invoker |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// AddRouters add routers |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">AddRouters</span>([]Router) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="filerouterfactory">FIleRouterFactory</h3> |
| <p>生成解析配置文件生成路由规则的工厂类。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// RouterFactory router creates factory use for parse config file |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> FileRouterFactory <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// NewFileRouter create file router with config file |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">NewFileRouter</span>([]<span style="color:#dc322f">byte</span>) (Router, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="routerfactory">RouterFactory</h3> |
| <p>通过配置信息生成路由规则的工厂类。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// RouterFactory router create factory |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> RouterFactory <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// NewRouter creates router instance with URL |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">NewRouter</span>(<span style="color:#719e07">*</span>common.URL) (Router, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h2 id="实现">实现</h2> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/router/router-design.png" alt="img"></p> |
| <p>实现路由规则以兼容 dubbo 为首要目标，降低使用者的学习成本为辅助目标。与配置中心模块相结合，实现路由规则远程统一管理与下发。</p> |
| <h3 id="规则类型">规则类型</h3> |
| <p>下面先来介绍一下有哪些具体的路由规则实现。</p> |
| <h4 id="条件路由">条件路由</h4> |
| <p>dubbo-go 中第一个支持的路由规则，允许用户通过配置文件及配置中心管理路由规则。</p> |
| <p>与之相似的一个概念是 dubbo-go 里面的 group 概念，但是条件路由提供了更加细粒度的控制手段和更加丰富的表达语义。比较典型的使用场景是黑白名单设置，灰度以及测试等。</p> |
| <h4 id="健康检查路由">健康检查路由</h4> |
| <p>在 RPC 调用中，我们希望尽可能地将请求命中到那些处理能力快、处于健康状态的实例，该路由的功能就是通过某种策略断定某个实例不健康，并将其排除在候选调用列表，优先调用那些健康的实例。这里的”健康”可以是我们自己定义的状态，默认实现即当错误比例到达某一个阈值时或者请求活跃数大于上限则认为其不健康，允许用户扩展健康检测策略。</p> |
| <p>在我们服务治理里面，核心的问题其实就在于如何判断一个实例是否可用。无论是负载均衡、</p> |
| <p>熔断还是限流，都是对这个问题的解答。所以，这个 feature 是一个很好的尝试。因为我们接下来计划提供的特性，基于规则的限流以及动态限流，都是要解决 “如何断定一个实例是否可用” 这么一个问题。</p> |
| <p>所以欢迎大家使用这个特性，并向社区反馈各自设定的健康指标。这对我们接下来的工作会有很大的帮助。</p> |
| <h4 id="标签路由">标签路由</h4> |
| <p>以 Provider 为维度，通过将某一个或多个服务的提供者划分到同一个分组，约束流量只在指定分组中流转，从而实现流量隔离的目的，可以作为蓝绿发布、灰度发布等场景的能力基础。</p> |
| <ul> |
| <li>静态打标：根据配置文件所配置的标签，固定给 Provider 设置标签。</li> |
| <li>动态打标：基于健康检查路由，根据服务不同时刻，不同状态，动态在 Provider 设置适合的标签。</li> |
| </ul> |
| <h3 id="分析">分析</h3> |
| <p>接着，以条件路由在 zookeeper 实现为例，对服务提供者与服务消费者进行整体流程分析。</p> |
| <h4 id="如何配置条件路由规则">如何配置条件路由规则</h4> |
| <p>配置条件路由规则既可以通过本地配置文件也能通过远程配置中心进行配置，配置生效流程都是：配置文件 =&gt; dubbo 内部协议 =&gt; 缓存至应用级内存 =&gt; 过滤出可调用节点。</p> |
| <p><strong>dubbo-admin</strong> 【服务治理/条件路由】增加路由规则配置，zookeeper 中会自动生成其对应配置节点，内容均为 <strong>dubbo-admin</strong> 中设置的配置。</p> |
| <p><strong>全局配置</strong></p> |
| <p>对应应用级全局路由规则配置。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>/dubbo/config/dubbo/user-info-server（应用名）.condition-router |
| </span></span></code></pre></div><p>应用名：只对 user-info-server 应用生效 .condition-router: 路由类型。除此之外，还有 .tag-router 表示标签路由。</p> |
| <p><strong>服务配置</strong></p> |
| <p>对应服务级所有路由规则配置。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>/dubbo/ com.ikurento.user.UserProvider（服务名） /routers |
| </span></span></code></pre></div><p>服务名：只对 com.ikurento.user.UserProvider 服务生效。</p> |
| <h3 id="实现-router">实现 Router</h3> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (c <span style="color:#719e07">*</span>ConditionRouter) <span style="color:#268bd2">Route</span>(invokers []protocol.Invoker, url <span style="color:#719e07">*</span>common.URL, invocation protocol.Invocation) []protocol.Invoker { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> !c.<span style="color:#268bd2">Enabled</span>() { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> invokers |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#b58900">len</span>(invokers) <span style="color:#719e07">==</span> <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> invokers |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> isMatchWhen <span style="color:#719e07">:=</span> c.<span style="color:#268bd2">MatchWhen</span>(url, invocation) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> !isMatchWhen { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> invokers |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> result []protocol.Invoker |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#b58900">len</span>(c.ThenCondition) <span style="color:#719e07">==</span> <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> result |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> _, invoker <span style="color:#719e07">:=</span> <span style="color:#719e07">range</span> invokers { |
| </span></span><span style="display:flex;"><span> invokerUrl <span style="color:#719e07">:=</span> invoker.<span style="color:#268bd2">GetUrl</span>() |
| </span></span><span style="display:flex;"><span> isMatchThen <span style="color:#719e07">:=</span> c.<span style="color:#268bd2">MatchThen</span>(<span style="color:#719e07">&amp;</span>invokerUrl, url) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> isMatchThen { |
| </span></span><span style="display:flex;"><span> result = <span style="color:#b58900">append</span>(result, invoker) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#b58900">len</span>(result) &gt; <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> result |
| </span></span><span style="display:flex;"><span> } <span style="color:#719e07">else</span> <span style="color:#719e07">if</span> c.Force { |
| </span></span><span style="display:flex;"><span> rult, _ <span style="color:#719e07">:=</span> url.<span style="color:#268bd2">GetParamAndDecoded</span>(constant.RULE_KEY) |
| </span></span><span style="display:flex;"><span> localIP, _ <span style="color:#719e07">:=</span> gxnet.<span style="color:#268bd2">GetLocalIP</span>() |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Warnf</span>(<span style="color:#2aa198">&#34;The route result is empty and force execute. consumer: %s, service: %s, route: %s&#34;</span>, localIP, url.<span style="color:#268bd2">Service</span>(), route) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> result |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> invokers |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>以下为必须实现的方法，以下方法用于获取过滤服务端节点配置。</p> |
| <ul> |
| <li>Route: 根据配置，调用节点与被调用节点，过滤出可调用节点。</li> |
| <li>Priority: 路由规则优先级，需要是个正整数。</li> |
| <li>URL: 通过路由规则转换出来的 dubbo 内部协议。</li> |
| </ul> |
| <p>更多实现参考：</p> |
| <p>路由规则：https://github.com/apache/dubbo-go/tree/master/cluster/router/condition</p> |
| <p>其中包含监听配置中心实现：https://github.com/apache/dubbo-go/blob/master/cluster/router/condition/listenable_router.go</p> |
| <h2 id="使用方法">使用方法</h2> |
| <p>经过上面设计与实现的分析，大概也能猜测到如何使用：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/protocol&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config_center/zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/router/condition&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span></code></pre></div><p>如图所示，使用路由规则并不复杂，只需要把对应的依赖引入进来。在包初始化的时候，会创建出来对应的路由规则的实现。比如说加载条件路由、健康检测路由或者标签作为路由规则：</p> |
| <h3 id="本地路由规则配置">本地路由规则配置</h3> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span>_ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/router/condition&#34;</span> |
| </span></span></code></pre></div><p>仅仅引用依赖包还不直接使用，还需要配置指定的配置文件： *<strong>router_config.yml*</strong> ，内容如下:</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span># dubbo router yaml configure file |
| </span></span><span style="display:flex;"><span>priority: <span style="color:#2aa198">1</span> |
| </span></span><span style="display:flex;"><span>force: <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span>conditions : [<span style="color:#2aa198">&#34;host = 1.1.1.1 =&gt; host = 192.168.199.214&#34;</span>] |
| </span></span></code></pre></div><p>更多配置方式：<a href="https://dubbo.apache.org/zh-cn/docsv2.7/user/examples/routing-rule/">条件路由配置</a></p> |
| <h3 id="配置中心配置">配置中心配置</h3> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span>_ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config_center/zookeeper&#34;</span> |
| </span></span></code></pre></div><p>目前仅支持 zookeeper 配置中心，与 dubbo-admin 结合即可使用。配置方式如下：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/router/dubbo-admin-1.png" alt="img"></p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/router/dubbo-admin-2.png" alt="img"></p> |
| <h2 id="总结">总结</h2> |
| <p>更加具体的实现，我就不详细论述，大家可以去看源码，欢迎大家持续关注，或者贡献代码。</p> |
| <p>整个路由规则功能，已经能跟上 dubbo 2.7.x 版本，已经支持本地及远端路由规则配置管理。从扩展性来说，是比较便利。目前已经支持条件路由、标签路由与健康检测路由，虽然能满足基本使用场景，距离完善还有还长远的路。</p> |
| <p>未来计划：</p> |
| <ol> |
| <li>更多的配置中心支持，理论上已经支持，但还没测试。</li> |
| <li>service-router（未支持）</li> |
| <li>标签路由-配置中心（未支持）</li> |
| <li>目前路由与配置中心结合的代码，对新增路由规则并不友好，有一定接入成本。</li> |
| </ol> |
| <p>欢迎大家关注或者贡献代码，https://github.com/apache/dubbo-go</p></description></item><item><title>Blog: 记一次对 dubbo-go-hessian2 的性能优化</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/12/%E8%AE%B0%E4%B8%80%E6%AC%A1%E5%AF%B9-dubbo-go-hessian2-%E7%9A%84%E6%80%A7%E8%83%BD%E4%BC%98%E5%8C%96/</link><pubDate>Tue, 12 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/12/%E8%AE%B0%E4%B8%80%E6%AC%A1%E5%AF%B9-dubbo-go-hessian2-%E7%9A%84%E6%80%A7%E8%83%BD%E4%BC%98%E5%8C%96/</guid><description> |
| <blockquote> |
| <p>dubbo-go-hessian2 是一个用 Go 实现的 hessian 协议 v2.0 版本的序列化库。从项目名称里可以看到主要用在 dubbo-go 这个项目里。hessian 协议作为 dubbo 的默认协议，因此对性能有比较高的要求。</p> |
| </blockquote> |
| <h2 id="立项">立项</h2> |
| <p>譬如有网文 基于Go的马蜂窝旅游网分布式IM系统技术实践 把 dubbo-go 与其他 RPC 框架对比如下：</p> |
| <ol> |
| <li>Go STDPRC: Go 标准库的 RPC，性能最优，但是没有治理；</li> |
| <li>RPCX: 性能优势 2*GRPC + 服务治理；</li> |
| <li>GRPC: 跨语言，但性能没有 RPCX 好；</li> |
| <li>TarsGo: 跨语言，性能 5*GRPC，缺点是框架较大，整合起来费劲；</li> |
| <li>Dubbo-Go: 性能稍逊一筹，比较适合 Go 和 Java 间通信场景使用</li> |
| </ol> |
| <p>有鉴于此，社区便开始组织部分人力，启动了对 dubbo-go 性能优化【同时也欢迎上文作者到钉钉群 23331795 与我们社区交流】。考察 dubbo-go 的各个组件，大家不约而同地决定首先优化比较独立的 dubbo-go-hessian2。</p> |
| <h2 id="起步">起步</h2> |
| <p>在最开始的时候，并没有太想清楚需要做什么，改哪个地方，要优化到何种程度，所以最简单的办法就是看看现状。</p> |
| <p>首先，写了一个简单的例子，把常见的类型到一个结构体里，然后测一下耗时。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> Mix <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> A <span style="color:#dc322f">int</span> |
| </span></span><span style="display:flex;"><span> B <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> CA time.Time |
| </span></span><span style="display:flex;"><span> CB <span style="color:#dc322f">int64</span> |
| </span></span><span style="display:flex;"><span> CC <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> CD []<span style="color:#dc322f">float64</span> |
| </span></span><span style="display:flex;"><span> D <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#268bd2">interface</span>{} |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>m <span style="color:#719e07">:=</span> Mix{A: <span style="color:#b58900">int</span>(<span style="color:#2aa198">&#39;a&#39;</span>), B: <span style="color:#2aa198">`hello`</span>} |
| </span></span><span style="display:flex;"><span>m.CD = []<span style="color:#dc322f">float64</span>{<span style="color:#2aa198">1</span>, <span style="color:#2aa198">2</span>, <span style="color:#2aa198">3</span>} |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// 再加一层，使得数据显得复杂一些 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>m.D = <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">`floats`</span>: m.CD, <span style="color:#2aa198">`A`</span>: m.A, <span style="color:#2aa198">`m`</span>: m} |
| </span></span></code></pre></div><blockquote> |
| <p>看起来这个结构体跟真实环境里可能不太一样，但是用来分析瓶颈应该是足够了。</p> |
| </blockquote> |
| <p>然后直接靠 Go Test 写个测试用例：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">BenchmarkEncode</span>(b <span style="color:#719e07">*</span>testing.B) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; i &lt; b.N; i<span style="color:#719e07">++</span> { |
| </span></span><span style="display:flex;"><span> _, _ = <span style="color:#268bd2">encodeTarget</span>(<span style="color:#719e07">&amp;</span>m) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">BenchmarkDecode</span>(b <span style="color:#719e07">*</span>testing.B) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; i &lt; b.N; i<span style="color:#719e07">++</span> { |
| </span></span><span style="display:flex;"><span> _, _ = <span style="color:#268bd2">NewDecoder</span>(bytes).<span style="color:#268bd2">Decode</span>() |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><blockquote> |
| <p>go test -benchmem -run=^$ github.com/apache/dubbo-go-hessian2 -bench &ldquo;^B&rdquo; -vet=off -v</p> |
| </blockquote> |
| <p>得到下面结果：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>BenchmarkEncode-8 89461 11485 ns/op 3168 B/op 122 allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkDecode-8 64914 19595 ns/op 7448 B/op 224 allocs/op |
| </span></span></code></pre></div><p>*<strong>注：基于 MacBook Pro 2018【主频 Intel Core i7 2.6 GHz】测试。*</strong></p> |
| <p>不与同类库作横向比较，仅仅从这个测试结果里的数字上无法得出任何结论。对我们来说更重要的是：它到底慢在哪里。首先想到的手段便是：借助 pprof 生成火焰图，定位 CPU 消耗。</p> |
| <p>pprof 工具的用法可以参考官网文档。本文测试时直接使用了 Goland 内置 <code>CPU Profiler</code> 的测试工具：测试函数左边的 <code>Run xx with 'CPU Profiler'</code>。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/hessian/p2.png" alt="img"></p> |
| <p>测试跑完后， Goland 直接显示火焰图如下：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/hessian/p3.png" alt="img"></p> |
| <p>从这个图里可以看到，测试代码大概占用了左边的70%，右边30%是运行时的一些消耗，运行时部分一般包括 gc、schedule 两大块，一般不能直接优化。图上左边可以清晰地看到 <code>encObject</code> 里 <code>RegisterPOJO</code> 和 <code>Encode</code> 各占了小一半。</p> |
| <p>完成序列化功能的 <code>Encode</code> 消耗 CPU 如此之多尚可理解，而直觉上，把类对象进行解析和注册 <code>RegisterPOJO</code> 是不应该成为消耗大户的。所以猜测这个地方要么注册有问题，要么有重复注册。</p> |
| <p>下一步分析，用了一个简单的办法：在这个函数里加日志。然后继续再跑一下 benchmark，可以看到性能瓶颈处：容器读写的地方。</p> |
| <p>既然知道这里做了许多重复的无用功，就很容易明确优化方法：加缓存。把已经解析过的结果缓存下来，下次需要的时候直接取出使用。改进后的代码简单如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">if</span> goName, ok <span style="color:#719e07">:=</span> pojoRegistry.j2g[o.<span style="color:#268bd2">JavaClassName</span>()]; ok { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> pojoRegistry.registry[goName].index |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>这里刚开始有个疑问，为什么要分两步先取 <code>JavaClassName</code> 再取 <code>GoName</code> 而不直接取后者？看起来好像是多此一举了，但其实 <code>JavaClassName</code> 是类直接定义的，而 <code>GoName</code> 却依赖一次反射。相较之下两次转换的消耗可以忽略了。改完之后再跑一下 benchmark：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>BenchmarkEncode-8 197593 5601 ns/op 1771 B/op 51 allocs/op |
| </span></span></code></pre></div><p>非常惊讶地看到，吞吐量大概是原来的 200%。与上面的火焰图对比，可以粗略的计算，<code>RegiserPOJO</code> 大概占了整体的30%，改进后应该也只有原来的 <code>1 / 0.7 * 100% = 140%</code> 才对。答案也可以在火焰图里找到：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/hessian/p4.png" alt="img"></p> |
| <p>除了 <code>RegisterPOJO</code> 被干掉以外，与上图对比，还有哪些区别呢？可以看到，原来占用将近 20% 的 <code>GC</code> 也几乎看不到了。所以真实的 CPU 利用率也要加上这部分的增长，大约 <code>1 / 0.5 * 100% = 200%</code>。</p> |
| <blockquote> |
| <p>需要提醒的是，benchmark 跑出来的结果并不算稳定，所以你自己压出来的结果跟我的可能不太一致，甚至多跑几次的结果也不完全一样。对于上面的数字你只要理解原因就好，上下浮动10%也都是正常范围。</p> |
| <p>反过来看，这也算是 GC 优化的一个角度。碰到 GC 占用CPU过高，除了去一个个换对象池，也可以重点看看那些被频繁调用的模块。当然更科学的方法是看 <code>pprof heap</code> / <code>memory profiler</code> 。</p> |
| </blockquote> |
| <p>针对这个结果，可以看到 <code>encObject</code> 以上都被切割成了不同的小格子，不再有像 <code>RegisterPOJO</code> 那样的大块占用，一般情况下，优化到这里就可以了。</p> |
| <p>看完了 <code>Encode</code> ，再来看看 <code>Decode</code> ，方法类似，直接看 Goland 生成的火焰图：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/hessian/p5.png" alt="img"></p> |
| <p>这个图有点迷惑性，好像也被分成差不多的小格子了。可以点开 <code>decObject</code> 这一层：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/hessian/p6.png" alt="img"></p> |
| <p>这个时候原来小的 <code>...</code> 会显示具体内容，需要注意的是里面有两个 <code>findField</code> ，在复杂的调用里经常会遇到这种情况：一个耗资源的函数被分到了许多函数里，导致在看火焰图时并不能直观地看到它就是瓶颈。比较常见的有序列化、日志、网络请求等每个模块都会干一点却又没有一个全局的函数只干他一件事。这个时候除了肉眼去找以外也可以借助于另外一个工具：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/hessian/p7.png" alt="img"></p> |
| <p>在这个 <code>Method List</code> 里可以明显看到 <code>findField</code> 已经被合并到一起了，总占用接近 CPU 的一半，看到这里你大概就知道它应该是个优化点了。</p> |
| <h2 id="进一步">进一步</h2> |
| <p>函数 <code>func findField(name string, typ reflect.Type) ([]int, error)</code> 的作用是在一个类型里寻找指定属性的位置（Index，反射包里用它来表示是第几个字段）。很容易想到，对于一个结构体来说，每个字段的位置从一开始就确定了，所以用缓存一样可以解决这个问题。一个简单的优化如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">findField</span>(name <span style="color:#dc322f">string</span>, typ reflect.Type) (indexes []<span style="color:#dc322f">int</span>, err <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> typCache, _ <span style="color:#719e07">:=</span> findFieldCache.<span style="color:#268bd2">LoadOrStore</span>(typ, <span style="color:#719e07">&amp;</span>sync.Map{}) |
| </span></span><span style="display:flex;"><span> indexes, _ <span style="color:#719e07">:=</span> typCache.(<span style="color:#719e07">*</span>sync.Map).<span style="color:#268bd2">Load</span>(name) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> <span style="color:#b58900">len</span>(indexes.([]<span style="color:#dc322f">int</span>)) <span style="color:#719e07">==</span> <span style="color:#2aa198">0</span> { |
| </span></span><span style="display:flex;"><span> err = perrors.<span style="color:#268bd2">Errorf</span>(<span style="color:#2aa198">&#34;failed to find field %s&#34;</span>, name) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> indexes.([]<span style="color:#dc322f">int</span>), err |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// ... |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>} |
| </span></span></code></pre></div><div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>- BenchmarkDecode-8 57723 17987 ns/op 7448 B/op 224 allocs/op |
| </span></span><span style="display:flex;"><span>+ BenchmarkDecode-8 82995 12272 ns/op 7224 B/op 126 allocs/op |
| </span></span></code></pre></div><p>可以看到，结果并不如预期的那样提升一倍效果。这个代码乍看起来，好像除了有一些啰嗦的断言，好像也没别的东西了，为什么只有60%的提升呢，我们还是借助下工具</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/hessian/p8.png" alt="img"></p> |
| <p>可以看到：读缓存耗费了 7% 的资源。其中，<code>sync.(*Map)</code> 不便优化，但 <code>newobejct</code> 是哪里来的呢？代码里可以看到，唯一定义新对象的地方就是函数第一行的 <code>&amp;sync.Map</code> ，我抱着试一试的心态把 <code>LoadOrStore</code> 拆成了两步</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span>typCache, ok <span style="color:#719e07">:=</span> findFieldCache.<span style="color:#268bd2">Load</span>(typ) |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">if</span> !ok { |
| </span></span><span style="display:flex;"><span> typCache = <span style="color:#719e07">&amp;</span>sync.Map{} |
| </span></span><span style="display:flex;"><span> findFieldCache.<span style="color:#268bd2">Store</span>(typ, typCache) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>- BenchmarkDecode-8 82995 12272 ns/op 7224 B/op 126 allocs/op |
| </span></span><span style="display:flex;"><span>+BenchmarkDecode-8 103876 12385 ns/op 6568 B/op 112 allocs/op |
| </span></span></code></pre></div><p>看结果，着实出乎意料。想起来以前看 Java 代码时经常碰到这样的代码：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">if</span> ( logLevel <span style="color:#719e07">&gt;=</span> <span style="color:#2aa198">`info`</span> ) { |
| </span></span><span style="display:flex;"><span> log.<span style="color:#268bd2">Info</span>(<span style="color:#719e07">...</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>以前一直觉得这个 <code>if</code> 真是浪费感情，现在想来，别是一番认知了。如果能提供一个 <code>LoadOrStore(key, func() interface{})</code> 的方法， 会不会更好一些？</p> |
| <p>到这里的话，我们做了两个比较大的优化，整体性能大约提升了一倍。如果仔细看火焰图，还会发现有很多小的优化点，但是由于没有什么特别质的飞跃，这里不再赘述。有兴趣的小伙伴可以到 PR Imp: cache in reflection 里阅读相关的讨论。</p> |
| <h2 id="更进一步">更进一步</h2> |
| <p>优化到此，依然藏着一个更深层次的问题：找一个可靠的参考基准，以衡量目前的工作结果【毕竟没有对比就没有伤害】。一个很容易想到的比较对象是 Go 语言官方的 <code>json</code> 标准库。</p> |
| <p>把 dubbo-go-hessian2 与 <code>json</code> 标准库做比较如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>$ go <span style="color:#b58900">test</span> -benchmem -run<span style="color:#719e07">=</span>^$ github.com/apache/dubbo-go-hessian2 -bench <span style="color:#2aa198">&#34;^B&#34;</span> -vet<span style="color:#719e07">=</span>off -v -count<span style="color:#719e07">=</span><span style="color:#2aa198">5</span> |
| </span></span><span style="display:flex;"><span>goos: darwin |
| </span></span><span style="display:flex;"><span>goarch: amd64 |
| </span></span><span style="display:flex;"><span>pkg: github.com/apache/dubbo-go-hessian2 |
| </span></span><span style="display:flex;"><span>BenchmarkJsonEncode |
| </span></span><span style="display:flex;"><span>BenchmarkJsonEncode-8 <span style="color:#2aa198">249114</span> <span style="color:#2aa198">4719</span> ns/op <span style="color:#2aa198">832</span> B/op <span style="color:#2aa198">15</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonEncode-8 <span style="color:#2aa198">252224</span> <span style="color:#2aa198">4862</span> ns/op <span style="color:#2aa198">832</span> B/op <span style="color:#2aa198">15</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonEncode-8 <span style="color:#2aa198">240582</span> <span style="color:#2aa198">4739</span> ns/op <span style="color:#2aa198">832</span> B/op <span style="color:#2aa198">15</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonEncode-8 <span style="color:#2aa198">213283</span> <span style="color:#2aa198">4784</span> ns/op <span style="color:#2aa198">832</span> B/op <span style="color:#2aa198">15</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonEncode-8 <span style="color:#2aa198">227101</span> <span style="color:#2aa198">4665</span> ns/op <span style="color:#2aa198">832</span> B/op <span style="color:#2aa198">15</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkEncode |
| </span></span><span style="display:flex;"><span>BenchmarkEncode-8 <span style="color:#2aa198">182184</span> <span style="color:#2aa198">5615</span> ns/op <span style="color:#2aa198">1771</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkEncode-8 <span style="color:#2aa198">183007</span> <span style="color:#2aa198">5565</span> ns/op <span style="color:#2aa198">1771</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkEncode-8 <span style="color:#2aa198">218664</span> <span style="color:#2aa198">5593</span> ns/op <span style="color:#2aa198">1771</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkEncode-8 <span style="color:#2aa198">214704</span> <span style="color:#2aa198">5886</span> ns/op <span style="color:#2aa198">1770</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkEncode-8 <span style="color:#2aa198">181861</span> <span style="color:#2aa198">5605</span> ns/op <span style="color:#2aa198">1770</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonDecode |
| </span></span><span style="display:flex;"><span>BenchmarkJsonDecode-8 <span style="color:#2aa198">123667</span> <span style="color:#2aa198">8412</span> ns/op <span style="color:#2aa198">1776</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonDecode-8 <span style="color:#2aa198">122796</span> <span style="color:#2aa198">8497</span> ns/op <span style="color:#2aa198">1776</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonDecode-8 <span style="color:#2aa198">132103</span> <span style="color:#2aa198">8471</span> ns/op <span style="color:#2aa198">1776</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonDecode-8 <span style="color:#2aa198">130687</span> <span style="color:#2aa198">8492</span> ns/op <span style="color:#2aa198">1776</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkJsonDecode-8 <span style="color:#2aa198">127668</span> <span style="color:#2aa198">8476</span> ns/op <span style="color:#2aa198">1776</span> B/op <span style="color:#2aa198">51</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkDecode |
| </span></span><span style="display:flex;"><span>BenchmarkDecode-8 <span style="color:#2aa198">107775</span> <span style="color:#2aa198">10092</span> ns/op <span style="color:#2aa198">6424</span> B/op <span style="color:#2aa198">98</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkDecode-8 <span style="color:#2aa198">110996</span> <span style="color:#2aa198">9950</span> ns/op <span style="color:#2aa198">6424</span> B/op <span style="color:#2aa198">98</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkDecode-8 <span style="color:#2aa198">111036</span> <span style="color:#2aa198">10760</span> ns/op <span style="color:#2aa198">6424</span> B/op <span style="color:#2aa198">98</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkDecode-8 <span style="color:#2aa198">113151</span> <span style="color:#2aa198">10063</span> ns/op <span style="color:#2aa198">6424</span> B/op <span style="color:#2aa198">98</span> allocs/op |
| </span></span><span style="display:flex;"><span>BenchmarkDecode-8 <span style="color:#2aa198">109197</span> <span style="color:#2aa198">10002</span> ns/op <span style="color:#2aa198">6424</span> B/op <span style="color:#2aa198">98</span> allocs/op |
| </span></span><span style="display:flex;"><span>PASS |
| </span></span><span style="display:flex;"><span>ok github.com/apache/dubbo-go-hessian2 28.680s |
| </span></span></code></pre></div><p>虽然每次的结果不稳定，但就整体而言，目前的序列化和反序列化性能大概都是JSON标准库的85%左右。这个成绩并不能说好，但短期内能花20分的精力得到一个80分的结果，应该也是可以接受的。至于剩下的20%，就不是靠改几行代码就能搞定了。内存分配是否合理、执行流程是否有冗余，都是需要一点一滴地去改进。</p> |
| <h2 id="总结">总结</h2> |
| <p>最后，我们来总结一下本文主要的优化步骤：</p> |
| <ul> |
| <li>利用火焰图 快速定位消耗 CPU 较高的模块；</li> |
| <li>利用缓存机制，快速消除重复的计算；</li> |
| <li>利用 CallTree、MethodList 等多种工具分析小段代码的精确消耗；</li> |
| <li>遵循二八定律，以最小的成本做出一个效果显著的收益。</li> |
| </ul> |
| <h3 id="欢迎加入-dubbo-go-社区">欢迎加入 dubbo-go 社区</h3> |
| <p>目前 dubbo-go 已经到了一个比较稳定成熟的状态。在接下来的版本里面，我们将集中精力在云原生上。下一个版本，我们将首先实现应用维度的服务注册，这是一个和现有注册模型完全不同的新的注册模型。也是我们朝着云原生努力的一个关键版本。</p> |
| <p>dubbo-go 钉钉群 <strong>23331795</strong> 欢迎你的加入。</p> |
| <h4 id="作者信息">作者信息</h4> |
| <p>张慧仁，github id: micln，任职 得到APP 后端开发。</p></description></item><item><title>Blog: Dubbo Go 踩坑记</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E8%B8%A9%E5%9D%91%E8%AE%B0/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E8%B8%A9%E5%9D%91%E8%AE%B0/</guid><description> |
| <h2 id="扯淡">扯淡</h2> |
| <h3 id="前尘">前尘</h3> |
| <p>由于我的一个项目需要做公司用户鉴权，而组内其他小伙伴刚好有一个 <em>dubbo</em> 的鉴权 <em>rpc</em> ，一开始我是打算直接的读 <em>redis</em> 数据然后自己做解密。工作进行到一半，由于考虑到如果以后这个服务有任何变动，我这边要有联动行为，所以改用 <em>go</em> 来调用 <em>dubbo</em> 的 <em>rpc</em> ，于是我在 <em>github</em> 上找到了 <a href="https://github.com/AlexStocks">雨神</a> 的 <code>https://github.com/apache/dubbo-go-samples/tree/master</code> (PS: 这个是 <em>dubbo-go</em> 前身)。不得不说，雨神是热心的人儿啊，当时还帮着我调试代码。最后也是接入了一个阉割版的吧，主要是当时 <em>hessian2</em> 对泛型支持的不怎么好。</p> |
| <h3 id="现在">现在</h3> |
| <p>目前 <a href="https://github.com/apache/dubbo-go">dubbo-go</a>隶属于 <em>apache</em> 社区，相比以前做了部分重构，并且维护也很活跃了。</p> |
| <h2 id="接入">接入</h2> |
| <h3 id="问题">问题</h3> |
| <p>目前整个项目在快速的迭代中，很多功能还没有完善，维护人员还没有时间来完善文档，所以在接入的时候要自己看源码或调试。</p> |
| <h3 id="说明">说明</h3> |
| <p>目前我司在使用 <em>dubbo</em> 的过程使用的 <em>zookeeper</em> 作为注册中心，序列化是 <em>hessian2</em> ，所以我们要做如下初始化：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span> <span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common/proxy/proxy_factory&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/protocol&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/filter/impl&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/cluster_impl&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/loadbalance&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> ) |
| </span></span></code></pre></div><h3 id="配置">配置</h3> |
| <p>由于我是接入客户端，所以我这边只配置了 <em>ConsumerConfig</em> 。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">dubbo</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># client</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">request_timeout</span>: <span style="color:#2aa198">&#34;3s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75"># connect timeout</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">connect_timeout</span>: <span style="color:#2aa198">&#34;3s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">check</span>: <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">application</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">organization</span>: <span style="color:#2aa198">&#34;dfire.com&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;soa.sso.ITokenService&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">module</span>: <span style="color:#2aa198">&#34;dubbogo token service client&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">version</span>: <span style="color:#2aa198">&#34;1.0.0&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">owner</span>: <span style="color:#2aa198">&#34;congbai&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registries</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;hangzhouzk&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">timeout</span>: <span style="color:#2aa198">&#34;3s&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;zk1.2dfire-daily.com:2181&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">username</span>: <span style="color:#2aa198">&#34;&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">password</span>: <span style="color:#2aa198">&#34;&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">references</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;ITokenService&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;hangzhouzk&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface</span>: <span style="color:#2aa198">&#34;com.dfire.soa.sso.ITokenService&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">version</span>: <span style="color:#2aa198">&#34;1.0.0&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;validate&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">&#34;3&#34;</span> |
| </span></span></code></pre></div><p>我这里是把 <em>dubbo-go</em> 作为第三方库来用，所以我没使用官方 <a href="https://github.com/apache/dubbo-go-samples/">dubbo-samples</a> 那样在 <em>init</em> 函数中读入配置。</p> |
| <p>配置代码如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span> <span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/protocol/dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> ) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">type</span> DubboCli <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">func</span> <span style="color:#268bd2">NewCli</span>(cconf config.ConsumerConfig) <span style="color:#719e07">*</span>DubboCli { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetConsumerConfig</span>(cconf) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> dubbo.<span style="color:#268bd2">SetClientConf</span>(dubbo.<span style="color:#268bd2">GetDefaultClientConfig</span>()) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>DubboCli{} |
| </span></span><span style="display:flex;"><span> } |
| </span></span></code></pre></div><h3 id="接入-1">接入</h3> |
| <p>好了，配置加载完就说明我们的准备工作已经做好了，接下来就要接入 <em>rpc</em> 接口了。</p> |
| <h4 id="返回值">返回值</h4> |
| <p>一般 <em>rpc</em> 调用的返回值都是自定义的，所以我们也要告诉 <em>dubbo-go</em> 长什么样子。这个结构体要跟 <em>java</em> 的类对应起来，这里我们是要实现 <em>hessian2</em> 的 <em>interface</em> :</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// POJO interface |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// !!! Pls attention that Every field name should be upper case. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// Otherwise the app may panic. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> POJO <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">JavaClassName</span>() <span style="color:#dc322f">string</span> <span style="color:#586e75">// got a go struct&#39;s Java Class package name which should be a POJO class. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>} |
| </span></span></code></pre></div><p>我的实现如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">type</span> Result <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> Model <span style="color:#268bd2">interface</span>{} <span style="color:#2aa198">`json:&#34;model,omitempty&#34;`</span> |
| </span></span><span style="display:flex;"><span> Models []<span style="color:#268bd2">interface</span>{} <span style="color:#2aa198">`json:&#34;models,omitempty&#34;`</span> |
| </span></span><span style="display:flex;"><span> ResultCode <span style="color:#dc322f">string</span> <span style="color:#2aa198">`json:&#34;resultCode&#34;`</span> |
| </span></span><span style="display:flex;"><span> Success <span style="color:#dc322f">bool</span> <span style="color:#2aa198">`json:&#34;success&#34;`</span> |
| </span></span><span style="display:flex;"><span> Message <span style="color:#dc322f">string</span> <span style="color:#2aa198">`json:&#34;message&#34;`</span> |
| </span></span><span style="display:flex;"><span> TotalRecord <span style="color:#dc322f">int</span> <span style="color:#2aa198">`json:&#34;totalRecord&#34;`</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (r Result) <span style="color:#268bd2">JavaClassName</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;com.twodfire.share.result.ResultSupport&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>这里的 <em>JavaClassName</em> 接口的意义就如函数签名一样，返回的就是 <em>java</em> 的类名。</p> |
| <h4 id="接口">接口</h4> |
| <p>要想调用 <em>dubbo</em> 的接口就必须实现下面这个 <em>interface</em></p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// rpc service interface |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> RPCService <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> <span style="color:#586e75">// rpc service id or reference id |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>} |
| </span></span></code></pre></div><p>所以我需要构造一个 <em>struct</em> 来做这个事情，比如：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">type</span> ITokenService <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> Validate <span style="color:#268bd2">func</span>(ctx context.Context, req []<span style="color:#268bd2">interface</span>{}, resp <span style="color:#719e07">*</span>Result) <span style="color:#dc322f">error</span> <span style="color:#2aa198">`dubbo:&#34;validate&#34;`</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (i <span style="color:#719e07">*</span>ITokenService) <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;ITokenService&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>这个结构体一般是不会有什么数据成员。</p> |
| <p>这里我们注意到 <em>Validate</em> 函数声明后面跟的 <em>dubbo tag</em> ，这个是为如果 <em>rpc</em> 名称的首字母是小写（比如我要调用的 <em>dubbo</em> 接口就是 <em>validate</em> )准备的 <em>MethodMapper</em> ，类似于 <em>json</em> 的映射 <em>tag</em> 功效。一开始我就是遇到这个坑，我按官方的例子实现，日志一直说找不到接口，后来我也在官方群里询问大佬才知道有这个功能。</p> |
| <h4 id="注册">注册</h4> |
| <p>好了，上面的准备全部完成后，我们要做最后一步，那就是告诉 <em>dubbo-go</em> 我们想要的是什么。代码如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span> <span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> hessian <span style="color:#2aa198">&#34;github.com/apache/dubbo-go-hessian2&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config&#34;</span> |
| </span></span><span style="display:flex;"><span> ) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">var</span> tokenProvider = <span style="color:#b58900">new</span>(ITokenService) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetConsumerService</span>(tokenProvider) |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>Result{}) |
| </span></span><span style="display:flex;"><span> } |
| </span></span></code></pre></div><h4 id="调用">调用</h4> |
| <p>接下来我们就可以完成我们的 <em>DubboCli</em> 接口了，代码如下：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">func</span> (d <span style="color:#719e07">*</span>DubboCli) <span style="color:#268bd2">CheckUser</span>(token, app <span style="color:#dc322f">string</span>) (<span style="color:#dc322f">bool</span>, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> args <span style="color:#719e07">:=</span> []<span style="color:#268bd2">interface</span>{}{token, app} |
| </span></span><span style="display:flex;"><span> resp <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>Result{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">:=</span> tokenProvider.<span style="color:#268bd2">Validate</span>(context.<span style="color:#268bd2">Background</span>(), args, resp); err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">false</span>, err |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> resp.Success { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp.Success, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> resp.Success, errors.<span style="color:#268bd2">New</span>(resp.Message) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>好了，至此我们就完成了 <em>dubbo-go</em> 的全部接入工作。 Happy Coding&hellip;</p> |
| <h2 id="写在最后">写在最后</h2> |
| <p>其实代码格式这个问题，我在接入的时候跟官方群里的维护者大佬提过，使用 <em>go</em> 官方的代码格式工具 <a href="https://github.com/golang/tools/tree/master/cmd/goimports">goimports</a> 来统一代码格式，这 样对于维护者以外的人提 <em>PR</em> 也是有利。我在接入的过程中遇到一个 <em>bug</em> ，我反馈给雨神，他就让我提了个 <em>PR</em> ，在整个过程就是这个 代码格式的问题，导致我反复的修改代码。</p></description></item><item><title>Blog: Dubbo Go 的前世今生</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E7%9A%84%E5%89%8D%E4%B8%96%E4%BB%8A%E7%94%9F/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E7%9A%84%E5%89%8D%E4%B8%96%E4%BB%8A%E7%94%9F/</guid><description> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-history.png" alt="img"></p> |
| <p>dubbo-go 是目前 Dubbo 多语言生态最火热的项目。dubbo-go 最早的版本应该要追溯到 2016 年，由社区于雨同学编写 dubbo-go 的初版。当时很多东西没有现成的轮子，如 Go 语言没有像 netty 一样的基于事件的网络处理引擎、 hessian2 协议没有 Go 语言版本实现，加上当时 Dubbo 也没有开始重新维护。所以从协议库到网络引擎，再到上层 dubbo-go ，其实都是从零开始写的。</p> |
| <p>在 2018 年，携程开始做 Go 语言的一些中间件以搭建内部的 Go 语言生态，需要有一个 Go 的服务框架可以与携程的现有 dubbo soa 生态互通。所以由我负责重构了 dubbo－go 并开源出这个版本。当时调研了很多开源的 Go 语言服务框架，当时能够支持 hessian2 协议的并跟 Dubbo 可以打通的仅找到了当时于雨写的 dubbo-go 早期版本。由于携程对社区版本的 Dubbo 做了挺多的扩展，源于对扩展性的需求我们 Go 语言版本需要一个更易于扩展的版本，加上当时这个版本本身的功能也比较简单，所以我们找到了作者合作重构了一个更好的版本。经过了大半年时间，在上图第三阶段 19 年 6 月的时候，基本上已经把 dubbo-go 重构了一遍，总体的思路是参考的 Dubbo 整体的代码架构，用Go语言完全重写了一个完整的具备服务端跟消费端的 Golang rpc/ 微服务框架。</p> |
| <p>后来我们将重构后的版本 dubbo-go 1.0 贡献给 Apache 基金会，到现在已经过去了两个多月的时间，近期社区发布了1.1版本。目前为止，已经有包括携程在内的公司已经在生产环境开始了试用和推广。</p> |
| <h3 id="start-dubbo-go">Start dubbo-go</h3> |
| <p>现在的 dubbo-go 已经能够跟 Java 版本做比较好的融合互通，同时 dubbo-go 自身也是一个完成的 Go 语言 rpc/ 微服务框架，它也可以脱离 java dubbo 来独立使用。</p> |
| <p>这边简单介绍一下用法，写一个 hello world 的例子。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/java-provider.png" alt="img"></p> |
| <p>上图是一个简单的 java service ，注册为一个 Dubbo 服务，是一个简单的获取用户信息的例子。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/go-consumer.png" alt="img"></p> |
| <p>上图是 dubbo-go 的客户端，来订阅和调用这个 Java 的 Dubbo 服务。Go 语言客户端需要显式调用 SetConsumerService 来注册需要订阅的服务，然后通过调用 dubbo-go-hessian2 库的 registerPOJO 方法来注册 user 对象，做 Java 和 Go 语言之间的自定义 pojo 类型转换。具体的服务调用方法就是声明一个的 GetUser 闭包，便可直接调用。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/go-provider.png" alt="img"></p> |
| <p>上图，同样的可以基于 dubbo-go 发布一个 GetUser 的服务端，使用方式类似，发布完后可以被 dubbo java 的客户端调用。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/java-go-interop.png" alt="img"></p> |
| <p>如上图所示，现在已经做到了这样一个程度，同样一份 dubbo-go 客户端代码，可以去调用 dubbo-go 的服务端，也可以去调用 Dubbo Java 的服务端；同样一份 dubbo-go 的服务端代码，可以被 dubbo-go 客户端和 Java 客户端调用，所以基本上使用 Dubbo 作为 PPC 框架的 Go 语言应用跟 Java 应用是没有什么阻碍的，是完全的跨语言 RPC 调用。更重要的是 dubbo-go 继承了 Dubbo 的许多优点，如易于扩展、服务治理功能强大，大家在用 Go 语言开发应用的过程中，如果也遇到类似需要与 Dubbo Java 打通的需求，或者需要找一个服务治理功能完备的 Go 微服务框架，可以看下我们 dubbo-go 项目。</p> |
| <h3 id="dubbo-go-的组成项目">dubbo-go 的组成项目</h3> |
| <p>下面介绍一下 dubbo-go 的组成项目，为了方便可以被其他项目直接复用， dubbo-go 拆分成了多个项目，并全部以 Apache 协议开源。</p> |
| <h5 id="apachedubbo-go">apache/dubbo-go</h5> |
| <p>dubbo-go 主项目， Dubbo 服务端、客户端完整 Go 语言实现。</p> |
| <h5 id="apachedubbo-go-hession2">apache/dubbo-go-hession2</h5> |
| <p>目前应用最广泛，与 Java 版本兼容程度最高的 hessian2 协议 Go 语言实现，已经被多个 GolangRPC &amp; Service Mesh 项目使用。</p> |
| <h5 id="dubbo-gogetty">dubbo-go/getty</h5> |
| <p>dubbo-go 异步网络 I/O 库，将网络处理层解耦。</p> |
| <h5 id="dubbo-gogost">dubbo-go/gost</h5> |
| <p>基本类库，定义了 timeWheel、hashSet、taskPool 等。</p> |
| <h5 id="dubbo-godubbo-go-benchmark">dubbo-go/dubbo-go-benchmark</h5> |
| <p>用于对 dubbo-go 进行简单的压力测试，性能测试。</p> |
| <h5 id="apachedubbo-go-hessian2">apache/dubbo-go-hessian2</h5> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-hessian2.png" alt="img"></p> |
| <p>先简单介绍一下 dubbo-go-hessian2 项目。该项目就是 hessian2 协议的 Go 语言实现，最基本的可以将 Java 的基本数据类型和复杂数据类型（如一些包装类和list接口实现类）与 golang 这边对应。</p> |
| <p>详情可以参考：</p> |
| <p><em><a href="https://github.com/hessian-group/hessian-type-mapping">https://github.com/hessian-group/hessian-type-mapping</a></em></p> |
| <p>另外 Dubbo Java 服务端可以不捕获异常，将异常类通过 hession2 协议序列化通过网络传输给消费端，消费端进行反序列化对该异常对象并进行捕获。我们经过一段时间的整理，目前已经支持在 Go 消费端定义对应 Java 的超过 40 种 exception 类，来实现对 Java 异常的捕获，即使用 dubbo-go 也可以做到直接捕获 Java 服务端抛出的异常。</p> |
| <p>另外对于 Java 端 BigDecimal 高精度计算类的支持。涉及到一些金融相关的计算会有类似的需求，所以也对这个类进行了支持。</p> |
| <p>其他的，还有映射 java 端的方法别名，主要的原因是 Go 这边语言的规约，需要被序列化的方法名必须是首字母大写。而 Java 这边没有这种规范，所以我们加了一个 hessian 标签的支持，可以允许用户手动映射 Java 端的方法名称。</p> |
| <p>基本上现在的 dubbo-go 已经满足绝大多数与 Java 的类型互通需求，我们近期也在实现对 Java 泛型的支持。</p> |
| <h5 id="dubbo-gogetty-1">dubbo-go/getty</h5> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-getty.png" alt="img"></p> |
| <p>Go 语言天生就是一个异步网络 I/O 模型，在 linux 上 Go 语言写的网络服务器也是采用的 epoll 作为最底层的数据收发驱动,这跟 java 在 linux 的 nio 实现是一样的。所以 Go 语言的网络处理天生就是异步的。我们需要封装的其实是基于 Go 的异步网络读写以及之后的处理中间层。getty 将网络数据处理分为三层，入向方向分别经过对网络 i/o 封装的 streaming 层、根据不同协议对数据进行序列化反序列化的 codec 层，以及最后数据上升到需要上层消费的 handler 层。出向方向基本与入向经过的相反。每个链接的 IO 协程是成对出现的，比如读协程负责读取、 codec 逻辑然后数据到 listener 层，然后最后的事件由业务协程池来处理。</p> |
| <p>该项目目前是与 dubbo-go 解耦出来的，所以大家如果有类似需求可以直接拿来用，目前已经有对于 tcp/udp/websocket 的支持。</p> |
| <h5 id="apachedubbo-go-1">apache/dubbo-go</h5> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-arch.png" alt="img"></p> |
| <p>dubbo-go 主项目，我们重构的这一版主要是基于 Dubbo 的分层代码设计，上图是 dubbo-go 的代码分层。基本上与 Java 版本 Dubbo 现有的分层一致，所以 dubbo－go 也继承了 Dubbo 的一些优良特性，比如整洁的代码架构、易于扩展、完善的服务治理功能。</p> |
| <p>我们携程这边，使用的是自己的注册中心，可以在 dubbo-go 扩展机制的基础上灵活扩展而无需去改动 dubbo-go 的源代码。</p> |
| <h3 id="dubbo-go-的功能介绍">dubbo-go 的功能介绍</h3> |
| <h5 id="dubbo-go-已实现功能">dubbo-go 已实现功能</h5> |
| <p>目前 dubbo-go 已经实现了 Dubbo 的常用功能（如负责均衡、集群策略、服务多版本多实现、服务多注册中心多协议发布、泛化调用、服务降级熔断等），其中服务注册发现已经支持 zookeeper/etcd/consul/nacos 主流注册中心。这里不展开详细介绍，目前 dubbo-go 支持的功能可以查看项目 readme 中的 feature list ，详情参考：<em><a href="https://github.com/apache/dubbo-go#feature-list">https://github.com/apache/dubbo-go#feature-list</a></em></p> |
| <p>目前社区正在开发中的功能，主要是早期用户使用过程中提出的一些需求，也是生产落地一些必需的需求，如监控、调用链跟踪以及服务路由、动态配置中心等更高级的服务治理需求。</p> |
| <h5 id="dubbo-go-功能介绍之泛化调用">dubbo-go 功能介绍之泛化调用</h5> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-generic-invoke.png" alt="img"></p> |
| <p>这里详细做几个重点功能的介绍。首先是泛化调用，如上图，这个也是社区同学提的需求。该同学公司内部有很多 Dubbo 服务，他们用 Go 做了一个 api gateway 网关，想要把 Dubbo 服务暴露成外网 http 接口。因为内部的 Dubbo 服务比较多，不可能每一个 Dubbo 服务都去做一个消费端接口去做适配，这样的话一旦服务端改动，客户端也要改。所以他这边的思路是做基于 dubbo-go 做泛化调用， api-gateway 解析出外网请求的地址，解析出想要调用的 Dubbo 服务的目标。基于dubbo-go consumer 泛化调用指定 service、method ，以及调用参数。</p> |
| <p>具体的原理是， dubbo-go 这边作为消费端，实际会通过本地 genericService.invoke 方法做代理，参数里面包含了 service name，method name ，还包含被调用目标 service 需要的参数类型、值等数据，这些数据后面会通过 dubbo-go-hession2 做转换，会将内容转化成 map 类型，经过网络发送到对应的 Java 服务端，然后 Java 那边是接收的 map 类型的参数，会自动反序列化成自己的 pojo 类型。这样就实现了 dubbo-go 作为客户端，泛化调用 Dubbo 服务端的目的。</p> |
| <h5 id="dubbo-go-功能介绍之降级熔断">dubbo-go 功能介绍之降级熔断</h5> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-curcuit-breaker.png" alt="img"></p> |
| <p>降级熔断这边是基于的是大家比较熟悉的 hystrix 的 Go 语言版本，基于 hystrix ，用户可以定义熔断规则和降级触发的代码段。降级熔断支持是作为一个独立的 dubbo-go filter ，可以灵活选择是否启用，如果不启用就可以在打包的时候不将依赖引入。Filter 层是 dubbo-go 中对于请求链路的一个责任链模式抽象，目前有许多功能都是基于动态扩展 filter 链来实现的，包括 trace、leastactive load balacne、log 等。降级熔断设计成一个服务调用端独立的filter可以灵活满足调用端视角对于微服务架构中“防雪崩“的服务治理需求。</p> |
| <h5 id="dubbo-go-功能介绍之动态配置">dubbo-go 功能介绍之动态配置</h5> |
| <p>关于动态配置中心， Dubbo 的 2.6 到 2.7 版本做了一个比较大的变化，从之前的 url 配置形式过渡到了支持配置中心 yaml 格式配置的形式，治理粒度也从单服务级别的配置支持到了应用级别的配置，不过在2.7版本中还是兼容 2.6 版本 url 形式进行服务配置的。dubbo-go 这边考虑到跟 Dubbo2.6 和 2.7 的互通性，同样支持 url 和配置文件方式的服务配置，同时兼容应用级别和服务级别的配置，跟 dubbo 保持一致，目前已经实现了zookeeper和apollo作为配置中心的支持。</p> |
| <h3 id="dubbo-go-roadmap-2019-2020">dubbo-go roadmap 2019-2020</h3> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-roadmap-2019.png" alt="img"></p> |
| <p>最后是大家比较关注的，社区关于 dubbo-go 2019 年下半年的计划，目前来看主要还是现有功能的补齐和一些问题的修复，我们的目标就是首先做到 Java 和 Go 在运行时的兼容互通和功能的一致，其次是查漏补缺 dubbo-go 作为一个完整 Go 语言微服务框架在功能上的可以改进之处。</p> |
| <p>另外值得关注的一点是，预计今年年底， dubbo-go 会发布一个支持 kubernetes 作为注册中心的扩展，积极拥抱云原生生态。关于云原生的支持，社区前期做了积极的工作，包括讨论关于 dubbo-go 与 Service Mesh 的关系以及在其中的定位，可以肯定的是， dubbo-go 将会配合 Dubbo 社区在 Service Mesh 方向的规划并扮演重要角色，我们初步预计会在明年给出与 Service Mesh开源社区项目集成的方案，请大家期待。</p> |
| <p>dubbo-go 社区目前属于快速健康成长状态，从捐赠给 Apache 后的不到3个月的时间里，吸引了大批量的活跃开发者和感兴趣的用户，欢迎各位同道在使用或者学习中遇到问题能够来社区讨论或者给予指正，也欢迎对 dubbo-go 有潜在需求或者对 dubbo-go 感兴趣的同道能加入到社区中。</p> |
| <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> |
| <iframe src="https://player.bilibili.com/player.html?aid=413770787&cid=210657864&page=1&as_wide=1&high_quality=1&danmaku=0" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title=""></iframe> |
| </div> |
| <h3 id="关于作者">关于作者</h3> |
| <p>何鑫铭，目前就职于携程，基础中台研发部技术专家，dubbo-go 共同发起人、主要作者，Apache Dubbo committer，关注互联网中台以及中间件领域。</p></description></item><item><title>Blog: 冲上云原生，Dubbo 发布 Go 版本</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/%E5%86%B2%E4%B8%8A%E4%BA%91%E5%8E%9F%E7%94%9Fdubbo-%E5%8F%91%E5%B8%83-go-%E7%89%88%E6%9C%AC/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/%E5%86%B2%E4%B8%8A%E4%BA%91%E5%8E%9F%E7%94%9Fdubbo-%E5%8F%91%E5%B8%83-go-%E7%89%88%E6%9C%AC/</guid><description> |
| <p>5 月 21 日，经过一年多的孵化，Apache Dubbo 从 Apache 软件基金会毕业，成为 Apache 顶级项目。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-tlp-twitter.jpg" alt="img"></p> |
| <p>Dubbo 是阿里于 2011 年开源的一款高性能 RPC 框架，在 Java 生态中具有不小的影响力。当初经历过一段被外界诟病的“停止维护”灰暗时光，后来在 2017 年 Dubbo 浪子回头，官方宣布重新重点维护。</p> |
| <p>重新启航的 Dubbo 将首要目标定位于重新激活社区，赢回开发者的信任，并且逐渐将 Dubbo 打造成一个国际化与现代化的项目，目前距离宣布重启已经过了一年半的时间。</p> |
| <p>在这个过程中，Dubbo 发布了多个版本，并逐渐从一个 RPC 框架向微服务生态系统转变，18 年年初 Dubbo 入驻 Apache 软件基金会孵化器，开始以 Apache 之道发展社区。</p> |
| <p>一年之后，Dubbo 在 Apache 孵化器中发布了重启维护以来的首个里程碑版本 2.7.0，添加了社区呼声很高的异步化支持，以及注册中心与配置中心分离等特性。</p> |
| <p>这期间 Dubbo 3.0 的开发工作也被提上了日程，今年 4 月中旬，官方正式公布了 Dubbo 3.0 的进度，此版本新特性包括支持 Filter 链的异步化、响应式编程、云原生/Service Mesh 方向的探索，以及与阿里内外融合。</p> |
| <p>然后，Dubbo 毕业了。毕业后的 Dubbo 近期有什么消息呢？生态还在发展，Dubbo 社区在前几日公开了 <a href="https://github.com/dubbo/awesome-dubbo/blob/master/slides/meetup/201905@beijing/DUBBO%20ROADMAP%202019.pdf">Dubbo Roadmap 2019</a>，计划在 2020 年 2 月份发布 Dubbo 3.0 正式版，感兴趣的同学可以详细查阅。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-roadmap-2019.jpg" alt="img"></p> |
| <p>而最近官方又<strong>宣布 Go 语言加入 Dubbo 生态</strong>，发布了 <a href="https://github.com/dubbo/go-for-apache-dubbo">dubbo-go</a> 项目。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dubbo-go-logo.jpg" alt="img"></p> |
| <p>在此之前 Dubbo 的跨语言可扩展性已经有一些实现，支持的语言包括 PHP、Node.js 与 Python，同时也基于标准 Java REST API - JAX-RS 2.0 实现了 REST 的调用支持，具体情况如下：</p> |
| <ul> |
| <li><strong>PHP</strong>：php-for-apache-dubbo，by 乐信，提供客户端和服务端</li> |
| <li><strong>Node.js</strong>：dubbo2.js，by 千米网，提供客户端</li> |
| <li><strong>Node.js</strong>：egg-dubbo-rpc，by 蚂蚁金服 egg 团队，提供客户端和服务端</li> |
| <li><strong>Python</strong>：py-client-for-apache-dubbo，by 千米网，提供客户端</li> |
| </ul> |
| <p>现在加入了 dubbo-go，Go 开发者也终于可以尝到 Dubbo 的滋味了。据悉，dubbo-go 项目将于<strong>本周完成往 Apache 软件基金会的迁移</strong>，作为 Apache Dubbo 顶级项目的子项目，届时 dubbo-go 项目的新地址也将变为：https://github.com/apache/dubbo-go。</p> |
| <p>关于项目的研发背景与具体技术细节等相关内容，我们第一时间采访了项目共同发起人，目前在携程基础中台研发部的何鑫铭。</p> |
| <p><em>OSCHINA</em>：dubbo-go 是什么，定位是什么，为什么做这个项目？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p><strong>dubbo-go 是 Dubbo 的完整 Go 语言实现</strong>。</p> |
| <p>我们知道 Dubbo 本身基于 Java，很多公司也都以 Java 开发为主，并且使用 Dubbo 作 RPC 或微服务开发框架。</p> |
| <p>而最近 Go 语言生态发展比较迅速，因其语言优势，我们已经有部门开始尝试使用 Go 开发一些新的项目，就会存在亟需解决的问题：</p> |
| <ul> |
| <li>如何实现 Go 项目和 Java &amp; Dubbo 项目的互通？</li> |
| <li>另外，Go 项目本身也有对 RPC 与微服务开发框架的诉求，如何解决？</li> |
| </ul> |
| <p>基于这两个问题，我们携程团队基于 dubbo-go 的早期项目，重构开发了更易于扩展且功能更加完善的 dubbo-go v1.0.0 版本，并贡献回了社区，它<strong>首要目的就是解决 Go 项目与 Java &amp; Dubbo 项目的互通问题，同时也为 Go 项目提供了一种 RPC 与微服务开发框架的选择</strong>。</p> |
| <p>dubbo-go 提供客户端与服务器端，目前 dubbo-go 社区作为 Dubbo 生态最活跃的社区之一，后面的定位需要配合 Dubbo 官方的要求与社区用户的需求。</p> |
| <p><em>OSCHINA</em>：我们知道 Dubbo 在 Java 生态上是有非常高的成就的，而目前 Go 生态本身也有一些知名的微服务框架，那 dubbo-go 之于 Go 生态，是否有与其它框架比拼的能力？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>我们最大的能力就是作为 Dubbo 的 Go 语言版本，打通了两种语言之间的 gap，<strong>让 Dubbo 更加贴近云原生</strong>，为开发者也提供了最大的灵活性，显著降低企业现有服务上云的成本，让企业在云原生时代多了一种选择。</p> |
| <p><em>OSCHINA</em>：Go 的特性有没有在 dubbo-go 中得到相应的体现？（比如 Go 的高并发是怎么从基于 Java 的 Dubbo 中改造到 dubbo-go 中的？）</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>我对于 Go 语言的认知是，首先学习成本比较小，相比于 Java 的学习成本，Go 语言更容易学习和上手。</p> |
| <p>其次 Go 在语言层面上，比如其 CSP 编程模型在高并发处理上的简单高效、轻量级协程的优势，相比较基于 JVM 的 Java 程序来说，基于 runtime 的 Go 程序瞬时启动能力等特性都吸引着很多开发者，这里就不详细阐述了。</p> |
| <p>最后就是作为云原生语言的优势，随着 Docker、k8s 与 Istio 等优秀项目的出现，云原生底层基本被 Go 语言统一了，相信企业在云原生模式下开发的日子已经不远了。我觉得 Go 语言的生态应该会越来越好，也会有越来越多的人使用它。</p> |
| <p>将基于 Java 的 Dubbo 引入到 Go 中，像前边讲的，dubbo-go 带来的优势就是可以快速融入云原生的领域。要说 Go 语言特性体现的话，可以参考一下 <strong>dubbo-go 中异步网络 I/O 模型的设计，这部分将 Go 语言轻量级协程的优势体现了出来</strong>。</p> |
| <p>这里也说一下 Go 语言不足的地方：</p> |
| <ul> |
| <li>Go 相对 Java 来说还是很年轻的语言，没有模板库可用，所以社区在编写并维护Hessian 2 协议库上付出了很高的开发成本；</li> |
| <li>比起 Java 的 try/catch 错误处理方式，Go 的 error 处理能力偏弱；</li> |
| <li>总体生态还是不如 Java，如没有像 Netty 一样的强有力网络 I/O 库。</li> |
| </ul> |
| <p>为什么提到这一点呢，因为 Dubbo 自身使用了 Netty 和 Hessian 2 协议官方 Java 库，而 dubbo-go 在开始做的时候这些都是没有的，这使得 <strong>dubbo-go 一路走来非常艰辛，但是社区最终都克服了，并且额外贡献了开源的 Getty 和 Hessian2 项目</strong>。</p> |
| <p>这里特别感谢 dubbo-go 社区早期的组织者于雨，项目的早期版本是 <strong>2016 年</strong>在其领导胡长城和同事刘畏三支持下开发的，他贡献的 Hessian2 和 Getty 项目，也为最新版本的 dubbo-go 打好了坚实的基础。</p> |
| <p><em>OSCHINA</em>：前不久 Dubbo 才宣布之后会在 3.0 中强调 Service Mesh ，这就是语言无关的了，那 dubbo-go 还有必要在这时候加入生态吗？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>Service Mesh 确实是微服务未来发展的的一个大方向，但是现阶段在国内大公司还没有看到非常成功的案例，很多中小公司自身微服务还未拆分完毕甚至于还未开始，目前 dubbo-go 社区优先解决这种类型企业微服务技术落地环节中遇到的问题，专注于补齐相关功能、优化整体性能和解决 bug。至于未来，我相信随着 Dubbo Mesh 在 Service Mesh 领域的探索，dubbo-go 肯定会跟进并扮演重要角色。</p> |
| <p><em>OSCHINA</em>：dubbo-go 与 Dubbo 的更新关系是怎么样的？是同步更新特性还是有自己的一些创新？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>我们现在发布的最新版本是 v1.0.0，我们在每一次 release 新的版本后，都会明确说明可以兼容的 Dubbo 版本。所以，dubbo-go 需要兼容对应 Dubbo 版本号的功能，会同步更新一些 Dubbo 特性。</p> |
| <p><em>OSCHINA</em>：新发布版本带来什么值得关注的特性？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>当前发布的 v1.0.0 版本支持的功能如下：</p> |
| <ul> |
| <li>角色：Consumer(√)、Provider(√)</li> |
| <li>传输协议：HTTP(√)、TCP(√)</li> |
| <li>序列化协议：JsonRPC v2(√)、Hessian v2(√)</li> |
| <li>注册中心：ZooKeeper(√)</li> |
| <li>集群策略：Failover(√)</li> |
| <li>负载均衡：Random(√)</li> |
| <li>过滤器：Echo Health Check(√)</li> |
| <li>extension 扩展机制</li> |
| </ul> |
| <p>dubbo-go v1.0.0 版本，主要由我和同在携程的同事<a href="https://github.com/fangyincheng">方银城</a>维护，社区成员<a href="https://github.com/u0x01">周子庆</a>与<a href="https://github.com/gaoxinge">高辛格</a>参与贡献，该版本沿用了 Dubbo 的代码分层解耦设计。Dubbo 2.6.x 的主要功能都会逐渐在 dubbo-go 中实现，包括 Dubbo 基于 SPI 的代码拓展机制，dubbo-go 也有对应的 extension 扩展机制与之对应。</p> |
| <p>我们在未来将逐渐推出目前可扩展模块的更多实现，如补齐更多的 Loadbalance 负载均衡、Cluster Strategy 集群策略实现（目前这些任务由社区伙伴主动认领，希望更多的 Go 语言爱好者朋友可以加入社区贡献）；又如云原生领域非常流行的 k8s，我们也将同步 Dubbo 的 roadmap，跟进 k8s 作为注册中心的支持，目前由社区成员<a href="https://github.com/NameHaibinZhang">张海彬</a>负责跟进。</p> |
| <p>当然广大开发者们也可以对这些模块接口进行新的实现，通过 extension 拓展，以完成自己的特殊需求而无需修改源代码。同时，我们非常欢迎开发者为社区贡献有用的拓展实现。</p> |
| <p>此版本解决了一大重点问题：与 <strong>Dubbo Java 版本互通的解决方案</strong>。我们将这部分提取出了 <a href="https://github.com/dubbogo/hessian2">Hessian2</a> 项目，该项目源自社区<a href="https://github.com/AlexStocks">于雨</a>的早期贡献，现在由社区成员<a href="https://github.com/wongoo">望哥</a>负责维护，<a href="https://github.com/u0x01">周子庆</a>与<a href="https://github.com/gaoxinge">高辛格</a>参与贡献。目前该项目已经完成了对 Java 大部分类型的兼容支持。大家也可以单独将该项目集成到自己的项目中，它的开源协议是 Apache-2.0。</p> |
| <p>另外一个比较重要的就是 <strong>dubbo-go 现在使用的 TCP 异步网络 I/O 库</strong>，该库也是基于于雨早期写的 Getty 项目，目前由社区的<a href="https://github.com/wongoo">望哥</a>与<a href="https://github.com/fangyincheng">方银城</a>负责维护，它同样也是 Apache-2.0 的开源协议。下一版本我们<strong>会针对 dubbo-go 和 Getty 的网络 I/O 与线程派发这一部分进行进一步优化</strong>。</p> |
| <p>除此之外，我们计划下一步支持 Dubbo 的另外几大重要功能，如：</p> |
| <ul> |
| <li>routing rule 路由规则(dubbo v2.6.x)</li> |
| <li>dynamic configuration 动态配置中心(dubbo v2.* 7.x)</li> |
| <li>metrics 指标与监控(dubbo v2.7.x)</li> |
| <li>trace 链路监控(dubbo ecos)</li> |
| </ul> |
| <p><em>OSCHINA</em>：目前项目的应用情况如何？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>dubbo-go 现在已经开始被一些企业尝试应用于 Go 语言应用融入企业已有 Java &amp; Dubbo 技术栈，以及搭建全新 Go 语言分布式应用等场景。比如中通快递内部 Go 调用 Java Dubbo 服务；作为携程 Go 语言应用的服务框架以及 Go、Java 应用互通。</p> |
| <p>具体的应用情况可以查看： <a href="https://github.com/dubbo/go-for-apache-dubbo/issues/2">https://github.com/dubbo/go-for-apache-dubbo/issues/2</a></p> |
| <p><em>OSCHINA</em>：接下来的演进方向是怎么样的？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>在 dubbo-go 迁往 Apache 软件基金会作为 Apache Dubbo 的子项目后，首先最重要的是<strong>性能的进一步优化</strong>，目前性能上虽然能够达到应用的生产级别要求，但我们觉得还没有发挥出 Go 语言的优势，还有比较大的优化空间。比如前边提到的 Getty，下一版本会针对 dubbo-go 应用 Getty 的网络 I/O 模型与线程派发做一些优化。</p> |
| <p>另外包含上面提到的我们近期需要补全一些重要功能，最大限度地在<strong>功能完整性</strong>上能够跟 Dubbo 兼容。关于未来 dubbo-go 的发展，也会向 Dubbo 2.7.x 版本这条线上的路线图演进。</p> |
| <p><em>OSCHINA</em>：说到性能，当前性能情况具体如何？</p> |
| <p><em>dubbo-go 何鑫铭</em>：</p> |
| <p>我们有做一个 <a href="https://github.com/dubbogo/go-for-apache-dubbo-benchmark">dubbo-go-benchmark</a> 项目，在 CPU 型号为 Intel(R) Xeon(R) CPU E5-2609 0 @2.40GHz，CPU 核心数为 4*8 的硬件水平下，发送 1k 并返回 1k 的数据，100 并发数，100w 总请求数，qps 可以达到 1.2 万左右。</p> |
| <p>CPU 性能换成比较高的配置如 Intel Core i9 2.9GHz，qps 可以到达 2 万左右。</p> |
| <p>我们后面会对 Hessian2 库和 Getty 库进行持续性能优化，以给广大使用者节约资源。</p> |
| <h4 id="采访嘉宾介绍">采访嘉宾介绍</h4> |
| <p><strong>何鑫铭</strong>，携程基础中台研发部技术专家，dubbo-go 主要作者。目前专注于 Golang &amp; Java、中台架构、中间件与区块链等技术。</p></description></item><item><title>Blog: Dubbo Go 快速开始</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B/</guid><description> |
| <h2 id="环境">环境</h2> |
| <ul> |
| <li>Go编程环境</li> |
| <li>启动zookeeper服务，也可以使用远程实例</li> |
| </ul> |
| <h2 id="从服务端开始">从服务端开始</h2> |
| <h3 id="第一步编写-provider-结构体和提供服务的方法">第一步：编写 <code>Provider</code> 结构体和提供服务的方法</h3> |
| <blockquote> |
| <p><a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-server/app/user.go">https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-server/app/user.go</a></p> |
| </blockquote> |
| <ol> |
| <li>编写需要被编码的结构体，由于使用 <code>Hessian2</code> 作为编码协议，<code>User</code> 需要实现 <code>JavaClassName</code> 方法，它的返回值在dubbo中对应User类的类名。</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> User <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> Id <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Name <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> Age <span style="color:#dc322f">int32</span> |
| </span></span><span style="display:flex;"><span> Time time.Time |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u User) <span style="color:#268bd2">JavaClassName</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;com.ikurento.user.User&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><ol> |
| <li>编写业务逻辑，<code>UserProvider</code> 相当于dubbo中的一个服务实现。需要实现 <code>Reference</code> 方法，返回值是这个服务的唯一标识，对应dubbo的 <code>beans</code> 和 <code>path</code> 字段。</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> UserProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u <span style="color:#719e07">*</span>UserProvider) <span style="color:#268bd2">GetUser</span>(ctx context.Context, req []<span style="color:#268bd2">interface</span>{}) (<span style="color:#719e07">*</span>User, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;req:%#v&#34;</span>, req) |
| </span></span><span style="display:flex;"><span> rsp <span style="color:#719e07">:=</span> User{<span style="color:#2aa198">&#34;A001&#34;</span>, <span style="color:#2aa198">&#34;hellowworld&#34;</span>, <span style="color:#2aa198">18</span>, time.<span style="color:#268bd2">Now</span>()} |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;rsp:%#v&#34;</span>, rsp) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>rsp, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u <span style="color:#719e07">*</span>UserProvider) <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><ol> |
| <li>注册服务和对象</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetProviderService</span>(<span style="color:#b58900">new</span>(UserProvider)) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// ------for hessian2------ |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="第二步编写主程序">第二步：编写主程序</h3> |
| <blockquote> |
| <p><a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-server/app/server.go">https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-server/app/server.go</a></p> |
| </blockquote> |
| <ol> |
| <li>引入必需的dubbo-go包</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> hessian <span style="color:#2aa198">&#34;github.com/apache/dubbo-go-hessian2&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/protocol&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common/proxy/proxy_factory&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/filter/impl&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/cluster_impl&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/loadbalance&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/protocol/dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span></code></pre></div><ol> |
| <li>main 函数</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="第三步编写配置文件并配置环境变量">第三步：编写配置文件并配置环境变量</h3> |
| <ol> |
| <li>参考 <a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-server/profiles/release/log.yml">log</a> 和 <a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-server/profiles/release/server.yml">server</a> 编辑配置文件。</li> |
| </ol> |
| <p>主要编辑以下部分：</p> |
| <ul> |
| <li><code>registries</code> 结点下需要配置zk的数量和地址</li> |
| <li><code>services</code> 结点下配置服务的具体信息，需要配置 <code>interface</code> 配置，修改为对应服务的接口名，服务的key对应第一步中 <code>Provider</code> 的 <code>Reference</code> 返回值</li> |
| </ul> |
| <ol> |
| <li>把上面的两个配置文件分别配置为环境变量</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">CONF_PROVIDER_FILE_PATH</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;xxx&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">APP_LOG_CONF_FILE</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;xxx&#34;</span> |
| </span></span></code></pre></div><h2 id="接着是客户端">接着是客户端</h2> |
| <h3 id="第一步编写客户端-provider">第一步：编写客户端 <code>Provider</code></h3> |
| <blockquote> |
| <p><a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/app/user.go">https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/app/user.go</a></p> |
| </blockquote> |
| <ol> |
| <li>参考服务端第一步的第一点。</li> |
| <li>与服务端不同的是，提供服务的方法作为结构体的参数，不需要编写具体业务逻辑。另外，<code>Provider</code> 不对应dubbo中的接口，而是对应一个实现。</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> UserProvider <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> GetUser <span style="color:#268bd2">func</span>(ctx context.Context, req []<span style="color:#268bd2">interface</span>{}, rsp <span style="color:#719e07">*</span>User) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (u <span style="color:#719e07">*</span>UserProvider) <span style="color:#268bd2">Reference</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><ol> |
| <li>注册服务和对象</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">SetConsumerService</span>(userProvider) |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="第二步编写客户端主程序">第二步：编写客户端主程序</h3> |
| <blockquote> |
| <p><a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/app/client.go">https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/app/client.go</a></p> |
| </blockquote> |
| <ol> |
| <li>引入必需的dubbo-go包</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> hessian <span style="color:#2aa198">&#34;github.com/apache/dubbo-go-hessian2&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/protocol&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common/proxy/proxy_factory&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/filter/impl&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/cluster_impl&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/loadbalance&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/protocol/dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span></code></pre></div><ol> |
| <li>main 函数</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> time.<span style="color:#268bd2">Sleep</span>(<span style="color:#2aa198">3e9</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;\n\n\nstart to test dubbo&#34;</span>) |
| </span></span><span style="display:flex;"><span> user <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>User{} |
| </span></span><span style="display:flex;"><span> err <span style="color:#719e07">:=</span> userProvider.<span style="color:#268bd2">GetUser</span>(context.<span style="color:#268bd2">TODO</span>(), []<span style="color:#268bd2">interface</span>{}{<span style="color:#2aa198">&#34;A001&#34;</span>}, user) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(err) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">println</span>(<span style="color:#2aa198">&#34;response result: %v\n&#34;</span>, user) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#b58900">println</span>(format <span style="color:#dc322f">string</span>, args <span style="color:#719e07">...</span><span style="color:#268bd2">interface</span>{}) { |
| </span></span><span style="display:flex;"><span> fmt.<span style="color:#268bd2">Printf</span>(<span style="color:#2aa198">&#34;\033[32;40m&#34;</span><span style="color:#719e07">+</span>format<span style="color:#719e07">+</span><span style="color:#2aa198">&#34;\033[0m\n&#34;</span>, args<span style="color:#719e07">...</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="第三步编写配置文件并配置环境变量-1">第三步：编写配置文件并配置环境变量</h3> |
| <ol> |
| <li>参考 <a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/profiles/release/log.yml">log</a> 和 <a href="https://github.com/dubbogo/dubbo-samples/blob/master/golang/helloworld/dubbo/go-client/profiles/release/client.yml">client</a> 编辑配置文件。</li> |
| </ol> |
| <p>主要编辑以下部分：</p> |
| <ul> |
| <li><code>registries</code> 结点下需要配置zk的数量和地址</li> |
| <li><code>references</code> 结点下配置服务的具体信息，需要配置 <code>interface</code> 配置，修改为对应服务的接口名，服务的key对应第一步中 <code>Provider</code> 的 <code>Reference</code> 返回值</li> |
| </ul> |
| <ol> |
| <li>把上面的两个配置文件费别配置为环境变量，为防止log的环境变量和服务端的log环境变量冲突，建议所有的环境变量不要做全局配置，在当前起效即可。</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">CONF_CONSUMER_FILE_PATH</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;xxx&#34;</span> |
| </span></span><span style="display:flex;"><span><span style="color:#b58900">export</span> <span style="color:#268bd2">APP_LOG_CONF_FILE</span><span style="color:#719e07">=</span><span style="color:#2aa198">&#34;xxx&#34;</span> |
| </span></span></code></pre></div></description></item><item><title>Blog: Dubbo Go 中 metrics 的设计</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E4%B8%AD-metrics-%E7%9A%84%E8%AE%BE%E8%AE%A1/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E4%B8%AD-metrics-%E7%9A%84%E8%AE%BE%E8%AE%A1/</guid><description> |
| <p>最近因为要在 Apache/dubbo-go（以下简称 dubbo-go ）里面实现类似的这个 metrics 功能，于是花了很多时间去了解现在 Dubbo 里面的 metrics 是怎么实现的。该部分，实际上是被放在一个独立的项目里面，即 |
| metrics ，见 <a href="https://github.com/flycash/dubbo-go/tree/feature/MetricsFilter">https://github.com/flycash/dubbo-go/tree/feature/MetricsFilter</a> 下 metrics 子目录。</p> |
| <p>总体上来说，Dubbo 的 metrics 是一个从设计到实现都非常优秀的模块，理论上来说，大部分的 Java 项目是可以直接使用 metrics 的。但也因为兼顾性能、扩展性等各种非功能特性，所以初看代码会有种无从下手的感觉。</p> |
| <p>今天这篇文章将会从比较大的概念和抽象上讨论一下 dubbo-go 中的 metrics 模块的设计——实际上也就是 Dubbo 中的 metrics 的设计。因为我仅仅是将 Dubbo 里面的相关内容在 dubbo-go 中复制一份。</p> |
| <p>目前 dubbo-go 的 metrics 刚刚开始起步，第一个 PR 是： <a href="https://github.com/apache/dubbo-go/pull/278">https://github.com/apache/dubbo-go/pull/278</a></p> |
| <h2 id="总体设计">总体设计</h2> |
| <h3 id="metrics">Metrics</h3> |
| <p>要想理解 metrics 的设计，首先要理解，我们需要收集一些什么数据。我们可以轻易列举出来在 RPC 领域里面我们所关心的各种指标，诸如每个服务的调用次数，响应时间；如果更加细致一点，还有各种响应时间的分布，平均响应时间，999线……</p> |
| <p>但是上面列举的是从数据的内容上划分的。 metrics 在抽象上，则是摒弃了这种划分方式，而是结合了数据的特性和表现形式综合划分的。</p> |
| <p>从源码里面很容易找到这种划分的抽象。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p1.png" alt="img"></p> |
| <p>metrics 设计了 Metric 接口作为所有数据的顶级抽象：</p> |
| <p>在 Dubbo 里面，其比较关键的子接口是：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p2.webp" alt="img"></p> |
| <p>为了大家理解，这里我抄一下这些接口的用途：</p> |
| <ul> |
| <li>Gauge: 一种实时数据的度量，反映的是瞬态的数据，不具有累加性，例如当前 JVM 的线程数；</li> |
| <li>Counter: 计数器型指标，适用于记录调用总量等类型的数据；</li> |
| <li>Histogram : 直方分布指标，例如，可以用于统计某个接口的响应时间，可以展示 50%, 70%, 90% 的请求响应时间落在哪个区间内；</li> |
| <li>Meter: 一种用于度量一段时间内吞吐率的计量器。例如，一分钟内，五分钟内，十五分钟内的qps指标；</li> |
| <li>Timer: Timer相当于Meter+Histogram的组合，同时统计一段代码，一个方法的qps，以及执行时间的分布情况；</li> |
| </ul> |
| <p>目前 dubbo-go 只实现了 FastCompass ，它也是 Metric 的子类：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p3.webp" alt="img"></p> |
| <p>这个接口功能很简单，就是用于收集一段时间之内的 subCategory 执行的次数和响应时间。 subCategory 是一个比较宽泛的概念，无论是在 Dubbo 还是在 dubbo-go 里面，一个典型的 subCategory |
| 就会是某个服务。</p> |
| <p>这里的设计要点在于，它是从什么角度上去做这些数据的抽象的。</p> |
| <p>很多人在开发这种采集数据的相关系统或者功能的时候，最容易陷入的就是从数据内容上做抽象，例如抽象一个接口，里面的方法就是获得服务的调用次数或者平均响应时间等。</p> |
| <p>这种抽象并非不可以，尤其是在简单系统里面，还非常好用。唯独在通用性和扩展性上要差很多。</p> |
| <h3 id="metricmanager">MetricManager</h3> |
| <p>在我们定义了 Metric 之后，很容易就想到，我要有一个东西来管理这些 Metric 。这就是 MetricManager ——对应到 Dubbo 里面的 IMetricManager 接口。</p> |
| <p>MetricManager 接口目前在 dubbo-go 里面还很简单：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p4.webp" alt="img"></p> |
| <p>本质上来说，我在前面提到的那些 Metric 的子类，都可以从这个 MetricManager 里面拿到。它是对外的唯一入口。</p> |
| <p>因此无论是上报采集的数据，还是某些功能要用这些采集的数据，最重要的就是获得一个 MetricManager 的实例。例如我们最近正在开发的接入 Prometheus 就是拿到这个 MetriManger 实例，而后从里面拿到 |
| FastCompass 的实例，而后采集这些数据：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p5.webp" alt="img"></p> |
| <h3 id="metricregistry">MetricRegistry</h3> |
| <p>MetricRegistry 是一个对 Metric 集合的抽象。 MetricManager 的默认实现里面，就是使用 MetricRegistry 来管理 Metric 的:</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p6.webp" alt="img"></p> |
| <p>所以，本质上它就是提供了一些注册 Metric 然后再从里面捞出来的方法。</p> |
| <p>于是，这就有一个问题了：为什么我在有了 MetricManager 之后，还有有一个MetricRegistry？似乎这两个功能有些重叠？</p> |
| <p>答案大概是两个方面：</p> |
| <p>1、除了管理所有的 Metric 之外，还承担着额外的功能，这些功能典型的就是 IsEnabled 。而实际上，在未来我们会赋予它管理生命周期的责任，比如说在 Dubbo 里面，该接口就还有一个 clear 方法；</p> |
| <p>2、 metrics 里面还有一个 group 的概念，而这只能由 MetricManager 来进行管理，至少交给 MetricRegistry 是不合适的。</p> |
| <p>metrics 的 group 说起来也很简单。比如在 Dubbo 框架里面采集的数据，都会归属于 Dubbo 这个 group 。也就是说，如果我想将非框架层面采集的数据——比如纯粹的业务数据——分隔出来，就可以借用一个 business |
| group 。又或者我采集到的机器自身的数据，可以将其归类到 system 这个 group 下。</p> |
| <p>所以 MetricManger 和 MetricRegistry 的关系是：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p7.webp" alt="img"></p> |
| <h3 id="clock">Clock</h3> |
| <p>Clock 抽象是一个初看没什么用，再看会觉得其抽象的很好。Clock 里面就两个方法：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p8.webp" alt="img"></p> |
| <p>一个是获得时间戳，另外一个则是获得时间周期(Tick)。比如通常采集数据可能是每一分钟采集一次，所以你得知道现在处在哪个时间周期里面。Clock 就提供了这种抽象。</p> |
| <p>很多人在实现自己的这种 metrics 的框架的时候，大多数都是直接使用系统的时钟，也就是系统的时间戳。于是所有的 Metic 在采集数据或者上报数据的时候，不得不自己去处理这种时钟方面的问题。</p> |
| <p>这样不同的 Metric 之间就很难做到时钟的同步。比如说可能在某个 Metric1 里面，采集周期是当前这一分钟，而 Metric2 是当前这一分钟的第三十秒到下一分钟的第三十秒。虽然它们都是一分钟采集一次，但是这个周期就对不上了。</p> |
| <p>另外一个有意思的地方在于，Clock 提供的这种抽象，允许我们不必真的按照现实时间的时间戳来处理。比如说，可以考虑按照 CPU 的运行时间来设计 Clock 的实现。</p> |
| <h2 id="例子">例子</h2> |
| <p>就用这一次 PR 的内容来展示一下这个设计。</p> |
| <p>在 dubbo-go 里面这次实现了 metricsFilter ，它主要就是收集调用次数和响应时间，其核心是：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p9.webp" alt="img"></p> |
| <p>report 其实就是把 metrics reports 给 MetricManager ：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p10.webp" alt="img"></p> |
| <p>所以，这里面可以看出来，如果我们要收集什么数据，也是要先获得 MetricManager 的实例。</p> |
| <p>FastCompass 的实现里面会将这一次调用的服务及其响应时间保存下来。而后在需要的时候再取出来。</p> |
| <p>所谓的需要的时候，通常就是上报给监控系统的时候。比如前面的提到的上报给 Prometheus。</p> |
| <p>所以这个流程可以抽象表达为：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p11.webp" alt="img"></p> |
| <p>这是一个更加宽泛的抽象。也就是意味着，我们除了可以从这个 metricFilter 里面收集数据，也可以从自身的业务里面去收集数据。比如说统计某段代码的执行时间，一样可以使用 FastCompass 。</p> |
| <p>而除了 Prometheus ，如果用户自己的公司里面有监控框架，那么他们可以自己实现自己的上报逻辑。而上报的数据则只需要拿到 MetricManager 实例就能拿到。</p> |
| <h2 id="总结">总结</h2> |
| <p>本质上来说，整个 metrics 可以看做是一个巨大无比的 provider-consumer 模型。</p> |
| <p>不同的数据会在不同的地方和不同时间点上被采集。有些人在读这些源码的时候会有点困惑，就是这些数据什么时间点会被采集呢？</p> |
| <p>它们只会在两类时间点采集：</p> |
| <p>1、实时采集。如我上面举例的 metricsFilter ，一次调用过来，它的数据就被采集了；</p> |
| <p>2、另外一个则是如同 Prometheus 。每次 Prometheus 触发了 collect 方法，那么它就会把每种（如 Meter, Gauge ）里面的数据收集过来，然后上报，可以称为是定时采集；</p> |
| <p>Dubbo 里面采集了非常多的数据：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/metrics/p12.webp" alt="img"></p> |
| <p>这些具体的实现，我就不一一讨论了，大家有兴趣可以去看看源码。这些数据，也是我们 dubbo-go 后面要陆续实现的东西，欢迎大家持续关注，或者来贡献代码。</p> |
| <h2 id="作者信息">作者信息</h2> |
| <p>邓明，毕业于南京大学，就职于 eBay Payment 部门，负责退款业务开发。</p></description></item><item><title>Blog: Dubbo Go 中的 TPS Limit 设计与实现</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E4%B8%AD%E7%9A%84-tps-limit-%E8%AE%BE%E8%AE%A1%E4%B8%8E%E5%AE%9E%E7%8E%B0/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E4%B8%AD%E7%9A%84-tps-limit-%E8%AE%BE%E8%AE%A1%E4%B8%8E%E5%AE%9E%E7%8E%B0/</guid><description> |
| <h1 id="前言">前言</h1> |
| <p><a href="https://links.jianshu.com/go?to=http%3A%2F%2Fdubbo.apache.org%2Fen-us%2F">Apache Dubbo</a>是由阿里开源的一个RPC框架，除了基本的RPC功能以外，还提供了一整套的服务治理相关功能。目前它已经是Apache基金会下的顶级项目。</p> |
| <p>而<a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fapache%2Fdubbo-go">dubbogo</a>则是dubbo的go语言实现。</p> |
| <p>最近在<code>dubbogo</code>的<code>todo list</code>上发现，它还没有实现<code>TPS Limit</code>的模块，于是就抽空实现了这个部分。</p> |
| <p><code>TPS limit</code>实际上就是限流，比如说限制一分钟内某个接口只能访问200次，超过这个次数，则会被拒绝服务。在<code>Dubbo</code>的Java版本上，只有一个实现，就是<code>DefaultTPSLimiter</code>。</p> |
| <p><code>DefaultTPSLimiter</code>是在服务级别上进行限流。虽然<code>dubbo</code>的官方文档里面声称可以在<code>method</code>级别上进行限流，但是我看了一下它的源码，实际上这个是做不到的。当然，如果自己通过实现<code>Filter</code>接口来实现<code>method</code>级别的限流，那么自然是可以的——这样暴露了<code>dubbo</code>Java版本实现的另外一个问题，就是<code>dubbo</code>的<code>TpsLimitFilter</code>实现，是不允许接入自己<code>TpsLimiter</code>的实现的。这从它的源码也可以看出来：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/tps-limit-filter.png" alt="img"></p> |
| <p>它直接写死了<code>TpsLimiter</code>的实现。</p> |
| <p>这个实现的目前只是合并到了<code>develop</code>上，等下次发布正式版本的时候才会发布出来。</p> |
| <p>Github: <a href="https://links.jianshu.com/go?to=https%3A%2F%2Fgithub.com%2Fapache%2Fdubbo-go%2Fpull%2F237">https://github.com/apache/dubbo-go/pull/237</a></p> |
| <h1 id="设计思路">设计思路</h1> |
| <p>于是我大概参考了一下<code>dubbo</code>已有的实现，做了一点改进。</p> |
| <p><code>dubbo</code>里面的核心抽象是<code>TpsLimiter</code>接口。<code>TpsLimitFilter</code>只是简单调用了一下这个接口的方法而已：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/tps-limiter.png" alt="img"></p> |
| <p>这个抽象是很棒的。但是还欠缺了一些抽象。</p> |
| <p>实际上，一个TPS Limit就要解决三个问题：</p> |
| <ol> |
| <li>对什么东西进行<code>limit</code>。比如说，对服务进行限流，或者对某个方法进行限流，或者对IP进行限流，或者对用户进行限流；</li> |
| <li>如何判断已经<code>over limitation</code>。这是从算法层面上考虑，即用什么算法来判断某个调用进来的时候，已经超过配置的上限了；</li> |
| <li>被拒绝之后该如何处理。如果一个请求被断定为已经<code>over limititation</code>了，那么该怎么处理；</li> |
| </ol> |
| <p>所以在<code>TpsLimiter</code>接口的基础上，我再加了两个抽象：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">type</span> TpsLimiter <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// IsAllowable will check whether this invocation should be enabled for further process |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">IsAllowable</span>(<span style="color:#719e07">*</span>common.URL, protocol.Invocation) <span style="color:#dc322f">bool</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">type</span> TpsLimitStrategy <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// IsAllowable will return true if this invocation is not over limitation |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">IsAllowable</span>() <span style="color:#dc322f">bool</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">type</span> RejectedExecutionHandler <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// RejectedExecution will be called if the invocation was rejected by some component. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">RejectedExecution</span>(url <span style="color:#719e07">*</span>common.URL, invocation protocol.Invocation) protocol.Result |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><code>TpsLimiter</code>对应到Java的<code>TpsLimiter</code>，两者是差不多。在我的设想里面，它既是顶级入口，还需要承担解决第一个问题的职责。</p> |
| <p>而<code>TpsLimitStrategy</code>则是第二个问题的抽象的接口定义。它代表的是纯粹的算法。该接口完全没有参数，实际上，所有的实现需要维护自身的状态——对于大部分实现而言，它大概只需要获取一下系统时间戳，所以不需要参数。</p> |
| <p>最后一个接口<code>RejectedExecutionHandler</code>代表的是拒绝策略。在<code>TpsLimitFilter</code>里面，如果它调用<code>TpsLimiter</code>的实现，发现该请求被拒绝，那么就会使用该接口的实现来获取一个返回值，返回给客户端。</p> |
| <h1 id="实现">实现</h1> |
| <p>其实实现没太多好谈的。不过有一些微妙的地方，我虽然在代码里面注释了，但是我觉得在这里再多说一点也是可以的。</p> |
| <p>首先提及的就是拒绝策略<code>RejectedExecutionHandler</code>，我就是提供了一种实现，就是随便log了一下，什么都没做。因为这个东西是强业务相关的，我也不能提供更加多的通用的实现。</p> |
| <h2 id="方法与服务双重支持的tpslimiter">方法与服务双重支持的TpsLimiter</h2> |
| <p><code>TpsLimiter</code>我只有一个实现，那就是<code>MethodServiceTpsLimiterImpl</code>。它就是根据配置，如果方法级别配置了参数，那么会在方法级别上进行限流。否则，如果在服务级别（ServiceKey）上有配置，那么会在服务级别进行限流。</p> |
| <p>举个最复杂的例子：服务A限制100，有四个方法，方法M1配置限制40，方法M2和方法M3无配置，方法M4配置限制-1：那么方法M1会单独限流40；M2和M3合并统计，被限制在100；方法M4则会被忽略。</p> |
| <p>用户可以配置具体的算法。比如说使用我接下来说的，我已经实现的三种实现。</p> |
| <h2 id="fixedwindow和threadsafefixedwindow">FixedWindow和ThreadSafeFixedWindow</h2> |
| <p><code>FixedWindow</code>直接对应到Java的<code>DefaultTpsLimiter</code>。它采用的是<code>fixed-window</code>算法：比如说配置了一分钟内只能调用100次。假如从00:00开始计时，那么00:00-01:00内，只能调用100次。只有到达01:00，才会开启新的窗口01:00-02:00。如图：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/fixed-window.png" alt="img"></p> |
| <p>Fixed-Window 实现</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// IsAllowable determines if the requests over the TPS limit within the interval. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// It is not thread-safe. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (impl <span style="color:#719e07">*</span>FixedWindowTpsLimitStrategyImpl) <span style="color:#268bd2">IsAllowable</span>() <span style="color:#dc322f">bool</span> { |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> current <span style="color:#719e07">:=</span> time.<span style="color:#268bd2">Now</span>().<span style="color:#268bd2">UnixNano</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> impl.timestamp<span style="color:#719e07">+</span>impl.interval &lt; current { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// it&#39;s a new window |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// if a lot of threads come here, the count will be set to 0 several times. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// so the return statement will be wrong. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> impl.timestamp = current |
| </span></span><span style="display:flex;"><span> impl.count = <span style="color:#2aa198">0</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// this operation is thread-safe, but count + 1 may be overflow |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">return</span> atomic.<span style="color:#268bd2">AddInt32</span>(<span style="color:#719e07">&amp;</span>impl.count, <span style="color:#2aa198">1</span>) <span style="color:#719e07">&lt;=</span> impl.rate |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>这里有一个很有意思的地方。就是这个实现，是一个几乎线程安全但是其实并不是线程安全的实现。</p> |
| <p>在所有的实现里面，它是最为简单，而且性能最高的。我在衡量了一番之后，还是没把它做成线程安全的。事实上，Java版本的也不是线程安全的。</p> |
| <p>它只会在多个线程通过第67行的检测之后，才会出现并发问题，这个时候就不是线程安全了。但是在最后的<code>return</code>语句中，那一整个是线程安全的。它因为不断计数往上加，所以多个线程同时跑到这里，其实不会有什么问题。</p> |
| <p>现在我要揭露一个最为奇诡的特性了：<strong>并发越高，那么这个<code>raise condition</code>就越严重，也就是说越不安全。</strong></p> |
| <p>但是从实际使用角度而言，有极端TPS的还是比较少的。对于那些TPS只有几百每秒的，是没什么问题的。</p> |
| <p><strong>为了保持和dubbo一致的特性，我把它作为默认的实现。</strong></p> |
| <p>此外，我还为它搞了一个线程安全版本，也就是<code>ThreadSafeFixedWindowTpsLimitStrategyImpl</code>，只是简单的用<code>sync</code>封装了一下，可以看做是一个<code>Decorator</code>模式的应用。</p> |
| <p>如果强求线程安全，可以考虑使用这个。</p> |
| <h2 id="slidingwindow">SlidingWindow</h2> |
| <p>这是我比较喜欢的实现。它跟网络协议里面的滑动窗口算法在理念上是比较接近的。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/sliding-window.png" alt="img"></p> |
| <p>具体来说，假如我设置的同样是一分钟1000次，它统计的永远是从当前时间点往前回溯一分钟内，已经被调用了多少次。如果这一分钟内，调用次数没超过1000，请求会被处理，如果已经超过，那么就会拒绝。</p> |
| <p>我再来描述一下，<code>SldingWindow</code>和<code>FixedWindow</code>两种算法的区别。这两者很多人会搞混。假如当前的时间戳是00:00，两个算法同时收到了第一个请求，开启第一个时间窗口。</p> |
| <p>那么<code>FixedWindow</code>就是00:00-01:00是第一个窗口，接下来依次是01:00-02:00, 02:00-03:00, &hellip;。当然假如说01:00之后的三十秒内都没有请求，在01:31又来了一个请求，那么时间窗口就是01:31-02:31。</p> |
| <p>而<code>SildingWindow</code>则没有这种概念。假如在01:30收到一个请求，那么<code>SlidingWindow</code>统计的则是00:30-01:30内有没有达到1000次。<strong>它永远计算的都是接收到请求的那一刻往前回溯一分钟的请求数量。</strong></p> |
| <p>如果还是觉得有困难，那么简单来说就是<code>FixedWindow</code>往后看一分钟，<code>SlidingWindow</code>回溯一分钟。</p> |
| <blockquote> |
| <p>这个说法并不严谨，只是为了方便理解。</p> |
| </blockquote> |
| <p>在真正写这个实现的时候，我稍微改了一点点：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// IsAllowable determins whether the number of requests within the time window overs the threshold |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// It is thread-safe. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (impl <span style="color:#719e07">*</span>SlidingWindowTpsLimitStrategyImpl) <span style="color:#268bd2">IsAllowable</span>() <span style="color:#dc322f">bool</span> { |
| </span></span><span style="display:flex;"><span> impl.mutex.<span style="color:#268bd2">Lock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">defer</span> impl.mutex.<span style="color:#268bd2">Unlock</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// quick path |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> size <span style="color:#719e07">:=</span> impl.queue.<span style="color:#268bd2">Len</span>() |
| </span></span><span style="display:flex;"><span> current <span style="color:#719e07">:=</span> time.<span style="color:#268bd2">Now</span>().<span style="color:#268bd2">UnixNano</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> size &lt; impl.rate { |
| </span></span><span style="display:flex;"><span> impl.queue.<span style="color:#268bd2">PushBack</span>(current) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// slow path |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> boundary <span style="color:#719e07">:=</span> current <span style="color:#719e07">-</span> impl.interval |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> timestamp <span style="color:#719e07">:=</span> impl.queue.<span style="color:#268bd2">Front</span>() |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// remove the element that out of the window |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#719e07">for</span> timestamp <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> <span style="color:#719e07">&amp;&amp;</span> timestamp.Value.(<span style="color:#dc322f">int64</span>) &lt; boundary { |
| </span></span><span style="display:flex;"><span> impl.queue.<span style="color:#268bd2">Remove</span>(timestamp) |
| </span></span><span style="display:flex;"><span> timestamp = impl.queue.<span style="color:#268bd2">Front</span>() |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> impl.queue.<span style="color:#268bd2">Len</span>() &lt; impl.rate { |
| </span></span><span style="display:flex;"><span> impl.queue.<span style="color:#268bd2">PushBack</span>(current) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">false</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>我用了一个队列来保存每次访问的时间戳。一般的写法，都是请求进来，先把已经不在窗口时间内的时间戳删掉，然后统计剩下的数量，也就是后面的<code>slow path</code>的那一堆逻辑。</p> |
| <p>但是我改了的一点是，我进来直接统计队列里面的数量——也就是请求数量，如果都小于上限，那么我可以直接返回<code>true</code>。即<code>quick path</code>。</p> |
| <p>这种改进的核心就是：我只有在检测到当前队列里面有超过上限数量的请求数量时候，才会尝试删除已经不在窗口内的时间戳。</p> |
| <p>这其实就是，是每个请求过来，我都清理一下队列呢？还是只有队列元素超出数量了，我才清理呢？我选择的是后者。</p> |
| <p>我认为这是一种改进……当然从本质上来说，整体开销是没有减少的——因为<code>golang</code>语言里面<code>List</code>的实现，一次多删除几个，和每次删除一个，多删几次，并没有多大的区别。</p> |
| <h3 id="算法总结">算法总结</h3> |
| <p>无论是<code>FixedWindow</code>算法还是<code>SlidingWindow</code>算法都有一个固有的缺陷，就是这个时间窗口难控制。</p> |
| <p>我们设想一下，假如说我们把时间窗口设置为一分钟，允许1000次调用。然而，在前十秒的时候就调用了1000次。在后面的五十秒，服务器虽然将所有的请求都处理完了，然是因为窗口还没到新窗口，所以这个时间段过来的请求，全部会被拒绝。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/busy-idle-time-window.png" alt="img"></p> |
| <p>解决的方案就是调小时间窗口，比如调整到一秒。但是时间窗口的缩小，会导致<code>FixedWindow</code>算法的<code>raise condition</code>情况加剧。<code>SlidingWindow</code>也会受影响，但是影响要小很多。</p> |
| <h2 id="那些没有实现的">那些没有实现的</h2> |
| <h3 id="基于特定业务对象的限流">基于特定业务对象的限流</h3> |
| <p>举例来说，某些特殊业务用的针对用户ID进行限流和针对IP进行限流，我就没有在<code>dubbogo</code>里面实现。有需要的可以通过实现<code>TpsLimiter</code>接口来完成。</p> |
| <h3 id="全局tps-limit">全局TPS limit</h3> |
| <p>这篇文章之前讨论的都是单机限流。如果全局限流，比如说针对某个客户，它购买的服务是每分钟调用100次，那么就需要全局限流——虽然这种case都不会用<code>Filter</code>方案，而是另外做一个<code>API</code>接入控制。</p> |
| <p>比如说，很常用的使用Redis进行限流的。针对某个客户，一分钟只能访问100次，那我就用客户ID做key，value设置成List，每次调用过来，随便塞一个值进去，设置过期时间一分钟。那么每次统计只需要统计当前key的存活的值的数量就可以了。</p> |
| <p>这种我也没实现，因为好像没什么需求。国内讨论TPS limit都是讨论单机TPS limit比较多。</p> |
| <p>这个同样可以通过实现<code>TpsLimiter</code>接口来实现。</p> |
| <h3 id="leaky-bucket算法">Leaky Bucket算法</h3> |
| <p>这个本来可以是<code>TpsLimitStrategy</code>的一种实现的。后来我觉得，它其实并没有特别大的优势——虽然号称可以做到均匀，但是其实并做不到真正的均匀。通过调整<code>SlidingWindow</code>的窗口大小，是可以接近它宣称的均匀消费的效果的。比如说调整到一秒，那其实就已经很均匀了。而这并不会带来多少额外的开销。</p></description></item><item><title>Blog: dubbo-go 中如何实现远程配置管理？</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E4%B8%AD%E5%A6%82%E4%BD%95%E5%AE%9E%E7%8E%B0%E8%BF%9C%E7%A8%8B%E9%85%8D%E7%BD%AE%E7%AE%A1%E7%90%86/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-%E4%B8%AD%E5%A6%82%E4%BD%95%E5%AE%9E%E7%8E%B0%E8%BF%9C%E7%A8%8B%E9%85%8D%E7%BD%AE%E7%AE%A1%E7%90%86/</guid><description> |
| <p>之前在 Apache/dubbo-go（以下简称 dubbo-go ）社区中，有同学希望配置文件不仅可以放于本地，还可以放于配置管理中心里。那么，放在本地和配置管理中心究竟有哪些不一样呢？</p> |
| <p>放在本地，每次更新需要重启，配置文件管理困难，无法做到实时更新即刻生效。此外，本地文件还依赖人工版本控制，在微服务的场景下，大大的增加了运维的成本与难度。</p> |
| <p>而配置管理中心提供了统一的配置文件管理，支持文件更新、实时同步、统一版本控制、权限管理等功能。</p> |
| <h2 id="目标">目标</h2> |
| <p>基于以上几个背景，可以总结出以下<strong>目标</strong></p> |
| <ul> |
| <li>与 Dubbo 现有的配置中心内的配置文件兼容，降低新增语言栈的学习成本；</li> |
| <li>支持多种配置文件格式；</li> |
| <li>支持主流配置中心，适应不一样的使用场景，实现高扩展的配置下发；</li> |
| </ul> |
| <h2 id="配置中心">配置中心</h2> |
| <p>配置中心在 dubbo-go 中主要承担以下场景的职责：</p> |
| <ol> |
| <li>作为外部化配置中心，即存储 dubbo.properties 配置文件，此时，key 值通常为文件名如 dubbo.properties , value 则为配置文件内容。</li> |
| <li>存储单个配置项，如各种开关项、常量值等。</li> |
| <li>存储服务治理规则，此时 key 通常按照 “服务名 + 规则类型” 的格式来组织，而 value 则为具体的治理规则。</li> |
| </ol> |
| <p>就目前而言，dubbo-go 首要支持的是 Dubbo 中支持的开源配置中心，包括：</p> |
| <ol> |
| <li>Apollo：携程框架部门研发的分布式配置中心，能够集中化管理应用不同环境、不同集群的配置，配置修改后能够实时推送到应用端，并且具备规范的权限、流程治理等特性，适用于微服务配置管理场景。</li> |
| <li>ZooKeeper：一个分布式的，开放源码的分布式应用程序协调服务，是 Google 的 Chubby 一个开源的实现，是 Hadoop 和 Hbase 的重要组件。它是一个为分布式应用提供一致性服务的软件，提供的功能包括：配置维护、域名服务、分布式同步、组服务等。</li> |
| <li>Nacos: Alibaba 开源的配置管理组件，提供了一组简单易用的特性集，帮助您实现动态服务发现、服务配置管理、服务及流量管理。</li> |
| </ol> |
| <p>而考虑到某些公司内部有自身的研发的配置中心，又或者当前流行而 Dubbo 尚未支持的配置中心，如 etcd，我们的核心在于设计一套机制，允许我们，也包括用户，可以通过扩展接口新的实现，来快速接入不同的配置中心。</p> |
| <p>那在 dubbo-go 中究竟怎么实现呢？我们的答案是：<strong>基于动态的插件机制在启动时按需加载配置中心的不同实现。</strong></p> |
| <p>实现该部分功能放置于一个独立的子项目中，见： <a href="https://github.com/apache/dubbo-go/tree/master/config_center">https://github.com/apache/dubbo-go/tree/master/config_center</a></p> |
| <h3 id="dubbo-go-设计">dubbo-go 设计</h3> |
| <p>原逻辑为：启动时读取本地配置文件，将其加载进内存，通过配置文件中的配置读取注册中心的信息获取服务提供者，注册服务消费者。</p> |
| <p>有些读者会有点困惑，不是说好了使用配置中心的，为什么现在又要读取本地配置呢？答案就是，读取的这部分信息分成两部分：</p> |
| <ul> |
| <li>使用什么作为配置中心；</li> |
| <li>该配置中心的元数据，比如说使用 zookeeper 作为配置中心，那么 zookeeper 的链接信息就是元数据，毕竟我们只有在知道了链接信息之后才能连上 zookeeper；</li> |
| </ul> |
| <p>在改造的时候，需要考虑以下的问题：</p> |
| <p><strong>1、如何实现支持多个配置中心？如何实现按需加载？</strong></p> |
| <p>通过抽象 DynamicConfiguration 让开发者可以快速支持多个配置中心。使用者导入指定的组件包后，在启动阶段将需要的组件加载进内存中，以便给程序按需调用，如下图绿色部分。</p> |
| <p><strong>2、配置中心的配置加载阶段在什么时候？</strong></p> |
| <p>应在读取配置文件阶段后，读取并解析本地配置文件中配置中心信息。初始化配置中心链接，读取 /dubbo/config/dubbo/dubbo.properties 与 /dubbo/config/dubbo/应用名/dubbo.properties ，并将其加载到内存之中覆盖原有配置，监听其变更，实时更新至内存，如下图蓝色部分: |
| <img src="https://dubbo.apache.org/imgs/blog/dubbo-go/config-center/config-center-class.jpg" alt="img"></p> |
| <h4 id="configcenterfactory">ConfigCenterFactory</h4> |
| <p>使用者加载对应配置中心模块后，在初始化阶段加入各配置中心模块往其中注册其初始化类。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#719e07">package</span> extension |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config_center&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">var</span> ( |
| </span></span><span style="display:flex;"><span> configCenterFactories = <span style="color:#b58900">make</span>(<span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]<span style="color:#268bd2">func</span>() config_center.DynamicConfigurationFactory) |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// SetConfigCenterFactory sets the DynamicConfigurationFactory with @name |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">SetConfigCenterFactory</span>(name <span style="color:#dc322f">string</span>, v <span style="color:#268bd2">func</span>() config_center.DynamicConfigurationFactory) { |
| </span></span><span style="display:flex;"><span> configCenterFactories[name] = v |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// GetConfigCenterFactory finds the DynamicConfigurationFactory with @name |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">GetConfigCenterFactory</span>(name <span style="color:#dc322f">string</span>) config_center.DynamicConfigurationFactory { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> configCenterFactories[name] <span style="color:#719e07">==</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#b58900">panic</span>(<span style="color:#2aa198">&#34;config center for &#34;</span> <span style="color:#719e07">+</span> name <span style="color:#719e07">+</span> <span style="color:#2aa198">&#34; is not existing, make sure you have import the package.&#34;</span>) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> configCenterFactories[name]() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h4 id="dynamicconfigurationfactory">DynamicConfigurationFactory</h4> |
| <p>整个动态配置中心的关键点就在 DynamicConfigurationFactory 上，其中通过解析内部自定义的 URL ，获取其协议类型，反射其参数，用于创建配置中心的链接。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#719e07">package</span> config_center |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// DynamicConfigurationFactory gets the DynamicConfiguration |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> DynamicConfigurationFactory <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">GetDynamicConfiguration</span>(<span style="color:#719e07">*</span>common.URL) (DynamicConfiguration, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>如：</p> |
| <p>配置文件中配置：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">config_center</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: zookeeper |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">127.0.0.1</span>:<span style="color:#2aa198">2181</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">namespace</span>: test |
| </span></span></code></pre></div><p>dubbo-go 内部会解析为：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>zookeeper://127.0.0.1:2181?namespace=test |
| </span></span></code></pre></div><p>在内部传递，用于初始化配置中心链接。</p> |
| <p><strong>PS:</strong> 在 dubbo-go 中到处可见这种内部协议，透彻理解这个内部协议对阅读 dubbo-go 代码很有帮助。</p> |
| <h4 id="dynamicconfiguration">DynamicConfiguration</h4> |
| <p>该接口规定了各个配置中心需要实现的功能：</p> |
| <ul> |
| <li>配置数据反序列化方式：目前只有 Properties 转换器，参见：DefaultConfigurationParser 。</li> |
| <li>增加监听器：用于增加监听数据变化后增加特定逻辑（受限于配置中心 client 端实现）。</li> |
| <li>删除监听器：删除已有监听器（受限于配置中心 client 端实现，目前所知 nacos client 没有提供该方法）。</li> |
| <li>获取路由配置：获取路由表配置。</li> |
| <li>获取应用级配置：获取应用层级配置，如：协议类型配置等。</li> |
| </ul> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#586e75">// DynamicConfiguration for modify listener and get properties file |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> DynamicConfiguration <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Parser</span>() parser.ConfigurationParser |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">SetParser</span>(parser.ConfigurationParser) |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">AddListener</span>(<span style="color:#dc322f">string</span>, ConfigurationListener, <span style="color:#719e07">...</span>Option) |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">RemoveListener</span>(<span style="color:#dc322f">string</span>, ConfigurationListener, <span style="color:#719e07">...</span>Option) |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// GetProperties get properties file |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">GetProperties</span>(<span style="color:#dc322f">string</span>, <span style="color:#719e07">...</span>Option) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// GetRule get Router rule properties file |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">GetRule</span>(<span style="color:#dc322f">string</span>, <span style="color:#719e07">...</span>Option) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// GetInternalProperty get value by key in Default properties file(dubbo.properties) |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">GetInternalProperty</span>(<span style="color:#dc322f">string</span>, <span style="color:#719e07">...</span>Option) (<span style="color:#dc322f">string</span>, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// PublishConfig will publish the config with the (key, group, value) pair |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">PublishConfig</span>(<span style="color:#dc322f">string</span>, <span style="color:#dc322f">string</span>, <span style="color:#dc322f">string</span>) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// RemoveConfig will remove the config white the (key, group) pair |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">RemoveConfig</span>(<span style="color:#dc322f">string</span>, <span style="color:#dc322f">string</span>) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// GetConfigKeysByGroup will return all keys with the group |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">GetConfigKeysByGroup</span>(group <span style="color:#dc322f">string</span>) (<span style="color:#719e07">*</span>gxset.HashSet, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h3 id="实现">实现</h3> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/config-center/design.png" alt="img"></p> |
| <p>优先考虑与现有 Dubbo 设计兼容，从而降低使用者的学习成本，dubbo-admin 作为服务提供者实现应用级配置管理， dubbo-go 作为消费端实现配置下发管理功能。下面以 ZooKeeper 为例，对服务提供者与服务消费者进行整体流程分析。</p> |
| <h4 id="如何存储配置管理">如何存储配置管理</h4> |
| <p>dubbo-admin 配置管理中增加 global 配置，ZooKeeper 中会自动生成其对应配置节点，内容均为 dubbo-admin 中设置的配置。</p> |
| <ol> |
| <li>/dubbo/config/dubbo/dubbo.properties 对应全局配置文件。</li> |
| <li>/dubbo/config/dubbo/ 应用名 /dubbo.properties 对应指定应用配置文件。</li> |
| </ol> |
| <h5 id="节点路径">节点路径</h5> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/config-center/key-struct.png" alt="img"></p> |
| <p>上图展示了 dubbo.properties 文件在 ZooKeeper 和 Apollo 中的存储结构：</p> |
| <p><strong>ZooKeeper</strong></p> |
| <ul> |
| <li>命名空间 namespace 都为：Dubbo</li> |
| <li>分组 group ：全局级别为 dubbo , 所有应用共享；应用级别为应用名 demo-provider ，只对该应用生效</li> |
| <li>key : dubbo.properties</li> |
| </ul> |
| <p><strong>Apollo</strong></p> |
| <ul> |
| <li>app_id : 自由指定，默认：dubbo ，最好与 zookeeper namespace 一致</li> |
| <li>cluster : 自由指定，最好与 zookeeper group 一致</li> |
| <li>命名空间 namespace : dubbo.properties</li> |
| </ul> |
| <p>ZooKeeper 与 Apollo 最大的不一样就在于 dubbo.properties 所在的节点。</p> |
| <h4 id="实现配置管理中心支持">实现配置管理中心支持</h4> |
| <p>以 Apollo 为例，简单的介绍，如何实现支持一个新的配置管理中心。</p> |
| <h5 id="选择配置管理中心-client--sdk">选择配置管理中心 Client / SDK</h5> |
| <p>本例中使用的 Apollo Go Client 为：https://github.com/zouyx/agollo 。</p> |
| <p><strong>PS:</strong> 如没找到，自己实现也是可以的哦。</p> |
| <h5 id="节点路径-1">节点路径</h5> |
| <p>因为每个配置管理中心的存储结构各有特点，导致 Dubbo 在使用外部配置管理中心时，存储配置节点的结构不一样。在 dubbo-configcenter 找到希望支持的配置管理中心，而本例中 Apollo 则在 ApolloDynamicConfiguration.java 。</p> |
| <p>注释中表明，Apollo namespace = governance (governance .properties) 用于治理规则，namespace = dubbo (dubbo.properties) 用于配置文件。</p> |
| <h5 id="实现-dynamicconfiguration">实现 DynamicConfiguration</h5> |
| <p>新建创建客户端方法，最好客户端保持为单例。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">newApolloConfiguration</span>(url <span style="color:#719e07">*</span>common.URL) (<span style="color:#719e07">*</span>apolloConfiguration, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> c <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>apolloConfiguration{ |
| </span></span><span style="display:flex;"><span> url: url, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> configAddr <span style="color:#719e07">:=</span> c.<span style="color:#268bd2">getAddressWithProtocolPrefix</span>(url) |
| </span></span><span style="display:flex;"><span> configCluster <span style="color:#719e07">:=</span> url.<span style="color:#268bd2">GetParam</span>(constant.CONFIG_CLUSTER_KEY, <span style="color:#2aa198">&#34;&#34;</span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> appId <span style="color:#719e07">:=</span> url.<span style="color:#268bd2">GetParam</span>(constant.CONFIG_APP_ID_KEY, <span style="color:#2aa198">&#34;&#34;</span>) |
| </span></span><span style="display:flex;"><span> namespaces <span style="color:#719e07">:=</span> <span style="color:#268bd2">getProperties</span>(url.<span style="color:#268bd2">GetParam</span>(constant.CONFIG_NAMESPACE_KEY, cc.DEFAULT_GROUP)) |
| </span></span><span style="display:flex;"><span> c.appConf = <span style="color:#719e07">&amp;</span>config.AppConfig{ |
| </span></span><span style="display:flex;"><span> AppID: appId, |
| </span></span><span style="display:flex;"><span> Cluster: configCluster, |
| </span></span><span style="display:flex;"><span> NamespaceName: namespaces, |
| </span></span><span style="display:flex;"><span> IP: configAddr, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> agollo.<span style="color:#268bd2">InitCustomConfig</span>(<span style="color:#268bd2">func</span>() (<span style="color:#719e07">*</span>config.AppConfig, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> c.appConf, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span> }) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> c, agollo.<span style="color:#268bd2">Start</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>以下为必须实现的方法，以下方法用于获取配置中心配置。</p> |
| <ul> |
| <li>GetInternalProperty：在配置文件（Apollo 为 namespace）中，根据 key 获取对应 value；</li> |
| <li>GetRule：获取治理配置文件（Apollo 为 namespace）；</li> |
| <li>GetProperties：获取整个配置文件（Apollo 为 namespace）；</li> |
| </ul> |
| <p>可选择实现的方法，如不实现，则不能动态更新 dubbo-go 中配置信息。</p> |
| <ul> |
| <li>RemoveListener</li> |
| <li>AddListener</li> |
| </ul> |
| <p>而 Parser &amp; SetParser 使用默认实现即可，默认为 Properties 转换器。</p> |
| <p>更多信息，参考：dubbo-go-apollo ，详情参考： <a href="https://github.com/apache/dubbo-go/tree/release-1.5/config_center/apollo">https://github.com/apache/dubbo-go/tree/release-1.5/config_center/apollo</a></p> |
| <h3 id="使用方法">使用方法</h3> |
| <p>从上面的设计里面，也能大概猜到怎么使用了：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/config-center/zookeeper-usercase.png" alt="img"></p> |
| <p>很显然，使用配置中心并不复杂，只需要把对应的依赖引入进来。在包初始化的时候，会创建出来对应的配置中心的实现。比如说加载 ZooKeeper 或者 Apollo 作为配置中心：</p> |
| <p><strong>ZooKeeper</strong></p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span>_ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config_center/zookeeper&#34;</span> |
| </span></span></code></pre></div><p><strong>Apollo</strong></p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-golang" data-lang="golang"><span style="display:flex;"><span>_ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config_center/apollo&#34;</span> |
| </span></span></code></pre></div><p>当然仅仅加载还不够，比如说虽然我加载了 zookeeper，但是我还需要知道怎么连上这个配置中心，即前面提到的配置中心的元数据，这部分信息是需要在本地配置出来的。比如说：</p> |
| <p><strong>ZooKeeper</strong></p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">config_center</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;127.0.0.1:2181&#34;</span> |
| </span></span></code></pre></div><p><strong>Apollo</strong></p> |
| <p>如果需要使用 Apollo 作为配置中心，请提前创建 namespace: dubbo.properties，用于配置管理。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">config_center</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol</span>: <span style="color:#2aa198">&#34;apollo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;127.0.0.1:8070&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">app_id</span>: test_app |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: dev |
| </span></span></code></pre></div><h2 id="总结">总结</h2> |
| <p>更加具体的实现，我就不详细论述，大家可以去看源码，欢迎大家持续关注，或者贡献代码。</p> |
| <p>整个配置中心的功能，麻雀虽小，但五脏俱全。目前并不算是十分完善，但是整个框架层面上来说，是走在了正确的路上。从扩展性来说，是比较便利。目前支持的配置中心还不够丰富，只有 ZooKeeper 与 Apollo ，支持的配置文件格式也只有 properties ，虽然能满足基本使用场景，距离完善还有还长远的路。</p> |
| <p><strong>未来计划：</strong></p> |
| <ul> |
| <li>Nacos（等待发布 ）</li> |
| <li>etcd（正在开发）</li> |
| <li>consul（未支持）</li> |
| <li>丰富的文件配置格式，如：yml , xml 等</li> |
| </ul> |
| <p><strong>本文作者：</strong> 邹毅贤，Github ID @zouyx，开源爱好者，就职于 SheIn 供应链部门，负责供应链开放平台</p></description></item><item><title>Blog: Dubbo Go Getty 开发日志</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-getty-%E5%BC%80%E5%8F%91%E6%97%A5%E5%BF%97/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/dubbo-go-getty-%E5%BC%80%E5%8F%91%E6%97%A5%E5%BF%97/</guid><description> |
| <h3 id="0-说明">0 说明</h3> |
| <p>[getty][3]是一个go语言实现的网络层引擎，可以处理TCP/UDP/websocket三种网络协议。</p> |
| <p>2016年6月我在上海做一个即时通讯项目时，接口层的底层网络驱动是当时的同事<a href="https://github.com/sanbit">sanbit</a>写的，原始网络层实现了TCP |
| Server，其命名规范学习了著名的netty。当时这个引擎比较简洁，随着我对这个项目的改进这个网络层引擎也就随之进化了（添加了TCP Client、抽象出了 TCP connection 和 TCP |
| session），至2016年8月份（又添加了websocket）其与原始实现已经大异其趣了，征得原作者和相关领导同意后就放到了github上。</p> |
| <p>将近两年的时间我不间断地对其进行改进，年齿渐增但记忆速衰，觉得有必要记录下一些开发过程中遇到的问题以及解决方法，以备将来回忆之参考。</p> |
| <h3 id="1-udp-connection">1 UDP connection</h3> |
| <p>2018年3月5日 起给 getty 添加了UDP支持。</p> |
| <h4 id="11-udp-connect">1.1 UDP connect</h4> |
| <p>UDP自身分为unconnected UDP和connected UDP两种，connected UDP的底层原理见下图。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/connected_udp_socket.gif" alt="img"></p> |
| <p>当一端的UDP endpoint调用connect之后，os就会在内部的routing table上把udp socket和另一个endpoint的地址关联起来，在发起connect的udp |
| endpoint端建立起一个单向的连接四元组：发出的datagram packet只能发往这个endpoint（不管sendto的时候是否指定了地址）且只能接收这个endpoint发来的udp datagram |
| packet（如图???发来的包会被OS丢弃）。</p> |
| <p>UDP endpoint发起connect后，OS并不会进行TCP式的三次握手，操作系统共仅仅记录下UDP socket的peer udp endpoint 地址后就理解返回，仅仅会核查对端地址是否存在网络中。</p> |
| <p>至于另一个udp endpoint是否为connected udp则无关紧要，所以称udp connection是单向的连接。如果connect的对端不存在或者对端端口没有进程监听，则发包后对端会返回ICMP “port |
| unreachable” 错误。</p> |
| <p>如果一个POSIX系统的进程发起UDP write时没有指定peer UDP address，则会收到ENOTCONN错误，而非EDESTADDRREQ。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/dns_udp.gif" alt="img"></p> |
| <p>一般发起connect的为 UDP client，典型的场景是DNS系统，DNS client根据/etc/resolv.conf里面指定的DNS server进行connect动作。</p> |
| <p>至于 UDP server 发起connect的情形有 TFTP，UDP client 和 UDP server 需要进行长时间的通信， client 和 server 都需要调用 connect 成为 connected UDP。</p> |
| <p>如果一个 connected UDP 需要更换 peer endpoint address，只需要重新 connect 即可。</p> |
| <h4 id="12-connected-udp-的性能">1.2 connected UDP 的性能</h4> |
| <p>connected UDP 的优势详见参考文档1。假设有两个 datagram 需要发送，unconnected UDP 的进行 write 时发送过程如下：</p> |
| <ul> |
| <li>Connect the socket</li> |
| <li>Output the first datagram</li> |
| <li>Unconnect the socket</li> |
| <li>Connect the socket</li> |
| <li>Output the second datagram</li> |
| <li>Unconnect the socket</li> |
| </ul> |
| <p>每发送一个包都需要进行 connect，操作系统到 routine table cache 中判断本次目的地地址是否与上次一致，如果不一致还需要修改 routine table。</p> |
| <p>connected UDP 的两次发送过程如下：</p> |
| <ul> |
| <li>Connect the socket</li> |
| <li>Output first datagram</li> |
| <li>Output second datagram</li> |
| </ul> |
| <p>这个 case 下，内核只在第一次设定下虚拟链接的 peer address，后面进行连续发送即可。所以 connected UDP 的发送过程减少了 1/3 的等待时间。</p> |
| <p>2017年5月7日 我曾用 python 程序(<code>https://github.com/alexStocks/python-practice/blob/master/tcp_udp_http_ws/udp/client.py</code>) |
| 对二者之间的性能做过测试，如果 client 和 server 都部署在本机，测试结果显示发送 100 000 量的 UDP datagram packet 时，connected UDP 比 unconnected UDP 少用了 2 |
| / 13 的时间。</p> |
| <p>这个测试的另一个结论是：不管是 connected UDP 还是 unconnected UDP，如果启用了 SetTimeout，则会增大发送延迟。</p> |
| <h4 id="13-go-udp">1.3 Go UDP</h4> |
| <p>Go 语言 UDP 编程也对 connected UDP 和 unconnected UDP 进行了明确区分，参考文档2 详细地列明了如何使用相关 |
| API，根据这篇文档个人也写一个 程序(<code>https://github.com/alexstocks/go-practice/blob/master/udp-tcp-http/udp/connected-udp.go</code>) 测试这些 |
| API，测试结论如下：</p> |
| <ul> |
| <li>connected UDP 读写方法是 Read 和 Write；</li> |
| <li>unconnected UDP 读写方法是 ReadFromUDP 和 WriteToUDP（以及 ReadFrom 和 WriteTo)；</li> |
| <li>unconnected UDP 可以调用 Read，只是无法获取 peer addr；</li> |
| <li>connected UDP 可以调用 ReadFromUDP（填写的地址会被忽略）</li> |
| <li>connected UDP 不能调用 WriteToUDP，”即使是相同的目标地址也不可以”，否则会得到错误 “use of WriteTo with pre-connected connection”；</li> |
| <li>unconnected UDP 不能调用 Write, “因为不知道目标地址”, error:”write: destination address requiredsmallnestMBP:udp smallnest”；</li> |
| <li>connected UDP 可以调用 WriteMsgUDP，但是地址必须为 nil；</li> |
| <li>unconnected UDP 可以调用 WriteMsgUDP，但是必须填写 peer endpoint address。</li> |
| </ul> |
| <p>综上结论，读统一使用 ReadFromUDP，写则统一使用 WriteMsgUDP。</p> |
| <h4 id="14-getty-udp">1.4 Getty UDP</h4> |
| <p>版本 v0.8.1 Getty 中添加 connected UDP 支持时，其连接函数 dialUDP(<code>https://github.com/alexstocks/getty/blob/master/client.go#L141</code>) |
| 这是简单调用了 net.DialUDP 函数，导致昨日（20180318 22:19 pm）测试的时候遇到一个怪现象：把 peer UDP endpoint 关闭，local udp endpoint 进行 connect 时 |
| net.DialUDP 函数返回成功，然后 lsof 命令查验结果时看到确实存在这个单链接：</p> |
| <pre><code>COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME |
| echo_clie 31729 alex 9u IPv4 0xa5d288135c97569d 0t0 UDP localhost:63410-&gt;localhost:10000 |
| </code></pre> |
| <p>然后当 net.UDPConn 进行 read 动作的时候，会得到错误 “read: connection refused”。</p> |
| <p>于是模仿C语言中对 TCP client connect 成功与否判断方法，对 dialUDP(<code>https://github.com/alexstocks/getty/blob/master/client.go#L141</code>) 改进如下：</p> |
| <ul> |
| <li>net.DialUDP 成功之后，判断其是否是自连接，是则退出；</li> |
| <li>connected UDP 向对端发送一个无用的 datagram packet【”ping”字符串，对端会因其非正确 datagram 而丢弃】，失败则退出；</li> |
| <li>connected UDP 发起读操作，如果对端返回 “read: connection refused” 则退出，否则就判断为 connect 成功。</li> |
| </ul> |
| <h3 id="2-compression">2 Compression</h3> |
| <p>去年给 getty 添加了 TCP/Websocket compression 支持，Websocket |
| 库使用的是 <a href="https://github.com/gorilla/websocket/">gorilla/websocket</a>，<a href="https://godoc.org/golang.org/x/net/websocket">Go |
| 官网</a>也推荐这个库，因为自 <code>This package(&quot;golang.org/x/net/websocket&quot;) currently lacks some features</code> |
| 。</p> |
| <h4 id="21-tcp-compression">2.1 TCP compression</h4> |
| <p>最近在对 Websocket compression 进行测试的时候，发现 CPU 很容易就跑到 100%，且程序启动后很快就 panic 退出了。</p> |
| <p>根据 panic 信息提示查到 <a href="https://github.com/gorilla/websocket/blob/master/conn.go#L1018">gorilla/websocket/conn.go:ReadMsg</a> |
| 函数调用 <a href="https://github.com/gorilla/websocket/blob/master/conn.go#L928">gorilla/websocket/conn.go:NextReader</a> 后就立即 panic |
| 退出了。panic 的 <code>表层原因</code> 到是很容易查明：</p> |
| <ul> |
| <li><a href="https://github.com/gorilla/websocket/blob/master/conn.go#L768">gorrilla/websocket:Conn::advanceFrame</a> 遇到读超时错误（io |
| timeout）;</li> |
| <li><a href="https://github.com/gorilla/websocket/blob/master/conn.go#L941">gorrilla/websocket:ConnConn.readErr</a>记录这个error；</li> |
| <li><a href="https://github.com/gorilla/websocket/blob/master/conn.go#L959">gorilla/websocket/conn.go:Conn::NextReader</a>开始读取之前则<a href="https://github.com/gorilla/websocket/blob/master/conn.go#L938">检查这个错误</a>，如以前发生过错误则不再读取 |
| websocket |
| frame，并对<a href="https://github.com/gorilla/websocket/blob/master/conn.go#L957">gorrilla/websocket:ConnConn.readErr累积计数</a>；</li> |
| <li><a href="https://github.com/gorilla/websocket/blob/master/conn.go#L958">当gorrilla/websocket:ConnConn.readErr数值大于 1000</a> |
| 的时候，程序就会panic 退出。</li> |
| </ul> |
| <p>但是为何发生读超时错误则毫无头绪。</p> |
| <p>2018/03/07 日测试 TCP compression 的时候发现启动 compression 后，程序 CPU 也会很快跑到 |
| 100%，进一步追查后发现函数 getty/conn.go:gettyTCPConn::read(<code>https://github.com/alexstocks/getty/blob/master/conn.go#L228</code>) 里面的 log |
| 有很多 “io timeout” error。当时查到这个错误很疑惑，因为我已经在 TCP read 之前进行了超时设置【SetReadDeadline】，难道启动 compression |
| 会导致超时设置失效使得socket成了非阻塞的socket？</p> |
| <p>于是在 getty/conn.go:gettyTCPConn::read(<code>https://github.com/alexstocks/getty/blob/master/conn.go#L228</code>) 中添加了一个逻辑：启用 TCP |
| compression 的时不再设置超时时间【默认情况下tcp connection是永久阻塞的】，CPU 100% 的问题很快就得到了解决。</p> |
| <p>至于为何 <code>启用 TCP compression 会导致 SetDeadline 失效使得socket成了非阻塞的socket</code>，囿于个人能力和精力，待将来追查出结果后再在此补充之。</p> |
| <h4 id="22-websocket-compression">2.2 Websocket compression</h4> |
| <p>TCP compression 的问题解决后，个人猜想 Websocket compression |
| 程序遇到的问题或许也跟 <code>启用 TCP compression 会导致 SetDeadline 失效使得socket成了非阻塞的socket</code> 有关。</p> |
| <p>于是借鉴 TCP 的解决方法，在 getty/conn.go:gettyWSConn::read(<code>https://github.com/alexstocks/getty/blob/master/conn.go#L527</code>) |
| 直接把超时设置关闭，然后 CPU 100% 被解决，且程序运转正常。</p> |
| <h3 id="a-name33-unix-socketa"><a name="3">3 unix socket</a></h3> |
| <p>本节与 getty 无关，仅仅是在使用 unix socket 过程中遇到一些 keypoint 的记录。</p> |
| <h4 id="31-reliable">3.1 reliable</h4> |
| <p>unix socket datagram 形式的包也是可靠的，每次写必然要求对应一次读，否则写方会被阻塞。如果是 stream 形式，则 buffer 没有满之前，写者是不会被阻塞的。datagram 的优势在于 api 简单。</p> |
| <blockquote> |
| <p>Unix sockets are reliable. If the reader doesn&rsquo;t read, the writer blocks. If the socket is a datagram socket, each write is paired with a read. If the socket is a stream socket, the kernel may buffer some bytes between the writer and the reader, but when the buffer is full, the writer will block. Data is never discarded, except for buffered data if the reader closes the connection before reading the buffer. &mdash;<a href="https://unix.stackexchange.com/questions/283323/do-unix-domain-sockets-overflow">Do UNIX Domain Sockets Overflow?</a></p> |
| </blockquote> |
| <blockquote> |
| <p>On most UNIX implementations, UNIX domain datagram sockets are always reliable and don&rsquo;t reorder datagrams. &mdash;<a href="http://www.man7.org/linux/man-pages/man7/unix.7.html">man 7 socketpair</a></p> |
| </blockquote> |
| <h4 id="32--buffer-size">3.2 buffer size</h4> |
| <p>datagram 形式的 unix socket 的单个 datagram 包最大长度是 130688 B。</p> |
| <blockquote> |
| <p>AF_UNIX SOCK_DATAGRAM/SOCK_SEQPACKET datagrams need contiguous memory. Contiguous physical memory is hard to find, and the allocation fails. The max size actually is 130688 B. &mdash; <a href="https://stackoverflow.com/questions/4729315/what-is-the-max-size-of-af-unix-datagram-message-that-can-be-sent-in-linux">the max size of AF_UNIX datagram message that can be sent in linux</a></p> |
| </blockquote> |
| <blockquote> |
| <p>It looks like AF_UNIX sockets don&rsquo;t support scatter/gather on current Linux. it is a fixed size 130688 B. &mdash; <a href="https://stackoverflow.com/questions/13953912/difference-between-unix-domain-stream-and-datagram-sockets">Difference between UNIX domain STREAM and DATAGRAM sockets?</a></p> |
| </blockquote> |
| <h3 id="a-name44-goroutine-poola"><a name="4">4 Goroutine Pool</a></h3> |
| <p>随着 [dubbogo/getty][1] 被 [apache/dubbo-go][2] 用作底层 tcp 的 transport 引擎，处于提高系统吞吐的需要，[dubbogo/getty][1] 面临着下一步的进化要求：[<strong>针对 |
| dubbo-go 和 Getty 的网络 I/O 与线程派发这一部分进行进一步优化</strong>][4]。其中的关键就是添加 Goroutine Pool【下文简称 gr pool】，以分离网络 I/O 和 逻辑处理。</p> |
| <p>Gr Pool 成员有任务队列【其数目为 M】和 Gr 数组【其数目为 N】以及任务【或者称之为消息】，根据 N 的数目变化其类型分为可伸缩与固定大小，可伸缩 Gr Pool 好处是可以随着任务数目变化增减 N 以节约 CPU |
| 和内存资源，但一般不甚常用，比人以前撸过一个后就躺在我的 [github repo][5] 里面了。</p> |
| <p>[dubbogo/getty][1] 只关注 N 值固定大小的 gr pool，且不考虑收到包后的处理顺序。譬如，[dubbogo/getty][1] 服务端收到了客户端发来的 A 和 B 两个网络包，不考虑处理顺序的 gr pool |
| 模型可能造成客户端先收到 B 包的 response，后才收到 A 包的 response。</p> |
| <p>如果客户端的每次请求都是独立的，没有前后顺序关系，则带有 gr pool 特性的 [dubbogo/getty][1] 不考虑顺序关系是没有问题的。如果上层用户关注 A 和 B 请求处理的前后顺序，则可以把 A 和 B |
| 两个请求合并为一个请求，或者把 gr pool 特性关闭。</p> |
| <h3 id="a-name4141-固定大小-gr-poola"><a name="4.1">4.1 固定大小 Gr Pool</a></h3> |
| <p>按照 M 与 N 的比例，固定大小 Gr Pool 又区分为 1:1、1:N、M:N 三类。</p> |
| <p>1:N 类型的 Gr Pool 最易实现，个人 2017 年在项目 [kafka-connect-elasticsearch][6] 中实现过此类型的 [Gr Pool][7]：作为消费者从 kafka 读取数据然后放入消息队列，然后各个 |
| worker gr 从此队列中取出任务进行消费处理。</p> |
| <p>向 [dubbogo/getty][1] 中添加 gr pool 时也曾实现过这个版本的 [gr pool][8]。这种模型的 gr pool 整个 pool 只创建一个 chan， 所有 gr 去读取这一个 |
| chan，其缺点是：队列读写模型是 一写多读，因为 go channel 的低效率【整体使用一个 mutex lock】造成竞争激烈，当然其网络包处理顺序更无从保证。</p> |
| <p>[dubbogo/getty][1] 初始版本的 [gr pool][9] 模型为 1:1，每个 gr 多有自己的 chan，其读写模型是一写一读，其优点是可保证网络包处理顺序性， 如读取 kafka 消息时候，按照 kafka |
| message 的 key 的 hash 值以取余方式【hash(message key) % N】将其投递到某个 task queue，则同一 key 的消息都可以保证处理有序。但望哥 |
| 指出了这种模型的缺陷：每个task处理要有时间，此方案会造成某个 gr 的 chan 里面有 task 堵塞，就算其他 gr 闲着，也没办法处理之【任务处理“饥饿”】。</p> |
| <p>[wenwei86][12] 给出了更进一步的 1:1 模型的改进方案：每个 gr 一个 chan，如果 gr 发现自己的 chan 没有请求，就去找别的 chan，发送方也尽量发往消费快的协程。这个方案类似于 go runtime 内部的 |
| MPG 调度算法，但是对我个人来说算法和实现均太复杂，故而没有采用。</p> |
| <p>[dubbogo/getty][1] 目前采用了 M:N 模型版本的 [gr pool][11]，每个 task queue 被 N/M 个 gr 消费，这种模型的优点是兼顾处理效率和锁压力平衡，可以做到总体层面的任务处理均衡。此版本下 |
| Task 派发采用 RoundRobin 方式。</p> |
| <h2 id="总结">总结</h2> |
| <p>本文总结了 [getty][3] 近期开发过程中遇到的一些问题，囿于个人水平只能给出目前自认为最好的解决方法【如何你有更好的实现，请留言】。</p> |
| <p>随着 [getty][3] 若有新的 improvement 或者新 feature，我会及时补加此文。</p> |
| <p>此记。</p> |
| <h2 id="参考文档">参考文档</h2> |
| <ol> |
| <li>connect Function with UDP(<code>http://www.masterraghu.com/subjects/np/introduction/unix_network_programming_v1.3/ch08lev1sec11.html</code>)</li> |
| <li><a href="http://colobu.com/2016/10/19/Go-UDP-Programming/">深入Go UDP编程</a></li> |
| </ol> |
| <blockquote> |
| <p>1: <a href="https://github.com/dubbogo/getty">https://github.com/dubbogo/getty</a></p> |
| <p>2: <a href="https://github.com/apache/dubbo-go/">https://github.com/apache/dubbo-go/</a></p> |
| <p>3: <a href="https://github.com/alexstocks/getty">https://github.com/alexstocks/getty</a></p> |
| <p>4: <a href="https://www.oschina.net/question/3820517_2306822">https://www.oschina.net/question/3820517_2306822</a></p> |
| <p>5: <a href="https://github.com/alexstocks/goext/blob/master/sync/pool/worker_pool.go">https://github.com/alexstocks/goext/blob/master/sync/pool/worker_pool.go</a></p> |
| <p>6: <a href="https://github.com/AlexStocks/kafka-connect-elasticsearch">https://github.com/AlexStocks/kafka-connect-elasticsearch</a></p> |
| <p>7: <a href="https://github.com/AlexStocks/kafka-connect-elasticsearch/blob/master/app/worker.go">https://github.com/AlexStocks/kafka-connect-elasticsearch/blob/master/app/worker.go</a></p> |
| <p>8: <a href="https://github.com/dubbogo/getty/pull/6/commits/4b32c61e65858b3eea9d88d8f1c154ab730c32f1">https://github.com/dubbogo/getty/pull/6/commits/4b32c61e65858b3eea9d88d8f1c154ab730c32f1</a></p> |
| <p>9: <a href="https://github.com/dubbogo/getty/pull/6/files/c4d06e2a329758a6c65c46abe464a90a3002e428#diff-9922b38d89e2ff9f820f2ce62f254162">https://github.com/dubbogo/getty/pull/6/files/c4d06e2a329758a6c65c46abe464a90a3002e428#diff-9922b38d89e2ff9f820f2ce62f254162</a></p> |
| <p>10: <a href="https://github.com/wongoo">https://github.com/wongoo</a></p> |
| <p>11: <a href="https://github.com/dubbogo/getty/pull/6/commits/1991056b300ba9804de0554dbb49b5eb04560c4b">https://github.com/dubbogo/getty/pull/6/commits/1991056b300ba9804de0554dbb49b5eb04560c4b</a></p> |
| <p>12: <a href="https://github.com/wenweihu86">https://github.com/wenweihu86</a></p> |
| </blockquote></description></item><item><title>Blog: 无缝衔接 gRPC 与 dubbo-go</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/%E6%97%A0%E7%BC%9D%E8%A1%94%E6%8E%A5-grpc-%E4%B8%8E-dubbo-go/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/%E6%97%A0%E7%BC%9D%E8%A1%94%E6%8E%A5-grpc-%E4%B8%8E-dubbo-go/</guid><description> |
| <p>最近我们 dubbo-go 社区里面，呼声很大的一个 feature 就是对 gRPC 的支持。在某位大佬的不懈努力之下，终于弄出来了。</p> |
| <p>今天我就给大家分析一下大佬是怎么连接 dubbo-go 和 gRPC 。</p> |
| <h2 id="grpc">gRPC</h2> |
| <p>先来简单介绍一下 gRPC 。它是 Google 推出来的一个 RPC 框架。gRPC是通过 IDL ( Interface Definition Language )——接口定义语言——编译成不同语言的客户端来实现的。可以说是RPC理论的一个非常非常标准的实现。</p> |
| <p>因而 gRPC 天然就支持多语言。这几年，它几乎成为了跨语言 RPC 框架的标准实现方式了，很多优秀的rpc框架，如 Spring Cloud 和 dubbo ，都支持 gRPC 。</p> |
| <p>server 端</p> |
| <p>在 Go 里面，server 端的用法是：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p1.webp" alt="img"></p> |
| <p>它的关键部分是：s := grpc.NewServer()和pb.RegisterGreeterServer(s, &amp;server{})两个步骤。第一个步骤很容易，唯独第二个步骤RegisterGreeterServer有点麻烦。为什么呢？</p> |
| <p>因为pb.RegisterGreeterServer(s, &amp;server{})这个方法是通过用户定义的protobuf编译出来的。</p> |
| <p>好在，这个编译出来的方法，本质上是：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p2.webp" alt="img"></p> |
| <p>也就是说，如果我们在 dubbo-go 里面拿到这个 _Greeter_serviceDesc ，就可以实现这个 server 的注册。因此，可以看到，在 dubbo-go 里面，要解决的一个关键问题就是如何拿到这个 serviceDesc 。</p> |
| <h2 id="client-端">Client 端</h2> |
| <p>Client 端的用法是：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p3.webp" alt="img"></p> |
| <p>这个东西要复杂一点：1、创建连接：conn, err := grpc.Dial(address)2、创建client：c := pb.NewGreeterClient(conn)3、调用方法：r, err := c.SayHello(ctx, &amp;pb.HelloRequest{Name: name})</p> |
| <p>第一个问题其实挺好解决的，毕竟我们可以从用户的配置里面读出 address ；</p> |
| <p>第二个问题就是最难的地方了。如同 RegisterGreeterServer 是被编译出来的那样，这个 NewGreeterClient 也是被编译出来的。</p> |
| <p>而第三个问题，乍一看是用反射就能解决，但是我们打开 SayHello 就能看到：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p4.webp" alt="img"></p> |
| <p>结合 greetClient 的定义，很容易看到，我们的关键就在于 err := c.cc.Invoke ( ctx, &ldquo;/helloworld.Greeter/SayHello&rdquo;, in, out, opts&hellip; )。换言之，我们只需要创建出来连接，并且拿到方法、参数就能通过类似的调用来模拟出 c.SayHello 。</p> |
| <p>通过对 gRPC 的简单分析，我们大概知道要怎么弄了。还剩下一个问题，就是我们的解决方案怎么和 dubbo-go 结合起来呢？</p> |
| <h2 id="设计">设计</h2> |
| <p>我们先来看一下 dubbo-go 的整体设计，思考一下，如果我们要做 gRPC 的适配，应该是在哪个层次上做适配。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p5.webp" alt="img"></p> |
| <p>我们根据前面介绍的 gRPC 的相关特性可以看出来，gRPC 已经解决了 codec 和 transport 两层的问题。</p> |
| <p>而从 cluster 往上，显然 gRPC 没有涉及。于是，从这个图里面我们就可以看出来，要做这种适配，那么 protocol 这一层是最合适的。即，我们可以如同 dubbo protocol 那般，扩展出来一个 grpc protocol 。</p> |
| <p>这个 gRPC protocol 大体上相当于一个适配器，将底层的 gRPC 的实现和我们自身的 dubbo-go 连接在一起。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p6.webp" alt="img"></p> |
| <h2 id="实现">实现</h2> |
| <p>在 dubbo-go 里面，和 gRPC 相关的主要是：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p7.webp" alt="img"></p> |
| <p>我们直接进去看看在 gRPC 小节里面提到的要点是如何实现的。</p> |
| <h3 id="server端">server端</h3> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p8.webp" alt="img"></p> |
| <p>这样看起来，还是很清晰的。如同 dubbo- go 其它的 protocol 一样，先拿到 service ，而后通过 service 来拿到 serviceDesc ，完成服务的注册。</p> |
| <p>注意一下上图我红线标准的 ds, ok := service.(DubboGrpcService) 这一句。</p> |
| <p>为什么我说这个地方有点奇怪呢？是因为理论上来说，我们这里注册的这个 service 实际上就是 protobuf 编译之后生成的 gRPC 服务端的那个 service ——很显然，单纯的编译一个 protobuf 接口，它肯定不会实现 DubboGrpcService 接口：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p9.webp" alt="img"></p> |
| <p>那么 ds, ok := service.(DubboGrpcService) 这一句，究竟怎么才能让它能够执行成功呢？</p> |
| <p>我会在后面给大家揭晓这个谜底。</p> |
| <h2 id="client端">Client端</h2> |
| <p>dubbo-go 设计了自身的 Client ，作为对 gRPC 里面 Client 的一种模拟与封装：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p10.webp" alt="img"></p> |
| <p>注意看，这个 Client 的定义与前面 greetClient 的定义及其相似。再看下面的 NewClient 方法，里面也无非就是创建了连接 conn ，而后利用 conn 里创建了一个 Client 实例。</p> |
| <p>注意的是，这里面维护的 invoker 实际上是一个 stub 。</p> |
| <p>当真正发起调用的时候：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p11.webp" alt="img"></p> |
| <p>红色框框框住的就是关键步骤。利用反射从 invoker ——也就是 stub ——里面拿到调用的方法，而后通过反射调用。</p> |
| <h3 id="代码生成">代码生成</h3> |
| <p>前面提到过 ds, ok := service.(DubboGrpcService) 这一句，面临的问题是如何让 protobuf 编译生成的代码能够实现 DubboGrpcService 接口呢？</p> |
| <p>有些小伙伴可能也注意到，在我贴出来的一些代码里面，反射操作会根据名字来获取method实例，比如NewClient方法里面的method := reflect.ValueOf(impl).MethodByName(&ldquo;GetDubboStub&rdquo;)这一句。这一句的impl，即指服务的实现，也是 protobuf 里面编译出来的，怎么让 protobuf 编译出来的代码里面含有这个 GetDubboStub 方法呢？</p> |
| <p>到这里，答案已经呼之欲出了：修改 protobuf 编译生成代码的逻辑！</p> |
| <p>庆幸的是，在 protobuf 里面允许我们通过插件的形式扩展我们自己的代码生成的逻辑。</p> |
| <p>所以我们只需要注册一个我们自己的插件：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p12.webp" alt="img"></p> |
| <p>然后这个插件会把我们所需要的代码给嵌入进去。比如说嵌入GetDubboStub方法：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p13.webp" alt="img"></p> |
| <p>还有DubboGrpcService接口：</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/grpc/p14.webp" alt="img"></p> |
| <p>这个东西，属于难者不会会者不难。就是如果你不知道可以通过plugin的形式来修改生成的代码，那就是真难；但是如果知道了，这个东西就很简单了——无非就是水磨工夫罢了。</p></description></item><item><title>Blog: 在dubbo-go中使用sentinel</title><link>https://dubbo.apache.org/zh-cn/blog/2021/01/11/%E5%9C%A8dubbo-go%E4%B8%AD%E4%BD%BF%E7%94%A8sentinel/</link><pubDate>Mon, 11 Jan 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/01/11/%E5%9C%A8dubbo-go%E4%B8%AD%E4%BD%BF%E7%94%A8sentinel/</guid><description> |
| <p>时至今日，Apache/dubbo-go（以下简称 dubbo-go ）项目在功能上已经逐步对齐java版本，稳定性也在不同的生产环境得到了验证。社区便开始再服务治理、监控等方向发力。随着 1.2和1.3 版本发布， dubbo-go 新增了大量此类新feature。</p> |
| <p>今天我们聊一聊限流相关话题，此前dubbo-go已经支持了<a href="https://github.com/apache/dubbo-go/pull/237">tps limit</a>、<a href="https://github.com/apache/dubbo-go/pull/246">execute limit </a>、<a href="https://github.com/apache/dubbo-go/pull/133">hystrix</a> 的内置filter，用户只要简单配置就能马上用上。但我们知道，在 java 的 dubbo 生态中，有一项限流工具被广泛使用，那就是sentinel。sentinel因为强大的动态规划配置、优秀的dashboard以及对dubbo的良好适配，成为众多使用dubbo的企业选用限流工具的不二之选。</p> |
| <p>就在前些日子，社区非常高兴得知 Sentinel Golang 首个版本 0.1.0 正式发布，这使得 dubbo-go也可以使用 sentinel 作为工具进行一些服务治理、监控的工作了。随着sentinel golang的健壮，我们相信用户马上可以像sentinel管理java dubbo服务那样管理dubbo-go的服务了。</p> |
| <p>完成sentinel golang的dubbo-adapter其实非常简单，这得益于dubbo-go早就完成了filter链的构造，用户可以自定义filter，并且灵活的安排其执行顺序。在1.3发布后，增加了filter中的context传递，构建sentinel/adapter/dubbo更为方便。</p> |
| <p>我们以其中的provider filter适配为例:</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/sentinel/dubbo-go-sentinel-provider-filter.png" alt="img"></p> |
| <p>此 filter 实现了 dubbo-go的filter接口，只要用户在服务启动时将此filter加载到dubbo-go中，即可使用此filter。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/dubbo-go/sentinel/sentinel-golang.png" alt="img"></p> |
| <p>sentinel实现原理与其他限流、熔断库大同小异，底层是用的滑动窗口算法。与hystrix等框架相比不同点是设计理念，Sentinel 的设计理念是让您自由选择控制的角度，并进行灵活组合，从而达到想要的效果。</p> |
| <p>下面我整理了完整的使用流程：(注意：dubbo-go版本请使用1.3.0-rc3及其以上版本)</p> |
| <p>在dubbo-go中使用sentinel主要分为以下几步：</p> |
| <ol> |
| <li>初始化sentinel</li> |
| <li>将sentinel注入dubbo-go的filter</li> |
| <li>初始化dubbo-go</li> |
| <li>配置规划</li> |
| </ol> |
| <h2 id="初始化sentinel">初始化sentinel</h2> |
| <p>示例代码：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> sentinel <span style="color:#2aa198">&#34;github.com/alibaba/sentinel-golang/api&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">initSentinel</span>() { |
| </span></span><span style="display:flex;"><span> err <span style="color:#719e07">:=</span> sentinel.<span style="color:#268bd2">InitWithLogDir</span>(confPath, logDir) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 初始化 Sentinel 失败 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> } |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h2 id="将sentinel注入dubbo-go的filter">将sentinel注入dubbo-go的filter</h2> |
| <p>你可以通过import包的形式执行，执行其中的init()来注入filter</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/alibaba/sentinel-golang/adapter/dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span></code></pre></div><p>也可以手动执行，给你的filter取上自己想要的名字</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common/extension&#34;</span> |
| </span></span><span style="display:flex;"><span> sd <span style="color:#2aa198">&#34;github.com/alibaba/sentinel-golang/adapter/dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>(){ |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetFilter</span>(<span style="color:#2aa198">&#34;myConsumerFilter&#34;</span>,sd.<span style="color:#268bd2">GetConsumerFilter</span>()) |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetFilter</span>(<span style="color:#2aa198">&#34;myProviderFilter&#34;</span>,sd.<span style="color:#268bd2">GetConsumerFilter</span>()) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>完成以上步骤，你就可以在需要的dubbo接口配置里写入sentinel的filterName,构建起接口的filter链条。比如以下以consumer.yml配置文件为例</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yml" data-lang="yml"><span style="display:flex;"><span><span style="color:#268bd2">references</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;UserProvider&#34;</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">registry</span>: <span style="color:#2aa198">&#34;hangzhouzk&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol </span>: <span style="color:#2aa198">&#34;dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">interface </span>: <span style="color:#2aa198">&#34;com.ikurento.user.UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">cluster</span>: <span style="color:#2aa198">&#34;failover&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">filter</span>: <span style="color:#2aa198">&#34;myConsumerFilter&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">methods </span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;GetUser&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">retries</span>: <span style="color:#2aa198">3</span> |
| </span></span></code></pre></div><h2 id="初始化dubbo-go">初始化dubbo-go</h2> |
| <p>到这一步，你只需要正常启动dubbo-go程序就完成了服务启动。用以下代码做一个较为完整举例</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> hessian <span style="color:#2aa198">&#34;github.com/apache/dubbo-go-hessian2&#34;</span> |
| </span></span><span style="display:flex;"><span> sd <span style="color:#2aa198">&#34;github.com/alibaba/sentinel-golang/adapter/dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">import</span> ( |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common/logger&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common/proxy/proxy_factory&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/config&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/filter/impl&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/protocol/dubbo&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/protocol&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/cluster_impl&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/cluster/loadbalance&#34;</span> |
| </span></span><span style="display:flex;"><span> _ <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/registry/zookeeper&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#2aa198">&#34;github.com/apache/dubbo-go/common/extension&#34;</span> |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> <span style="color:#268bd2">main</span>() { |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> hessian.<span style="color:#268bd2">RegisterPOJO</span>(<span style="color:#719e07">&amp;</span>User{}) |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetFilter</span>(<span style="color:#2aa198">&#34;myConsumerFilter&#34;</span>,sd.<span style="color:#268bd2">GetConsumerFilter</span>()) |
| </span></span><span style="display:flex;"><span> extension.<span style="color:#268bd2">SetFilter</span>(<span style="color:#2aa198">&#34;myProviderFilter&#34;</span>,sd.<span style="color:#268bd2">GetConsumerFilter</span>()) |
| </span></span><span style="display:flex;"><span> config.<span style="color:#268bd2">Load</span>() |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// init finish, do your work |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">test</span>() |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h2 id="规划配置">规划配置</h2> |
| <p>sentinel以强大的规划配置吸引了很多使用者，其提供动态数据源接口进行扩展，用户可以通过动态文件或 etcd 等配置中心来动态地配置规则。但目前sentinel-golang作为破蛋版本，动态配置还在开发中</p> |
| <h3 id="动态数据源">动态数据源</h3> |
| <p>（开发中）Sentinel 提供动态数据源接口进行扩展，用户可以通过动态文件或 etcd 等配置中心来动态地配置规则。</p> |
| <h3 id="硬编码方式">硬编码方式</h3> |
| <p>Sentinel 也支持原始的硬编码方式加载规则，可以通过各个模块的 <code>LoadRules(rules)</code> 方法加载规则。以下是硬编码方式对某个method在consumer端的QPS流控：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span>_, err <span style="color:#719e07">:=</span> flow.<span style="color:#268bd2">LoadRules</span>([]<span style="color:#719e07">*</span>flow.FlowRule{ |
| </span></span><span style="display:flex;"><span> { |
| </span></span><span style="display:flex;"><span> ID: <span style="color:#2aa198">666</span>, |
| </span></span><span style="display:flex;"><span> Resource: <span style="color:#2aa198">&#34;dubbo:consumer:com.ikurento.user.UserProvider:myGroup:1.0.0:hello()&#34;</span>, |
| </span></span><span style="display:flex;"><span> MetricType: flow.QPS, |
| </span></span><span style="display:flex;"><span> Count: <span style="color:#2aa198">10</span>, |
| </span></span><span style="display:flex;"><span> ControlBehavior: flow.Reject, |
| </span></span><span style="display:flex;"><span> }, |
| </span></span><span style="display:flex;"><span>}) |
| </span></span><span style="display:flex;"><span><span style="color:#719e07">if</span> err <span style="color:#719e07">!=</span> <span style="color:#cb4b16">nil</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// 加载规则失败，进行相关处理 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>} |
| </span></span></code></pre></div><h1 id="总结">总结</h1> |
| <p>更加具体的实现，我就不详细论述，大家可以去看源码进一步了解。</p> |
| <p>最后，欢迎大家持续关注，或者贡献代码，期待dubbo-go在2020年在云原生领域继续突破。</p> |
| <p>dubbo-go仓库地址：https://github.com/apache/dubbo-go</p></description></item></channel></rss> |
