| <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Dubbo – Pixiu</title><link>https://dubbo.apache.org/zh-cn/tags/pixiu/</link><description>Recent content in Pixiu on Apache Dubbo</description><generator>Hugo -- gohugo.io</generator><language>zh-cn</language><lastBuildDate>Sat, 19 Feb 2022 00:00:00 +0000</lastBuildDate><atom:link href="https://dubbo.apache.org/zh-cn/tags/pixiu/index.xml" rel="self" type="application/rss+xml"/><item><title>Blog: 谈谈Pixiu的Filter</title><link>https://dubbo.apache.org/zh-cn/blog/2022/02/19/%E8%B0%88%E8%B0%88pixiu%E7%9A%84filter/</link><pubDate>Sat, 19 Feb 2022 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2022/02/19/%E8%B0%88%E8%B0%88pixiu%E7%9A%84filter/</guid><description> |
| <h2 id="filter的生命周期"><strong>Filter的生命周期</strong></h2> |
| <p>Pixiu作为一个面向云原生的gateway,通过简单的配置即可代理Http to Dubbo 2、Tripe甚至是Spring Cloud的请求。那Filter是怎样运行的呢?</p> |
| <p>首先<strong>Filter Plugin</strong>向<strong>Filter Manager</strong>注册自己**,<strong>然后</strong>Filter Manager<strong>根据配置创建好</strong>Filter Factory<strong>并持有它们,等待请求来临时,<strong>Manager</strong>创建一个一次性的用于此次请求的Filter Chain,然后利用</strong>Factory<strong>创建好</strong>Decode/Encode Filter<strong>并把它们加入链中,然后按照顺序去运行Decode Filter,然后去请求</strong>Upstream**,拿到Response再反向运行Encode Filter,让Filter可以访问到Response。</p> |
| <p>几个关键的概念:</p> |
| <p><strong>Filter Manager</strong></p> |
| <blockquote> |
| <p>Filter的Manger。。。</p> |
| </blockquote> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// FilterManager manage filters |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">type</span> FilterManager <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> filters <span style="color:#268bd2">map</span>[<span style="color:#dc322f">string</span>]HttpFilterFactory |
| </span></span><span style="display:flex;"><span> filtersArray []<span style="color:#719e07">*</span>HttpFilterFactory |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><strong>Filter Plugin</strong>:定义了Filter的(唯一的)名字和描述如何去创建一个Filter Factory。</p> |
| <blockquote> |
| <p>其实结合Filter Factory的定义,可以认为Plugin是Filter Factory的Factory</p> |
| </blockquote> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// HttpFilterPlugin describe plugin |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>HttpFilterPlugin <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Kind returns the unique kind name to represent itself. |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Kind</span>() <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// CreateFilterFactory return the filter factory |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">CreateFilterFactory</span>() (HttpFilterFactory, <span style="color:#dc322f">error</span>) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><strong>Filter Factory</strong>:定义了Filter自身的配置,并且在请求来临时创建真实的Filter并把它添加到FilterChain中</p> |
| <blockquote> |
| <ul> |
| <li>Config() 的目的是能让Filter Manager能够有机会把配置交给Factory(此时golang泛型还没有落地)</li> |
| <li>Apply() 在配置被注入到Factory后,有机会对config做一些检查和提前做一些初始化的工作</li> |
| <li>PrepareFilterChain() 创建Filter并加入Filter Chain</li> |
| </ul> |
| </blockquote> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// HttpFilterFactory describe http filter |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>HttpFilterFactory <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Config Expose the config so that Filter Manger can inject it, so it must be a pointer |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Config</span>() <span style="color:#268bd2">interface</span>{} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Apply After the config is injected, check it or make it to default |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">Apply</span>() <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// PrepareFilterChain create filter and append it to FilterChain |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// Be Careful !!! Do not pass the Factory&#39;s config pointer to the Filter instance, |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#586e75">// Factory&#39;s config may be updated by FilterManager |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> <span style="color:#268bd2">PrepareFilterChain</span>(ctx <span style="color:#719e07">*</span>http.HttpContext, chain FilterChain) <span style="color:#dc322f">error</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p><strong>Decode/Encode Filter:<strong>Filter分为两个部分,<strong>Decode</strong>在实际请求</strong>Upstream</strong>之前,所以可以做一些鉴权、限流,把请求在gateway层拦截掉。<strong>Eecode</strong>则运行在获得<strong>Upstream</strong>的Response之后,所以可以对返回Log甚至修改Response。</p> |
| <blockquote> |
| <p>一个Filter可以即是Decode Filter,又是Encode Filter,没有限制!</p> |
| <p>假设有A、B、C三个Filter,都是Decode/Encode Filter,如果配置的顺序是A、B、C,那么运行将会是下面这样</p> |
| <p>在Decode阶段 A-&gt;B-&gt;C,而在Encode阶段,顺序将会反过来!C-&gt;B-&gt;A</p> |
| </blockquote> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">// decode filters will be invoked in the config order: A、B、C, and decode filters will be |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// invoked in the reverse order: C、B、A |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>HttpDecodeFilter <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Decode</span>(ctx <span style="color:#719e07">*</span>http.HttpContext) FilterStatus |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// HttpEncodeFilter after invoke upstream, |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75">// decode filters will be invoked in the reverse order |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span>HttpEncodeFilter <span style="color:#268bd2">interface</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">Encode</span>(ctx <span style="color:#719e07">*</span>http.HttpContext) FilterStatus |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>更详细的,每个Decode/Encode Filter可以返回一个FilterStatus来决定继续还是就在这里停下!比如JWT鉴权,token无效时就要及时把401返回给Downstream。当然Decode Filter发出的停止命令只会终止Decode阶段,至于为什么?想想如何做一个Access Log Filter,能在请求失败时也把失败的结果记录下吧来!</p> |
| <h2 id="怎样编写一个自定义filter"><strong>怎样编写一个自定义Filter</strong></h2> |
| <p>我们来尝试写一个简单的Filter,这个Filter将会有简单的配置,在Decode阶段把请求的Body Log出来,并翻转后作为Mock的返回值。最后在Encode阶段根据配置把返回值Log出来。</p> |
| <p>1.首先创建一个Filter</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> DemoFilter <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> logPrefix <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilter) <span style="color:#268bd2">Decode</span>(ctx <span style="color:#719e07">*</span>contexthttp.HttpContext) filter.FilterStatus { |
| </span></span><span style="display:flex;"><span> body, _ <span style="color:#719e07">:=</span> ioutil.<span style="color:#268bd2">ReadAll</span>(ctx.Request.Body) |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;request body: %s&#34;</span>, body) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//reverse res str |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> runes <span style="color:#719e07">:=</span> []<span style="color:#b58900">rune</span>(<span style="color:#b58900">string</span>(body)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; i &lt; <span style="color:#b58900">len</span>(runes)<span style="color:#719e07">/</span><span style="color:#2aa198">2</span>; i <span style="color:#719e07">+=</span> <span style="color:#2aa198">1</span> { |
| </span></span><span style="display:flex;"><span> runes[i], runes[<span style="color:#b58900">len</span>(runes)<span style="color:#719e07">-</span><span style="color:#2aa198">1</span><span style="color:#719e07">-</span>i] = runes[<span style="color:#b58900">len</span>(runes)<span style="color:#719e07">-</span><span style="color:#2aa198">1</span><span style="color:#719e07">-</span>i], runes[i] |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> reverse <span style="color:#719e07">:=</span> <span style="color:#b58900">string</span>(runes) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//mock response |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ctx.<span style="color:#268bd2">SendLocalReply</span>(<span style="color:#2aa198">200</span>, []<span style="color:#b58900">byte</span>(reverse)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> filter.Stop |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilter) <span style="color:#268bd2">Encode</span>(ctx <span style="color:#719e07">*</span>contexthttp.HttpContext) filter.FilterStatus { |
| </span></span><span style="display:flex;"><span> res <span style="color:#719e07">:=</span> ctx.SourceResp.(<span style="color:#dc322f">string</span>) |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;%s: %s&#34;</span>, f.logPrefix, res) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> filter.Continue |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>2.创建Filter Factory</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> ( |
| </span></span><span style="display:flex;"><span> DemoFilterFactory <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> conf <span style="color:#719e07">*</span>Config |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Config describe the config of Filter |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> Config <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> LogPrefix <span style="color:#dc322f">string</span> <span style="color:#2aa198">`yaml:&#34;logPrefix,omitempty&#34;`</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilterFactory) <span style="color:#268bd2">PrepareFilterChain</span>(ctx <span style="color:#719e07">*</span>contexthttp.HttpContext, chain filter.FilterChain) <span style="color:#dc322f">error</span> { |
| </span></span><span style="display:flex;"><span> demo <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>DemoFilter{logPrefix: f.conf.LogPrefix} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> chain.<span style="color:#268bd2">AppendDecodeFilters</span>(demo) |
| </span></span><span style="display:flex;"><span> chain.<span style="color:#268bd2">AppendEncodeFilters</span>(demo) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilterFactory) <span style="color:#268bd2">Config</span>() <span style="color:#268bd2">interface</span>{} { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> f.conf |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilterFactory) <span style="color:#268bd2">Apply</span>() <span style="color:#dc322f">error</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>3.创建Filter Plugin,并注册自己</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">//important |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> filter.<span style="color:#268bd2">RegisterHttpFilter</span>(<span style="color:#719e07">&amp;</span>Plugin{}) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">type</span> Plugin <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>Plugin) <span style="color:#268bd2">Kind</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;dgp.filters.demo&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>Plugin) <span style="color:#268bd2">CreateFilterFactory</span>() (filter.HttpFilterFactory, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>DemoFilterFactory{conf: <span style="color:#719e07">&amp;</span>Config{}}, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>4.配置文件中配置此Filter,并启动Pixiu</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">static_resources</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">listeners</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;net/http&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol_type</span>: <span style="color:#2aa198">&#34;HTTP&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">socket_address</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;0.0.0.0&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">port</span>: <span style="color:#2aa198">8888</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">filter_chains</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">filters</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: dgp.filter.httpconnectionmanager |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">config</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">route_config</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">routes</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">match</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">prefix</span>: <span style="color:#2aa198">&#34;/&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">http_filters</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: dgp.filters.demo |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">config</span>: |
| </span></span></code></pre></div><p>5.访问并查看日志与结果</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>curl localhost:8888/demo -d <span style="color:#2aa198">&#34;eiv al tse’c&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>c’est la vie% |
| </span></span></code></pre></div><p>日志</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>2022-02-19T20:20:11.900+0800 INFO demo/demo.go:62 request body: eiv al tse’c |
| </span></span><span style="display:flex;"><span>2022-02-19T20:20:11.900+0800 INFO demo/demo.go:71 : eiv al tse’c |
| </span></span></code></pre></div></description></item><item><title>Blog: Dubbo 跨语言调用神兽:dubbo-go-pixiu</title><link>https://dubbo.apache.org/zh-cn/blog/2021/08/25/dubbo-%E8%B7%A8%E8%AF%AD%E8%A8%80%E8%B0%83%E7%94%A8%E7%A5%9E%E5%85%BDdubbo-go-pixiu/</link><pubDate>Wed, 25 Aug 2021 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/blog/2021/08/25/dubbo-%E8%B7%A8%E8%AF%AD%E8%A8%80%E8%B0%83%E7%94%A8%E7%A5%9E%E5%85%BDdubbo-go-pixiu/</guid><description> |
| <h2 id="pixiu-是什么">Pixiu 是什么</h2> |
| <p>在回答 Pixiu 是什么之前,我们简单解释一下 Dubbo 是什么。Dubbo 是一个开源的高性能 RPC 框架,有着丰富的服务治理能力以及优秀的扩展能力。Dubbo 更扩展出 Dubbo-go【1】,为用户提供了 Golang 的 Dubbo 解决方案,打通了两种语言之间的隔阂,使 Dubbo 更加贴近云原生。</p> |
| <p>Dubbo-go 作为 Golang 服务,实现与 Dubbo 服务之间的相互调用。然而,在日常使用场景中,用户往往有把 Dubbo 服务以 RESTful 风格向外暴露的需求同时也要兼顾内部 Dubbo 调用。为了解决这种场景,作为 Dubbo API 网关的 Pixiu【2】 (中文: 貔貅, 曾用名 dubbo-go-proxy) 便应运而生。之所以采用 Pixiu 这个名称,是因为 Java 同类产品 Zuul 的意象是一个西方怪兽,Pixiu 作为一个国产产品,就用了我们中国的一个类似的神兽貔貅作为项目名称。也同时表达了 Dubbo 社区希望扩展出一整套云原生生态链的决心。</p> |
| <p>目前 Dubbo 多语言生态,发展最好的自然是 Java,其次是 Golang,其他语言都差强人意。dubbo-go-pixiu 项目是一个基于 dubbo-go 发展起来的项目,目前接口协议层支持的是七层的 HTTP 请求调用,计划在未来的 0.5 版本中支持 gRPC 请求调用,其另外一个使命是作为一种新的 dubbo 多语言解决方案。</p> |
| <h2 id="为什么使用-pixiu">为什么使用 Pixiu</h2> |
| <p>Pixiu 是基于 Dubbogo 的云原生、高性能、可扩展的微服务 API 网关。作为一款网关产品,Pixiu 帮助用户轻松创建、发布、维护、监控和保护任意规模的 API ,接受和处理成千上万个并发 API 调用,包括流量管理、 CORS 支持、授权和访问控制、限制、监控,以及 API 版本管理。除此以外,作为 Dubbo 的衍生产品,Pixiu 可以帮助 Dubbo 用户进行协议转换,实现跨系统、跨协议的服务能力互通。</p> |
| <p>Pixiu 的整体设计遵守以下原则:</p> |
| <ul> |
| <li>High performance: 高吞吐量以及毫秒级的延时。</li> |
| <li>可扩展: 通过 go-plugin,用户可以根据自己的需求延展 Pixiu 的功能。</li> |
| <li>简单可用: 用户通过少量配置,即可上线。</li> |
| </ul> |
| <h2 id="pixiu-的特性及核心功能">Pixiu 的特性及核心功能</h2> |
| <ul> |
| <li>为 RESTful API 和 Dubbo API 提供支持</li> |
| </ul> |
| <p>非 RESTful 风格的 API 和 Dubbo 协议的服务往往需要修改才可以以 RESTful API 风格对外开放。Pixiu 提供协议转换功能,通过 Pixiu,开发者可以将自己的 HTTP API 或 Dubbo API 通过配置,以 RESTful API 风格对外开放。v0.2.1 版本已支持基于泛化调用的 HTTP 至 Dubbo 的协议转换以及 HTTP 协议的转发。在后续的版本,社区将会增加对 gRPC 和 http2 协议的支持。</p> |
| <ul> |
| <li>面向用户的配置方式</li> |
| </ul> |
| <p>一般的网关的配置往往繁琐且复杂。Pixiu,目标作为一款易用的网关产品,在设计上拥有三层配置层级,Gateway 层全局配置, API resource 层配置以及 HTTP verbs 方法层配置。通过三个不同层级的配置,既可以实现深度的定制,亦支持统一的默认配置;同时,支持本地的配置文件,亦可使用统一配置服务器。另外,还提供控制台模块,通过控制台模块,支持配置的热更新。Pixiu 配套配套的控制台界面也在同步开发中。</p> |
| <ul> |
| <li>通用功能的集成</li> |
| </ul> |
| <p>重试、熔断、流量控制、访问控制等通用功能不再需要在每个后端服务上重复实现。使用 Pixiu,通过配置 filter ,开发者可以进行全局的控制,亦可以根据 API 配置各自的规则。因此开发者可以专注于业务逻辑和服务,而不是将时间用在维护基础设施上。</p> |
| <ul> |
| <li>可扩展</li> |
| </ul> |
| <p>不同的使用场景有着各自独特的需求。为满足不同用户的定制化需求,Pixiu 使用了插件模式。开发者可以通过编写 go plugin,将自身特有的业务逻辑以 filter 形式内嵌至 Pixiu 网关中,实现诸如企业登录鉴权等功能。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/1/01/01/dubbo-go-pixiu/fd38da297d095e4c3af1c89b18804ef1.webp" alt="img"></p> |
| <p>图 1: Pixiu 核心功能列表</p> |
| <h2 id="pixiu-的架构设计">Pixiu 的架构设计</h2> |
| <p><img src="https://dubbo.apache.org/imgs/blog/1/01/01/dubbo-go-pixiu/2b2fd6ea1cc0375392919d9e0c181f2b.webp" alt="img"></p> |
| <p>图 2: Pixiu 架构</p> |
| <p>貔貅: 即 dubbo-go-pixiu,由四个主要模块:Listener、Router、Filters 和 Clients 组成;</p> |
| <ul> |
| <li>Dubbo Cluster: Dubbo 服务所在集群,包含一个或多个 Dubbo Services;</li> |
| <li>Other Cluster: Dubbo 以外的服务所在集群,现支持 HTTP 服务,未来将拓展支持 gRPC 等其他服务;</li> |
| <li>Registry Center: 注册中心,维护每个业务服务的调用地址信息;</li> |
| <li>Metadata Center: 元数据中心,维护每个业务服务的配置信息以及存储 Pixiu 本身的配置信息。</li> |
| </ul> |
| <p>作为 Dubbo 所衍生的 API 网关,Pixiu 使用 Golang 搭建,主要因为: 1. Golang 的 G-M-P,net poller 等特性使 Golang 非常适合构建 IO 密集型应用;2. 使用 Golang 可以直接引入 Dubbo-go 中的一些组建,简化开发。</p> |
| <p>整个 Pixiu 大致可以拆分为四个主要模块:Listener、Router、Filters 和 Client。</p> |
| <h3 id="1listener">1、Listener</h3> |
| <p>在 Pixiu 中,Listener 代表外部可以访问 Pixiu 的方式。通过配置指定协议类型,地址,端口等属性,暴露 Gateway。现阶段暂支持 HTTP 协议,未来将会加入 gRPC。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>listeners: |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> - name: &#34;net/http&#34; |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> address: |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> socket_address: |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> protocol_type: &#34;HTTP&#34; |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> address: &#34;0.0.0.0&#34; |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> port: 8888 |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> config: |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> idle_timeout: 5s |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> read_timeout: 5s |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> write_timeout: 5s |
| </span></span></code></pre></div><h3 id="2router">2、Router</h3> |
| <p>Router 是 Pixiu 的路由组件。根据配置文件,Pixiu 将对外暴露的 URLs 以树的形势存储于内存中,当请求到了 router 组件时,即会根据 URL 及 HTTP 方法查找到对应的后端服务及其 API 配置,并将信息封装于请求中,为后续 filter,及 client 的调用提供足够的内容。</p> |
| <p>现阶段,Router 提供以下功能:</p> |
| <ul> |
| <li>支持请求一对一转发路由配置或 wildcard 路由配置。</li> |
| <li>支持 HTTP 请求的转发到后端 HTTP 服务。</li> |
| <li>支持 HTTP 请求转化为 dubbo 泛化调用请求。</li> |
| </ul> |
| <h3 id="3filters">3、Filters</h3> |
| <p>Filter 是 Pixiu 实现额外功能及其扩展性的主要组件。其实现类似于 Dubbo-go 中的 filter,根据配置中 filter 的指定,生成调用链,从而在调用后端服务前,将各 filter 中的逻辑运行一遍,实现节流,日志等功能。</p> |
| <p>用户如果需要客制化的 filter,可通过编写 go-plugin 实现。在配置中,可通过类似如下配置,加载 .so 文件,并在 API config 中指定使用的 plugin group,plugin name 实现。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>pluginFilePath: &#34;&#34; |
| </span></span><span style="display:flex;"><span>pluginsGroup: |
| </span></span><span style="display:flex;"><span> - groupName: &#34;group1&#34; |
| </span></span><span style="display:flex;"><span> plugins: |
| </span></span><span style="display:flex;"><span> - name: &#34;rate limit&#34; |
| </span></span><span style="display:flex;"><span> version: &#34;0.0.1&#34; |
| </span></span><span style="display:flex;"><span> priority: 1000 |
| </span></span><span style="display:flex;"><span> externalLookupName: &#34;ExternalPluginRateLimit&#34; |
| </span></span><span style="display:flex;"><span> - name: &#34;access&#34; |
| </span></span><span style="display:flex;"><span> version: &#34;0.0.1&#34; |
| </span></span><span style="display:flex;"><span> priority: 1000 |
| </span></span><span style="display:flex;"><span> externalLookupName: &#34;ExternalPluginAccess&#34; |
| </span></span><span style="display:flex;"><span> - groupName: &#34;group2&#34; |
| </span></span><span style="display:flex;"><span> plugins: |
| </span></span><span style="display:flex;"><span> - name: &#34;blacklist&#34; |
| </span></span><span style="display:flex;"><span> version: &#34;0.0.1&#34; |
| </span></span><span style="display:flex;"><span> priority: 1000 |
| </span></span><span style="display:flex;"><span> externalLookupName: &#34;ExternalPluginBlackList&#34; |
| </span></span></code></pre></div><h3 id="4client">4、Client</h3> |
| <p>Client 负责调用具体服务。现阶段,Pixiu 支持 HTTP 与 Dubbo 的后端服务。社区将逐渐增加 gRPC 等其他 Client 以满足不同的协议。</p> |
| <p>HTTP client 的实现相对简单,根据 Router 中获取的后端服务信息,通过 Golang 官方包 net/http 生成请求并调用。</p> |
| <p>Dubbo client 的实现对比 HTTP client 会稍微复杂,其基础为 Dubbo 服务的泛化调用。泛化调用技术是 Dubbo 提供的一个很基础的功能只需要知道调用的方法名、参数类型和返回值类型,即可发起服务调用。客户端对服务端的泛化调用既可以通过注册中心发现服务,也可以直连服务端,实现对服务的动态调用。</p> |
| <p>如下面代码所示,Pixiu 通过动态配置 referenceConfig,然后通过 GetRPCService 生成 Dubbo 的 Generic Client(泛化调用客户端)进行下一步的调用。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>referenceConfig := dg.NewReferenceConfig(irequest.Interface, context.TODO()) |
| </span></span><span style="display:flex;"><span> referenceConfig.InterfaceName = irequest.Interface |
| </span></span><span style="display:flex;"><span> referenceConfig.Cluster = constant.DEFAULT_CLUSTER |
| </span></span><span style="display:flex;"><span> var registers []string |
| </span></span><span style="display:flex;"><span> for k := range dgCfg.Registries { |
| </span></span><span style="display:flex;"><span> registers = append(registers, k) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> referenceConfig.Registry = strings.Join(registers, &#34;,&#34;) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> if len(irequest.DubboBackendConfig.Protocol) == 0 { |
| </span></span><span style="display:flex;"><span> referenceConfig.Protocol = dubbo.DUBBO |
| </span></span><span style="display:flex;"><span> } else { |
| </span></span><span style="display:flex;"><span> referenceConfig.Protocol = irequest.DubboBackendConfig.Protocol |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> referenceConfig.Version = irequest.DubboBackendConfig.Version |
| </span></span><span style="display:flex;"><span> referenceConfig.Group = irequest.Group |
| </span></span><span style="display:flex;"><span> referenceConfig.Generic = true |
| </span></span><span style="display:flex;"><span> if len(irequest.DubboBackendConfig.Retries) == 0 { |
| </span></span><span style="display:flex;"><span> referenceConfig.Retries = &#34;3&#34; |
| </span></span><span style="display:flex;"><span> } else { |
| </span></span><span style="display:flex;"><span> referenceConfig.Retries = irequest.DubboBackendConfig.Retries |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> dc.lock.Lock() |
| </span></span><span style="display:flex;"><span> defer dc.lock.Unlock() |
| </span></span><span style="display:flex;"><span> referenceConfig.GenericLoad(key) |
| </span></span><span style="display:flex;"><span> clientService := referenceConfig.GetRPCService().(*dg.GenericService) |
| </span></span></code></pre></div><p>实际上,在泛化调用的客户端中,实际执行泛化调用的关键步骤是 Dubbo-go 中的 generic_filter (如下代码片段)。在调用 generic_filter 的 Invoke 时,约定 invocation 参数列表第一个为方法名,第二个为参数类型列表,第三个为参数值列表。generic_filter 将用户请求的参数值列表转化为统一格式的 map(代码中的 struct2MapAll ),将类( golang 中为 struct )的正反序列化操作变成 map 的正反序列化操作。这使得无需 POJO 描述通过硬编码注入 hessain 库,从而完成 Dubbo 服务的泛化调用。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>func (ef *GenericFilter) Invoke(ctx context.Context, invoker protocol.Invoker, invocation protocol.Invocation) protocol.Result { |
| </span></span><span style="display:flex;"><span> if invocation.MethodName() == constant.GENERIC &amp;&amp; len(invocation.Arguments()) == 3 { |
| </span></span><span style="display:flex;"><span> oldArguments := invocation.Arguments() |
| </span></span><span style="display:flex;"><span> if oldParams, ok := oldArguments[2].([]interface{}); ok { |
| </span></span><span style="display:flex;"><span> newParams := make([]hessian.Object, 0, len(oldParams)) |
| </span></span><span style="display:flex;"><span> for i := range oldParams { |
| </span></span><span style="display:flex;"><span> newParams = append(newParams, hessian.Object(struct2MapAll(oldParams[i]))) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> newArguments := []interface{}{ |
| </span></span><span style="display:flex;"><span> oldArguments[0], |
| </span></span><span style="display:flex;"><span> oldArguments[1], |
| </span></span><span style="display:flex;"><span> newParams, |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> newInvocation := invocation2.NewRPCInvocation(invocation.MethodName(), newArguments, invocation.Attachments()) |
| </span></span><span style="display:flex;"><span> newInvocation.SetReply(invocation.Reply()) |
| </span></span><span style="display:flex;"><span> return invoker.Invoke(ctx, newInvocation) |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> return invoker.Invoke(ctx, invocation) |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><h2 id="总结">总结</h2> |
| <p>通过上面的四个模块以及注册中心的简单介绍不难发现,当请求通过 listener 被 Pixiu 接收后,请求被传入 router 中。router 根据接口的配置,从原请求中找到目标后端服务连同相关 API 配置下发到 filter 组件。filter 组件根据原请求、 API 配置等信息顺序执行,最终请求到达 client, 通过 client 调用后端服务。</p> |
| <h3 id="pixiu-的未来">Pixiu 的未来</h3> |
| <p><img src="https://dubbo.apache.org/imgs/blog/1/01/01/dubbo-go-pixiu/e57050f224f658b96cd6bd917050b259.webp" alt="img"> |
| 图 3: Pixiu 迭代里程碑</p> |
| <p>Pixiu 作为网关产品外,其衍生项目也会在我们的未来计划中,主要目的是提供更好的可用性。例如,由于 Golang 语言缺乏原生的注解, 因此 Dubbo-go 需要通过配置文件方式生成服务的元数据写入注册中心。开课啦教育公司相关同学写了一个扫描代码的工具 <a href="https://github.com/jack15083/dubbo-go-proxy-tool">https://github.com/jack15083/dubbo-go-proxy-tool</a>,在每个 RPC 服务方法前加上对应的注释,从而在服务启动前通过扫描注释生成元数据。Pixiu 也计划在未来的版本上通过提供 package,允许服务通过注释借助 <a href="https://github.com/MarcGrol/golangAnnotations">https://github.com/MarcGrol/golangAnnotations</a> 生成 API 配置并注册到 Pixiu 上。</p> |
| <p>Pixiu 目前的定位是一个七层协议网关,其最初版本是被定义成一个 Dubbo 的服务网关。作为云时代的产品,Pixiu 的发展方向必然是面向云原生的。现在的版本为 0.2.1, 已经实现基本的 Dubbo/Http 服务代理和部分的网关通用功能。目前正在开发中的 0.4 及其后续版本支持 gRPC 和 Spring Cloud 服务调用, 后续还将提供 MQ 服务支持。另外,社区将继续优化配置方式,降低用户的使用难度,继续优化官方的 filter,使 Pixiu 可以在官方层面实现更多的网关通用功能。</p> |
| <p><img src="https://dubbo.apache.org/imgs/blog/1/01/01/dubbo-go-pixiu/0c1afe00699eb3e5cc022e48966ef5a6.webp" alt="img"></p> |
| <p>在未来的一年内,社区计划支持 xDS API,将 Pixiu 演化为 Dubbo mesh 的 sidecar。其最终目的就是:在现有的 dubbo mesh 形态中演化出 Proxy Service Mesh 形态。基于这个形态,Js、Python、PHP、Ruby 和 Perl 等脚本语言程序除了收获 dubbo mesh 原有的技术红利之外,大概率还能收获性能上的提升。</p> |
| <p>Pixiu 在 Dubbo Mesh 中的终极目的是:把东西向和南北向数据面流量逐步统一 Pixiu 中的同时,让它逐步具备 Application Runtime 的能力,作为 Dubbo 多语言生态的关键解决方案。</p> |
| <p>相关链接:</p> |
| <p>【1】Dubbo-go:https://github.com/apache/dubbo-go</p> |
| <p>【2】Pixiu:https://github.com/apache/dubbo-go-pixiu</p> |
| <p>冯振宇,Apache Dubbo Committer,目前负责管理香港一家消费品公司的 IT 部门整个团队。2020 年夏天 偶然看到了介绍 dubbogo 的文章后加入了 dubbogo 社区,目前在主导 Pixiu 0.4.0 版本的开发。</p></description></item></channel></rss> |