| <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Dubbo – 开发者指南</title><link>https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/</link><description>Recent content in 开发者指南 on Apache Dubbo</description><generator>Hugo -- gohugo.io</generator><language>zh-cn</language><atom:link href="https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/index.xml" rel="self" type="application/rss+xml"/><item><title>Overview: Pixiu Filter体系介绍</title><link>https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/filter-extension/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/filter-extension/</guid><description> |
| <h2 id="怎样编写一个filter"><strong>怎样编写一个Filter</strong></h2> |
| <p><code>更详细的信息，请移步Blog《谈谈Pixiu的Filter》</code></p> |
| <p>我们来尝试写一个简单的Filter，这个Filter将会有简单的配置，在Decode阶段把请求的Body Log出来，并翻转后作为Mock的返回值。最后在Encode阶段根据配置把返回值Log出来。</p> |
| <p>1.首先创建一个Filter</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> DemoFilter <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> logPrefix <span style="color:#dc322f">string</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// Decode阶段，发生在调用Upstream之前 |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilter) <span style="color:#268bd2">Decode</span>(ctx <span style="color:#719e07">*</span>contexthttp.HttpContext) filter.FilterStatus { |
| </span></span><span style="display:flex;"><span> body, _ <span style="color:#719e07">:=</span> ioutil.<span style="color:#268bd2">ReadAll</span>(ctx.Request.Body) |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;request body: %s&#34;</span>, body) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//reverse res str |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> runes <span style="color:#719e07">:=</span> []<span style="color:#b58900">rune</span>(<span style="color:#b58900">string</span>(body)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">for</span> i <span style="color:#719e07">:=</span> <span style="color:#2aa198">0</span>; i &lt; <span style="color:#b58900">len</span>(runes)<span style="color:#719e07">/</span><span style="color:#2aa198">2</span>; i <span style="color:#719e07">+=</span> <span style="color:#2aa198">1</span> { |
| </span></span><span style="display:flex;"><span> runes[i], runes[<span style="color:#b58900">len</span>(runes)<span style="color:#719e07">-</span><span style="color:#2aa198">1</span><span style="color:#719e07">-</span>i] = runes[<span style="color:#b58900">len</span>(runes)<span style="color:#719e07">-</span><span style="color:#2aa198">1</span><span style="color:#719e07">-</span>i], runes[i] |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> reverse <span style="color:#719e07">:=</span> <span style="color:#b58900">string</span>(runes) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">//mock response |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> ctx.<span style="color:#268bd2">SendLocalReply</span>(<span style="color:#2aa198">200</span>, []<span style="color:#b58900">byte</span>(reverse)) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> filter.Stop |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#586e75">// Encode阶段，此时可以获取到Upstream的Response |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilter) <span style="color:#268bd2">Encode</span>(ctx <span style="color:#719e07">*</span>contexthttp.HttpContext) filter.FilterStatus { |
| </span></span><span style="display:flex;"><span> res <span style="color:#719e07">:=</span> ctx.SourceResp.(<span style="color:#dc322f">string</span>) |
| </span></span><span style="display:flex;"><span> logger.<span style="color:#268bd2">Infof</span>(<span style="color:#2aa198">&#34;%s: %s&#34;</span>, f.logPrefix, res) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> filter.Continue |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>2.创建Filter Factory</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#268bd2">type</span> ( |
| </span></span><span style="display:flex;"><span> DemoFilterFactory <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> conf <span style="color:#719e07">*</span>Config |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span> <span style="color:#586e75">// Config describe the config of Filter |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span> Config <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span> LogPrefix <span style="color:#dc322f">string</span> <span style="color:#2aa198">`yaml:&#34;logPrefix,omitempty&#34;`</span> |
| </span></span><span style="display:flex;"><span> } |
| </span></span><span style="display:flex;"><span>) |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilterFactory) <span style="color:#268bd2">PrepareFilterChain</span>(ctx <span style="color:#719e07">*</span>contexthttp.HttpContext, chain filter.FilterChain) <span style="color:#dc322f">error</span> { |
| </span></span><span style="display:flex;"><span> demo <span style="color:#719e07">:=</span> <span style="color:#719e07">&amp;</span>DemoFilter{logPrefix: f.conf.LogPrefix} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span> chain.<span style="color:#268bd2">AppendDecodeFilters</span>(demo) |
| </span></span><span style="display:flex;"><span> chain.<span style="color:#268bd2">AppendEncodeFilters</span>(demo) |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilterFactory) <span style="color:#268bd2">Config</span>() <span style="color:#268bd2">interface</span>{} { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> f.conf |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (f <span style="color:#719e07">*</span>DemoFilterFactory) <span style="color:#268bd2">Apply</span>() <span style="color:#dc322f">error</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>3.创建Filter Plugin，并注册自己</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#586e75">//important |
| </span></span></span><span style="display:flex;"><span><span style="color:#586e75"></span><span style="color:#268bd2">func</span> <span style="color:#268bd2">init</span>() { |
| </span></span><span style="display:flex;"><span> filter.<span style="color:#268bd2">RegisterHttpFilter</span>(<span style="color:#719e07">&amp;</span>Plugin{}) |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">type</span> Plugin <span style="color:#268bd2">struct</span> { |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>Plugin) <span style="color:#268bd2">Kind</span>() <span style="color:#dc322f">string</span> { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#2aa198">&#34;dgp.filters.demo&#34;</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span><span style="color:#268bd2">func</span> (p <span style="color:#719e07">*</span>Plugin) <span style="color:#268bd2">CreateFilterFactory</span>() (filter.HttpFilterFactory, <span style="color:#dc322f">error</span>) { |
| </span></span><span style="display:flex;"><span> <span style="color:#719e07">return</span> <span style="color:#719e07">&amp;</span>DemoFilterFactory{conf: <span style="color:#719e07">&amp;</span>Config{}}, <span style="color:#cb4b16">nil</span> |
| </span></span><span style="display:flex;"><span>} |
| </span></span></code></pre></div><p>4.配置文件中配置此Filter，并启动Pixiu</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#268bd2">static_resources</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">listeners</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: <span style="color:#2aa198">&#34;net/http&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">protocol_type</span>: <span style="color:#2aa198">&#34;HTTP&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">socket_address</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">address</span>: <span style="color:#2aa198">&#34;0.0.0.0&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">port</span>: <span style="color:#2aa198">8888</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">filter_chains</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">filters</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: dgp.filter.httpconnectionmanager |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">config</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">route_config</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">routes</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">match</span>: |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">prefix</span>: <span style="color:#2aa198">&#34;/&#34;</span> |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">http_filters</span>: |
| </span></span><span style="display:flex;"><span> - <span style="color:#268bd2">name</span>: dgp.filters.demo |
| </span></span><span style="display:flex;"><span> <span style="color:#268bd2">config</span>: |
| </span></span></code></pre></div><p>5.访问并查看日志与结果</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>curl localhost:8888/demo -d <span style="color:#2aa198">&#34;eiv al tse’c&#34;</span> |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>c’est la vie% |
| </span></span></code></pre></div><p>日志</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>2022-02-19T20:20:11.900+0800 INFO demo/demo.go:62 request body: eiv al tse’c |
| </span></span><span style="display:flex;"><span>2022-02-19T20:20:11.900+0800 INFO demo/demo.go:71 : eiv al tse’c |
| </span></span></code></pre></div></description></item><item><title>Overview: dubbo-pilot Control Plane 部署</title><link>https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/dubbo-pilot/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/dubbo-pilot/</guid><description> |
| <ul> |
| <li><a href="#target">1.总体目标</a></li> |
| <li><a href="#basic">2.基本流程</a></li> |
| <li><a href="#detail">3.详细步骤</a> |
| <ul> |
| <li><a href="#env">3.1 环境要求</a></li> |
| <li><a href="#native_deploy">3.2 istio 本地部署</a> |
| <ul> |
| <li><a href="#nbuild">3.2.1 编译</a></li> |
| <li><a href="#ndeploy">3.2.2 部署 &amp; debug</a></li> |
| </ul> |
| </li> |
| <li><a href="#docker_deploy">3.3 istio 容器部署</a> |
| <ul> |
| <li><a href="#dbuild">3.3.1 编译</a></li> |
| <li><a href="#ddeploy">3.3.2 部署 &amp; debug</a></li> |
| </ul> |
| </li> |
| </ul> |
| </li> |
| </ul> |
| <h2 id="target">1 总体目标</h2> |
| <ul> |
| <li>控制面编译和镜像构建</li> |
| <li>使用 istioctl 在 kubernetes 环境部署</li> |
| <li>如何对控制面程序 debug</li> |
| </ul> |
| <h2 id="basic">2 基本流程</h2> |
| 这个例子将演示如何在编译 dubbo-pilot 控制平面并在 kubernetes 环境下如何使用 istioctl 进行部署 |
| <ol> |
| <li>本地启动控制平面，对 dubbo-pilot 进行启动和 debug</li> |
| <li>使用 istioctl 在 k8s 环境启动和 debug</li> |
| </ol> |
| <h2 id="detail">3 详细步骤</h2> |
| <h3 id="env">3.1 环境要求</h3> |
| <ul> |
| <li>Golang</li> |
| <li>Docker</li> |
| <li>Minikube/Kind</li> |
| <li>Kubectl</li> |
| <li>Dlv</li> |
| </ul> |
| <h3 id="native_deploy">3.2 本地部署</h3> |
| <h4 id="nbuild">3.2.1 编译</h4> |
| <ol> |
| <li>编译 docker-builder</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>cd dubbo-go-pixiu/tools/docker-builder &amp;&amp; go install |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>docker-builder -h: |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>Builds Istio docker images |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>Usage: |
| </span></span><span style="display:flex;"><span> [flags] |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>Flags: |
| </span></span><span style="display:flex;"><span> --architecures strings architectures to build (default [linux/amd64]) |
| </span></span><span style="display:flex;"><span> --base-version string base version to use (default &#34;latest&#34;) |
| </span></span><span style="display:flex;"><span> --builder string type of builder to use. options are crane or docker (default &#34;docker&#34;) |
| </span></span><span style="display:flex;"><span> -h, --help help for this command |
| </span></span><span style="display:flex;"><span> --hub strings docker hub(s) (default [localhost:5000]) |
| </span></span><span style="display:flex;"><span> --istio-version string istio version to use (default &#34;1.14-dev&#34;) |
| </span></span><span style="display:flex;"><span> --kind-load kind cluster to load into |
| </span></span><span style="display:flex;"><span> --no-cache disable caching |
| </span></span><span style="display:flex;"><span> --no-clobber do not allow pushing images that already exist |
| </span></span><span style="display:flex;"><span> --proxy-version string proxy version to use (default &#34;7ae8e27f274b33dc2f4d83100aea5971ed6698d3&#34;) |
| </span></span><span style="display:flex;"><span> --push push targets to registry |
| </span></span><span style="display:flex;"><span> --save save targets to tar.gz |
| </span></span><span style="display:flex;"><span> --tag strings docker tag(s) (default [latest]) |
| </span></span><span style="display:flex;"><span> --targets strings targets to build (default [app,app_sidecar_centos_7,app_sidecar_debian_11,app_sidecar_ubuntu_jammy,app_sidecar_ubuntu_xenial,ext-authz,install-cni,istioctl,operator,pilot,proxyv2]) |
| </span></span><span style="display:flex;"><span> --variants strings variants to build (default [default]) |
| </span></span><span style="display:flex;"><span> --version show build version |
| </span></span></code></pre></div><ol start="2"> |
| <li>使用 docker-builder 自动编译 &amp;&amp; 构建镜像</li> |
| </ol> |
| <p>编译 istioctl</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>docker-builder --targets istioctl |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>编译完成： |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>ls out/linux_amd64/ |
| </span></span><span style="display:flex;"><span>istioctl logs pilot-agent pilot-discovery |
| </span></span></code></pre></div><p>编译 dubbo-pilot 并推送到私有镜像仓库</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>tools/docker-builder/docker-builder --targets pilot --hub docker.io/bobtthp --push |
| </span></span></code></pre></div><h4 id="ndeploy">3.2.2 本地部署</h4> |
| <p>本地启动方式：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>./out/linux_amd64/pilot-discovery |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>启动日志： |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751245Z info FLAG: --caCertFile=&#34;&#34; |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751277Z info FLAG: --clusterAliases=&#34;[]&#34; |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751280Z info FLAG: --clusterID=&#34;Kubernetes&#34; |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751282Z info FLAG: --clusterRegistriesNamespace=&#34;istio-system&#34; |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751284Z info FLAG: --configDir=&#34;&#34; |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751286Z info FLAG: --ctrlz_address=&#34;localhost&#34; |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751289Z info FLAG: --ctrlz_port=&#34;9876&#34; |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.751291Z info FLAG: --domain=&#34;cluster.local&#34; |
| </span></span><span style="display:flex;"><span>... |
| </span></span><span style="display:flex;"><span>2022-09-24T15:31:56.753814Z info initializing mesh configuration ./etc/istio/config/mesh |
| </span></span></code></pre></div><h3 id="docker_deploy">3.3 容器部署</h3> |
| <h4 id="dbuild">3.3.1 镜像构建</h4> |
| <p>构建远程 debug 镜像</p> |
| <ol> |
| <li>下载 dlv</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>git clone https://github.com/go-delve/delve.git |
| </span></span><span style="display:flex;"><span>make install |
| </span></span><span style="display:flex;"><span> |
| </span></span><span style="display:flex;"><span>which dlv |
| </span></span><span style="display:flex;"><span>/root/go/bin/dlv |
| </span></span></code></pre></div><ol start="2"> |
| <li>Dockerfile 增加dlv dubbo-go-pixiu/pilot/docker/Dockerfile.pilot:</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>COPY ${TARGETARCH:-amd64}/dlv /usr/local/bin/dlv |
| </span></span></code></pre></div><ol start="3"> |
| <li>拷贝 dlv 至镜像挂载目录中：</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>cp /root/go/bin/dlv out/linux_amd64/dockerx_build/build.docker.pilot/amd64/ |
| </span></span></code></pre></div><ol start="4"> |
| <li>debug 镜像构建并推送:</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>docker-builder --targets pilot --hub docker.io/bobtthp --push --tag debug |
| </span></span></code></pre></div><ol start="5"> |
| <li>本地也可以查看镜像构建情况：</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>[root~master-1] /tmp/dubbo-go-pixiu/tools/docker-builder&gt; docker images |
| </span></span><span style="display:flex;"><span>REPOSITORY TAG IMAGE ID CREATED SIZE |
| </span></span><span style="display:flex;"><span>bobtthp/pilot latest 7b1aadd55120 13 minutes ago 262MB |
| </span></span></code></pre></div><h4 id="ddeploy">3.3.2 部署</h4> |
| <ol> |
| <li>使用刚构建的镜像部署:</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>out/linux_amd64/istioctl --set .values.pilot.image=bobtthp/pilot:debug install |
| </span></span></code></pre></div><ol start="2"> |
| <li>查看部署情况：</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>[root~master] /tmp/dubbo-go-pixiu/tools/docker-builder&gt; kubectl get po -n istio-system |
| </span></span><span style="display:flex;"><span>NAME READY STATUS RESTARTS AGE |
| </span></span><span style="display:flex;"><span>istiod-fd5d9f77-2ncjq 1/1 Running 0 18m |
| </span></span></code></pre></div><ol start="3"> |
| <li>进入容器远程 debug:</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>[root~master-1] /tmp/dubbo-go-pixiu&gt; kubectl exec -it -n istio-system istiod-fd5d9f77-2ncjq bash |
| </span></span><span style="display:flex;"><span>kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. |
| </span></span></code></pre></div><ol start="4"> |
| <li>启动 dlv:</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>istio-proxy@istiod-fd5d9f77-2ncjq:/$ dlv --listen=:8015 --headless=true --api-version=2 --log attach `ps -ef |grep pilot-discovery| awk &#39;{print $2}&#39;` |
| </span></span><span style="display:flex;"><span>2022-11-04T15:43:14Z error layer=debugger could not create config directory: mkdir /home/istio-proxy/.config: read-only file system |
| </span></span><span style="display:flex;"><span>API server listening at: [::]:8015 |
| </span></span><span style="display:flex;"><span>2022-11-04T15:43:14Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted) |
| </span></span><span style="display:flex;"><span>2022-11-04T15:43:14Z info layer=debugger attaching to pid 1 |
| </span></span></code></pre></div><ol start="5"> |
| <li>对外暴露端口：</li> |
| </ol> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-fallback" data-lang="fallback"><span style="display:flex;"><span>[root~master-1] /tmp&gt; kubectl port-forward -n istio-system istiod-fd5d9f77-2ncjq 8015:8015 |
| </span></span><span style="display:flex;"><span>Forwarding from 127.0.0.1:8015 -&gt; 8015 |
| </span></span><span style="display:flex;"><span>Forwarding from [::1]:8015 -&gt; 8015 |
| </span></span></code></pre></div><ol start="6"> |
| <li>可以进行远程调试</li> |
| </ol></description></item><item><title>Overview: Trie 前缀树介绍</title><link>https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/trie/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://dubbo.apache.org/zh-cn/overview/mannual/dubbo-go-pixiu/dev/trie/</guid><description> |
| <h1 id="简介">简介</h1> |
| <p><img src="https://dubbo.apache.org/imgs/pixiu/trie-1.png" alt="image.png"><br />网关的核心之一是路由逻辑，决定一个请求需要经过怎样的加工，被转发到哪个下行服务。<br />其中 80% 的路由需求表达都以 URL 为基础。需要描述清楚具有某个特征的 URL 或者 URL 集合对应怎样的一系列下游处理策略。</p> |
| <p>例如，&rsquo;/test/<strong>&rsquo; 开头的 URL 路由到测试环境集群，&rsquo;/release/user/</strong>&rsquo; 开头的 URL 会被路由到正式环境的 user 服务集群。</p> |
| <p>同时网关作为所有请求的入口，每一毫秒的延时都会做用在全量的业务下，在 mesh 场景下，延时还会随着调用链路的加深，被倍数放大。按照生产环境业务相应 &lt;=7 毫秒的标准来看，规则匹配的性能要求也是十分苛刻的。一定不能随着规则数目的增加而性能退化。</p> |
| <h1 id="使用介绍">使用介绍</h1> |
| <p>仅从使用方的角度阐述 pixiu 的配置文件如何描述 URL 相关的路由规则。（下面，我们介绍一下如何配置 URL 路由规则）<br />如下是一份 pixiu 的 api 配置文件，这份配置文件会被解析后生成一份对应的内存模型，作为 pixiu 路由相关配置的初始状态。之后由 RDS 协议修改解析后得到的内存模型，实现路由逻辑动态生效的效果。RDS 协议（RDS：xDS 协议下描述路由规则的部分）相关内容是后话不详细阐述。我们把注意力聚焦到resource部分。<br />resource 下 path 部分就是上文阐述的，URL 相关的路由描述。意思是满足 path 描述特征的 URL 会被成功匹配。</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-json" data-lang="json"><span style="display:flex;"><span>name: server |
| </span></span><span style="display:flex;"><span>description: server sample |
| </span></span><span style="display:flex;"><span>resources: |
| </span></span><span style="display:flex;"><span> - path: &#39;/api/v<span style="color:#2aa198">1</span>/test-dubbo/user/name/:name&#39; |
| </span></span><span style="display:flex;"><span> type: restful |
| </span></span><span style="display:flex;"><span> description: user |
| </span></span><span style="display:flex;"><span> methods: |
| </span></span><span style="display:flex;"><span> - httpVerb: GET |
| </span></span><span style="display:flex;"><span> enable: <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> timeout: <span style="color:#2aa198">1000</span>ms |
| </span></span><span style="display:flex;"><span> inboundRequest: |
| </span></span><span style="display:flex;"><span> requestType: http |
| </span></span><span style="display:flex;"><span> uri: |
| </span></span><span style="display:flex;"><span> - name: name |
| </span></span><span style="display:flex;"><span> required: <span style="color:#cb4b16">true</span> |
| </span></span><span style="display:flex;"><span> integrationRequest: |
| </span></span><span style="display:flex;"><span> requestType: dubbo |
| </span></span><span style="display:flex;"><span> mappingParams: |
| </span></span><span style="display:flex;"><span> - name: uri.name |
| </span></span><span style="display:flex;"><span> mapTo: <span style="color:#2aa198">0</span> |
| </span></span><span style="display:flex;"><span> mapType: <span style="color:#2aa198">&#34;string&#34;</span> |
| </span></span><span style="display:flex;"><span> applicationName: <span style="color:#2aa198">&#34;UserProvider&#34;</span> |
| </span></span><span style="display:flex;"><span> interface: <span style="color:#2aa198">&#34;com.dubbogo.server.UserService&#34;</span> |
| </span></span><span style="display:flex;"><span> method: <span style="color:#2aa198">&#34;GetUserByName&#34;</span> |
| </span></span><span style="display:flex;"><span> group: <span style="color:#2aa198">&#34;test&#34;</span> |
| </span></span><span style="display:flex;"><span> version: <span style="color:#2aa198">1.0</span>.<span style="color:#2aa198">0</span> |
| </span></span><span style="display:flex;"><span> clusterName: <span style="color:#2aa198">&#34;test_dubbo&#34;</span> |
| </span></span></code></pre></div><p>被匹配后的请求会被转化成 dubbo 协议转发到 test_dubbo 集群调用 com.dubbogo.server.UserService 下的 GetUserByName 服务。<br />我们继续聚焦到如下范围：</p> |
| <div class="highlight"><pre tabindex="0" style="color:#93a1a1;background-color:#002b36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-json" data-lang="json"><span style="display:flex;"><span>path: &#39;/api/v<span style="color:#2aa198">1</span>/test-dubbo/user/name/:name&#39; |
| </span></span></code></pre></div><p>为了描述清楚一个 URL 或者一组 URL，路由引擎需要拥有以下能力：</p> |
| <ol> |
| <li>URL 可以包含变量，&rsquo;/api/v1/test-dubbo/user/name/:name&rsquo; 代表 URL 用“/”分割后，第六个部分的值作为变量 name 的值，向下游 filter 传递供filter使用。</li> |
| <li>需要有通配符 |
| <ol> |
| <li> |
| <ul> |
| <li>代表一个层级任意字符的通配 &lsquo;/api/*/test-dubbo/user/name/:name&rsquo; 这样的一个 path 描述代表了可能并不关心具体的版本，不论什么版本下的 URL 只要匹配都使用相同的逻辑加工数据并转发。</li> |
| </ul> |
| </li> |
| <li>** 代表多个层级的通配，从这个层级以后，子层级也可以是任意字符，<strong>只可能存在于 URL 的末尾，不然会有二义性。&rsquo;/api/v1/</strong>&rsquo; 这样的 path 表达了所有 V1 版本下的 URL 都采用相同的逻辑。</li> |
| </ol> |
| </li> |
| </ol> |
| <p>为了正确的使用 pixiu 您可能还需要了解如下内容。</p> |
| <h2 id="优先级">优先级</h2> |
| <p>并非是独创的，类似 java 下的 spring 以及其他框架统一具有的优先级逻辑：</p> |
| <ol> |
| <li>通配的优先级低于特指 。 &lsquo;/api/v1/**&rsquo; 低于 &lsquo;/api/v1/test-dubbo/user/name/:name&rsquo; 的优先级，假设有两个 resource 分别采用了如上两个path 配置，request 为 &lsquo;/api/v1/test-dubbo/user/name/yqxu&rsquo; 的请求到达pixiu 后应该生效哪个 resource？按照通配低于特指的原则，&rsquo;/api/v1/test-dubbo/user/name/:name&rsquo; 这条规则会生效。</li> |
| <li>深度更深的，优先级更高 。 &lsquo;/api/v1/<strong>&rsquo; 对比 /api/v1/test-dubbo/</strong>&rsquo; ，如果请求同时满足如上两个描述， &lsquo;/api/v1/test-dubbo/**&rsquo; 深度更深，会生效。</li> |
| <li>通配符之间 &lsquo;/*&rsquo; 优先级高于 &lsquo;/**&rsquo;</li> |
| <li>变量等同于通配。</li> |
| </ol> |
| <h2 id="冲突处理">冲突处理</h2> |
| <p>优先级规则只是冲突解决策略的一种，才同时匹配多个url描述时，优先级更高的那一种将会生效，然而优先级策略并不能涵盖所有的情况。<br />如果强行配置两条 resource path 完全相同，但是转发到不同的下游服务，这时候就会冲突。pixiu 下应对冲突的方案是 failfast，在 pixiu 初始化阶段，发现配置文件中有冲突的两项规则，则启动失败，让开发者今早发现问题并处理。</p> |
| <h1 id="原理介绍">原理介绍</h1> |
| <p>技术选型之初，以及确定使用pixiu后为了处理一些突发情况，以及应付一些pixiu自身可能存在的bug，开发者需要对pixiu 的路由原理有更深刻的了解。<br />下面，我们将详细介绍路由引擎的相关原理和实现，供感兴趣的同学了解。<br />相信阅读这部分内容的同学一定会有人下意识联想到字典树这个结构。使用字典树这个结构能实现存量规则数无关的匹配性能优化。</p> |
| <p>一个存放字符串作为node的字典树，具有表达url 的能力。<br /><img src="https://dubbo.apache.org/imgs/pixiu/trie-2.png" alt="img"><br />如上图描述等价于URL集合 &lsquo;/api/v1&rsquo; ，&rsquo;/api/v2&rsquo; ，&rsquo;/web&rsquo;</p> |
| <p>维护一个标准字典树有几个关键的操作</p> |
| <ol> |
| <li>字典树指定节点的查找（find）: 从root 开始遍历字典书，&rsquo;/api/v2&rsquo; 称之为路径，在当前层级寻找指定路径，如果存在就继续在子树下完成剩下的路径匹配。&rsquo;/api/v2&rsquo; 先从 logic root 找到 &lsquo;/api&rsquo; ，并在 &lsquo;/api&rsquo; 的子树下继续查找剩下的路径 &lsquo;/v2&rsquo; 。</li> |
| <li>字典树节点的添加（add）： 尝试查找指定节点，如果指定节点不存在则新建一个节点。假设一个空树状态下添加 &lsquo;/api/v1&rsquo; ，因为是空树那么logic root 下查找 &lsquo;/api&rsquo; 一定不存在，则在 root 下创建 &lsquo;/api&rsquo; ，继续在创建的 &lsquo;/api&rsquo; 节点下查找 &lsquo;/v1&rsquo; 因为 &lsquo;/api&rsquo; 是新建的 v1 一定也不存在，则继续创建v1</li> |
| <li>字典树url匹配（match）：在这个最简单的版本下，匹配逻辑与指定节点的查找逻辑没有区别。</li> |
| </ol> |
| <p>还有一些不涉及递归或者复用上面逻辑递归操作的简单操作</p> |
| <ol start="4"> |
| <li>修改字典树节点（modify）：通过 find 逻辑找到指定节点，调用 set 方法或者直接赋值的方式修改节点内容。</li> |
| <li>删除字典树节点（delete）: 通过 modify 逻辑修改 isdeleted 标为 true，并把节点内容 modify 为空。节点本身的内存不释放。</li> |
| <li>重建字典树（rebuild）：遍历所有节点，添加到新树，如果 isdeleted为 true 则不添加到新树，通过rebuild 操作创建副本。</li> |
| </ol> |
| <p>由上可知，标准字典树结构距离通用的路由引擎底层数据结构能力还有一定差距，缺乏统配描述能力，缺乏变量表达的能力，下面我们来看一下如何进行改进。</p> |
| <p>添加 描述统配逻辑的子树，作为子树中默认存在的一部分<br /><img src="https://dubbo.apache.org/imgs/pixiu/trie-3.png" alt="img"><br />现在我们的变种字典树多了变量表达能力<br />&rsquo;/web/:appname/test/*&rsquo; 这样的url 在图中应该怎么表达？<br />没错就是这个路径<br /></p> |
| <p><img src="https://dubbo.apache.org/imgs/pixiu/trie-4.png" alt="img"></p> |
| <p>继续分析字典树几个关键的操作是否需要做变化？</p> |
| <ol> |
| <li>字典树指定节点的查找 ： |
| <ol> |
| <li>如果不改动使用前一版本逻辑在 &lsquo;/<em>&rsquo; 节点处理之前都不会有问题： 从root 开始遍历字典书，&rsquo;/api/v2/</em>&rsquo; 称之为路径，在当前层级寻找指定路径，如果存在就继续在子树下完成剩下的路径匹配。 /api/v2 先从 logic root 找到 &lsquo;/api&rsquo; ，并在 &lsquo;/api&rsquo; 的子树下继续查找剩下的路径 &lsquo;/v2&rsquo; 。</li> |
| <li>这版本我们加上对 &lsquo;/<em>&rsquo; 节点的处理：&rsquo;/v2&rsquo; 后是 &lsquo;/</em>&rsquo; ，&rsquo;/<em>&rsquo; 对应单级通配节点，继续递归查找 &lsquo;/v2&rsquo; 节点下一级通配节点是否为空。如果 path 是 &lsquo;/api/v2/</em>/test2&rsquo; 这样的路径则继续在统配子树下完成递归过程。</li> |
| </ol> |
| </li> |
| <li>字典树节点的添加 ： |
| <ol> |
| <li>在添加 &lsquo;/<em>&rsquo; 节点之前，所有逻辑上一版本就足够处理：尝试查找指定节点，如果指定节点不存在则新建一个节点。假设一个空树状态下添加 &lsquo;/api/v1/</em>&rsquo; ，因为是空树那么 logic root 下查找 &lsquo;/api&rsquo; 一定不存在，则在 root 下创建 &lsquo;/api&rsquo; ，继续在创建的 &lsquo;/api&rsquo; 节点下查找 &lsquo;/v1&rsquo; 因为 &lsquo;/api&rsquo; 是新建的 v1 一定也不存在，则继续创建 v1。</li> |
| <li>这版本加上 &lsquo;/*&rsquo; 的特殊处理 ：&rsquo;/v1&rsquo; 新建后，查看通配子树，通配子树不存在，则为V1 节点添加内容为空的单级通配子树并在子树中继续递归。</li> |
| </ol> |
| </li> |
| <li>字典树url匹配：在这个版本下，对比查找逻辑需要增加回朔逻辑。 |
| <ol> |
| <li>在遇到通配节点前逻辑与find 依旧相同 ： 从root 开始遍历字典书，&rsquo;/api/v2/*&rsquo; 称之为路径，在当前层级寻找指定路径，如果存在就继续在子树下完成剩下的路径匹配。 &lsquo;/api/v2&rsquo; 先从 logic root 找到 &lsquo;/api&rsquo; ，并在 &lsquo;/api&rsquo; 的子树下继续查找剩下的路径 &lsquo;/v2&rsquo; 。</li> |
| <li>在处理统配节点的时候会与 find 逻辑有所不同：&rsquo;/v2&rsquo; 下普通子树无匹配节点，回朔到通配子树，查看是否能匹配，这个例子中 &lsquo;/v2&rsquo; 下无通配子树，查询不到节点 。值得注意的是回朔逻辑的先后顺序，是先找普通子树再回朔到通配子树还是先查找通配子树再回朔到普通子树是取决于优先级规则的，按照需求必须是先查找普通子树。</li> |
| </ol> |
| </li> |
| </ol> |
| <p>但是我们目前还是缺乏 &lsquo;/<strong>&rsquo; 这种通配的表达能力代表了多级通配，可以分析需求得到结论，这种通配符，一定不存在子树，是一种特殊的叶子结点，仅用于 match 逻辑回朔时做特殊判断。继续加点特殊 node 后演化为：<br /><img src="https://dubbo.apache.org/imgs/pixiu/trie-5.png" alt="img"><br />好了至此，需求都能满足了。<br />&rsquo;/api/</strong>&rsquo; 等价路径为：<br /><img src="https://dubbo.apache.org/imgs/docs3-v2/dubbo-go-pixiu/dev/trie/1642993180981-51a0df19-bb03-49c8-9128-a6e95dbabfcd.png" alt="img"><br />其他逻辑大同小异，match 逻辑回朔再多一级判断，如果一级通配子树也匹配不到结果，则再看一下多级通配子树是否为空（其实留一个标位就可以，为了统一模型好理解，还是用一个子树去描述）</p> |
| <p>到目前这个版本所有上文提到的能力已经都能有效支撑，回头分析一下时间复杂读。<br />url 被 &lsquo;/&rsquo; 分割出一个一个的段，容易理解在匹配一个url 过程中复杂度是 O(n) n= url 段数。与树中存有的规则数量无关。再分析 n 的范围，n 其实不是一个可以无限大的数字，一部分浏览器甚至约束 url 长度必须小于 2000，按照一个单词长度为 5 来计算，可以大概估计段数最多会在 400 左右，n 如果可以被视为一个常数，那么复杂读可以看作是 O（1）。</p> |
| <p>稍微解释一下find 和 match 有什么不同，为什么需要两种查找节点的方法。看下这个例子 :假设树中已经add 了 &lsquo;/api/v1/:name/add&rsquo; 这个 path，那么<br />find（&quot;/api/v1/:name/add&quot;），find（&quot;/api/v1/*/add&quot;）两个调用应该能够拿到结果，在add 的过程中用于冲突判断。<br />假设有请求进来url 为 &lsquo;/api/v1/:name/add&rsquo; 那么match（&quot;/api/v1/:name/list&quot;）也应该能 match 到结果且变量name 为 :name。<br />再假设有请求进来 url 为 &lsquo;/api/v1/yq/add&rsquo; 那么match（&quot;/api/v1/yq/list&quot;）也应该能 match 到结果且变量name 为 yq 。find（&quot;/api/v1/yq/add&quot; ） 则不会匹配到结果。</p> |
| <h1 id="后续改进">后续改进</h1> |
| <p>目前实现在读树和写树之前，竞争一把全局锁，竞争失败后自旋直到竞争成功，然后完成读写。<br />解释一下为什么读都需要上锁，因为代码中大量运用了go 的 map 结构。这个结构只要并发读写直接会报如下错误：concurrent map read and map write<br />目前实现如下<br /><img src="https://dubbo.apache.org/imgs/pixiu/trie-6.png" alt="img"><br />引入 command 队列，所有对 trie 的用户写操作先入列，同时做读写分离，分为读树和写树，维护一个线程负责追 log 把 command 写入到写树，读树因为只读，没有写入线程操作读树所以可以不加锁。写树因为只有一条线程向树内写入，没有竞争问题，也可以不加锁。（写入操作并不会很频繁单线程完全能负荷）<br />定义一个配置延迟生效的时间，比如3s<br />每3秒，读树和写树角色切换，每个 trie 分别维护一个 command 队列的游标，游标代表当前这个 trie，追 log 追到了哪条记录，写入线程每3s 切换写游标引用。<br /></p> |
| <p><img src="https://dubbo.apache.org/imgs/pixiu/trie-7.png" alt="img"></p> |
| <p>如上图，最上面部分是一个先进先出的 command 队列，追 log 线程从这个队列中读取用户写操作，这个队列维护了两个游标 index1，index2，index1 代表了trie1 追 log 追到了 index1 的位置，index2 代表了 trie2 追 log 追到了 index2 的位置。追 log 线程同一时间内只会使用一个引用进行写操作，每次写完树对应的 index 游标下移一格，另一个 trie 引用将被用于读操作，一切读请求将从读引用对应的树中读取。因为追的是同一份 log ，最终一致性是能保证的。</p> |
| <p>切换逻辑：</p> |
| <ol> |
| <li>先使追 log 线程空转（不挂起，避免上下文切换，因为马上要恢复）</li> |
| <li>保证两个树都没有写入线程操作</li> |
| <li>切换读引用到另一个树</li> |
| <li>切换写引用到另一个树</li> |
| <li>恢复追 log 线程</li> |
| </ol> |
| <p>pr：<br /><a href="https://github.com/apache/dubbo-go-pixiu/pull/262">https://github.com/apache/dubbo-go-pixiu/pull/262</a><br />pkg/common/router/trie/trie.go:26</p></description></item></channel></rss> |
