[SCB-1295] Add Syncer (#548)
* Initial core code of Syncer
Signed-off-by: linzhinan <linzhinan@huawei.com>
* add License declaration to files in syncer
Signed-off-by: linzhinan <linzhinan@huawei.com>
* Add cmd and main.go for syncer
Signed-off-by: MabinGo <bin.ma@huawei.com>
* Add code comments
* fix bug when testing
* correct SyncerArch png addr
Signed-off-by: linzhinan <linzhinan@huawei.com>
* Correct peer to be serf, Rename store to datacenter.
Signed-off-by: linzhinan <linzhinan@huawei.com>
* Correct peer to Serf
Signed-off-by: linzhinan <linzhinan@huawei.com>
* Refactoring syncer arch, mv storage to syncer server, remove extra events, rename repository to datacenter, uncouple grpc client and server
Signed-off-by: linzhinan <linzhinan@huawei.com>
* fix bug when initialize grpc client/storing data to the hard disk
* Supplementary test case
* add copyright to files
* Fix bugs, when multiple DCs have instances of the same id, Syner overwrites the registered instance
* Decouple status of service-center to its plugin, run gofmt/goimports fot syncer
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* for test
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Add TODO file
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Updata README.md and TODO.md
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Updata README.md and TODO.md
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Update README.md
Fix table issue.
* Update README-ZH.md
Fix table issue.
* Change dos to unix in TODO.md and TODO-ZH.md, add *.iml to gitignore
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Add License of memberlist and serf, introduced by syncer
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* fix typos after run go tools
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* mv storage to data-center, fix bug of clear expired instance in a tick
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Break entry of syncer from service-center to keep syncer independent
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* fix bug to close grpc connection if pull failed
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Add logs to key process
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* correct cmdlilne message
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* Adjust the test code, when syncer independented
* correct datacenter to be servicecenter
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* correct data-center to be service-center in docs
Signed-off-by: Zen Lin <linzhinan@huawei.com>
* correct command line in README.md
Signed-off-by: Zen Lin <linzhinan@huawei.com>
diff --git a/.gitignore b/.gitignore
index df2efa7..3a3ad28 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,8 +4,9 @@
**/coverage.out
**/coverage.txt
**/*.log
+syncer/syncer
-service-center.iml
+*.iml
.idea/
**/*junit.xml
**/*.exe
@@ -17,6 +18,8 @@
# for local UT
**/conf/
+data
+syncer-data
!etc/conf/
frontend/app/bower_components
etc/data/
diff --git a/go.mod b/go.mod
index e0e9c08..bd8dc7b 100644
--- a/go.mod
+++ b/go.mod
@@ -33,23 +33,22 @@
github.com/eapache/go-resiliency v1.1.0 // indirect
github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21 // indirect
github.com/eapache/queue v1.1.0 // indirect
- github.com/fatih/color v1.7.0 // indirect
github.com/fsnotify/fsnotify v1.4.7 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/go-chassis/paas-lager v0.0.0-20180727081842-50655443dc96
github.com/go-logfmt/logfmt v0.3.0 // indirect
- github.com/gogo/protobuf v1.1.1 // indirect
+ github.com/gogo/protobuf v1.1.1
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // indirect
github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef // indirect
github.com/golang/protobuf v1.0.0
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db // indirect
- github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c // indirect
github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf // indirect
github.com/googleapis/gnostic v0.2.0 // indirect
github.com/gorilla/websocket v1.2.0
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.3.0 // indirect
- github.com/hashicorp/golang-lru v0.5.0 // indirect
+ github.com/hashicorp/memberlist v0.1.3
+ github.com/hashicorp/serf v0.8.3
github.com/howeyc/gopass v0.0.0-20170109162249-bf9dde6d0d2c // indirect
github.com/hpcloud/tail v1.0.0 // indirect
github.com/imdario/mergo v0.3.6 // indirect
@@ -61,7 +60,6 @@
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515 // indirect
github.com/labstack/echo v3.2.2-0.20180316170059-a5d81b8d4a62+incompatible
github.com/labstack/gommon v0.2.1 // indirect
- github.com/mattn/go-colorable v0.0.9 // indirect
github.com/mattn/go-isatty v0.0.4 // indirect
github.com/mattn/go-runewidth v0.0.3 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
@@ -97,7 +95,7 @@
go.uber.org/atomic v1.3.2 // indirect
go.uber.org/multierr v1.1.0 // indirect
go.uber.org/zap v1.9.0
- golang.org/x/net v0.0.0-20180824152047-4bcd98cce591
+ golang.org/x/net v0.0.0-20181201002055-351d144fa1fc
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6 // indirect
golang.org/x/text v0.0.0-20170627122817-6353ef0f9243 // indirect
golang.org/x/time v0.0.0-20170424234030-8be79e1e0910 // indirect
diff --git a/scripts/release/LICENSE b/scripts/release/LICENSE
index 0e9dd49..a983724 100644
--- a/scripts/release/LICENSE
+++ b/scripts/release/LICENSE
@@ -748,6 +748,22 @@
You can find a copy of the License at licenses/LICENSE-imdario-mergo
================================================================
+For hashicorp/memberlist (v0.1.3)
+================================================================
+This product bundles the hashicorp/memberlist library which is licensed under
+the MPL 2.0 License.
+For details, see https://github.com/hashicorp/memberlist
+You can find a copy of the License at licenses/LICENSE-hashicorp-memberlist
+
+================================================================
+For hashicorp/serf (v0.8.3)
+================================================================
+This product bundles the hashicorp/serf library which is licensed under
+the MPL 2.0 License.
+For details, see https://github.com/hashicorp/serf
+You can find a copy of the License at licenses/LICENSE-hashicorp-serf
+
+================================================================
For hashicorp/golang-lru (20f1fb78b0740ba8c3cb143a61e86ba5c8669768)
================================================================
This product bundles the hashicorp/golang-lru library which is licensed under
diff --git a/scripts/release/licenses/LICENSE-hashicorp-memberlist b/scripts/release/licenses/LICENSE-hashicorp-memberlist
new file mode 100644
index 0000000..cdff15c
--- /dev/null
+++ b/scripts/release/licenses/LICENSE-hashicorp-memberlist
@@ -0,0 +1,353 @@
+Mozilla Public License, version 2.0
+
+1. Definitions
+
+1.1. “Contributor”
+
+ means each individual or legal entity that creates, contributes to the
+ creation of, or owns Covered Software.
+
+1.2. “Contributor Version”
+
+ means the combination of the Contributions of others (if any) used by a
+ Contributor and that particular Contributor’s Contribution.
+
+1.3. “Contribution”
+
+ means Covered Software of a particular Contributor.
+
+1.4. “Covered Software”
+
+ means Source Code Form to which the initial Contributor has attached the
+ notice in Exhibit A, the Executable Form of such Source Code Form, and
+ Modifications of such Source Code Form, in each case including portions
+ thereof.
+
+1.5. “Incompatible With Secondary Licenses”
+ means
+
+ a. that the initial Contributor has attached the notice described in
+ Exhibit B to the Covered Software; or
+
+ b. that the Covered Software was made available under the terms of version
+ 1.1 or earlier of the License, but not also under the terms of a
+ Secondary License.
+
+1.6. “Executable Form”
+
+ means any form of the work other than Source Code Form.
+
+1.7. “Larger Work”
+
+ means a work that combines Covered Software with other material, in a separate
+ file or files, that is not Covered Software.
+
+1.8. “License”
+
+ means this document.
+
+1.9. “Licensable”
+
+ means having the right to grant, to the maximum extent possible, whether at the
+ time of the initial grant or subsequently, any and all of the rights conveyed by
+ this License.
+
+1.10. “Modifications”
+
+ means any of the following:
+
+ a. any file in Source Code Form that results from an addition to, deletion
+ from, or modification of the contents of Covered Software; or
+
+ b. any new file in Source Code Form that contains any Covered Software.
+
+1.11. “Patent Claims” of a Contributor
+
+ means any patent claim(s), including without limitation, method, process,
+ and apparatus claims, in any patent Licensable by such Contributor that
+ would be infringed, but for the grant of the License, by the making,
+ using, selling, offering for sale, having made, import, or transfer of
+ either its Contributions or its Contributor Version.
+
+1.12. “Secondary License”
+
+ means either the GNU General Public License, Version 2.0, the GNU Lesser
+ General Public License, Version 2.1, the GNU Affero General Public
+ License, Version 3.0, or any later versions of those licenses.
+
+1.13. “Source Code Form”
+
+ means the form of the work preferred for making modifications.
+
+1.14. “You” (or “Your”)
+
+ means an individual or a legal entity exercising rights under this
+ License. For legal entities, “You” includes any entity that controls, is
+ controlled by, or is under common control with You. For purposes of this
+ definition, “control” means (a) the power, direct or indirect, to cause
+ the direction or management of such entity, whether by contract or
+ otherwise, or (b) ownership of more than fifty percent (50%) of the
+ outstanding shares or beneficial ownership of such entity.
+
+
+2. License Grants and Conditions
+
+2.1. Grants
+
+ Each Contributor hereby grants You a world-wide, royalty-free,
+ non-exclusive license:
+
+ a. under intellectual property rights (other than patent or trademark)
+ Licensable by such Contributor to use, reproduce, make available,
+ modify, display, perform, distribute, and otherwise exploit its
+ Contributions, either on an unmodified basis, with Modifications, or as
+ part of a Larger Work; and
+
+ b. under Patent Claims of such Contributor to make, use, sell, offer for
+ sale, have made, import, and otherwise transfer either its Contributions
+ or its Contributor Version.
+
+2.2. Effective Date
+
+ The licenses granted in Section 2.1 with respect to any Contribution become
+ effective for each Contribution on the date the Contributor first distributes
+ such Contribution.
+
+2.3. Limitations on Grant Scope
+
+ The licenses granted in this Section 2 are the only rights granted under this
+ License. No additional rights or licenses will be implied from the distribution
+ or licensing of Covered Software under this License. Notwithstanding Section
+ 2.1(b) above, no patent license is granted by a Contributor:
+
+ a. for any code that a Contributor has removed from Covered Software; or
+
+ b. for infringements caused by: (i) Your and any other third party’s
+ modifications of Covered Software, or (ii) the combination of its
+ Contributions with other software (except as part of its Contributor
+ Version); or
+
+ c. under Patent Claims infringed by Covered Software in the absence of its
+ Contributions.
+
+ This License does not grant any rights in the trademarks, service marks, or
+ logos of any Contributor (except as may be necessary to comply with the
+ notice requirements in Section 3.4).
+
+2.4. Subsequent Licenses
+
+ No Contributor makes additional grants as a result of Your choice to
+ distribute the Covered Software under a subsequent version of this License
+ (see Section 10.2) or under the terms of a Secondary License (if permitted
+ under the terms of Section 3.3).
+
+2.5. Representation
+
+ Each Contributor represents that the Contributor believes its Contributions
+ are its original creation(s) or it has sufficient rights to grant the
+ rights to its Contributions conveyed by this License.
+
+2.6. Fair Use
+
+ This License is not intended to limit any rights You have under applicable
+ copyright doctrines of fair use, fair dealing, or other equivalents.
+
+2.7. Conditions
+
+ Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
+ Section 2.1.
+
+
+3. Responsibilities
+
+3.1. Distribution of Source Form
+
+ All distribution of Covered Software in Source Code Form, including any
+ Modifications that You create or to which You contribute, must be under the
+ terms of this License. You must inform recipients that the Source Code Form
+ of the Covered Software is governed by the terms of this License, and how
+ they can obtain a copy of this License. You may not attempt to alter or
+ restrict the recipients’ rights in the Source Code Form.
+
+3.2. Distribution of Executable Form
+
+ If You distribute Covered Software in Executable Form then:
+
+ a. such Covered Software must also be made available in Source Code Form,
+ as described in Section 3.1, and You must inform recipients of the
+ Executable Form how they can obtain a copy of such Source Code Form by
+ reasonable means in a timely manner, at a charge no more than the cost
+ of distribution to the recipient; and
+
+ b. You may distribute such Executable Form under the terms of this License,
+ or sublicense it under different terms, provided that the license for
+ the Executable Form does not attempt to limit or alter the recipients’
+ rights in the Source Code Form under this License.
+
+3.3. Distribution of a Larger Work
+
+ You may create and distribute a Larger Work under terms of Your choice,
+ provided that You also comply with the requirements of this License for the
+ Covered Software. If the Larger Work is a combination of Covered Software
+ with a work governed by one or more Secondary Licenses, and the Covered
+ Software is not Incompatible With Secondary Licenses, this License permits
+ You to additionally distribute such Covered Software under the terms of
+ such Secondary License(s), so that the recipient of the Larger Work may, at
+ their option, further distribute the Covered Software under the terms of
+ either this License or such Secondary License(s).
+
+3.4. Notices
+
+ You may not remove or alter the substance of any license notices (including
+ copyright notices, patent notices, disclaimers of warranty, or limitations
+ of liability) contained within the Source Code Form of the Covered
+ Software, except that You may alter any license notices to the extent
+ required to remedy known factual inaccuracies.
+
+3.5. Application of Additional Terms
+
+ You may choose to offer, and to charge a fee for, warranty, support,
+ indemnity or liability obligations to one or more recipients of Covered
+ Software. However, You may do so only on Your own behalf, and not on behalf
+ of any Contributor. You must make it absolutely clear that any such
+ warranty, support, indemnity, or liability obligation is offered by You
+ alone, and You hereby agree to indemnify every Contributor for any
+ liability incurred by such Contributor as a result of warranty, support,
+ indemnity or liability terms You offer. You may include additional
+ disclaimers of warranty and limitations of liability specific to any
+ jurisdiction.
+
+4. Inability to Comply Due to Statute or Regulation
+
+ If it is impossible for You to comply with any of the terms of this License
+ with respect to some or all of the Covered Software due to statute, judicial
+ order, or regulation then You must: (a) comply with the terms of this License
+ to the maximum extent possible; and (b) describe the limitations and the code
+ they affect. Such description must be placed in a text file included with all
+ distributions of the Covered Software under this License. Except to the
+ extent prohibited by statute or regulation, such description must be
+ sufficiently detailed for a recipient of ordinary skill to be able to
+ understand it.
+
+5. Termination
+
+5.1. The rights granted under this License will terminate automatically if You
+ fail to comply with any of its terms. However, if You become compliant,
+ then the rights granted under this License from a particular Contributor
+ are reinstated (a) provisionally, unless and until such Contributor
+ explicitly and finally terminates Your grants, and (b) on an ongoing basis,
+ if such Contributor fails to notify You of the non-compliance by some
+ reasonable means prior to 60 days after You have come back into compliance.
+ Moreover, Your grants from a particular Contributor are reinstated on an
+ ongoing basis if such Contributor notifies You of the non-compliance by
+ some reasonable means, this is the first time You have received notice of
+ non-compliance with this License from such Contributor, and You become
+ compliant prior to 30 days after Your receipt of the notice.
+
+5.2. If You initiate litigation against any entity by asserting a patent
+ infringement claim (excluding declaratory judgment actions, counter-claims,
+ and cross-claims) alleging that a Contributor Version directly or
+ indirectly infringes any patent, then the rights granted to You by any and
+ all Contributors for the Covered Software under Section 2.1 of this License
+ shall terminate.
+
+5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
+ license agreements (excluding distributors and resellers) which have been
+ validly granted by You or Your distributors under this License prior to
+ termination shall survive termination.
+
+6. Disclaimer of Warranty
+
+ Covered Software is provided under this License on an “as is” basis, without
+ warranty of any kind, either expressed, implied, or statutory, including,
+ without limitation, warranties that the Covered Software is free of defects,
+ merchantable, fit for a particular purpose or non-infringing. The entire
+ risk as to the quality and performance of the Covered Software is with You.
+ Should any Covered Software prove defective in any respect, You (not any
+ Contributor) assume the cost of any necessary servicing, repair, or
+ correction. This disclaimer of warranty constitutes an essential part of this
+ License. No use of any Covered Software is authorized under this License
+ except under this disclaimer.
+
+7. Limitation of Liability
+
+ Under no circumstances and under no legal theory, whether tort (including
+ negligence), contract, or otherwise, shall any Contributor, or anyone who
+ distributes Covered Software as permitted above, be liable to You for any
+ direct, indirect, special, incidental, or consequential damages of any
+ character including, without limitation, damages for lost profits, loss of
+ goodwill, work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses, even if such party shall have been
+ informed of the possibility of such damages. This limitation of liability
+ shall not apply to liability for death or personal injury resulting from such
+ party’s negligence to the extent applicable law prohibits such limitation.
+ Some jurisdictions do not allow the exclusion or limitation of incidental or
+ consequential damages, so this exclusion and limitation may not apply to You.
+
+8. Litigation
+
+ Any litigation relating to this License may be brought only in the courts of
+ a jurisdiction where the defendant maintains its principal place of business
+ and such litigation shall be governed by laws of that jurisdiction, without
+ reference to its conflict-of-law provisions. Nothing in this Section shall
+ prevent a party’s ability to bring cross-claims or counter-claims.
+
+9. Miscellaneous
+
+ This License represents the complete agreement concerning the subject matter
+ hereof. If any provision of this License is held to be unenforceable, such
+ provision shall be reformed only to the extent necessary to make it
+ enforceable. Any law or regulation which provides that the language of a
+ contract shall be construed against the drafter shall not be used to construe
+ this License against a Contributor.
+
+
+10. Versions of the License
+
+10.1. New Versions
+
+ Mozilla Foundation is the license steward. Except as provided in Section
+ 10.3, no one other than the license steward has the right to modify or
+ publish new versions of this License. Each version will be given a
+ distinguishing version number.
+
+10.2. Effect of New Versions
+
+ You may distribute the Covered Software under the terms of the version of
+ the License under which You originally received the Covered Software, or
+ under the terms of any subsequent version published by the license
+ steward.
+
+10.3. Modified Versions
+
+ If you create software not governed by this License, and you want to
+ create a new license for such software, you may create and use a modified
+ version of this License if you rename the license and remove any
+ references to the name of the license steward (except to note that such
+ modified license differs from this License).
+
+10.4. Distributing Source Code Form that is Incompatible With Secondary Licenses
+ If You choose to distribute Source Code Form that is Incompatible With
+ Secondary Licenses under the terms of this version of the License, the
+ notice described in Exhibit B of this License must be attached.
+
+Exhibit A - Source Code Form License Notice
+
+ This Source Code Form is subject to the
+ terms of the Mozilla Public License, v.
+ 2.0. If a copy of the MPL was not
+ distributed with this file, You can
+ obtain one at
+ http://mozilla.org/MPL/2.0/.
+
+If it is not possible or desirable to put the notice in a particular file, then
+You may include the notice in a location (such as a LICENSE file in a relevant
+directory) where a recipient would be likely to look for such a notice.
+
+You may add additional accurate notices of copyright ownership.
+
+Exhibit B - “Incompatible With Secondary Licenses” Notice
+
+ This Source Code Form is “Incompatible
+ With Secondary Licenses”, as defined by
+ the Mozilla Public License, v. 2.0.
\ No newline at end of file
diff --git a/scripts/release/licenses/LICENSE-hashicorp-serf b/scripts/release/licenses/LICENSE-hashicorp-serf
new file mode 100644
index 0000000..cdff15c
--- /dev/null
+++ b/scripts/release/licenses/LICENSE-hashicorp-serf
@@ -0,0 +1,353 @@
+Mozilla Public License, version 2.0
+
+1. Definitions
+
+1.1. “Contributor”
+
+ means each individual or legal entity that creates, contributes to the
+ creation of, or owns Covered Software.
+
+1.2. “Contributor Version”
+
+ means the combination of the Contributions of others (if any) used by a
+ Contributor and that particular Contributor’s Contribution.
+
+1.3. “Contribution”
+
+ means Covered Software of a particular Contributor.
+
+1.4. “Covered Software”
+
+ means Source Code Form to which the initial Contributor has attached the
+ notice in Exhibit A, the Executable Form of such Source Code Form, and
+ Modifications of such Source Code Form, in each case including portions
+ thereof.
+
+1.5. “Incompatible With Secondary Licenses”
+ means
+
+ a. that the initial Contributor has attached the notice described in
+ Exhibit B to the Covered Software; or
+
+ b. that the Covered Software was made available under the terms of version
+ 1.1 or earlier of the License, but not also under the terms of a
+ Secondary License.
+
+1.6. “Executable Form”
+
+ means any form of the work other than Source Code Form.
+
+1.7. “Larger Work”
+
+ means a work that combines Covered Software with other material, in a separate
+ file or files, that is not Covered Software.
+
+1.8. “License”
+
+ means this document.
+
+1.9. “Licensable”
+
+ means having the right to grant, to the maximum extent possible, whether at the
+ time of the initial grant or subsequently, any and all of the rights conveyed by
+ this License.
+
+1.10. “Modifications”
+
+ means any of the following:
+
+ a. any file in Source Code Form that results from an addition to, deletion
+ from, or modification of the contents of Covered Software; or
+
+ b. any new file in Source Code Form that contains any Covered Software.
+
+1.11. “Patent Claims” of a Contributor
+
+ means any patent claim(s), including without limitation, method, process,
+ and apparatus claims, in any patent Licensable by such Contributor that
+ would be infringed, but for the grant of the License, by the making,
+ using, selling, offering for sale, having made, import, or transfer of
+ either its Contributions or its Contributor Version.
+
+1.12. “Secondary License”
+
+ means either the GNU General Public License, Version 2.0, the GNU Lesser
+ General Public License, Version 2.1, the GNU Affero General Public
+ License, Version 3.0, or any later versions of those licenses.
+
+1.13. “Source Code Form”
+
+ means the form of the work preferred for making modifications.
+
+1.14. “You” (or “Your”)
+
+ means an individual or a legal entity exercising rights under this
+ License. For legal entities, “You” includes any entity that controls, is
+ controlled by, or is under common control with You. For purposes of this
+ definition, “control” means (a) the power, direct or indirect, to cause
+ the direction or management of such entity, whether by contract or
+ otherwise, or (b) ownership of more than fifty percent (50%) of the
+ outstanding shares or beneficial ownership of such entity.
+
+
+2. License Grants and Conditions
+
+2.1. Grants
+
+ Each Contributor hereby grants You a world-wide, royalty-free,
+ non-exclusive license:
+
+ a. under intellectual property rights (other than patent or trademark)
+ Licensable by such Contributor to use, reproduce, make available,
+ modify, display, perform, distribute, and otherwise exploit its
+ Contributions, either on an unmodified basis, with Modifications, or as
+ part of a Larger Work; and
+
+ b. under Patent Claims of such Contributor to make, use, sell, offer for
+ sale, have made, import, and otherwise transfer either its Contributions
+ or its Contributor Version.
+
+2.2. Effective Date
+
+ The licenses granted in Section 2.1 with respect to any Contribution become
+ effective for each Contribution on the date the Contributor first distributes
+ such Contribution.
+
+2.3. Limitations on Grant Scope
+
+ The licenses granted in this Section 2 are the only rights granted under this
+ License. No additional rights or licenses will be implied from the distribution
+ or licensing of Covered Software under this License. Notwithstanding Section
+ 2.1(b) above, no patent license is granted by a Contributor:
+
+ a. for any code that a Contributor has removed from Covered Software; or
+
+ b. for infringements caused by: (i) Your and any other third party’s
+ modifications of Covered Software, or (ii) the combination of its
+ Contributions with other software (except as part of its Contributor
+ Version); or
+
+ c. under Patent Claims infringed by Covered Software in the absence of its
+ Contributions.
+
+ This License does not grant any rights in the trademarks, service marks, or
+ logos of any Contributor (except as may be necessary to comply with the
+ notice requirements in Section 3.4).
+
+2.4. Subsequent Licenses
+
+ No Contributor makes additional grants as a result of Your choice to
+ distribute the Covered Software under a subsequent version of this License
+ (see Section 10.2) or under the terms of a Secondary License (if permitted
+ under the terms of Section 3.3).
+
+2.5. Representation
+
+ Each Contributor represents that the Contributor believes its Contributions
+ are its original creation(s) or it has sufficient rights to grant the
+ rights to its Contributions conveyed by this License.
+
+2.6. Fair Use
+
+ This License is not intended to limit any rights You have under applicable
+ copyright doctrines of fair use, fair dealing, or other equivalents.
+
+2.7. Conditions
+
+ Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
+ Section 2.1.
+
+
+3. Responsibilities
+
+3.1. Distribution of Source Form
+
+ All distribution of Covered Software in Source Code Form, including any
+ Modifications that You create or to which You contribute, must be under the
+ terms of this License. You must inform recipients that the Source Code Form
+ of the Covered Software is governed by the terms of this License, and how
+ they can obtain a copy of this License. You may not attempt to alter or
+ restrict the recipients’ rights in the Source Code Form.
+
+3.2. Distribution of Executable Form
+
+ If You distribute Covered Software in Executable Form then:
+
+ a. such Covered Software must also be made available in Source Code Form,
+ as described in Section 3.1, and You must inform recipients of the
+ Executable Form how they can obtain a copy of such Source Code Form by
+ reasonable means in a timely manner, at a charge no more than the cost
+ of distribution to the recipient; and
+
+ b. You may distribute such Executable Form under the terms of this License,
+ or sublicense it under different terms, provided that the license for
+ the Executable Form does not attempt to limit or alter the recipients’
+ rights in the Source Code Form under this License.
+
+3.3. Distribution of a Larger Work
+
+ You may create and distribute a Larger Work under terms of Your choice,
+ provided that You also comply with the requirements of this License for the
+ Covered Software. If the Larger Work is a combination of Covered Software
+ with a work governed by one or more Secondary Licenses, and the Covered
+ Software is not Incompatible With Secondary Licenses, this License permits
+ You to additionally distribute such Covered Software under the terms of
+ such Secondary License(s), so that the recipient of the Larger Work may, at
+ their option, further distribute the Covered Software under the terms of
+ either this License or such Secondary License(s).
+
+3.4. Notices
+
+ You may not remove or alter the substance of any license notices (including
+ copyright notices, patent notices, disclaimers of warranty, or limitations
+ of liability) contained within the Source Code Form of the Covered
+ Software, except that You may alter any license notices to the extent
+ required to remedy known factual inaccuracies.
+
+3.5. Application of Additional Terms
+
+ You may choose to offer, and to charge a fee for, warranty, support,
+ indemnity or liability obligations to one or more recipients of Covered
+ Software. However, You may do so only on Your own behalf, and not on behalf
+ of any Contributor. You must make it absolutely clear that any such
+ warranty, support, indemnity, or liability obligation is offered by You
+ alone, and You hereby agree to indemnify every Contributor for any
+ liability incurred by such Contributor as a result of warranty, support,
+ indemnity or liability terms You offer. You may include additional
+ disclaimers of warranty and limitations of liability specific to any
+ jurisdiction.
+
+4. Inability to Comply Due to Statute or Regulation
+
+ If it is impossible for You to comply with any of the terms of this License
+ with respect to some or all of the Covered Software due to statute, judicial
+ order, or regulation then You must: (a) comply with the terms of this License
+ to the maximum extent possible; and (b) describe the limitations and the code
+ they affect. Such description must be placed in a text file included with all
+ distributions of the Covered Software under this License. Except to the
+ extent prohibited by statute or regulation, such description must be
+ sufficiently detailed for a recipient of ordinary skill to be able to
+ understand it.
+
+5. Termination
+
+5.1. The rights granted under this License will terminate automatically if You
+ fail to comply with any of its terms. However, if You become compliant,
+ then the rights granted under this License from a particular Contributor
+ are reinstated (a) provisionally, unless and until such Contributor
+ explicitly and finally terminates Your grants, and (b) on an ongoing basis,
+ if such Contributor fails to notify You of the non-compliance by some
+ reasonable means prior to 60 days after You have come back into compliance.
+ Moreover, Your grants from a particular Contributor are reinstated on an
+ ongoing basis if such Contributor notifies You of the non-compliance by
+ some reasonable means, this is the first time You have received notice of
+ non-compliance with this License from such Contributor, and You become
+ compliant prior to 30 days after Your receipt of the notice.
+
+5.2. If You initiate litigation against any entity by asserting a patent
+ infringement claim (excluding declaratory judgment actions, counter-claims,
+ and cross-claims) alleging that a Contributor Version directly or
+ indirectly infringes any patent, then the rights granted to You by any and
+ all Contributors for the Covered Software under Section 2.1 of this License
+ shall terminate.
+
+5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
+ license agreements (excluding distributors and resellers) which have been
+ validly granted by You or Your distributors under this License prior to
+ termination shall survive termination.
+
+6. Disclaimer of Warranty
+
+ Covered Software is provided under this License on an “as is” basis, without
+ warranty of any kind, either expressed, implied, or statutory, including,
+ without limitation, warranties that the Covered Software is free of defects,
+ merchantable, fit for a particular purpose or non-infringing. The entire
+ risk as to the quality and performance of the Covered Software is with You.
+ Should any Covered Software prove defective in any respect, You (not any
+ Contributor) assume the cost of any necessary servicing, repair, or
+ correction. This disclaimer of warranty constitutes an essential part of this
+ License. No use of any Covered Software is authorized under this License
+ except under this disclaimer.
+
+7. Limitation of Liability
+
+ Under no circumstances and under no legal theory, whether tort (including
+ negligence), contract, or otherwise, shall any Contributor, or anyone who
+ distributes Covered Software as permitted above, be liable to You for any
+ direct, indirect, special, incidental, or consequential damages of any
+ character including, without limitation, damages for lost profits, loss of
+ goodwill, work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses, even if such party shall have been
+ informed of the possibility of such damages. This limitation of liability
+ shall not apply to liability for death or personal injury resulting from such
+ party’s negligence to the extent applicable law prohibits such limitation.
+ Some jurisdictions do not allow the exclusion or limitation of incidental or
+ consequential damages, so this exclusion and limitation may not apply to You.
+
+8. Litigation
+
+ Any litigation relating to this License may be brought only in the courts of
+ a jurisdiction where the defendant maintains its principal place of business
+ and such litigation shall be governed by laws of that jurisdiction, without
+ reference to its conflict-of-law provisions. Nothing in this Section shall
+ prevent a party’s ability to bring cross-claims or counter-claims.
+
+9. Miscellaneous
+
+ This License represents the complete agreement concerning the subject matter
+ hereof. If any provision of this License is held to be unenforceable, such
+ provision shall be reformed only to the extent necessary to make it
+ enforceable. Any law or regulation which provides that the language of a
+ contract shall be construed against the drafter shall not be used to construe
+ this License against a Contributor.
+
+
+10. Versions of the License
+
+10.1. New Versions
+
+ Mozilla Foundation is the license steward. Except as provided in Section
+ 10.3, no one other than the license steward has the right to modify or
+ publish new versions of this License. Each version will be given a
+ distinguishing version number.
+
+10.2. Effect of New Versions
+
+ You may distribute the Covered Software under the terms of the version of
+ the License under which You originally received the Covered Software, or
+ under the terms of any subsequent version published by the license
+ steward.
+
+10.3. Modified Versions
+
+ If you create software not governed by this License, and you want to
+ create a new license for such software, you may create and use a modified
+ version of this License if you rename the license and remove any
+ references to the name of the license steward (except to note that such
+ modified license differs from this License).
+
+10.4. Distributing Source Code Form that is Incompatible With Secondary Licenses
+ If You choose to distribute Source Code Form that is Incompatible With
+ Secondary Licenses under the terms of this version of the License, the
+ notice described in Exhibit B of this License must be attached.
+
+Exhibit A - Source Code Form License Notice
+
+ This Source Code Form is subject to the
+ terms of the Mozilla Public License, v.
+ 2.0. If a copy of the MPL was not
+ distributed with this file, You can
+ obtain one at
+ http://mozilla.org/MPL/2.0/.
+
+If it is not possible or desirable to put the notice in a particular file, then
+You may include the notice in a location (such as a LICENSE file in a relevant
+directory) where a recipient would be likely to look for such a notice.
+
+You may add additional accurate notices of copyright ownership.
+
+Exhibit B - “Incompatible With Secondary Licenses” Notice
+
+ This Source Code Form is “Incompatible
+ With Secondary Licenses”, as defined by
+ the Mozilla Public License, v. 2.0.
\ No newline at end of file
diff --git a/scripts/ut_test_in_docker.sh b/scripts/ut_test_in_docker.sh
index 14640af..43cc825 100755
--- a/scripts/ut_test_in_docker.sh
+++ b/scripts/ut_test_in_docker.sh
@@ -46,6 +46,7 @@
[ $? == 0 ] && ut_for_dir pkg
[ $? == 0 ] && ut_for_dir server
[ $? == 0 ] && ut_for_dir scctl
+[ $? == 0 ] && ut_for_dir syncer
ret=$?
if [ ${ret} == 0 ]; then
diff --git a/syncer/README-ZH.md b/syncer/README-ZH.md
new file mode 100644
index 0000000..9e99212
--- /dev/null
+++ b/syncer/README-ZH.md
@@ -0,0 +1,82 @@
+ServiceCenter Syncer
+-------
+[Introduction to English](./README.md)
+
+### 1. 什么是ServiceCenter Syncer
+Syncer是一个多服务中心的同步工具，专为大型微服务架构设计，支持异构服务中心。它坚持以下信念：
+- 对应用程序透明。同步工具启停，不应该对应用程序的原有流程产生影响。
+- 为多服务中心提供对等网络。他们之间是松耦合的，成员可以自由加入与退出。
+- 对异构服务中心提供支持。插件化的形式支持多种服务中心驱动，用户可便捷的接入自定义插件。
+
+##### 名词释义
+- Gossip - Syncer使用来自serf的Gossip协议进行集群间的广播。Gossip协议使用UDP在集群节点间接力式随机传播，它是一种病毒式传播的协议。
+- Serf - Gossip协议的一种具体实现，是一种用于集群成员管理、故障检测和编排的工具，具有分布式、容错性、高可用性的特点。
+- Service center - 服务中心中心。可以是单个微服务注册发现中心，也可以是一个微服务注册中心集群。我们定义的服务中心是一个拥有统一实例数据的网络区域。
+
+### 2. ServiceCenter Syncer架构
+Syncer的运行时架构图如下：
+
+如图所示，我们可以看到被标记为“A”、"B"、“C”的三个服务中心。
+
+- 在每个服务中心内，均部署了一套服务注册中心（ServiceCenter、Eurake或者其他）集群，该集群管理其所属服务中心的所有微服务实例，并且是彼此隔离的。同时在每个服務中心里各自部署了一个Syncer集群，它负责从注册中心发现实例，并向服务中心注册来自其他服务中心的实例信息。
+- 在多个服务中心间，多个Syncer组成对等网络，维持一个Gossip池。Gossip协议的使用主要带来了以下便捷：
+ - Syncer仅需任意一个池成员信息即可加入网络，其他成员信息的发现是自动完成的；
+ - 成员的故障检测是分布式的，是池中多个成员互相协作完成的，这相对于简单的心跳更加准确、完善；
+ - 提供集群消息传递，主要用于实例数据更新事件通知、指定实例跨服务中心查询。
+- Syncer提供RPC服务，用于传输微服务实例信息。当Syncer接收到其他成员的事件通知或跨服务中心查询请求，可通过RPC服务提供的Pull和Push接口进行数据的同步。
+
+### 3. 快速入门
+##### 3.1 获取并启动服务中心
+
+以Service-center为例，详细参见：[ServiceCenter 快速入门](https://github.com/apache/servicecomb-service-center#quick-start)
+
+##### 3.2 从源码构建和运行ServiceCenter Syncer
+必要条件：golang 版本 1.11+
+```bash
+# 从github获取Service-center Syncer最新源码
+$ git clone https://github.com/apache/servicecomb-service-center.git $GOPATH/src/github.com/apache/servicecomb-service-center
+$cd $GOPATH/src/github.com/apache/servicecomb-service-center
+
+# go mod 编译设置
+$ GO111MODULE=on go mod download
+$ GO111MODULE=on go mod vendor
+
+# 编译
+$ cd syncer
+$ go build
+```
+
+##### 3.3 启动ServiceCenter Syncer
+###### 命令行参数说明:
+- sc-addr: 服务中心地址，即服务注册中心地址。支持集群模式，多个地址间使用英文半角","隔开。
+例：--sc-addr http://10.0.0.10:30100, http://10.0.0.11:30100
+- bind：Syncer在Gossip协议内P2P地址，用于在P2P网络上和其他Syncer成员之间进行通信。
+例：--bind 10.0.0.10:30190
+- rpc-addr：Syncer的数据通讯地址，用于Syncer间同步微服务实例数据。
+例：--rpc-addr 10.0.0.10:30191
+- join：用于指定待加入的P2P网络，P2P中的任何成员的地址， 启动第一个Syncer时不需要该参数。
+例：--join 10.0.0.10:30191
+
+
+假设有2个服务中心，每个服务中心都有一个用于微服务发现和注册的服务中心集群，如下所示：
+
+| Service center | Local address |
+| :--------------------: | :-----------: |
+| http://10.0.0.10:30100 | 10.0.0.10 |
+| http://10.0.0.11:30100 | 10.0.0.11 |
+
+以下将启动两个Syncer来完成这两个服务中心之间的微服务数据同步:
+
+**在10.0.0.10的机器上执行以下命令启动ServiceCenter Syncer**
+
+```bash
+$ ./syncer daemon --sc-addr http://10.0.0.10:30100 --bind 10.0.0.10:30190 --rpc-addr 10.0.0.10:30191
+```
+
+**在10.0.0.10的机器上执行以下命令启动ServiceCenter Syncer，并加入10.0.0.10的gossip池**
+```bash
+$ ./syncer daemon --sc-addr http://10.0.0.11:30100 --bind 10.0.0.11:30190 --rpc-addr 10.0.0.11:30191 --join 10.0.0.10:30191
+```
+
+**结果验证**
+将微服务实例注册到其中一个ServiceCener后30秒，可以从每个ServiceCenter获取有关该实例的信息。
diff --git a/syncer/README.md b/syncer/README.md
new file mode 100644
index 0000000..bbb4196
--- /dev/null
+++ b/syncer/README.md
@@ -0,0 +1,96 @@
+ServiceCenter Syncer
+-------
+[中文简介](./README-ZH.md)
+
+### 1. What is ServiceCenter Syncer
+Syncer is a multiple servicecenters synchronization tool designed for large microservice architectures,supporting differ-structure servicecenters. The project adheres to the following beliefst:
+- Transparent to the application. Regardless of whether the tool is running or not, it should not affect the original microservices.
+- Provide peer-to-peer networks for multiple service-centers. Syncers are loosely coupled and members can join and quit freely.
+- Support for different service-centers. Plug-in form, users can easily implement plug-ins for different service-centers.
+
+##### Glossary
+- Gossip - Syncer uses a gossip protocol by serf for inter-cluster broadcasts. The Gossip protocol uses UDP to indirectly force random propagation between cluster nodes, which is a viral propagation protocol.
+- Serf - An implementation of the Gossip protocol, a tool for cluster member management, fault detection, and orchestration, which is distributed, fault tolerant and highly available.
+- service center - It can be a single microservice discovery and registration center or a cluster of microservice discovery and registration centers. We define a service center to be a network area with unified instances data.
+
+### 2. ServiceCenter Syncer Architecture
+Syncer's runtime architecture diagram is as follows,
+
+There are three service-centers, labeled "A" and "B" and "C".
+
+- Within each service center, a service registry (ServiceCenter, Eurake, or other) cluster is deployed that manages all microservice instances of the service center to which it belongs, and the service centers are isolated from each other. At the same time, a Syner cluster is deployed in each service center, which is responsible for discovering instances from the registry and registering instance information from other service centers to its own service centers.
+- Between multiple service centers, multiple Syncers form a peer-to-peer network that maintains a Gossip pool. The use of the Gossip protocol mainly brings the following conveniences,
+ - Syncer only needs any pool member information to join the network, and the discovery of other member information is done automatically.
+ - The fault detection of members is distributed, and multiple members in the pool cooperate with each other, which is more accurate and perfect than simple heartbeat.
+ - Provides cluster messaging, mainly for event notification of instances data.
+- Syncer provides RPC service for transmitting microservice instances information. When a Syncer receives event notifications from other members or queries across service centers, the data can be synchronized through the Pull and Push interfaces provided by the RPC service.
+
+### 3. Quick Start
+##### 3.1 Getting & Running Service center
+
+Take Service-center as an example, reference to [ServiceCenter Quick Start](https://github.com/apache/servicecomb-service-center#quick-start)
+
+##### 3.2 Building Service-Center Syncer from Source
+Requirements, Golang version 1.11+
+```bash
+# Get Service-center Syncer source code from github
+$ git clone https://github.com/apache/servicecomb-service-center.git $GOPATH/src/github.com/apache/servicecomb-service-center
+$cd $GOPATH/src/github.com/apache/servicecomb-service-center
+
+# Set "go mod" compile env
+$ GO111MODULE=on go mod download
+$ GO111MODULE=on go mod vendor
+
+# Build it
+$ cd syncer
+$ go build
+```
+
+##### 3.3 Running ServiceCenter Syncer
+###### Parameter Description
+- dc-addr
+
+ Service center address, which is the service registry address. Cluster mode is supported, and multiple addresses are separated by commas.
+ Example `--dc-addr http://10.0.0.10:30100,http://10.0.0.11:30100`
+
+- bind
+
+ P2P address of Syncer for communication with other Syner members in P2P networks by Gossip.
+ Example `--bind 10.0.0.10:30190`
+
+- rpc-addr
+
+ Address of Syncer for data transmission, used to synchronize microservices data between Syncers.
+ Example `--rpc-addr 10.0.0.10:30191`
+
+- join
+
+ The address of any member of the P2P network, to enable itself join the specified P2P network, ignore this parameter when starting the first syncer on a P2P network.
+ Example `--join 10.0.0.10:30191 `
+
+
+Suppose there are 2 Service centers, each of them with a Service-center cluster for microservices discovery and registry, as following,
+
+| Servicecenter | Local address |
+| :--------------------: | :-----------: |
+| http://10.0.0.10:30100 | 10.0.0.10 |
+| http://10.0.0.11:30100 | 10.0.0.11 |
+
+
+Start Service-center Syncer to enable communication between 2 service centers,
+
+**Start the ServiceCenter Syner by executing the following command on the 10.0.0.10 machine**
+
+```bash
+$ ./syncer daemon --dc-addr http://10.0.0.10:30100 --bind 10.0.0.10:30190 --rpc-addr 10.0.0.10:30191
+```
+
+**Start the ServiceCenter Syncer by executing the following command on the 10.0.0.10 machine and join the 10.0.0.10 gossip pool**
+
+```bash
+$ ./syncer daemon --dc-addr http://10.0.0.11:30100 --bind 10.0.0.11:30190 --rpc-addr 10.0.0.11:30191 --join 10.0.0.10:30191
+```
+
+**Verification**
+30 seconds after registering a microservice to one of the Service-centers, the information about it can be get from the other one.
+
diff --git a/syncer/TODO-ZH.md b/syncer/TODO-ZH.md
new file mode 100644
index 0000000..ec4a18d
--- /dev/null
+++ b/syncer/TODO-ZH.md
@@ -0,0 +1,36 @@
+# 服务中心 Syncer的 待完成任务列表
+
+[Reference to English version](./TODO.md)
+
+## 主体功能
+
+- 完善Service-center 插件功能，Service-center之间进行数据同步时，除Serivce和Instance外的数据进行无损同步
+- 增量数据同步
+- 异构支持SpringCloud Eureka，Eureka注册的微服务可与Service-center之间进行跨DC数据通信
+- 异构支持Istio
+- 异构支持K8S etcd
+- 异构支持Consule
+- 支持跨云的数据同步
+
+## 管理功能
+
+- Syncer集群的生命周期管理，增删改查Syncer集群成员
+- 跨服务中心微服务黑名单管理
+- 跨服务中心微服务白名单管理
+- 按需同步管理
+- 支持配置文件中读取配置
+
+## 可靠性
+
+- 服务中心内部 Syncer 多实例生命周期管理
+- Syncer的微服务实例数据和映射关系表 存储到 etcd
+
+## 安全
+
+- 通信加密
+- 权限管理
+- 鉴权管理
+
+## 部署
+
+- Docker部署
\ No newline at end of file
diff --git a/syncer/TODO.md b/syncer/TODO.md
new file mode 100644
index 0000000..94d96fd
--- /dev/null
+++ b/syncer/TODO.md
@@ -0,0 +1,36 @@
+# ToDo List of Service-center Syncer
+
+[中文版](./TODO-ZH.md)
+
+## Functionality
+
+- Improve Service-center plugin to perform lossless data synchronization between Service-center
+- Incremental data synchronization
+- Support SpringCloud Eureka
+- Support Istio
+- Support K8S etcd
+- Support Consule
+- Support data synchronization cross datacenters
+
+## Management
+
+- Lifecycle management of syncer cluster
+- Microservices blacklist of cross service-center
+- Microservices whitelist of cross service-center-center
+- Management of on-demand service-center synchronization
+- Support read configuration from config file
+
+## Reliable
+
+- Lifecycle management for syncer's multi-instances in a Data-center
+- Store syncer data to etcd
+
+## Security
+
+- Communication encryption
+- Authority management
+- Authentication management
+
+## Deployment
+
+- Docker deployment
\ No newline at end of file
diff --git a/syncer/cmd/daemon.go b/syncer/cmd/daemon.go
new file mode 100644
index 0000000..51984b9
--- /dev/null
+++ b/syncer/cmd/daemon.go
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package cmd
+
+import (
+ "context"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/syncer/config"
+ "github.com/apache/servicecomb-service-center/syncer/server"
+ "github.com/spf13/cobra"
+)
+
+var (
+ conf = config.DefaultConfig()
+)
+
+var syncerCmd = &cobra.Command{
+ Use: "daemon",
+ Short: "Start a syncer daemon",
+ Run: runSyncer,
+}
+
+func init() {
+ rootCmd.AddCommand(syncerCmd)
+
+ syncerCmd.Flags().StringVar(&conf.Mode, "mode", conf.Mode,
+ "run mode")
+
+ syncerCmd.Flags().StringVar(&conf.NodeName, "node", conf.NodeName,
+ "node name")
+
+ syncerCmd.Flags().StringVar(&conf.BindAddr, "bind", conf.BindAddr,
+ "address to bind listeners to")
+
+ syncerCmd.Flags().StringVar(&conf.RPCAddr, "rpc-addr", conf.RPCAddr,
+ "port to bind RPC listener to")
+
+ syncerCmd.Flags().StringVar(&conf.JoinAddr, "join", conf.JoinAddr,
+ "address to join the cluster by specifying at least one existing member")
+
+ syncerCmd.Flags().StringVar(&conf.SCAddr, "sc-addr", conf.SCAddr,
+ "address to monitor the service-center")
+}
+
+// runSyncer Runs the Syncer service.
+func runSyncer(cmd *cobra.Command, args []string) {
+ if err := conf.Verification(); err != nil {
+ log.Errorf(err, "verification syncer config failed")
+ return
+ }
+
+ server.NewServer(conf).Run(context.Background())
+}
diff --git a/syncer/cmd/root.go b/syncer/cmd/root.go
new file mode 100644
index 0000000..96b2086
--- /dev/null
+++ b/syncer/cmd/root.go
@@ -0,0 +1,22 @@
+package cmd
+
+import (
+ "log"
+
+ "github.com/spf13/cobra"
+)
+
+var rootCmd = &cobra.Command{
+ Use: "syncer",
+ Short: "Syncer is a synchronization tool for mutilple service centers",
+ Long: "Syncer is a synchronization tool for mutilple service centers, supported: ServiceComb service-center, " +
+ "going to be supported: SpringCloud Eureka, Istio, K8S",
+}
+
+func Execute() error {
+ err := rootCmd.Execute()
+ if err != nil {
+ log.Println(err)
+ }
+ return err
+}
diff --git a/syncer/config/config.go b/syncer/config/config.go
new file mode 100644
index 0000000..15a4613
--- /dev/null
+++ b/syncer/config/config.go
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package config
+
+import (
+ "fmt"
+ "os"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/syncer/pkg/utils"
+ "github.com/apache/servicecomb-service-center/syncer/plugins/servicecenter"
+ "github.com/apache/servicecomb-service-center/syncer/serf"
+)
+
+var (
+ DefaultMode = "service-center"
+ DefaultDCPort = 30100
+ DefaultTickerInterval = 30
+)
+
+// Config is the configuration that can be set for Syncer. Some of these
+// configurations are exposed as command-line flags.
+type Config struct {
+ // Wraps the serf config
+ *serf.Config
+ LogFile string `yaml:"log_file"`
+
+ // Mode is the type of servicecenter, currently supports "service-center"
+ Mode string `yaml:"mode"`
+
+ // SCAddr servicecenter address, which is the service registry address.
+ // Cluster mode is supported, and multiple addresses are separated by an English ",".
+ SCAddr string `yaml:"dc_addr"`
+
+ // JoinAddr The management address of one gossip pool member.
+ JoinAddr string `yaml:"join_addr"`
+ TickerInterval int `yaml:"ticker_interval"`
+ Profile string `yaml:"profile"`
+ EnableCompression bool `yaml:"enable_compression"`
+ AutoSync bool `yaml:"auto_sync"`
+ ServicecenterPlugin string `json:"servicecenter_plugin"`
+}
+
+// DefaultConfig returns the default config
+func DefaultConfig() *Config {
+ serfConf := serf.DefaultConfig()
+ hostname, err := os.Hostname()
+ if err != nil {
+ log.Errorf(err, "Error determining hostname: %s", err)
+ return nil
+ }
+ serfConf.NodeName = hostname
+ return &Config{
+ Mode: DefaultMode,
+ SCAddr: fmt.Sprintf("127.0.0.1:%d", DefaultDCPort),
+ TickerInterval: DefaultTickerInterval,
+ Config: serfConf,
+ ServicecenterPlugin: servicecenter.PluginName,
+ }
+}
+
+// Verification Provide config verification
+func (c *Config) Verification() error {
+ ip, port, err := utils.SplitHostPort(c.BindAddr, serf.DefaultBindPort)
+ if err != nil {
+ return err
+ }
+ if ip == "127.0.0.1" {
+ c.BindAddr = fmt.Sprintf("0.0.0.0:%d", port)
+ }
+
+ ip, port, err = utils.SplitHostPort(c.RPCAddr, serf.DefaultRPCPort)
+ if err != nil {
+ return err
+ }
+ c.RPCPort = port
+ if ip == "127.0.0.1" {
+ c.RPCAddr = fmt.Sprintf("0.0.0.0:%d", c.RPCPort)
+ }
+ return nil
+}
diff --git a/syncer/config/config_test.go b/syncer/config/config_test.go
new file mode 100644
index 0000000..629c102
--- /dev/null
+++ b/syncer/config/config_test.go
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package config
+
+import "testing"
+
+func TestDefaultVerification(t *testing.T) {
+ conf := DefaultConfig()
+ conf.Verification()
+}
+
+func TestBindAddrVerification(t *testing.T) {
+ conf := DefaultConfig()
+ conf.BindAddr = "abcde"
+ conf.Verification()
+}
+
+func TestRPCAddrAddrVerification(t *testing.T) {
+ conf := DefaultConfig()
+ conf.RPCAddr = "abcde"
+ conf.Verification()
+}
+
+func TestLocalBindAddrVerification(t *testing.T) {
+ conf := DefaultConfig()
+ conf.BindAddr = "127.0.0.1"
+ conf.Verification()
+}
+
+func TestLocalRPCAddrVerification(t *testing.T) {
+ conf := DefaultConfig()
+ conf.RPCAddr = "127.0.0.1"
+ conf.Verification()
+}
diff --git a/syncer/grpc/client.go b/syncer/grpc/client.go
new file mode 100644
index 0000000..f57ab5c
--- /dev/null
+++ b/syncer/grpc/client.go
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package grpc
+
+import (
+ "context"
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+ "google.golang.org/grpc"
+)
+
+var (
+ clients = make(map[string]*Client)
+ lock sync.RWMutex
+)
+
+// Client struct
+type Client struct {
+ addr string
+ conn *grpc.ClientConn
+ cli pb.SyncClient
+}
+
+func Pull(ctx context.Context, addr string) (*pb.SyncData, error) {
+ cli := getClient(addr)
+
+ data, err := cli.cli.Pull(ctx, &pb.PullRequest{})
+ if err != nil {
+ log.Errorf(err, "Pull from grpc failed, going to close the client")
+ closeClient(addr)
+ }
+ return data, err
+}
+
+func closeClient(addr string) {
+ lock.RLock()
+ cli, ok := clients[addr]
+ lock.RUnlock()
+ if ok {
+
+ cli.conn.Close()
+ lock.Lock()
+ delete(clients, addr)
+ lock.Unlock()
+ log.Infof("Close grpc connection to %s", addr)
+ }
+}
+
+// GetClient Get the client from the client caches with addr
+func getClient(addr string) *Client {
+ lock.RLock()
+ cli, ok := clients[addr]
+ lock.RUnlock()
+ if !ok {
+ log.Infof("Create new grpc connection to %s", addr)
+ conn, err := grpc.Dial(addr, grpc.WithInsecure())
+ if err != nil {
+ return nil
+ }
+ cli = &Client{conn: conn, cli: pb.NewSyncClient(conn), addr: addr}
+ lock.Lock()
+ clients[addr] = cli
+ lock.Unlock()
+ }
+ return cli
+}
diff --git a/syncer/grpc/server.go b/syncer/grpc/server.go
new file mode 100644
index 0000000..8941871
--- /dev/null
+++ b/syncer/grpc/server.go
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package grpc
+
+import (
+ "context"
+ "net"
+
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+ "google.golang.org/grpc"
+)
+
+type GRPCHandler interface {
+ Discovery() *pb.SyncData
+}
+type PullHandle func() *pb.SyncData
+
+// Server struct
+type Server struct {
+ lsn net.Listener
+ addr string
+ handler GRPCHandler
+}
+
+// NewServer new grpc server
+func NewServer(addr string, handler GRPCHandler) *Server {
+ return &Server{addr: addr, handler: handler}
+}
+
+// Provide consumers with an interface to pull data
+func (s *Server) Pull(ctx context.Context, in *pb.PullRequest) (*pb.SyncData, error) {
+ return s.handler.Discovery(), nil
+}
+
+// Stop grpc server
+func (s *Server) Stop() {
+ if s.lsn == nil {
+ return
+ }
+ s.lsn.Close()
+}
+
+// Run grpc server
+func (s *Server) Run() (err error) {
+ s.lsn, err = net.Listen("tcp", s.addr)
+ if err != nil {
+ return err
+ }
+ svc := grpc.NewServer()
+ pb.RegisterSyncServer(svc, s)
+ gopool.Go(func(ctx context.Context) {
+ svc.Serve(s.lsn)
+ })
+ return nil
+}
diff --git a/syncer/images/SyncerArchitecture.png b/syncer/images/SyncerArchitecture.png
new file mode 100644
index 0000000..b65ce52
--- /dev/null
+++ b/syncer/images/SyncerArchitecture.png
Binary files differ
diff --git a/syncer/pkg/syssig/signal.go b/syncer/pkg/syssig/signal.go
new file mode 100644
index 0000000..13b6251
--- /dev/null
+++ b/syncer/pkg/syssig/signal.go
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package syssig
+
+import (
+ "context"
+ "fmt"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "os"
+ "os/signal"
+ "sync"
+ "syscall"
+)
+
+var (
+ once sync.Once
+ lock sync.RWMutex
+ handlerMap = map[os.Signal][]func(){
+ syscall.SIGHUP: {},
+ syscall.SIGINT: {},
+ syscall.SIGKILL: {},
+ syscall.SIGTERM: {},
+ }
+)
+
+// Run start system signal listening
+func Run(ctx context.Context) {
+ once.Do(func() {
+ listenSignals := make([]os.Signal, 0, 10)
+ for key := range handlerMap {
+ listenSignals = append(listenSignals, key)
+ }
+
+ sigChan := make(chan os.Signal, 1)
+ signal.Notify(sigChan, listenSignals...)
+
+ select {
+ case <-ctx.Done():
+
+ case sig := <-sigChan:
+ log.Infof("system signal: %s", sig.String())
+ calls := callbacks(sig)
+ for _, call := range calls {
+ call()
+ }
+ }
+
+ })
+}
+
+// AddSignalsHandler add system signal listener with types
+func AddSignalsHandler(handler func(), signals ...os.Signal) error {
+ for _, sig := range signals {
+ lock.RLock()
+ handlers, ok := handlerMap[sig]
+ lock.RUnlock()
+ if !ok {
+ return fmt.Errorf("system signal %s is not notify", sig.String())
+ }
+ handlers = append(handlers, handler)
+ lock.Lock()
+ handlerMap[sig] = handlers
+ lock.Unlock()
+ }
+ return nil
+}
+
+// callbacks Callback system listeners
+func callbacks(signal os.Signal) []func() {
+ lock.RLock()
+ calls := handlerMap[signal]
+ lock.RUnlock()
+ return calls
+}
diff --git a/syncer/pkg/syssig/signal_test.go b/syncer/pkg/syssig/signal_test.go
new file mode 100644
index 0000000..d519e4a
--- /dev/null
+++ b/syncer/pkg/syssig/signal_test.go
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package syssig
+
+import (
+ "context"
+ "syscall"
+ "testing"
+ "time"
+)
+
+func TestSignalsHandler(t *testing.T) {
+ AddSignalsHandler(func() {}, syscall.SIGHUP, syscall.SIGINT, syscall.SIGKILL, syscall.SIGTERM)
+ AddSignalsHandler(func() {}, syscall.Signal(999))
+ ctx, cancel := context.WithCancel(context.Background())
+ go func() { Run(ctx) }()
+ time.Sleep(time.Second)
+ cancel()
+}
diff --git a/syncer/pkg/ticker/ticker.go b/syncer/pkg/ticker/ticker.go
new file mode 100644
index 0000000..ac861a4
--- /dev/null
+++ b/syncer/pkg/ticker/ticker.go
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package ticker
+
+import (
+ "context"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "sync"
+ "time"
+)
+
+// TaskTicker task of ticker struct
+type TaskTicker struct {
+ interval time.Duration
+ handler func(ctx context.Context)
+ once sync.Once
+ ticker *time.Ticker
+}
+
+// NewTaskTicker new task ticker with interval
+func NewTaskTicker(interval int, handler func(ctx context.Context)) *TaskTicker {
+ return &TaskTicker{
+ interval: time.Second * time.Duration(interval),
+ handler: handler,
+ }
+}
+
+// Start start task ticker
+func (t *TaskTicker) Start(ctx context.Context) {
+ t.once.Do(func() {
+ t.handler(ctx)
+ t.ticker = time.NewTicker(t.interval)
+ for {
+ select {
+ case <-t.ticker.C:
+ t.handler(ctx)
+ case <-ctx.Done():
+ t.Stop()
+ return
+ }
+ }
+ })
+}
+
+// Start stop task ticker
+func (t *TaskTicker) Stop() {
+ if t.ticker == nil {
+ return
+ }
+ t.ticker.Stop()
+ log.Info("ticker stop")
+ t.ticker = nil
+}
diff --git a/syncer/pkg/ticker/ticker_test.go b/syncer/pkg/ticker/ticker_test.go
new file mode 100644
index 0000000..50374b3
--- /dev/null
+++ b/syncer/pkg/ticker/ticker_test.go
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package ticker
+
+import (
+ "context"
+ "testing"
+ "time"
+)
+
+func TestNewTaskTicker(t *testing.T) {
+ ticker := NewTaskTicker(2, func(ctx context.Context) {
+ t.Log("test new task ticker over")
+ })
+ ctx, cancel := context.WithCancel(context.Background())
+ go func() {
+ ticker.Start(ctx)
+ }()
+ time.Sleep(time.Second * 3)
+ ticker.Stop()
+ cancel()
+}
diff --git a/syncer/pkg/utils/addr.go b/syncer/pkg/utils/addr.go
new file mode 100644
index 0000000..f84c4fe
--- /dev/null
+++ b/syncer/pkg/utils/addr.go
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package utils
+
+import (
+ "fmt"
+ "net"
+)
+
+// SplitHostPort returns the parts of the address and port. If the port does not exist, use defaultPort.
+func SplitHostPort(address string, defaultPort int) (string, int, error) {
+ _, _, err := net.SplitHostPort(address)
+ if ae, ok := err.(*net.AddrError); ok && ae.Err == "missing port in address" {
+ address = fmt.Sprintf("%s:%d", address, defaultPort)
+ _, _, err = net.SplitHostPort(address)
+ }
+ if err != nil {
+ return "", 0, err
+ }
+
+ addr, err := net.ResolveTCPAddr("tcp", address)
+ if err != nil {
+ return "", 0, err
+ }
+
+ return addr.IP.String(), addr.Port, nil
+}
diff --git a/syncer/pkg/utils/addr_test.go b/syncer/pkg/utils/addr_test.go
new file mode 100644
index 0000000..bf5ba78
--- /dev/null
+++ b/syncer/pkg/utils/addr_test.go
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package utils
+
+import "testing"
+
+func TestSplitHostPort(t *testing.T) {
+ _, _, err := SplitHostPort("", 0)
+ if err != nil {
+ t.Logf("split host port failed, error: %s", err)
+ }
+}
diff --git a/syncer/pkg/utils/files.go b/syncer/pkg/utils/files.go
new file mode 100644
index 0000000..da1c0c3
--- /dev/null
+++ b/syncer/pkg/utils/files.go
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package utils
+
+import (
+ "os"
+ "path/filepath"
+)
+
+// IsDirExist checks if a dir exists
+func IsDirExist(path string) bool {
+ fi, err := os.Stat(path)
+ return err == nil && fi.IsDir() || os.IsExist(err)
+}
+
+// IsFileExist checks if a file exists
+func IsFileExist(path string) bool {
+ fi, err := os.Stat(path)
+ return err == nil && !fi.IsDir() || os.IsExist(err)
+}
+
+// OpenFile if file not exist auto create
+func OpenFile(path string) (*os.File, error) {
+ if IsFileExist(path) {
+ return os.Create(path)
+ }
+
+ dir := filepath.Dir(path)
+ if !IsDirExist(dir) {
+ err := os.MkdirAll(dir, 0666)
+ if err != nil {
+ return nil, err
+ }
+ }
+ return os.Create(path)
+}
diff --git a/syncer/pkg/utils/files_test.go b/syncer/pkg/utils/files_test.go
new file mode 100644
index 0000000..0eee21c
--- /dev/null
+++ b/syncer/pkg/utils/files_test.go
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package utils
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+)
+
+func TestExist(t *testing.T) {
+ if IsDirExist("./test/file") {
+ t.Error("dir exist failed")
+ }
+
+ if IsFileExist("./test/file") {
+ t.Error("file exist failed")
+ }
+}
+
+func TestOpenFile(t *testing.T) {
+ fileName := "file"
+ f, err := OpenFile(fileName)
+ if err != nil {
+ t.Errorf("open file failed: %s", err)
+ }
+ f.Close()
+
+ f, err = OpenFile(fileName)
+ if err != nil {
+ t.Errorf("open file failed: %s", err)
+ }
+ f.Close()
+ os.RemoveAll(fileName)
+
+ fileName = "./test/file"
+ f, err = OpenFile(fileName)
+ if err != nil {
+ t.Logf("open file failed: %s", err)
+ }
+ f.Close()
+ os.RemoveAll(filepath.Dir(fileName))
+}
diff --git a/syncer/plugins/export.go b/syncer/plugins/export.go
new file mode 100644
index 0000000..123681f
--- /dev/null
+++ b/syncer/plugins/export.go
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package plugins
+
+type PluginType int
+
+const (
+ PluginServicecenter PluginType = iota
+ pluginTotal
+
+ BUILDIN = "buildin"
+ STATIC = "static"
+ DYNAMIC = "dynamic"
+ keyPluginName = "name"
+)
+
+func (p PluginType) String() string {
+ switch p {
+ case PluginServicecenter:
+ return "servicecenter"
+ default:
+ return ""
+ }
+}
+
+func (m Manager) Servicecenter() Adaptor {
+ return m.Instance(PluginServicecenter).(Adaptor)
+}
diff --git a/syncer/plugins/plugin.go b/syncer/plugins/plugin.go
new file mode 100644
index 0000000..564059c
--- /dev/null
+++ b/syncer/plugins/plugin.go
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package plugins
+
+import (
+ pg "plugin"
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/pkg/plugin"
+)
+
+var (
+ pluginMgr Manager
+ pluginConfig map[string]string
+)
+
+type Manager map[PluginType]*pluginMap
+
+type PluginInstance interface{}
+
+type pluginMap struct {
+ plugins map[string]*Plugin
+ dynamic bool
+ instance PluginInstance
+ lock sync.RWMutex
+}
+
+type Plugin struct {
+ Kind PluginType
+ Name string
+ New func() PluginInstance
+}
+
+func init() {
+ pluginMgr = make(Manager, pluginTotal)
+ for t := PluginType(0); t != pluginTotal; t++ {
+ pluginMgr[t] = &pluginMap{plugins: map[string]*Plugin{}}
+ }
+
+ pluginConfig = make(map[string]string, pluginTotal)
+}
+
+func (m Manager) Register(p *Plugin) {
+ pm, ok := m[p.Kind]
+ if !ok {
+ pm = &pluginMap{plugins: map[string]*Plugin{}}
+ }
+ pm.plugins[p.Name] = p
+ m[p.Kind] = pm
+}
+
+func (m Manager) Get(pn PluginType, name string) *Plugin {
+ pm, ok := m[pn]
+ if !ok {
+ return nil
+ }
+ return pm.plugins[name]
+}
+
+func (m Manager) Instance(pn PluginType) PluginInstance {
+ pm := m[pn]
+ pm.lock.RLock()
+ if pm.instance != nil {
+ pm.lock.RUnlock()
+ return pm.instance
+ }
+ pm.lock.RUnlock()
+
+ pm.lock.Lock()
+ if pm.instance != nil {
+ pm.lock.Unlock()
+ return pm.instance
+ }
+ m.New(pn)
+ pm.lock.Unlock()
+ return pm.instance
+}
+
+func (m Manager) New(pn PluginType) {
+ var (
+ //title = STATIC
+ f func() PluginInstance
+ )
+
+ wi := m[pn]
+ p := m.existDynamicPlugin(pn)
+ if p != nil {
+ wi.dynamic = true
+ //title = DYNAMIC
+ f = p.New
+ } else {
+ wi.dynamic = false
+ pm, ok := m[pn]
+ if !ok {
+ return
+ }
+
+ name, ok := pluginConfig[pn.String()]
+ if !ok {
+ name = BUILDIN
+ }
+
+ p, ok = pm.plugins[name]
+ if !ok {
+ return
+ }
+
+ f = p.New
+ }
+ wi.instance = f()
+}
+
+func (m Manager) existDynamicPlugin(pn PluginType) *Plugin {
+ pm, ok := m[pn]
+ if !ok {
+ return nil
+ }
+ // 'buildin' implement of all plugins should call DynamicPluginFunc()
+ if plugin.PluginLoader().Exist(pn.String()) {
+ return pm.plugins[BUILDIN]
+ }
+ return nil
+}
+
+func DynamicPluginFunc(pn PluginType, funcName string) pg.Symbol {
+ if wi, ok := pluginMgr[pn]; ok && !wi.dynamic {
+ return nil
+ }
+
+ f, err := plugin.FindFunc(pn.String(), funcName)
+ if err != nil {
+ return nil
+ }
+ return f
+}
+func Plugins() Manager {
+ return pluginMgr
+}
+
+func RegisterPlugin(p *Plugin) {
+ pluginMgr.Register(p)
+}
+
+func LoadPlugins() {
+ for t := PluginType(0); t != pluginTotal; t++ {
+ pluginMgr.Instance(t)
+ }
+}
+
+func SetPluginConfig(key, val string) {
+ pluginConfig[key] = val
+}
diff --git a/syncer/plugins/plugin_test.go b/syncer/plugins/plugin_test.go
new file mode 100644
index 0000000..cfaa778
--- /dev/null
+++ b/syncer/plugins/plugin_test.go
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+//package plugins_test
+package plugins
+
+import (
+ "context"
+ "testing"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+type mockPlugin struct{}
+
+func newAdaptor() PluginInstance { return &mockAdaptor{} }
+
+type mockAdaptor struct{}
+
+func (*mockAdaptor) New(endpoints []string) (Servicecenter, error) {
+ return &mockRepository{}, nil
+}
+
+type mockRepository struct{}
+
+func (r *mockRepository) GetAll(ctx context.Context) (data *pb.SyncData, err error) { return }
+
+func (r *mockRepository) CreateService(ctx context.Context, domainProject string, service *scpb.MicroService) (str string, err error) {
+ return
+}
+
+func (r *mockRepository) DeleteService(ctx context.Context, domainProject, serviceId string) (err error) {
+ return
+}
+
+func (r *mockRepository) ServiceExistence(ctx context.Context, domainProject string, service *scpb.MicroService) (str string, err error) {
+ return
+}
+
+func (r *mockRepository) RegisterInstance(ctx context.Context, domainProject, serviceId string, instance *scpb.MicroServiceInstance) (str string, err error) {
+ return
+}
+
+func (r *mockRepository) UnregisterInstance(ctx context.Context, domainProject, serviceId, instanceId string) (err error) {
+ return
+}
+
+func (r *mockRepository) DiscoveryInstances(ctx context.Context, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule string) (list []*scpb.MicroServiceInstance, err error) {
+ return
+}
+
+func (r *mockRepository) Heartbeat(ctx context.Context, domainProject, serviceId, instanceId string) (err error) {
+ return
+}
+
+func TestManager_New(t *testing.T) {
+ pm := Plugins()
+
+ notfound := PluginType(999)
+ notfound.String()
+
+ p := pm.Get(notfound, BUILDIN)
+ if p != nil {
+ t.Fatalf("get %s %s failed", notfound, BUILDIN)
+ }
+
+ instanceNil := pm.Instance(PluginServicecenter)
+ if instanceNil != pm.Instance(PluginServicecenter) {
+ t.Fatalf("instance storage plugin: %s failed", PluginServicecenter)
+ }
+
+ getNil := pm.Get(PluginServicecenter, BUILDIN)
+ if getNil != nil {
+ t.Fatalf("get %s %s failed", PluginServicecenter, BUILDIN)
+ }
+
+ RegisterPlugin(&Plugin{Kind: PluginServicecenter, Name: "mock", New: newAdaptor})
+ SetPluginConfig(PluginServicecenter.String(), "mock")
+
+ repositoryInstance := pm.Instance(PluginServicecenter)
+ if repositoryInstance != pm.Instance(PluginServicecenter) {
+ t.Fatalf("instance storage plugin: %s failed", PluginServicecenter)
+ }
+ pm.Servicecenter()
+
+ RegisterPlugin(&Plugin{Kind: notfound, Name: "mock", New: func() PluginInstance { return &mockPlugin{} }})
+
+ LoadPlugins()
+
+ DynamicPluginFunc(notfound, "mock")
+}
diff --git a/syncer/plugins/servicecenter.go b/syncer/plugins/servicecenter.go
new file mode 100644
index 0000000..8797e44
--- /dev/null
+++ b/syncer/plugins/servicecenter.go
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package plugins
+
+import (
+ "context"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+// Adaptor the plugin adaptor of repository
+type Adaptor interface {
+ New(endpoints []string) (Servicecenter, error)
+}
+
+// Servicecenter servicecenter interface
+type Servicecenter interface {
+ GetAll(ctx context.Context) (*pb.SyncData, error)
+ CreateService(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error)
+ DeleteService(ctx context.Context, domainProject, serviceId string) error
+ ServiceExistence(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error)
+ RegisterInstance(ctx context.Context, domainProject, serviceId string, instance *scpb.MicroServiceInstance) (string, error)
+ UnregisterInstance(ctx context.Context, domainProject, serviceId, instanceId string) error
+ DiscoveryInstances(ctx context.Context, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule string) ([]*scpb.MicroServiceInstance, error)
+ Heartbeat(ctx context.Context, domainProject, serviceId, instanceId string) error
+}
diff --git a/syncer/plugins/servicecenter/instance.go b/syncer/plugins/servicecenter/instance.go
new file mode 100644
index 0000000..05d21f3
--- /dev/null
+++ b/syncer/plugins/servicecenter/instance.go
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+)
+
+// RegisterInstance register instance to servicecenter
+func (c *Client) RegisterInstance(ctx context.Context, domainProject, serviceId string, instance *scpb.MicroServiceInstance) (string, error) {
+ instanceID, err := c.cli.RegisterInstance(ctx, domainProject, serviceId, instance)
+ if err != nil {
+ return "", err
+ }
+ return instanceID, nil
+}
+
+// UnregisterInstance unregister instance from servicecenter
+func (c *Client) UnregisterInstance(ctx context.Context, domainProject, serviceId, instanceId string) error {
+ err := c.cli.UnregisterInstance(ctx, domainProject, serviceId, instanceId)
+ if err != nil {
+ return err
+ }
+ return nil
+}
+
+// DiscoveryInstances discoveries instances from servicecenter
+func (c *Client) DiscoveryInstances(ctx context.Context, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule string) ([]*scpb.MicroServiceInstance, error) {
+ instances, err := c.cli.DiscoveryInstances(ctx, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule)
+ if err != nil {
+ return nil, err
+ }
+ return instances, nil
+}
+
+// Heartbeat sends heartbeat to servicecenter
+func (c *Client) Heartbeat(ctx context.Context, domainProject, serviceId, instanceId string) error {
+ err := c.cli.Heartbeat(ctx, domainProject, serviceId, instanceId)
+ if err != nil {
+ return err
+ }
+ return nil
+}
diff --git a/syncer/plugins/servicecenter/instance_test.go b/syncer/plugins/servicecenter/instance_test.go
new file mode 100644
index 0000000..0df4d5e
--- /dev/null
+++ b/syncer/plugins/servicecenter/instance_test.go
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+ "testing"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+)
+
+func TestClient_RegisterInstance(t *testing.T) {
+ svr, repo := newServiceCenter(t)
+ _, err := repo.RegisterInstance(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", &scpb.MicroServiceInstance{})
+ if err != nil {
+ t.Errorf("register instance failed, error: %s", err)
+ }
+
+ svr.Close()
+ _, err = repo.RegisterInstance(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", &scpb.MicroServiceInstance{})
+ if err != nil {
+ t.Logf("register instance failed, error: %s", err)
+ }
+}
+
+func TestClient_UnregisterInstance(t *testing.T) {
+ svr, repo := newServiceCenter(t)
+ err := repo.UnregisterInstance(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", "7a6be9f861a811e9b3f6fa163eca30e0")
+ if err != nil {
+ t.Errorf("unregister instance failed, error: %s", err)
+ }
+
+ svr.Close()
+ err = repo.UnregisterInstance(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", "7a6be9f861a811e9b3f6fa163eca30e0")
+ if err != nil {
+ t.Logf("unregister instance failed, error: %s", err)
+ }
+}
+
+func TestClient_DiscoveryInstances(t *testing.T) {
+ svr, repo := newServiceCenter(t)
+ _, err := repo.DiscoveryInstances(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", "default", "testservice", "1.0.1")
+ if err != nil {
+ t.Errorf("discovery instances failed, error: %s", err)
+ }
+
+ svr.Close()
+ _, err = repo.DiscoveryInstances(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", "default", "testservice", "1.0.1")
+ if err != nil {
+ t.Logf("discovery instances failed, error: %s", err)
+ }
+}
+
+func TestClient_Heartbeat(t *testing.T) {
+ svr, repo := newServiceCenter(t)
+ err := repo.Heartbeat(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", "7a6be9f861a811e9b3f6fa163eca30e0")
+ if err != nil {
+ t.Errorf("send instance heartbeat failed, error: %s", err)
+ }
+
+ svr.Close()
+ err = repo.Heartbeat(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd", "7a6be9f861a811e9b3f6fa163eca30e0")
+ if err != nil {
+ t.Logf("send instance heartbeat, error: %s", err)
+ }
+}
diff --git a/syncer/plugins/servicecenter/service.go b/syncer/plugins/servicecenter/service.go
new file mode 100644
index 0000000..129f904
--- /dev/null
+++ b/syncer/plugins/servicecenter/service.go
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+)
+
+// CreateService creates the service of servicecenter
+func (c *Client) CreateService(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error) {
+ serviceID, err := c.cli.CreateService(ctx, domainProject, service)
+ if err != nil {
+ return "", err
+ }
+ return serviceID, nil
+}
+
+// DeleteService deletes service from servicecenter
+func (c *Client) DeleteService(ctx context.Context, domainProject, serviceId string) error {
+ err := c.cli.DeleteService(ctx, domainProject, serviceId)
+ if err != nil {
+ return err
+ }
+ return nil
+}
+
+// ServiceExistence Checkes service exists in servicecenter
+func (c *Client) ServiceExistence(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error) {
+ serviceID, err := c.cli.ServiceExistence(ctx, domainProject, service.AppId, service.ServiceName, service.Version, service.Environment)
+ if err != nil {
+ return "", err
+ }
+ return serviceID, nil
+}
diff --git a/syncer/plugins/servicecenter/service_test.go b/syncer/plugins/servicecenter/service_test.go
new file mode 100644
index 0000000..8f4e343
--- /dev/null
+++ b/syncer/plugins/servicecenter/service_test.go
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+ "testing"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+)
+
+func TestClient_CreateService(t *testing.T) {
+ svr, repo := newServiceCenter(t)
+ _, err := repo.CreateService(context.Background(), "default/deault", &scpb.MicroService{})
+ if err != nil {
+ t.Errorf("create service failed, error: %s", err)
+ }
+
+ svr.Close()
+ _, err = repo.CreateService(context.Background(), "default/deault", &scpb.MicroService{})
+ if err != nil {
+ t.Logf("create service failed, error: %s", err)
+ }
+}
+
+func TestClient_ServiceExistence(t *testing.T) {
+ svr, repo := newServiceCenter(t)
+ _, err := repo.ServiceExistence(context.Background(), "default/deault", &scpb.MicroService{})
+ if err != nil {
+ t.Errorf("check service existence failed, error: %s", err)
+ }
+
+ svr.Close()
+ _, err = repo.ServiceExistence(context.Background(), "default/deault", &scpb.MicroService{})
+ if err != nil {
+ t.Logf("check service existence failed, error: %s", err)
+ }
+}
+
+func TestClient_DeleteService(t *testing.T) {
+ svr, repo := newServiceCenter(t)
+ err := repo.DeleteService(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd")
+ if err != nil {
+ t.Errorf("delete service failed, error: %s", err)
+ }
+
+ svr.Close()
+ err = repo.DeleteService(context.Background(), "default/deault",
+ "4042a6a3e5a2893698ae363ea99a69eb63fc51cd")
+ if err != nil {
+ t.Logf("delete service failed, error: %s", err)
+ }
+}
diff --git a/syncer/plugins/servicecenter/servicecenter.go b/syncer/plugins/servicecenter/servicecenter.go
new file mode 100644
index 0000000..8d3154a
--- /dev/null
+++ b/syncer/plugins/servicecenter/servicecenter.go
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+
+ "github.com/apache/servicecomb-service-center/pkg/client/sc"
+ "github.com/apache/servicecomb-service-center/syncer/plugins"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+const PluginName = "servicecenter"
+
+func init() {
+ // Register self as a repository plugin
+ plugins.RegisterPlugin(&plugins.Plugin{
+ Kind: plugins.PluginServicecenter,
+ Name: PluginName,
+ New: New,
+ })
+}
+
+type adaptor struct{}
+
+func New() plugins.PluginInstance {
+ return &adaptor{}
+}
+
+// New repository with endpoints
+func (*adaptor) New(endpoints []string) (plugins.Servicecenter, error) {
+ cli, err := sc.NewSCClient(sc.Config{Endpoints: endpoints})
+ if err != nil {
+ return nil, err
+ }
+ return &Client{cli: cli}, nil
+}
+
+type Client struct {
+ cli *sc.SCClient
+}
+
+// GetAll get and transform servicecenter data to SyncData
+func (c *Client) GetAll(ctx context.Context) (*pb.SyncData, error) {
+ cache, err := c.cli.GetScCache(ctx)
+ if err != nil {
+ return nil, err
+ }
+ return transform(cache), nil
+
+}
diff --git a/syncer/plugins/servicecenter/servicecenter_test.go b/syncer/plugins/servicecenter/servicecenter_test.go
new file mode 100644
index 0000000..d0a5636
--- /dev/null
+++ b/syncer/plugins/servicecenter/servicecenter_test.go
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+ "net/http/httptest"
+ "testing"
+
+ "github.com/apache/servicecomb-service-center/syncer/plugins"
+ "github.com/apache/servicecomb-service-center/syncer/test/servicecenter"
+)
+
+func TestClient_GetAll(t *testing.T) {
+ svr, sc := newServiceCenter(t)
+ _, err := sc.GetAll(context.Background())
+ if err != nil {
+ t.Errorf("get all from %s server failed, error: %s", PluginName, err)
+ }
+
+ svr.Close()
+ _, err = sc.GetAll(context.Background())
+ if err != nil {
+ t.Logf("get all from %s server failed, error: %s", PluginName, err)
+ }
+}
+
+func newServiceCenter(t *testing.T) (*httptest.Server, plugins.Servicecenter) {
+ plugins.SetPluginConfig(plugins.PluginServicecenter.String(), PluginName)
+ adaptor := plugins.Plugins().Servicecenter()
+ if adaptor == nil {
+ t.Errorf("get repository adaptor %s failed", PluginName)
+ }
+ svr := servicecenter.NewMockServer()
+ if svr == nil {
+ t.Error("new httptest server failed")
+ }
+ repo, err := adaptor.New([]string{svr.URL})
+ if err != nil {
+ t.Errorf("new repository %s failed, error: %s", PluginName, err)
+ }
+ return svr, repo
+}
diff --git a/syncer/plugins/servicecenter/transform.go b/syncer/plugins/servicecenter/transform.go
new file mode 100644
index 0000000..984d678
--- /dev/null
+++ b/syncer/plugins/servicecenter/transform.go
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "strings"
+
+ "github.com/apache/servicecomb-service-center/server/admin/model"
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+// transform servicecenter service cache to SyncData
+func transform(cache *model.Cache) (data *pb.SyncData) {
+ data = &pb.SyncData{Services: make([]*pb.SyncService, 0, 10)}
+
+ for _, svc := range cache.Microservices {
+ instances := instancesFromService(svc.Value, cache.Instances)
+ if len(instances) == 0 {
+ continue
+ }
+
+ data.Services = append(data.Services, &pb.SyncService{
+ DomainProject: strings.Join(strings.Split(svc.Key, "/")[4:6], "/"),
+ Service: svc.Value,
+ Instances: instances,
+ })
+ }
+ return
+}
+
+// instancesFromService Extract instance information from the service cache of the servicecenter
+func instancesFromService(service *scpb.MicroService, instances []*model.Instance) []*scpb.MicroServiceInstance {
+ instList := make([]*scpb.MicroServiceInstance, 0, 10)
+ for _, inst := range instances {
+ if inst.Value.Status != "UP" {
+ continue
+ }
+
+ if inst.Value.ServiceId == service.ServiceId {
+ instList = append(instList, inst.Value)
+ }
+ }
+ return instList
+}
diff --git a/syncer/proto/event.pb.go b/syncer/proto/event.pb.go
new file mode 100644
index 0000000..fc5eb0c
--- /dev/null
+++ b/syncer/proto/event.pb.go
@@ -0,0 +1,118 @@
+// Code generated by protoc-gen-go. DO NOT EDIT.
+// source: event.proto
+
+/*
+Package proto is a generated protocol buffer package.
+
+It is generated from these files:
+ event.proto
+ syncer.proto
+
+It has these top-level messages:
+ Member
+ Discover
+ PullRequest
+ SyncService
+ SyncData
+*/
+package proto
+
+import proto1 "github.com/golang/protobuf/proto"
+import fmt "fmt"
+import math "math"
+
+// Reference imports to suppress errors if they are not otherwise used.
+var _ = proto1.Marshal
+var _ = fmt.Errorf
+var _ = math.Inf
+
+// This is a compile-time assertion to ensure that this generated file
+// is compatible with the proto package it is being compiled against.
+// A compilation error at this line likely means your copy of the
+// proto package needs to be updated.
+const _ = proto1.ProtoPackageIsVersion2 // please upgrade the proto package
+
+type Member struct {
+ NodeName string `protobuf:"bytes,1,opt,name=nodeName" json:"nodeName,omitempty"`
+ RPCPort int32 `protobuf:"varint,2,opt,name=RPCPort" json:"RPCPort,omitempty"`
+ Time string `protobuf:"bytes,3,opt,name=time" json:"time,omitempty"`
+}
+
+func (m *Member) Reset() { *m = Member{} }
+func (m *Member) String() string { return proto1.CompactTextString(m) }
+func (*Member) ProtoMessage() {}
+func (*Member) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{0} }
+
+func (m *Member) GetNodeName() string {
+ if m != nil {
+ return m.NodeName
+ }
+ return ""
+}
+
+func (m *Member) GetRPCPort() int32 {
+ if m != nil {
+ return m.RPCPort
+ }
+ return 0
+}
+
+func (m *Member) GetTime() string {
+ if m != nil {
+ return m.Time
+ }
+ return ""
+}
+
+type Discover struct {
+ ServiceName string `protobuf:"bytes,1,opt,name=serviceName" json:"serviceName,omitempty"`
+ Options string `protobuf:"bytes,2,opt,name=options" json:"options,omitempty"`
+ Time string `protobuf:"bytes,3,opt,name=time" json:"time,omitempty"`
+}
+
+func (m *Discover) Reset() { *m = Discover{} }
+func (m *Discover) String() string { return proto1.CompactTextString(m) }
+func (*Discover) ProtoMessage() {}
+func (*Discover) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{1} }
+
+func (m *Discover) GetServiceName() string {
+ if m != nil {
+ return m.ServiceName
+ }
+ return ""
+}
+
+func (m *Discover) GetOptions() string {
+ if m != nil {
+ return m.Options
+ }
+ return ""
+}
+
+func (m *Discover) GetTime() string {
+ if m != nil {
+ return m.Time
+ }
+ return ""
+}
+
+func init() {
+ proto1.RegisterType((*Member)(nil), "proto.Member")
+ proto1.RegisterType((*Discover)(nil), "proto.Discover")
+}
+
+func init() { proto1.RegisterFile("event.proto", fileDescriptor0) }
+
+var fileDescriptor0 = []byte{
+ // 154 bytes of a gzipped FileDescriptorProto
+ 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4e, 0x2d, 0x4b, 0xcd,
+ 0x2b, 0xd1, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x05, 0x53, 0x4a, 0x41, 0x5c, 0x6c, 0xbe,
+ 0xa9, 0xb9, 0x49, 0xa9, 0x45, 0x42, 0x52, 0x5c, 0x1c, 0x79, 0xf9, 0x29, 0xa9, 0x7e, 0x89, 0xb9,
+ 0xa9, 0x12, 0x8c, 0x0a, 0x8c, 0x1a, 0x9c, 0x41, 0x70, 0xbe, 0x90, 0x04, 0x17, 0x7b, 0x50, 0x80,
+ 0x73, 0x40, 0x7e, 0x51, 0x89, 0x04, 0x93, 0x02, 0xa3, 0x06, 0x6b, 0x10, 0x8c, 0x2b, 0x24, 0xc4,
+ 0xc5, 0x52, 0x92, 0x99, 0x9b, 0x2a, 0xc1, 0x0c, 0xd6, 0x01, 0x66, 0x2b, 0x45, 0x71, 0x71, 0xb8,
+ 0x64, 0x16, 0x27, 0xe7, 0x97, 0xa5, 0x16, 0x09, 0x29, 0x70, 0x71, 0x17, 0xa7, 0x16, 0x95, 0x65,
+ 0x26, 0x23, 0x1b, 0x8c, 0x2c, 0x04, 0x32, 0x3b, 0xbf, 0xa0, 0x24, 0x33, 0x3f, 0xaf, 0x18, 0x6c,
+ 0x36, 0x67, 0x10, 0x8c, 0x8b, 0xcd, 0xec, 0x24, 0x36, 0xb0, 0xb3, 0x8d, 0x01, 0x01, 0x00, 0x00,
+ 0xff, 0xff, 0x00, 0xd3, 0xab, 0xa6, 0xcc, 0x00, 0x00, 0x00,
+}
diff --git a/syncer/proto/event.proto b/syncer/proto/event.proto
new file mode 100644
index 0000000..65aed26
--- /dev/null
+++ b/syncer/proto/event.proto
@@ -0,0 +1,15 @@
+syntax = "proto3";
+
+package proto;
+
+message Member {
+ string nodeName = 1;
+ int32 RPCPort = 2;
+ string time = 3;
+}
+
+message Discover {
+ string serviceName = 1;
+ string options = 2;
+ string time = 3;
+}
\ No newline at end of file
diff --git a/syncer/proto/syncer.pb.go b/syncer/proto/syncer.pb.go
new file mode 100644
index 0000000..7f0082b
--- /dev/null
+++ b/syncer/proto/syncer.pb.go
@@ -0,0 +1,200 @@
+// Code generated by protoc-gen-go. DO NOT EDIT.
+// source: syncer.proto
+
+package proto
+
+import proto1 "github.com/golang/protobuf/proto"
+import fmt "fmt"
+import math "math"
+import proto2 "github.com/apache/servicecomb-service-center/server/core/proto"
+
+import (
+ context "golang.org/x/net/context"
+ grpc "google.golang.org/grpc"
+)
+
+// Reference imports to suppress errors if they are not otherwise used.
+var _ = proto1.Marshal
+var _ = fmt.Errorf
+var _ = math.Inf
+
+type PullRequest struct {
+ ServiceName string `protobuf:"bytes,1,opt,name=serviceName" json:"serviceName,omitempty"`
+ Options string `protobuf:"bytes,2,opt,name=options" json:"options,omitempty"`
+ Time string `protobuf:"bytes,3,opt,name=time" json:"time,omitempty"`
+}
+
+func (m *PullRequest) Reset() { *m = PullRequest{} }
+func (m *PullRequest) String() string { return proto1.CompactTextString(m) }
+func (*PullRequest) ProtoMessage() {}
+func (*PullRequest) Descriptor() ([]byte, []int) { return fileDescriptor1, []int{0} }
+
+func (m *PullRequest) GetServiceName() string {
+ if m != nil {
+ return m.ServiceName
+ }
+ return ""
+}
+
+func (m *PullRequest) GetOptions() string {
+ if m != nil {
+ return m.Options
+ }
+ return ""
+}
+
+func (m *PullRequest) GetTime() string {
+ if m != nil {
+ return m.Time
+ }
+ return ""
+}
+
+type SyncService struct {
+ DomainProject string `protobuf:"bytes,1,opt,name=domainProject" json:"domainProject,omitempty"`
+ Service *proto2.MicroService `protobuf:"bytes,2,opt,name=service" json:"service,omitempty"`
+ Instances []*proto2.MicroServiceInstance `protobuf:"bytes,3,rep,name=instances" json:"instances,omitempty"`
+}
+
+func (m *SyncService) Reset() { *m = SyncService{} }
+func (m *SyncService) String() string { return proto1.CompactTextString(m) }
+func (*SyncService) ProtoMessage() {}
+func (*SyncService) Descriptor() ([]byte, []int) { return fileDescriptor1, []int{1} }
+
+func (m *SyncService) GetDomainProject() string {
+ if m != nil {
+ return m.DomainProject
+ }
+ return ""
+}
+
+func (m *SyncService) GetService() *proto2.MicroService {
+ if m != nil {
+ return m.Service
+ }
+ return nil
+}
+
+func (m *SyncService) GetInstances() []*proto2.MicroServiceInstance {
+ if m != nil {
+ return m.Instances
+ }
+ return nil
+}
+
+type SyncData struct {
+ Services []*SyncService `protobuf:"bytes,1,rep,name=services" json:"services,omitempty"`
+}
+
+func (m *SyncData) Reset() { *m = SyncData{} }
+func (m *SyncData) String() string { return proto1.CompactTextString(m) }
+func (*SyncData) ProtoMessage() {}
+func (*SyncData) Descriptor() ([]byte, []int) { return fileDescriptor1, []int{2} }
+
+func (m *SyncData) GetServices() []*SyncService {
+ if m != nil {
+ return m.Services
+ }
+ return nil
+}
+
+func init() {
+ proto1.RegisterType((*PullRequest)(nil), "proto.PullRequest")
+ proto1.RegisterType((*SyncService)(nil), "proto.SyncService")
+ proto1.RegisterType((*SyncData)(nil), "proto.SyncData")
+}
+
+// Reference imports to suppress errors if they are not otherwise used.
+var _ context.Context
+var _ grpc.ClientConn
+
+// This is a compile-time assertion to ensure that this generated file
+// is compatible with the grpc package it is being compiled against.
+const _ = grpc.SupportPackageIsVersion4
+
+// Client API for Sync service
+
+type SyncClient interface {
+ Pull(ctx context.Context, in *PullRequest, opts ...grpc.CallOption) (*SyncData, error)
+}
+
+type syncClient struct {
+ cc *grpc.ClientConn
+}
+
+func NewSyncClient(cc *grpc.ClientConn) SyncClient {
+ return &syncClient{cc}
+}
+
+func (c *syncClient) Pull(ctx context.Context, in *PullRequest, opts ...grpc.CallOption) (*SyncData, error) {
+ out := new(SyncData)
+ err := grpc.Invoke(ctx, "/proto.Sync/Pull", in, out, c.cc, opts...)
+ if err != nil {
+ return nil, err
+ }
+ return out, nil
+}
+
+// Server API for Sync service
+
+type SyncServer interface {
+ Pull(context.Context, *PullRequest) (*SyncData, error)
+}
+
+func RegisterSyncServer(s *grpc.Server, srv SyncServer) {
+ s.RegisterService(&_Sync_serviceDesc, srv)
+}
+
+func _Sync_Pull_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
+ in := new(PullRequest)
+ if err := dec(in); err != nil {
+ return nil, err
+ }
+ if interceptor == nil {
+ return srv.(SyncServer).Pull(ctx, in)
+ }
+ info := &grpc.UnaryServerInfo{
+ Server: srv,
+ FullMethod: "/proto.Sync/Pull",
+ }
+ handler := func(ctx context.Context, req interface{}) (interface{}, error) {
+ return srv.(SyncServer).Pull(ctx, req.(*PullRequest))
+ }
+ return interceptor(ctx, in, info, handler)
+}
+
+var _Sync_serviceDesc = grpc.ServiceDesc{
+ ServiceName: "proto.Sync",
+ HandlerType: (*SyncServer)(nil),
+ Methods: []grpc.MethodDesc{
+ {
+ MethodName: "Pull",
+ Handler: _Sync_Pull_Handler,
+ },
+ },
+ Streams: []grpc.StreamDesc{},
+ Metadata: "syncer.proto",
+}
+
+func init() { proto1.RegisterFile("syncer.proto", fileDescriptor1) }
+
+var fileDescriptor1 = []byte{
+ // 259 bytes of a gzipped FileDescriptorProto
+ 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x6c, 0x90, 0x3f, 0x4f, 0xc3, 0x30,
+ 0x10, 0xc5, 0x09, 0x09, 0xb4, 0x3d, 0xf3, 0x47, 0x3a, 0x16, 0xab, 0x2c, 0x51, 0xc4, 0xd0, 0xa5,
+ 0x19, 0x82, 0x18, 0x60, 0x66, 0x61, 0x00, 0x55, 0xee, 0xcc, 0x60, 0xcc, 0x0d, 0x46, 0x8d, 0x5d,
+ 0x6c, 0x17, 0xa9, 0x1f, 0x86, 0xef, 0x8a, 0xe2, 0x38, 0x10, 0xa4, 0x4e, 0x89, 0xdf, 0xfd, 0xfc,
+ 0xde, 0xf9, 0xc1, 0x99, 0xdf, 0x1b, 0x45, 0xae, 0xde, 0x3a, 0x1b, 0x2c, 0x9e, 0xc4, 0xcf, 0xfc,
+ 0xc2, 0x93, 0xfb, 0xd2, 0x8a, 0x7c, 0x2f, 0x57, 0xaf, 0xc0, 0x56, 0xbb, 0xcd, 0x46, 0xd0, 0xe7,
+ 0x8e, 0x7c, 0xc0, 0x12, 0x58, 0x02, 0x5e, 0x64, 0x4b, 0x3c, 0x2b, 0xb3, 0xc5, 0x4c, 0x8c, 0x25,
+ 0xe4, 0x30, 0xb1, 0xdb, 0xa0, 0xad, 0xf1, 0xfc, 0x38, 0x4e, 0x87, 0x23, 0x22, 0x14, 0x41, 0xb7,
+ 0xc4, 0xf3, 0x28, 0xc7, 0xff, 0xea, 0x3b, 0x03, 0xb6, 0xde, 0x1b, 0xb5, 0xee, 0x1d, 0xf0, 0x06,
+ 0xce, 0xdf, 0x6d, 0x2b, 0xb5, 0x59, 0x39, 0xfb, 0x41, 0x2a, 0xa4, 0x84, 0xff, 0x22, 0x2e, 0x61,
+ 0x92, 0x22, 0x63, 0x06, 0x6b, 0xae, 0xfa, 0x6d, 0xeb, 0x67, 0xad, 0x9c, 0x4d, 0x5e, 0x62, 0x60,
+ 0xf0, 0x1e, 0x66, 0xda, 0xf8, 0x20, 0x8d, 0x22, 0xcf, 0xf3, 0x32, 0x5f, 0xb0, 0xe6, 0xfa, 0xc0,
+ 0x85, 0xa7, 0xc4, 0x88, 0x3f, 0xba, 0x7a, 0x80, 0x69, 0xb7, 0xde, 0xa3, 0x0c, 0x12, 0x6b, 0x98,
+ 0x0e, 0xe5, 0xf0, 0x2c, 0xba, 0x60, 0x72, 0x19, 0xbd, 0x40, 0xfc, 0x32, 0xcd, 0x1d, 0x14, 0xdd,
+ 0x00, 0x97, 0x50, 0x74, 0x15, 0xe2, 0x40, 0x8f, 0xfa, 0x9c, 0x5f, 0x8e, 0x1c, 0xba, 0x90, 0xea,
+ 0xe8, 0xed, 0x34, 0x2a, 0xb7, 0x3f, 0x01, 0x00, 0x00, 0xff, 0xff, 0x48, 0xd4, 0x2e, 0x71, 0x9f,
+ 0x01, 0x00, 0x00,
+}
diff --git a/syncer/proto/syncer.proto b/syncer/proto/syncer.proto
new file mode 100644
index 0000000..6f95ab9
--- /dev/null
+++ b/syncer/proto/syncer.proto
@@ -0,0 +1,25 @@
+syntax = "proto3";
+import "services.proto";
+//import "../../server/core/proto/services.proto";
+package proto;
+
+message PullRequest {
+ string serviceName = 1;
+ string options = 2;
+ string time = 3;
+}
+
+service Sync {
+ rpc Pull(PullRequest) returns (SyncData) {}
+}
+
+message SyncService {
+ string domainProject = 1;
+ MicroService service = 2;
+ repeated MicroServiceInstance instances = 3;
+}
+
+message SyncData {
+ repeated SyncService services = 1;
+}
+
diff --git a/syncer/proto/types.go b/syncer/proto/types.go
new file mode 100644
index 0000000..88988ae
--- /dev/null
+++ b/syncer/proto/types.go
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package proto
+
+type SyncMapping []*MappingItem
+
+type MappingItem struct {
+ NodeName string `json:"node_name"`
+ DomainProject string `json:"domain_project"`
+ OrgServiceID string `json:"org_service_id"`
+ OrgInstanceID string `json:"org_instance_id"`
+ CurServiceID string `json:"cur_service_id"`
+ CurInstanceID string `json:"cur_instance_id"`
+}
+
+func (s SyncMapping) OriginIndex(instanceID string) int {
+ for index, val := range s {
+ if val.OrgInstanceID == instanceID {
+ return index
+ }
+ }
+ return -1
+}
+
+func (s SyncMapping) CurrentIndex(instanceID string) int {
+ for index, val := range s {
+ if val.CurInstanceID == instanceID {
+ return index
+ }
+ }
+ return -1
+}
diff --git a/syncer/serf/agent.go b/syncer/serf/agent.go
new file mode 100644
index 0000000..60b4195
--- /dev/null
+++ b/syncer/serf/agent.go
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package serf
+
+import (
+ "context"
+ "io"
+ "os"
+
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/hashicorp/serf/cmd/serf/command/agent"
+ "github.com/hashicorp/serf/serf"
+)
+
+// Agent warps the serf agent
+type Agent struct {
+ *agent.Agent
+ conf *Config
+}
+
+// Create create serf agent with config
+func Create(conf *Config, logOutput io.Writer) (*Agent, error) {
+ if logOutput == nil {
+ logOutput = os.Stderr
+ }
+
+ // config cover to serf config
+ serfConf, err := conf.convertToSerf()
+ if err != nil {
+ return nil, err
+ }
+
+ // create serf agent with serf config
+ serfAgent, err := agent.Create(conf.Config, serfConf, logOutput)
+ if err != nil {
+ return nil, err
+ }
+ return &Agent{Agent: serfAgent, conf: conf}, nil
+}
+
+// Start agent
+func (a *Agent) Start(ctx context.Context) {
+ err := a.Agent.Start()
+ if err != nil {
+ log.Errorf(err, "start serf agent failed")
+ }
+
+ gopool.Go(func(ctx context.Context) {
+ select {
+ case <-ctx.Done():
+ a.Leave()
+ a.Shutdown()
+ }
+ })
+}
+
+// Leave from Serf
+func (a *Agent) Leave() error {
+ return a.Agent.Leave()
+}
+
+// Shutdown Serf server
+func (a *Agent) Shutdown() error {
+ return a.Agent.Shutdown()
+}
+
+// ShutdownCh returns a channel that can be selected to wait
+// for the agent to perform a shutdown.
+func (a *Agent) ShutdownCh() <-chan struct{} {
+ return a.Agent.ShutdownCh()
+}
+
+func (a *Agent) LocalMember() *serf.Member {
+ serfAgent := a.Agent.Serf()
+ if serfAgent != nil {
+ member := serfAgent.LocalMember()
+ return &member
+ }
+ serfAgent.State()
+ return nil
+}
+
+// Member get member information with node
+func (a *Agent) Member(node string) *serf.Member {
+ serfAgent := a.Agent.Serf()
+ if serfAgent != nil {
+ ms := serfAgent.Members()
+ for _, m := range ms {
+ if m.Name == node {
+ return &m
+ }
+ }
+ }
+ return nil
+}
+
+// SerfConfig get serf config
+func (a *Agent) SerfConfig() *serf.Config {
+ return a.Agent.SerfConfig()
+}
+
+// Join serf clusters through one or more members
+func (a *Agent) Join(addrs []string, replay bool) (n int, err error) {
+ return a.Agent.Join(addrs, replay)
+}
+
+// ForceLeave forced to leave from serf
+func (a *Agent) ForceLeave(node string) error {
+ return a.Agent.ForceLeave(node)
+}
+
+// UserEvent sends a UserEvent on Serf
+func (a *Agent) UserEvent(name string, payload []byte, coalesce bool) error {
+ return a.Agent.UserEvent(name, payload, coalesce)
+}
+
+// Query sends a Query on Serf
+func (a *Agent) Query(name string, payload []byte, params *serf.QueryParam) (*serf.QueryResponse, error) {
+ return a.Agent.Query(name, payload, params)
+}
diff --git a/syncer/serf/agent_test.go b/syncer/serf/agent_test.go
new file mode 100644
index 0000000..cf56565
--- /dev/null
+++ b/syncer/serf/agent_test.go
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package serf
+
+import (
+ "context"
+ "testing"
+ "time"
+
+ "github.com/hashicorp/serf/serf"
+)
+
+func TestAgent(t *testing.T) {
+ conf := DefaultConfig()
+ agent, err := Create(conf, nil)
+ if err != nil {
+ t.Errorf("create agent failed, error: %s", err)
+ }
+ ctx, cancel := context.WithCancel(context.Background())
+ agent.Start(ctx)
+
+ go func() {
+ agent.ShutdownCh()
+ }()
+ time.Sleep(time.Second)
+
+ err = agent.UserEvent("test", []byte("test"), true)
+ if err != nil {
+ t.Errorf("send user event failed, error: %s", err)
+ }
+
+ _, err = agent.Query("test", []byte("test"), &serf.QueryParam{})
+ if err != nil {
+ t.Errorf("query for other node failed, error: %s", err)
+ }
+ agent.LocalMember()
+
+ agent.Member("testnode")
+
+ agent.SerfConfig()
+
+ _, err = agent.Join([]string{"127.0.0.1:9999"}, true)
+ if err != nil {
+ t.Logf("join to other node failed, error: %s", err)
+ }
+
+ err = agent.Leave()
+ if err != nil {
+ t.Errorf("angent leave failed, error: %s", err)
+ }
+
+ err = agent.ForceLeave("testnode")
+ if err != nil {
+ t.Errorf("angent force leave failed, error: %s", err)
+ }
+
+ err = agent.Shutdown()
+ if err != nil {
+ t.Errorf("angent shutdown failed, error: %s", err)
+ }
+ cancel()
+}
diff --git a/syncer/serf/config.go b/syncer/serf/config.go
new file mode 100644
index 0000000..f9dd94c
--- /dev/null
+++ b/syncer/serf/config.go
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package serf
+
+import (
+ "fmt"
+
+ "github.com/apache/servicecomb-service-center/syncer/pkg/utils"
+ "github.com/hashicorp/memberlist"
+ "github.com/hashicorp/serf/cmd/serf/command/agent"
+ "github.com/hashicorp/serf/serf"
+)
+
+const (
+ DefaultBindPort = 30190
+ DefaultRPCPort = 30191
+)
+
+// DefaultConfig default config
+func DefaultConfig() *Config {
+ agentConf := agent.DefaultConfig()
+ agentConf.BindAddr = fmt.Sprintf("0.0.0.0:%d", DefaultBindPort)
+ agentConf.RPCAddr = fmt.Sprintf("0.0.0.0:%d", DefaultRPCPort)
+ return &Config{Config: agentConf}
+}
+
+// Config struct
+type Config struct {
+ // config from serf agent
+ *agent.Config
+ RPCPort int `yaml:"-"`
+}
+
+// readConfigFile reads configuration from config file
+func (c *Config) readConfigFile(filepath string) error {
+ if filepath != "" {
+ // todo:
+ }
+ return nil
+}
+
+// convertToSerf convert Config to serf.Config
+func (c *Config) convertToSerf() (*serf.Config, error) {
+ serfConf := serf.DefaultConfig()
+
+ bindIP, bindPort, err := utils.SplitHostPort(c.BindAddr, DefaultBindPort)
+ if err != nil {
+ return nil, fmt.Errorf("invalid bind address: %s", err)
+ }
+
+ switch c.Profile {
+ case "lan":
+ serfConf.MemberlistConfig = memberlist.DefaultLANConfig()
+ case "wan":
+ serfConf.MemberlistConfig = memberlist.DefaultWANConfig()
+ case "local":
+ serfConf.MemberlistConfig = memberlist.DefaultLocalConfig()
+ default:
+ serfConf.MemberlistConfig = memberlist.DefaultLANConfig()
+ }
+
+ serfConf.MemberlistConfig.BindAddr = bindIP
+ serfConf.MemberlistConfig.BindPort = bindPort
+ serfConf.NodeName = c.NodeName
+ return serfConf, nil
+}
diff --git a/syncer/server/handler.go b/syncer/server/handler.go
new file mode 100644
index 0000000..f65e0ca
--- /dev/null
+++ b/syncer/server/handler.go
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package server
+
+import (
+ "context"
+ "fmt"
+ "time"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/syncer/grpc"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+ "github.com/gogo/protobuf/proto"
+ "github.com/hashicorp/serf/serf"
+)
+
+const (
+ EventDiscovered = "discovered"
+)
+
+// tickHandler Timed task handler
+func (s *Server) tickHandler(ctx context.Context) {
+ log.Debugf("Handle Tick")
+ // Flush data to the storage of servicecenter
+ s.servicecenter.FlushData()
+
+ event, _ := proto.Marshal(&pb.Member{
+ NodeName: s.conf.NodeName,
+ RPCPort: int32(s.conf.RPCPort),
+ Time: fmt.Sprintf("%d", time.Now().UTC().Second()),
+ })
+
+ // sends a UserEvent on Serf, the event will be broadcast between members
+ err := s.agent.UserEvent(EventDiscovered, event, true)
+ if err != nil {
+ log.Errorf(err, "Syncer send user event failed")
+ }
+}
+
+// GetData Sync Data to GRPC
+func (s *Server) Discovery() *pb.SyncData {
+ return s.servicecenter.Discovery()
+}
+
+// HandleEvent Handles events from serf
+func (s *Server) HandleEvent(event serf.Event) {
+ switch event.EventType() {
+ case serf.EventUser:
+ s.userEvent(event.(serf.UserEvent))
+ case serf.EventQuery:
+ s.queryEvent(event.(*serf.Query))
+ }
+}
+
+// userEvent Handles "EventUser" notification events, no response required
+func (s *Server) userEvent(event serf.UserEvent) {
+ m := &pb.Member{}
+ err := proto.Unmarshal(event.Payload, m)
+ if err != nil {
+ log.Errorf(err, "trigger user event '%s' handler failed", event.EventType())
+ return
+ }
+
+ // Excludes notifications from self, as the gossip protocol inevitably has redundant notifications
+ if s.agent.LocalMember().Name == m.NodeName {
+ return
+ }
+
+ // Get member information and get synchronized data from it
+ member := s.agent.Member(m.NodeName)
+ // Get dta from remote member
+ endpoint := fmt.Sprintf("%s:%d", member.Addr, m.RPCPort)
+ log.Debugf("Going to pull data from %s %s", m.NodeName, endpoint)
+ data, err := grpc.Pull(context.Background(), endpoint)
+ if err != nil {
+ log.Errorf(err, "Pull other serf instances failed, node name is '%s'", m.NodeName)
+ return
+ }
+ // Registry instances to servicecenter and update storage of it
+ s.servicecenter.Registry(m.NodeName, data)
+}
+
+// queryEvent Handles "EventQuery" query events and respond if conditions are met
+func (s *Server) queryEvent(query *serf.Query) {
+ // todo: Get instances requested
+}
diff --git a/syncer/server/server.go b/syncer/server/server.go
new file mode 100644
index 0000000..0e3bbdc
--- /dev/null
+++ b/syncer/server/server.go
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package server
+
+import (
+ "context"
+ "io"
+ "os"
+ "strings"
+ "syscall"
+
+ "github.com/apache/servicecomb-service-center/pkg/gopool"
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/syncer/config"
+ "github.com/apache/servicecomb-service-center/syncer/servicecenter"
+ "github.com/apache/servicecomb-service-center/syncer/grpc"
+ "github.com/apache/servicecomb-service-center/syncer/pkg/syssig"
+ "github.com/apache/servicecomb-service-center/syncer/pkg/ticker"
+ "github.com/apache/servicecomb-service-center/syncer/pkg/utils"
+ "github.com/apache/servicecomb-service-center/syncer/plugins"
+ "github.com/apache/servicecomb-service-center/syncer/serf"
+ "github.com/apache/servicecomb-service-center/syncer/storage"
+)
+
+// Server struct for syncer
+type Server struct {
+ // Syncer configuration
+ conf *config.Config
+
+ // Ticker for Syncer
+ tick *ticker.TaskTicker
+
+ // Wrap the servicecenter
+ servicecenter servicecenter.Servicecenter
+
+ storage *storage.Storage
+
+ // Wraps the serf agent
+ agent *serf.Agent
+
+ // Wraps the grpc server
+ grpc *grpc.Server
+}
+
+// NewServer new server with Config
+func NewServer(conf *config.Config) *Server {
+ return &Server{
+ conf: conf,
+ }
+}
+
+// Run syncer Server
+func (s *Server) Run(ctx context.Context) {
+ s.initPlugin()
+
+ if err := s.initialization(); err != nil {
+ log.Errorf(err, "syncer server initialization faild: %s")
+ return
+ }
+
+ s.agent.RegisterEventHandler(s)
+
+ // Start serf/grpc/tick services
+ s.startServers(ctx)
+
+ s.waitQuit(ctx)
+}
+
+// Stop Syncer Server
+func (s *Server) Stop() {
+ if s.tick != nil {
+ s.tick.Stop()
+ }
+
+ if s.agent != nil {
+ // removes the serf eventHandler
+ s.agent.DeregisterEventHandler(s)
+ //Leave from Serf
+ s.agent.Leave()
+ // closes this serf agent
+ s.agent.Shutdown()
+ }
+
+ if s.grpc != nil {
+ s.grpc.Stop()
+ }
+
+ if s.storage != nil {
+ s.storage.Stop()
+ }
+
+ // Closes all goroutines in the pool
+ gopool.CloseAndWait()
+}
+
+// initPlugin Initialize the plugin and load the external plugin according to the configuration
+func (s *Server) initPlugin() {
+ plugins.SetPluginConfig(plugins.PluginServicecenter.String(), s.conf.ServicecenterPlugin)
+ plugins.LoadPlugins()
+}
+
+// initialization Initialize the starter of the syncer
+func (s *Server) initialization() (err error) {
+ s.storage = storage.New()
+
+ s.tick = ticker.NewTaskTicker(s.conf.TickerInterval, s.tickHandler)
+
+ s.servicecenter, err = servicecenter.NewServicecenter(strings.Split(s.conf.SCAddr, ","), s.storage)
+ if err != nil {
+ return err
+ }
+
+ s.agent, err = serf.Create(s.conf.Config, createLogFile(s.conf.LogFile))
+ if err != nil {
+ return err
+ }
+
+ s.grpc = grpc.NewServer(s.conf.RPCAddr, s)
+ return nil
+}
+
+// startServers Start all internal services
+func (s *Server) startServers(ctx context.Context) {
+ // start serf agent service to wait for
+ s.agent.Start(ctx)
+
+ if s.conf.JoinAddr != "" {
+ _, err := s.agent.Join([]string{s.conf.JoinAddr}, false)
+ if err != nil {
+ log.Errorf(err, "Syncer join serf cluster failed")
+ }
+ }
+
+ s.grpc.Run()
+
+ gopool.Go(s.tick.Start)
+}
+
+// waitQuit Waiting for system quit signal
+func (s *Server) waitQuit(ctx context.Context) {
+ err := syssig.AddSignalsHandler(func() {
+ s.Stop()
+ }, syscall.SIGINT, syscall.SIGKILL, syscall.SIGTERM)
+ if err != nil {
+ log.Errorf(err, "Syncer add signals handler failed")
+ return
+ }
+ syssig.Run(ctx)
+}
+
+// createLogFile create log file
+func createLogFile(logFile string) (fw io.Writer) {
+ fw = os.Stderr
+ if logFile == "" {
+ return
+ }
+
+ f, err := utils.OpenFile(logFile)
+ if err != nil {
+ log.Errorf(err, "Syncer open log file %s failed", logFile)
+ return
+ }
+ return f
+}
diff --git a/syncer/servicecenter/exclude.go b/syncer/servicecenter/exclude.go
new file mode 100644
index 0000000..d3f0f21
--- /dev/null
+++ b/syncer/servicecenter/exclude.go
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+// exclude instances from other servicecenter, exclude the expired instances in the maps
+func (s *servicecenter) exclude(data *pb.SyncData, mapping pb.SyncMapping) (*pb.SyncData, pb.SyncMapping) {
+ services := make([]*pb.SyncService, 0, 10)
+ maps := make(pb.SyncMapping, 0, len(mapping))
+ for _, svc := range data.Services {
+
+ nis := make([]*scpb.MicroServiceInstance, 0, len(svc.Instances))
+ for _, inst := range svc.Instances {
+ if index := mapping.CurrentIndex(inst.InstanceId); index != -1 {
+ // exclude the expired instances in the maps
+ maps = append(maps, mapping[index])
+ continue
+ }
+ // exclude instances from other servicecenter
+ nis = append(nis, inst)
+ }
+
+ svc.Instances = nis
+ services = append(services, svc)
+ }
+ data.Services = services
+
+ return data, maps
+}
diff --git a/syncer/servicecenter/servicecenter.go b/syncer/servicecenter/servicecenter.go
new file mode 100644
index 0000000..74b6cfa
--- /dev/null
+++ b/syncer/servicecenter/servicecenter.go
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/syncer/plugins"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+// Store interface of servicecenter
+type Servicecenter interface {
+ FlushData()
+ Registry(nodeName string, data *pb.SyncData)
+ Discovery() *pb.SyncData
+}
+
+type servicecenter struct {
+ servicecenter plugins.Servicecenter
+ storage Storage
+}
+
+type Storage interface {
+ GetData() (data *pb.SyncData)
+ UpdateData(data *pb.SyncData)
+ GetMaps() (maps pb.SyncMapping)
+ UpdateMaps(maps pb.SyncMapping)
+ GetMapByNode(nodeName string) (mapping pb.SyncMapping)
+ UpdateMapByNode(nodeName string, mapping pb.SyncMapping)
+}
+
+// NewServicecenter new store with endpoints
+func NewServicecenter(endpoints []string, storage Storage) (Servicecenter, error) {
+ dc, err := plugins.Plugins().Servicecenter().New(endpoints)
+ if err != nil {
+ return nil, err
+ }
+
+ return &servicecenter{
+ servicecenter: dc,
+ storage: storage,
+ }, nil
+}
+
+// FlushData flush data to servicecenter, update mapping data
+func (s *servicecenter) FlushData() {
+ data, err := s.servicecenter.GetAll(context.Background())
+ if err != nil {
+ log.Errorf(err, "Syncer discover instances failed")
+ return
+ }
+
+ maps := s.storage.GetMaps()
+
+ data, maps = s.exclude(data, maps)
+ s.storage.UpdateData(data)
+ s.storage.UpdateMaps(maps)
+}
+
+// Registry registry data to the servicecenter, update mapping data
+func (s *servicecenter) Registry(nodeName string, data *pb.SyncData) {
+ mapping := s.storage.GetMapByNode(nodeName)
+ for _, svc := range data.Services {
+ log.Debugf("trying to do registration of service, serviceID = %s", svc.Service.ServiceId)
+ // If the svc is in the mapping, just do nothing, if not, created it in servicecenter and get the new serviceID
+ svcID := s.createService(svc)
+ for _, inst := range svc.Instances {
+ // If inst is in the mapping, just heart beat it in servicecenter
+ log.Debugf("trying to do registration of instance, instanceID = %s", inst.InstanceId)
+ if s.heartbeatInstances(mapping, inst) {
+ continue
+ }
+
+ // If inst is not in the mapping, that is because this the first time syncer get the instance data
+ // in this case, we should registry it to the servicecenter and get the new instanceID
+ item := &pb.MappingItem{
+ DomainProject: svc.DomainProject,
+ OrgServiceID: inst.ServiceId,
+ OrgInstanceID: inst.InstanceId,
+ CurServiceID: svcID,
+ NodeName: nodeName,
+ }
+ item.CurInstanceID = s.registryInstances(svc.DomainProject, svcID, inst)
+
+ // Use new serviceID and instanceID to update mapping data in this servicecenter
+ if item.CurInstanceID != "" {
+ mapping = append(mapping, item)
+ }
+ }
+ }
+ // UnRegistry instances that is not in the data which means the instance in the mapping is no longer actived
+ mapping = s.unRegistryInstances(data, mapping)
+ // Update mapping data of the node to the storage of the servicecenter
+ s.storage.UpdateMapByNode(nodeName, mapping)
+}
+
+// Discovery discovery data from storage
+func (s *servicecenter) Discovery() *pb.SyncData {
+ return s.storage.GetData()
+}
diff --git a/syncer/servicecenter/servicecenter_test.go b/syncer/servicecenter/servicecenter_test.go
new file mode 100644
index 0000000..71f379c
--- /dev/null
+++ b/syncer/servicecenter/servicecenter_test.go
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+ "errors"
+ "github.com/apache/servicecomb-service-center/syncer/storage"
+ "testing"
+
+ "github.com/apache/servicecomb-service-center/server/core/proto"
+ "github.com/apache/servicecomb-service-center/syncer/config"
+ "github.com/apache/servicecomb-service-center/syncer/plugins"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+ "github.com/apache/servicecomb-service-center/syncer/test/dcmock"
+)
+
+func TestNewServicecenter(t *testing.T) {
+ defer func() {
+ err := recover()
+ if err != nil {
+ t.Log(err)
+ }
+ }()
+ _, err := NewServicecenter([]string{"127.0.0.1:30100"}, nil)
+ if err != nil {
+ t.Log(err)
+ }
+
+ _, err = NewServicecenter([]string{"127.0.0.1:30100"}, nil)
+ if err != nil {
+ t.Fatal(err)
+ return
+ }
+}
+
+func TestOnEvent(t *testing.T) {
+ conf := config.DefaultConfig()
+ conf.ServicecenterPlugin = dcmock.PluginName
+ initPlugin(conf)
+ dc, err := NewServicecenter([]string{"http://127.0.0.1:30100"}, storage.New())
+ if err != nil {
+ t.Fatal(err)
+ return
+ }
+
+ dcmock.SetGetAll(func(ctx context.Context) (data *pb.SyncData, e error) {
+ return nil, errors.New("test error")
+ })
+
+ dc.FlushData()
+ data := dc.Discovery()
+ if err != nil {
+ t.Log(err)
+ }
+
+ dcmock.SetGetAll(nil)
+
+ dc.FlushData()
+ data = dc.Discovery()
+ if err != nil {
+ t.Fatal(err)
+ return
+ }
+
+ nodeName := "test_node"
+ dc.Registry(nodeName, data)
+
+ dcmock.SetGetAll(dcmock.NewGetAll)
+ dc.FlushData()
+ newData := dc.Discovery()
+ if err != nil {
+ t.Fatal(err)
+ return
+ }
+
+ dc.Registry(nodeName, newData)
+
+ dcmock.SetRegisterInstance(func(ctx context.Context, domainProject, serviceId string, instance *proto.MicroServiceInstance) (s string, e error) {
+ return "", errors.New("test error")
+ })
+
+ dc.Registry(nodeName, data)
+
+ dcmock.SetRegisterInstance(nil)
+
+ dc.Registry(nodeName, data)
+
+ dc.Registry(nodeName, data)
+
+ dcmock.SetHeartbeat(func(ctx context.Context, domainProject, serviceId, instanceId string) error {
+ return errors.New("test error")
+ })
+
+ dc.Registry(nodeName, data)
+}
+
+func initPlugin(conf *config.Config) {
+ plugins.SetPluginConfig(plugins.PluginServicecenter.String(), conf.ServicecenterPlugin)
+}
diff --git a/syncer/servicecenter/sync.go b/syncer/servicecenter/sync.go
new file mode 100644
index 0000000..56b26eb
--- /dev/null
+++ b/syncer/servicecenter/sync.go
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package servicecenter
+
+import (
+ "context"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+// Send an instance heartbeat if the instance has already been registered
+func (s *servicecenter) heartbeatInstances(mapping pb.SyncMapping, instance *scpb.MicroServiceInstance) bool {
+ index := mapping.OriginIndex(instance.InstanceId)
+ if index == -1 {
+ return false
+ }
+
+ item := mapping[index]
+ err := s.servicecenter.Heartbeat(context.Background(), item.DomainProject, item.CurServiceID, item.CurInstanceID)
+ if err != nil {
+ log.Errorf(err, "Servicecenter heartbeat instance failed")
+ }
+ log.Debugf("Instance %s is already exist, sent heartbeat to service-center")
+ instance.InstanceId = item.CurInstanceID
+ return true
+}
+
+func (s *servicecenter) createService(service *pb.SyncService) string {
+ ctx := context.Background()
+ serviceID, _ := s.servicecenter.ServiceExistence(ctx, service.DomainProject, service.Service)
+ if serviceID != "" {
+ return serviceID
+ }
+ service.Service.ServiceId = ""
+ serviceID, err := s.servicecenter.CreateService(ctx, service.DomainProject, service.Service)
+ if err != nil {
+ log.Errorf(err, "Servicecenter create service failed")
+ return ""
+ }
+ log.Debugf("Create service successful, serviceID = %s", serviceID)
+ service.Service.ServiceId = serviceID
+ return serviceID
+}
+
+func (s *servicecenter) registryInstances(domainProject, serviceId string, instance *scpb.MicroServiceInstance) string {
+ instance.ServiceId = serviceId
+ instance.InstanceId = ""
+ instanceID, err := s.servicecenter.RegisterInstance(context.Background(), domainProject, serviceId, instance)
+ if err != nil {
+ log.Errorf(err, "Servicecenter registry instance failed")
+ return ""
+ }
+ log.Debugf("Registered instance successful, instanceID = %s", instanceID)
+ instance.InstanceId = instanceID
+ return instanceID
+}
+
+// DeleteInstances Unregister instances of mapping table that has been unregistered from other servicecenter
+func (s *servicecenter) unRegistryInstances(data *pb.SyncData, mapping pb.SyncMapping) pb.SyncMapping {
+ ctx := context.Background()
+ nm := make(pb.SyncMapping, 0, len(mapping))
+next:
+ for _, val := range mapping {
+ for _, svc := range data.Services {
+ for _, inst := range svc.Instances {
+ if val.CurInstanceID == inst.InstanceId {
+ nm = append(nm, val)
+ continue next
+ }
+ }
+ }
+
+ err := s.servicecenter.UnregisterInstance(ctx, val.DomainProject, val.CurServiceID, val.CurInstanceID)
+ if err != nil {
+ log.Errorf(err, "Servicecenter delete instance failed")
+ }
+ log.Debugf("Unregistered instance, InstanceID = %s", val.CurInstanceID)
+ }
+ return nm
+}
diff --git a/syncer/storage/storage.go b/syncer/storage/storage.go
new file mode 100644
index 0000000..4084ce0
--- /dev/null
+++ b/syncer/storage/storage.go
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package storage
+
+import (
+ "encoding/json"
+ "io/ioutil"
+ "sync"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/syncer/pkg/utils"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+var (
+ defaultMapping = make(pb.SyncMapping, 0)
+ snapshotPath = "./data/syncer-snapshot"
+)
+
+func New() *Storage {
+ return &Storage{
+ data: &pb.SyncData{},
+ maps: loadSnapshot(),
+ }
+}
+
+// Storage of Syncer
+type Storage struct {
+ data *pb.SyncData
+ // mapping table for other servicecenter instances
+ maps map[string]pb.SyncMapping
+ lock sync.RWMutex
+}
+
+// loadSnapshot Load snapshot of mapping table
+func loadSnapshot() map[string]pb.SyncMapping {
+ mapping := make(map[string]pb.SyncMapping)
+ data, err := ioutil.ReadFile(snapshotPath)
+ if err != nil {
+ log.Warnf("get syncer snapshot from '%s' failed, error: %s", snapshotPath, err)
+ return mapping
+ }
+ err = json.Unmarshal(data, &mapping)
+ if err != nil {
+ log.Warnf("unmarshal syncer snapshot failed, error: %s", err)
+ }
+ log.Infof("Loaded maps from disk to storage, maps = %s", data)
+ return mapping
+}
+
+func (s *Storage) Stop() {
+ s.flush()
+}
+
+// flush Refresh the mapping table to the hard disk
+func (s *Storage) flush() {
+ data, err := json.Marshal(&s.maps)
+ if err != nil {
+ log.Warnf("marshal syncer snapshot failed, error: %s", err)
+ return
+ }
+
+ f, err := utils.OpenFile(snapshotPath)
+ if err != nil {
+ log.Warnf("open syncer snapshot file '%s' failed, error: %s", snapshotPath, err)
+ return
+ }
+ defer f.Close()
+
+ _, err = f.Write(data)
+ if err != nil {
+ log.Warnf("flush syncer snapshot to '%s' failed, error: %s", snapshotPath, err)
+ return
+ }
+ log.Infof("Flushed maps to disk before exit, data = %s", data)
+}
+
+// UpdateData Update data to storage
+func (s *Storage) UpdateData(data *pb.SyncData) {
+ s.lock.Lock()
+ s.data = data
+ s.lock.Unlock()
+}
+
+// GetData Get data from storage
+func (s *Storage) GetData() (data *pb.SyncData) {
+ s.lock.RLock()
+ data = s.data
+ s.lock.RUnlock()
+ return
+}
+
+// UpdateMapByNode update map to storage by nodeName of other node
+func (s *Storage) UpdateMapByNode(nodeName string, mapping pb.SyncMapping) {
+ s.lock.Lock()
+ s.maps[nodeName] = mapping
+ s.lock.Unlock()
+}
+
+// GetMapByNode get map by nodeName of other node
+func (s *Storage) GetMapByNode(nodeName string) (mapping pb.SyncMapping) {
+ s.lock.RLock()
+ data, ok := s.maps[nodeName]
+ if !ok {
+ data = defaultMapping
+ }
+ s.lock.RUnlock()
+ return data
+}
+
+func (s *Storage) UpdateMaps(maps pb.SyncMapping) {
+ s.lock.Lock()
+ mappings := make(map[string]pb.SyncMapping)
+ for _, item := range maps {
+ mapping, ok := mappings[item.NodeName]
+ if !ok {
+ mapping = make(pb.SyncMapping, 0, 10)
+ }
+ mapping = append(mapping, item)
+ mappings[item.NodeName] = mapping
+ }
+ s.maps = mappings
+ s.lock.Unlock()
+}
+
+// GetMaps Get maps from storage
+func (s *Storage) GetMaps() (mapping pb.SyncMapping) {
+ s.lock.RLock()
+ mapping = make(pb.SyncMapping, 0, 10)
+ for _, data := range s.maps {
+ if data != nil {
+ mapping = append(mapping, data...)
+ }
+ }
+ s.lock.RUnlock()
+ return
+}
diff --git a/syncer/storage/storage_test.go b/syncer/storage/storage_test.go
new file mode 100644
index 0000000..3b3e0bc
--- /dev/null
+++ b/syncer/storage/storage_test.go
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package storage
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+func TestSyncData(t *testing.T) {
+ storage := New()
+ defer func() {
+ storage.Stop()
+ os.RemoveAll(filepath.Dir(snapshotPath))
+ }()
+ data := storage.GetData()
+ if len(data.Services) > 0 {
+ t.Error("default sync data was wrong")
+ }
+ data = getSyncData()
+ storage.UpdateData(data)
+ nd := storage.GetData()
+ if nd == nil {
+ t.Error("save sync data failed!")
+ }
+ storage.Stop()
+
+ New()
+}
+
+func TestSyncMapping(t *testing.T) {
+ storage := New()
+ defer func() {
+ storage.Stop()
+ os.RemoveAll(filepath.Dir(snapshotPath))
+ }()
+ nodeName := "testnode"
+ data := storage.GetMapByNode(nodeName)
+ if len(data) > 0 {
+ t.Error("default sync mapping was wrong")
+ }
+ data = getSyncMapping(nodeName)
+ storage.UpdateMapByNode(nodeName, data)
+ nd := storage.GetMapByNode(nodeName)
+
+ if index := nd.CurrentIndex(nodeName); index == -1 {
+ t.Error("save sync mapping failed!")
+ }
+
+ all := storage.GetMaps()
+ if index := all.CurrentIndex(nodeName); index == -1 {
+ t.Errorf("all mapping has not node name %s!", nodeName)
+ }
+
+ storage.UpdateMaps(all)
+}
+
+func getSyncMapping(nodeName string) pb.SyncMapping {
+ return pb.SyncMapping{&pb.MappingItem{
+ DomainProject: "default/default",
+ OrgServiceID: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ OrgInstanceID: "4d41a637471f11e9888cfa163eca30e0",
+ CurServiceID: nodeName,
+ CurInstanceID: nodeName,
+ }}
+}
+
+func getSyncData() *pb.SyncData {
+ return &pb.SyncData{
+ Services: []*pb.SyncService{
+ {
+ DomainProject: "default/default",
+ Service: &scpb.MicroService{
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ AppId: "default",
+ ServiceName: "SERVICECENTER",
+ Version: "1.1.0",
+ Level: "BACK",
+ Schemas: []string{
+ "servicecenter.grpc.api.ServiceCtrl",
+ "servicecenter.grpc.api.ServiceInstanceCtrl",
+ },
+ Status: "UP",
+ Properties: map[string]string{
+ "allowCrossApp": "true",
+ },
+ Timestamp: "1552626180",
+ ModTimestamp: "1552626180",
+ Environment: "production",
+ },
+ Instances: []*scpb.MicroServiceInstance{
+ {
+ InstanceId: "4d41a637471f11e9888cfa163eca30e0",
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ Endpoints: []string{
+ "rest://127.0.0.1:30100/",
+ },
+ HostName: "testmock",
+ Status: "UP",
+ HealthCheck: &scpb.HealthCheck{
+ Mode: "push",
+ Interval: 30,
+ Times: 3,
+ },
+ Timestamp: "1552653537",
+ ModTimestamp: "1552653537",
+ Version: "1.1.0",
+ },
+ },
+ },
+ },
+ }
+}
diff --git a/syncer/syncer.go b/syncer/syncer.go
new file mode 100644
index 0000000..52fb88b
--- /dev/null
+++ b/syncer/syncer.go
@@ -0,0 +1,15 @@
+package main
+
+import (
+ "os"
+
+ "github.com/apache/servicecomb-service-center/pkg/log"
+ "github.com/apache/servicecomb-service-center/syncer/cmd"
+)
+
+func main() {
+ if err := cmd.Execute(); err != nil {
+ log.Error("Failed to execute syncer command", err)
+ os.Exit(-1)
+ }
+}
diff --git a/syncer/test/dcmock/instance.go b/syncer/test/dcmock/instance.go
new file mode 100644
index 0000000..ed52838
--- /dev/null
+++ b/syncer/test/dcmock/instance.go
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dcmock
+
+import (
+ "context"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+)
+
+var (
+ registerInstance func(ctx context.Context, domainProject, serviceId string, instance *scpb.MicroServiceInstance) (string, error)
+ unregisterInstance func(ctx context.Context, domainProject, serviceId, instanceId string) error
+ discoveryInstances func(ctx context.Context, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule string) ([]*scpb.MicroServiceInstance, error)
+ heartbeat func(ctx context.Context, domainProject, serviceId, instanceId string) error
+)
+
+func SetRegisterInstance(handler func(ctx context.Context, domainProject, serviceId string, instance *scpb.MicroServiceInstance) (string, error)) {
+ registerInstance = handler
+}
+
+func SetUnregisterInstance(handler func(ctx context.Context, domainProject, serviceId, instanceId string) error) {
+ unregisterInstance = handler
+}
+
+func SetDiscoveryInstances(handler func(ctx context.Context, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule string) ([]*scpb.MicroServiceInstance, error)) {
+ discoveryInstances = handler
+}
+
+func SetHeartbeat(handler func(ctx context.Context, domainProject, serviceId, instanceId string) error) {
+ heartbeat = handler
+}
+
+func (c *mockPlugin) RegisterInstance(ctx context.Context, domainProject, serviceId string, instance *scpb.MicroServiceInstance) (string, error) {
+ if registerInstance != nil {
+ return registerInstance(ctx, domainProject, serviceId, instance)
+ }
+ return "4d41a637471f11e9888cfa163eca30e0", nil
+}
+
+func (c *mockPlugin) UnregisterInstance(ctx context.Context, domainProject, serviceId, instanceId string) error {
+ if unregisterInstance != nil {
+ return unregisterInstance(ctx, domainProject, serviceId, instanceId)
+ }
+ return nil
+}
+
+func (c *mockPlugin) DiscoveryInstances(ctx context.Context, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule string) ([]*scpb.MicroServiceInstance, error) {
+ if discoveryInstances != nil {
+ return discoveryInstances(ctx, domainProject, consumerId, providerAppId, providerServiceName, providerVersionRule)
+ }
+ return []*scpb.MicroServiceInstance{
+ {
+ InstanceId: "4d41a637471f11e9888cfa163eca30e0",
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ Endpoints: []string{
+ "rest://127.0.0.1:30100/",
+ },
+ HostName: "testmock",
+ Status: "UP",
+ HealthCheck: &scpb.HealthCheck{
+ Mode: "push",
+ Interval: 30,
+ Times: 3,
+ },
+ Timestamp: "1552653537",
+ ModTimestamp: "1552653537",
+ Version: "1.1.0",
+ },
+ }, nil
+}
+
+func (c *mockPlugin) Heartbeat(ctx context.Context, domainProject, serviceId, instanceId string) error {
+ if heartbeat != nil {
+ return heartbeat(ctx, domainProject, serviceId, instanceId)
+ }
+ return nil
+}
diff --git a/syncer/test/dcmock/service.go b/syncer/test/dcmock/service.go
new file mode 100644
index 0000000..a4ac462
--- /dev/null
+++ b/syncer/test/dcmock/service.go
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dcmock
+
+import (
+ "context"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+)
+
+var (
+ createServiceHandler func(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error)
+ deleteService func(ctx context.Context, domainProject, serviceId string) error
+ serviceExistence func(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error)
+)
+
+func SetCreateService(handler func(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error)) {
+ createServiceHandler = handler
+}
+
+func SetDeleteService(handler func(ctx context.Context, domainProject, serviceId string) error) {
+ deleteService = handler
+}
+
+func SetServiceExistence(handler func(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error)) {
+ serviceExistence = handler
+}
+
+func (c *mockPlugin) CreateService(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error) {
+ if createServiceHandler != nil {
+ return createServiceHandler(ctx, domainProject, service)
+ }
+ return "5db1b794aa6f8a875d6e68110260b5491ee7e223", nil
+}
+
+func (c *mockPlugin) DeleteService(ctx context.Context, domainProject, serviceId string) error {
+ if createServiceHandler != nil {
+ return deleteService(ctx, domainProject, serviceId)
+ }
+ return nil
+}
+
+func (c *mockPlugin) ServiceExistence(ctx context.Context, domainProject string, service *scpb.MicroService) (string, error) {
+ if serviceExistence != nil {
+ return serviceExistence(ctx, domainProject, service)
+ }
+ return "5db1b794aa6f8a875d6e68110260b5491ee7e223", nil
+}
diff --git a/syncer/test/dcmock/servicecenter.go b/syncer/test/dcmock/servicecenter.go
new file mode 100644
index 0000000..821aeaa
--- /dev/null
+++ b/syncer/test/dcmock/servicecenter.go
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dcmock
+
+import (
+ "context"
+
+ scpb "github.com/apache/servicecomb-service-center/server/core/proto"
+ "github.com/apache/servicecomb-service-center/syncer/plugins"
+ pb "github.com/apache/servicecomb-service-center/syncer/proto"
+)
+
+const PluginName = "testmock"
+
+var (
+ scCacheHandler func(ctx context.Context) (*pb.SyncData, error)
+)
+
+func SetGetAll(handler func(ctx context.Context) (*pb.SyncData, error)) {
+ scCacheHandler = handler
+}
+
+func init() {
+ plugins.RegisterPlugin(&plugins.Plugin{
+ Kind: plugins.PluginServicecenter,
+ Name: PluginName,
+ New: New,
+ })
+}
+
+type adaptor struct{}
+
+func New() plugins.PluginInstance {
+ return &adaptor{}
+}
+
+func (*adaptor) New(endpoints []string) (plugins.Servicecenter, error) {
+ return &mockPlugin{}, nil
+}
+
+type mockPlugin struct {
+}
+
+func NewGetAll(ctx context.Context) (*pb.SyncData, error) {
+ return &pb.SyncData{
+ Services: []*pb.SyncService{
+ {
+ DomainProject: "default/default",
+ Service: &scpb.MicroService{
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ AppId: "default",
+ ServiceName: "SERVICECENTER",
+ Version: "1.1.0",
+ Level: "BACK",
+ Schemas: []string{
+ "servicecenter.grpc.api.ServiceCtrl",
+ "servicecenter.grpc.api.ServiceInstanceCtrl",
+ },
+ Status: "UP",
+ Properties: map[string]string{
+ "allowCrossApp": "true",
+ },
+ Timestamp: "1552626180",
+ ModTimestamp: "1552626180",
+ Environment: "production",
+ },
+ Instances: []*scpb.MicroServiceInstance{
+ {
+ InstanceId: "4d41a637471f11e9888cfa163eca30ab",
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ Endpoints: []string{
+ "rest://127.0.0.1:30100/",
+ },
+ HostName: "testmock",
+ Status: "UP",
+ HealthCheck: &scpb.HealthCheck{
+ Mode: "push",
+ Interval: 30,
+ Times: 3,
+ },
+ Timestamp: "1552653537",
+ ModTimestamp: "1552653537",
+ Version: "1.1.0",
+ },{
+ InstanceId: "4d41a637471f11e9888cfa163eca30e0",
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ Endpoints: []string{
+ "rest://127.0.0.1:30100/",
+ },
+ HostName: "testmock",
+ Status: "UP",
+ HealthCheck: &scpb.HealthCheck{
+ Mode: "push",
+ Interval: 30,
+ Times: 3,
+ },
+ Timestamp: "1552653537",
+ ModTimestamp: "1552653537",
+ Version: "1.1.0",
+ },
+ },
+ },{
+ DomainProject: "default/default",
+ Service: &scpb.MicroService{
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e211",
+ AppId: "default",
+ ServiceName: "SERVICECENTER",
+ Version: "1.1.0",
+ Level: "BACK",
+ Schemas: []string{
+ "servicecenter.grpc.api.ServiceCtrl",
+ "servicecenter.grpc.api.ServiceInstanceCtrl",
+ },
+ Status: "UP",
+ Properties: map[string]string{
+ "allowCrossApp": "true",
+ },
+ Timestamp: "1552626180",
+ ModTimestamp: "1552626180",
+ Environment: "production",
+ },
+ Instances: []*scpb.MicroServiceInstance{
+ {
+ InstanceId: "4d41a637471f11e9888cfa163eca30ab",
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e211",
+ Endpoints: []string{
+ "rest://127.0.0.1:30100/",
+ },
+ HostName: "testmock",
+ Status: "UP",
+ HealthCheck: &scpb.HealthCheck{
+ Mode: "push",
+ Interval: 30,
+ Times: 3,
+ },
+ Timestamp: "1552653537",
+ ModTimestamp: "1552653537",
+ Version: "1.1.0",
+ },
+ },
+ },
+ },
+ }, nil
+}
+
+func (c *mockPlugin) GetAll(ctx context.Context) (*pb.SyncData, error) {
+ if scCacheHandler != nil {
+ return scCacheHandler(ctx)
+ }
+ return &pb.SyncData{
+ Services: []*pb.SyncService{
+ {
+ DomainProject: "default/default",
+ Service: &scpb.MicroService{
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ AppId: "default",
+ ServiceName: "SERVICECENTER",
+ Version: "1.1.0",
+ Level: "BACK",
+ Schemas: []string{
+ "servicecenter.grpc.api.ServiceCtrl",
+ "servicecenter.grpc.api.ServiceInstanceCtrl",
+ },
+ Status: "UP",
+ Properties: map[string]string{
+ "allowCrossApp": "true",
+ },
+ Timestamp: "1552626180",
+ ModTimestamp: "1552626180",
+ Environment: "production",
+ },
+ Instances: []*scpb.MicroServiceInstance{
+ {
+ InstanceId: "4d41a637471f11e9888cfa163eca30e0",
+ ServiceId: "5db1b794aa6f8a875d6e68110260b5491ee7e223",
+ Endpoints: []string{
+ "rest://127.0.0.1:30100/",
+ },
+ HostName: "testmock",
+ Status: "UP",
+ HealthCheck: &scpb.HealthCheck{
+ Mode: "push",
+ Interval: 30,
+ Times: 3,
+ },
+ Timestamp: "1552653537",
+ ModTimestamp: "1552653537",
+ Version: "1.1.0",
+ },
+ },
+ },
+ },
+ }, nil
+}
diff --git a/syncer/test/servicecenter/instance.go b/syncer/test/servicecenter/instance.go
new file mode 100644
index 0000000..960eed6
--- /dev/null
+++ b/syncer/test/servicecenter/instance.go
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package servicecenter
+
+import (
+ "net/http"
+)
+
+func (m *mockServer) DiscoveryInstances(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{
+ "instances": [
+ {
+ "instanceId": "7a6be9f861a811e9b3f6fa163eca30e0",
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd",
+ "endpoints": [
+ "rest://192.168.88.75:30100/"
+ ],
+ "hostName": "chenzhu",
+ "status": "UP",
+ "healthCheck": {
+ "mode": "push",
+ "interval": 30,
+ "times": 3
+ },
+ "timestamp": "1555571184",
+ "modTimestamp": "1555571184",
+ "version": "0.0.1"
+ },
+ {
+ "instanceId": "8e0fe4b961a811e981a6fa163e86b81a",
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd",
+ "endpoints": [
+ "rest://192.168.88.109:30100/"
+ ],
+ "hostName": "sunlisen",
+ "status": "UP",
+ "healthCheck": {
+ "mode": "push",
+ "interval": 30,
+ "times": 3
+ },
+ "timestamp": "1555571221",
+ "modTimestamp": "1555571221",
+ "version": "0.0.1"
+ }
+ ]
+}`))
+}
+
+func (m *mockServer) RegisterInstance(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{"instanceId": "8e0fe4b961a811e981a6fa163e86b81a"}`))
+}
+
+func (m *mockServer) UnregisterInstance(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{"instanceId": "8e0fe4b961a811e981a6fa163e86b81a"}`))
+}
+
+func (m *mockServer) Heartbeat(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{"instanceId": "8e0fe4b961a811e981a6fa163e86b81a"}`))
+}
diff --git a/syncer/test/servicecenter/service.go b/syncer/test/servicecenter/service.go
new file mode 100644
index 0000000..623035c
--- /dev/null
+++ b/syncer/test/servicecenter/service.go
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package servicecenter
+
+import (
+ "net/http"
+)
+
+func (m *mockServer) ServiceExistence(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd"
+}`))
+}
+
+func (m *mockServer) CreateService(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd"
+}`))
+}
+
+func (m *mockServer) DeleteService(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cc"
+}`))
+}
diff --git a/syncer/test/servicecenter/servicecenter.go b/syncer/test/servicecenter/servicecenter.go
new file mode 100644
index 0000000..cdadf63
--- /dev/null
+++ b/syncer/test/servicecenter/servicecenter.go
@@ -0,0 +1,165 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package servicecenter
+
+import (
+ "errors"
+ "net/http"
+ "net/http/httptest"
+ "strconv"
+
+ "github.com/apache/servicecomb-service-center/server/interceptor"
+
+ "github.com/apache/servicecomb-service-center/pkg/rest"
+)
+
+const (
+ WantStatus = "X-Want-Status"
+ WantContent = "X-Want-Content"
+)
+
+func NewMockServer() *httptest.Server {
+ ms := &mockServer{}
+ interceptor.RegisterInterceptFunc(ms.WantHandler)
+ rest.RegisterServant(ms)
+ return httptest.NewServer(rest.GetRouter())
+}
+
+type mockServer struct{}
+
+func (m *mockServer) URLPatterns() []rest.Route {
+ return []rest.Route{
+ {rest.HTTP_METHOD_GET, "/v4/:project/admin/dump", m.GetAll},
+ {rest.HTTP_METHOD_GET, "/v4/:project/registry/existence", m.ServiceExistence},
+ {rest.HTTP_METHOD_POST, "/v4/:project/registry/microservices", m.CreateService},
+ {rest.HTTP_METHOD_DELETE, "/v4/:project/registry/microservices/:serviceId", m.DeleteService},
+ {rest.HTTP_METHOD_GET, "/v4/:project/registry/instances", m.DiscoveryInstances},
+ {rest.HTTP_METHOD_POST, "/v4/:project/registry/microservices/:serviceId/instances", m.RegisterInstance},
+ {rest.HTTP_METHOD_DELETE, "/v4/:project/registry/microservices/:serviceId/instances/:instanceId", m.UnregisterInstance},
+ {rest.HTTP_METHOD_PUT, "/v4/:project/registry/microservices/:serviceId/instances/:instanceId/heartbeat", m.Heartbeat},
+ }
+}
+
+func (m *mockServer) WantHandler(rw http.ResponseWriter, req *http.Request) error {
+ statusCode, err := strconv.Atoi(req.Header.Get(WantStatus))
+ if err != nil || statusCode == 0 {
+ statusCode = http.StatusOK
+ }
+ rw.WriteHeader(statusCode)
+ data := req.Header.Get(WantContent)
+ if data == "" {
+ return nil
+ }
+ rw.Write([]byte(data))
+ return errors.New(data)
+}
+
+func (m *mockServer) GetAll(rw http.ResponseWriter, req *http.Request) {
+ rw.Write([]byte(`{
+ "cache": {
+ "services": [
+ {
+ "key": "/cse-sr/ms/files/default/default/4042a6a3e5a2893698ae363ea99a69eb63fc51cd",
+ "rev": 5,
+ "cluster": "sr-0",
+ "value": {
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd",
+ "appId": "default",
+ "serviceName": "SERVICECENTER",
+ "version": "0.0.1",
+ "level": "BACK",
+ "schemas": [
+ "servicecenter.grpc.api.ServiceCtrl",
+ "servicecenter.grpc.api.ServiceInstanceCtrl"
+ ],
+ "status": "UP",
+ "properties": {
+ "allowCrossApp": "true"
+ },
+ "timestamp": "1555571184",
+ "alias": "SERVICECENTER",
+ "modTimestamp": "1555571184",
+ "environment": "production"
+ }
+ }
+ ],
+ "serviceIndexes": [
+ {
+ "key": "/cse-sr/ms/indexes/default/default/production/default/SERVICECENTER/0.0.1",
+ "rev": 5,
+ "cluster": "sr-0",
+ "value": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd"
+ }
+ ],
+ "serviceAliases": [
+ {
+ "key": "/cse-sr/ms/alias/default/default/production/default/SERVICECENTER/0.0.1",
+ "rev": 5,
+ "cluster": "sr-0",
+ "value": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd"
+ }
+ ],
+ "instances": [
+ {
+ "key": "/cse-sr/inst/files/default/default/4042a6a3e5a2893698ae363ea99a69eb63fc51cd/7a6be9f861a811e9b3f6fa163eca30e0",
+ "rev": 8,
+ "cluster": "sr-0",
+ "value": {
+ "instanceId": "7a6be9f861a811e9b3f6fa163eca30e0",
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd",
+ "endpoints": [
+ "rest://192.168.88.75:30100/"
+ ],
+ "hostName": "chenzhu",
+ "status": "UP",
+ "healthCheck": {
+ "mode": "push",
+ "interval": 30,
+ "times": 3
+ },
+ "timestamp": "1555571184",
+ "modTimestamp": "1555571184",
+ "version": "0.0.1"
+ }
+ },
+ {
+ "key": "/cse-sr/inst/files/default/default/4042a6a3e5a2893698ae363ea99a69eb63fc51cd/8e0fe4b961a811e981a6fa163e86b81a",
+ "rev": 9,
+ "cluster": "sr-0",
+ "value": {
+ "instanceId": "8e0fe4b961a811e981a6fa163e86b81a",
+ "serviceId": "4042a6a3e5a2893698ae363ea99a69eb63fc51cd",
+ "endpoints": [
+ "rest://192.168.88.109:30100/"
+ ],
+ "hostName": "sunlisen",
+ "status": "UP",
+ "healthCheck": {
+ "mode": "push",
+ "interval": 30,
+ "times": 3
+ },
+ "timestamp": "1555571221",
+ "modTimestamp": "1555571221",
+ "version": "0.0.1"
+ }
+ }
+ ]
+ }
+}`))
+}
