Merge branch 'S4-88' into piper
diff --git a/.gitignore b/.gitignore
index 941182e..a13ec8f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,3 +25,6 @@
# Java
*.class
+
+# apache RAT
+.rat-excludes
\ No newline at end of file
diff --git a/DISCLAIMER b/DISCLAIMER
new file mode 100644
index 0000000..789090b
--- /dev/null
+++ b/DISCLAIMER
@@ -0,0 +1,15 @@
+Apache S4 is an effort undergoing incubation at the Apache Software
+Foundation (ASF), sponsored by the Apache Incubator PMC.
+
+Incubation is required of all newly accepted projects until a further review
+indicates that the infrastructure, communications, and decision making process
+have stabilized in a manner consistent with other successful ASF projects.
+
+While incubation status is not necessarily a reflection of the completeness
+or stability of the code, it does indicate that the project has yet to be
+fully endorsed by the ASF.
+
+For more information about the incubation status of the S4 project you
+can go to the following page:
+
+http://incubator.apache.org/s4
\ No newline at end of file
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..810a231
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,483 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+----------------------------------------------------------------------------
+
+This product uses Kryo, which uses the following license:
+
+Copyright (c) 2008, Nathan Sweet All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+list of conditions and the following disclaimer. * Redistributions in binary
+form must reproduce the above copyright notice, this list of conditions and the
+following disclaimer in the documentation and/or other materials provided with
+the distribution. * Neither the name of Esoteric Software nor the names of its
+contributors may be used to endorse or promote products derived from this
+software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+----------------------------------------------------------------------------
+
+This product uses Logback, which uses the following license (EPL1.0):
+
+Logback: the reliable, generic, fast and flexible logging framework. Copyright
+(C) 1999-2012, QOS.ch. All rights reserved.
+
+This program and the accompanying materials are dual-licensed under either the
+terms of the Eclipse Public License v1.0 as published by the Eclipse Foundation
+
+or (per the licensee's choosing)
+
+under the terms of the GNU Lesser General Public License version 2.1 as
+published by the Free Software Foundation.
+
+------------- Here is the corresponding EPL 1.0 license:
+
+Eclipse Public License - v 1.0 THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE
+TERMS OF THIS ECLIPSE PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR
+DISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS
+AGREEMENT.
+
+1. DEFINITIONS
+
+"Contribution" means:
+
+a) in the case of the initial Contributor, the initial code and documentation
+distributed under this Agreement, and b) in the case of each subsequent
+Contributor:
+
+i)changes to the Program, and
+
+ii)additions to the Program;
+
+where such changes and/or additions to the Program originate from and are
+distributed by that particular Contributor. A Contribution 'originates' from a
+Contributor if it was added to the Program by such Contributor itself or anyone
+acting on such Contributor's behalf. Contributions do not include additions to
+the Program which: (i) are separate modules of software distributed in
+conjunction with the Program under their own license agreement, and (ii) are not
+derivative works of the Program.
+
+"Contributor" means any person or entity that distributes the Program.
+
+"Licensed Patents " mean patent claims licensable by a Contributor which are
+necessarily infringed by the use or sale of its Contribution alone or when
+combined with the Program.
+
+"Program" means the Contributions distributed in accordance with this Agreement.
+
+"Recipient" means anyone who receives the Program under this Agreement,
+including all Contributors.
+
+2. GRANT OF RIGHTS
+
+a) Subject to the terms of this Agreement, each Contributor hereby grants
+Recipient a non-exclusive, worldwide, royalty-free copyright license to
+reproduce, prepare derivative works of, publicly display, publicly perform,
+distribute and sublicense the Contribution of such Contributor, if any, and such
+derivative works, in source code and object code form.
+
+b) Subject to the terms of this Agreement, each Contributor hereby grants
+Recipient a non-exclusive, worldwide, royalty-free patent license under Licensed
+Patents to make, use, sell, offer to sell, import and otherwise transfer the
+Contribution of such Contributor, if any, in source code and object code form.
+This patent license shall apply to the combination of the Contribution and the
+Program if, at the time the Contribution is added by the Contributor, such
+addition of the Contribution causes such combination to be covered by the
+Licensed Patents. The patent license shall not apply to any other combinations
+which include the Contribution. No hardware per se is licensed hereunder.
+
+c) Recipient understands that although each Contributor grants the licenses to
+its Contributions set forth herein, no assurances are provided by any
+Contributor that the Program does not infringe the patent or other intellectual
+property rights of any other entity. Each Contributor disclaims any liability to
+Recipient for claims brought by any other entity based on infringement of
+intellectual property rights or otherwise. As a condition to exercising the
+rights and licenses granted hereunder, each Recipient hereby assumes sole
+responsibility to secure any other intellectual property rights needed, if any.
+For example, if a third party patent license is required to allow Recipient to
+distribute the Program, it is Recipient's responsibility to acquire that license
+before distributing the Program.
+
+d) Each Contributor represents that to its knowledge it has sufficient copyright
+rights in its Contribution, if any, to grant the copyright license set forth in
+this Agreement.
+
+3. REQUIREMENTS
+
+A Contributor may choose to distribute the Program in object code form under its
+own license agreement, provided that:
+
+a) it complies with the terms and conditions of this Agreement; and
+
+b) its license agreement:
+
+i) effectively disclaims on behalf of all Contributors all warranties and
+conditions, express and implied, including warranties or conditions of title and
+non-infringement, and implied warranties or conditions of merchantability and
+fitness for a particular purpose;
+
+ii) effectively excludes on behalf of all Contributors all liability for
+damages, including direct, indirect, special, incidental and consequential
+damages, such as lost profits;
+
+iii) states that any provisions which differ from this Agreement are offered by
+that Contributor alone and not by any other party; and
+
+iv) states that source code for the Program is available from such Contributor,
+and informs licensees how to obtain it in a reasonable manner on or through a
+medium customarily used for software exchange.
+
+When the Program is made available in source code form:
+
+a) it must be made available under this Agreement; and
+
+b) a copy of this Agreement must be included with each copy of the Program.
+
+Contributors may not remove or alter any copyright notices contained within the
+Program.
+
+Each Contributor must identify itself as the originator of its Contribution, if
+any, in a manner that reasonably allows subsequent Recipients to identify the
+originator of the Contribution.
+
+4. COMMERCIAL DISTRIBUTION
+
+Commercial distributors of software may accept certain responsibilities with
+respect to end users, business partners and the like. While this license is
+intended to facilitate the commercial use of the Program, the Contributor who
+includes the Program in a commercial product offering should do so in a manner
+which does not create potential liability for other Contributors. Therefore, if
+a Contributor includes the Program in a commercial product offering, such
+Contributor ("Commercial Contributor") hereby agrees to defend and indemnify
+every other Contributor ("Indemnified Contributor") against any losses, damages
+and costs (collectively "Losses") arising from claims, lawsuits and other legal
+actions brought by a third party against the Indemnified Contributor to the
+extent caused by the acts or omissions of such Commercial Contributor in
+connection with its distribution of the Program in a commercial product
+offering. The obligations in this section do not apply to any claims or Losses
+relating to any actual or alleged intellectual property infringement. In order
+to qualify, an Indemnified Contributor must: a) promptly notify the Commercial
+Contributor in writing of such claim, and b) allow the Commercial Contributor to
+control, and cooperate with the Commercial Contributor in, the defense and any
+related settlement negotiations. The Indemnified Contributor may participate in
+any such claim at its own expense.
+
+For example, a Contributor might include the Program in a commercial product
+offering, Product X. That Contributor is then a Commercial Contributor. If that
+Commercial Contributor then makes performance claims, or offers warranties
+related to Product X, those performance claims and warranties are such
+Commercial Contributor's responsibility alone. Under this section, the
+Commercial Contributor would have to defend claims against the other
+Contributors related to those performance claims and warranties, and if a court
+requires any other Contributor to pay any damages as a result, the Commercial
+Contributor must pay those damages.
+
+5. NO WARRANTY
+
+EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR
+IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE,
+NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each
+Recipient is solely responsible for determining the appropriateness of using and
+distributing the Program and assumes all risks associated with its exercise of
+rights under this Agreement , including but not limited to the risks and costs
+of program errors, compliance with applicable laws, damage to or loss of data,
+programs or equipment, and unavailability or interruption of operations.
+
+6. DISCLAIMER OF LIABILITY
+
+EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY
+CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST
+PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS
+GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+7. GENERAL
+
+If any provision of this Agreement is invalid or unenforceable under applicable
+law, it shall not affect the validity or enforceability of the remainder of the
+terms of this Agreement, and without further action by the parties hereto, such
+provision shall be reformed to the minimum extent necessary to make such
+provision valid and enforceable.
+
+If Recipient institutes patent litigation against any entity (including a
+cross-claim or counterclaim in a lawsuit) alleging that the Program itself
+(excluding combinations of the Program with other software or hardware)
+infringes such Recipient's patent(s), then such Recipient's rights granted under
+Section 2(b) shall terminate as of the date such litigation is filed.
+
+All Recipient's rights under this Agreement shall terminate if it fails to
+comply with any of the material terms or conditions of this Agreement and does
+not cure such failure in a reasonable period of time after becoming aware of
+such noncompliance. If all Recipient's rights under this Agreement terminate,
+Recipient agrees to cease use and distribution of the Program as soon as
+reasonably practicable. However, Recipient's obligations under this Agreement
+and any licenses granted by Recipient relating to the Program shall continue and
+survive.
+
+Everyone is permitted to copy and distribute copies of this Agreement, but in
+order to avoid inconsistency the Agreement is copyrighted and may only be
+modified in the following manner. The Agreement Steward reserves the right to
+publish new versions (including revisions) of this Agreement from time to time.
+No one other than the Agreement Steward has the right to modify this Agreement.
+The Eclipse Foundation is the initial Agreement Steward. The Eclipse Foundation
+may assign the responsibility to serve as the Agreement Steward to a suitable
+separate entity. Each new version of the Agreement will be given a
+distinguishing version number. The Program (including Contributions) may always
+be distributed subject to the version of the Agreement under which it was
+received. In addition, after a new version of the Agreement is published,
+Contributor may elect to distribute the Program (including its Contributions)
+under the new version. Except as expressly stated in Sections 2(a) and 2(b)
+above, Recipient receives no rights or licenses to the intellectual property of
+any Contributor under this Agreement, whether expressly, by implication,
+estoppel or otherwise. All rights in the Program not expressly granted under
+this Agreement are reserved.
+
+This Agreement is governed by the laws of the State of New York and the
+intellectual property laws of the United States of America. No party to this
+Agreement will bring a legal action under this Agreement more than one year
+after the cause of action arose. Each party waives its rights to a jury trial in
+any resulting litigation.
+
+
+
+----------------------------------------------------------------------------
+
+This product uses slf4j, which uses the following license (MIT):
+
+Copyright (c) 2004-2011 QOS.ch
+ All rights reserved.
+
+ Permission is hereby granted, free of charge, to any person obtaining
+ a copy of this software and associated documentation files (the
+ "Software"), to deal in the Software without restriction, including
+ without limitation the rights to use, copy, modify, merge, publish,
+ distribute, sublicense, and/or sell copies of the Software, and to
+ permit persons to whom the Software is furnished to do so, subject to
+ the following conditions:
+
+ The above copyright notice and this permission notice shall be
+ included in all copies or substantial portions of the Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
\ No newline at end of file
diff --git a/NOTES.md b/NOTES.md
deleted file mode 100644
index c096e43..0000000
--- a/NOTES.md
+++ /dev/null
@@ -1,101 +0,0 @@
-# Manually packaging an S4 app.
-
-The app should extend io.s4.core.App:
-
-<pre>
-import io.s4.core.App;
-
-
-public class HelloApp extends App {
-
- @Override
- protected void start() {
- System.out.println("Hello App! I'm starting...");
- }
-
- @Override
- protected void init() {
- System.out.println("Hello App! I'm initing...");
-
- }
-
- @Override
- protected void close() {
- System.out.println("Hello App! I'm closing...");
-
- }
-}
-</pre>
-
-We want to indicate the entry point in the jar. To do this I create a manifest file.
-
-* Edit MANIFEST.MF:
-
-<pre>
-Manifest-Version: 1.0
-provider: gradle
-Implementation-Url: http://incubator.apache.org/projects/s4.html
-Implementation-Version: 0.5.0-SNAPSHOT
-Implementation-Vendor: Apache S4
-Implementation-Vendor-Id: io.s4
-S4-App-Class: HelloApp
-</pre>
-
-We will use the attribute prefix "S4-App-" for all S3-related properties.
-
-* Files:
- HelloApp.class
- HelloApp.java
- MANIFEST.MF
-
-* Create the jar file:
- jar cmf MANIFEST.MF HelloApp.jar HelloApp.class
-
-* Check:
- jar tvf HelloApp.jar
- 0 Mon Oct 17 16:31:18 PDT 2011 META-INF/
- 275 Mon Oct 17 16:31:18 PDT 2011 META-INF/MANIFEST.MF
- 702 Mon Oct 17 13:36:50 PDT 2011 HelloApp.class
-
-## Create Counter Example JAR
-
-Copy counter example classfiles:
-
-<pre>
-AgeKeyFinder.class CountKeyFinder.class GenderKeyFinder.class Module.class PrintPE.class UserEvent.class
-CountEvent.class CounterPE.class GenerateUserEventPE.class MyApp.class README.md UserIDKeyFinder.class
-
-$ cat MANIFEST.MF
-Manifest-Version: 1.0
-provider: gradle
-Implementation-Url: http://incubator.apache.org/projects/s4.html
-Implementation-Version: 0.5.0-SNAPSHOT
-Implementation-Vendor: Apache S4
-Implementation-Vendor-Id: io.s4
-S4-App-Class: io.s4.example.counter.MyApp
-
-$ jar cmf MANIFEST.MF CounterExample.s4r org
-$ jar tfv CounterExample.s4r
- 0 Sat Oct 22 19:21:06 PDT 2011 META-INF/
- 307 Sat Oct 22 19:21:06 PDT 2011 META-INF/MANIFEST.MF
- 0 Wed Oct 19 08:55:18 PDT 2011 org/
- 0 Wed Oct 19 08:55:18 PDT 2011 org/apache/
- 0 Wed Oct 19 08:55:36 PDT 2011 org/apache/s4/
- 0 Wed Oct 19 08:55:44 PDT 2011 org/apache/s4/example/
- 0 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/
- 1240 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/AgeKeyFinder.class
- 1708 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/CounterPE.class
- 1330 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/CountEvent.class
- 1197 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/CountKeyFinder.class
- 1254 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/GenderKeyFinder.class
- 2378 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/GenerateUserEventPE.class
- 3414 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/Module.class
- 4196 Sat Oct 22 19:19:14 PDT 2011 org/apache/s4/example/counter/MyApp.class
- 897 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/PrintPE.class
- 532 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/README.md
- 1233 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/UserEvent.class
- 1198 Wed Oct 19 08:55:18 PDT 2011 org/apache/s4/example/counter/UserIDKeyFinder.class
-
-</pre>
-
-
diff --git a/NOTICE.txt b/NOTICE.txt
index b0a385f..63abaeb 100644
--- a/NOTICE.txt
+++ b/NOTICE.txt
@@ -1,13 +1,17 @@
-=======================================================================
-NOTICE file for use with, and corresponding to Section 4 of,
-the Apache License, Version 2.0, in this case for the S4 project.
-=========================================================================
+Apache S4
+Copyright 2012 The Apache Software Foundation
- This product includes software developed by
- Yahoo! Inc. (www.yahoo.com)
- Copyright (c) 2010, 2011 Yahoo! Inc. All rights reserved.
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
- This product includes software developed by
- The S4 Project community (http://s4.io)
- Copyright (c) 2011 The S4 Project All rights reserved.
-
\ No newline at end of file
+This product includes kryo, developed by Nathan Sweet
+(http://www.kryo.googlecode.com)
+
+This product include software developed in the netty project
+in binary form (http://netty.io)
+
+This product includes logback, developed by QOS.ch, in binary form
+(http://logback.qos.ch/)
+
+This product includes slf4j, developed by QOS.ch, in binary form
+(http://www.slf4j.org)
\ No newline at end of file
diff --git a/README.md b/README.md
index 931ddbc..2bd8ad0 100644
--- a/README.md
+++ b/README.md
@@ -1,52 +1,99 @@
-S4-Piper Design
-===============
+Apache S4
+=========
-The goal of this project is to incorporate feedback from various users and create a prototype with a cleaner, simpler API.
-The base classes are the foundation on top of which higher level layers and tools can be built. For example, one could write
-a domain specific language to create the application graph. Look at this as the foundation and not as the end product.
+Overview
+--------
+S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable
+platform that allows programmers to easily develop applications for processing continuous
+unbounded streams of data.
-Here is a list of ideas:
+S4 0.5.0 is a complete refactoring of the previous version of S4. It grounds on the same
+concepts (partitioning inspired by map-reduce, actors-like distribution model),
+but with the following objectives:
-- Eliminate string identifiers completely and make it work nicely with Guice.
-- Take advantage of Java strong typing using generics in the framework. (But keep it simple for the app developer.)
-- Use dependency injection, no more "new".
-- Limit property files to simple parameters. Config file should not be
-a programming language. (No more XML.)
-- Configure the app in a GuiceModule.
-- Make fields final private whenever possible and use constructors
-instead of setter methods.
-- Make PEs as immutable as possible.
-- Hide details about partitioning from app developer. The idea is to
-create a graph for the prototypes and have the base classes deal with
-the distributed processing details.
-- Use osgi to create s4 app bundles that can be dropped in a directory
-and get loaded by s4. (This will take a bit longer to do.)
+- cleaner and simpler API
+- robust configuration through statically defined modules
+- cleaner architecture
+- robust codebase
+- easier to develop S4 apps, to test, and to use the platform
-Please post feedback and suggestions in the s4 forum.
+We added the following core features:
-The repo is here:
+- TCP-based communications
+- state recovery through a flexible checkpointing mechanism
+- inter-cluster/app communications through a pub-sub model
+- dynamic application deployment
+- toolset for easily starting S4 nodes, testing, packaging, deploying and monitoring S4 apps
-https://github.com/leoneu/s4-piper
-In this version:
+Documentation
+-------------
-- Everything is a POJO.
-- Key framework classes (io.s4) are:
- - Event: all messages sent between PEs inherit from Event.
- - ProcessingElement: base class for PEs.
- - Stream: takes an Event subclass as the parameter type for the stream. A stream holds references to the target PEs. Streams are the edges between PEs.
- - Key: is a helper class that can return the value of a key for a specific type of event. For each type of event, one will create a KeyFinder class that knows how to get the value from the Event class.
- - App: the container for the app.
+For the latest information about S4, please visit our website at:
-Note that when we create a PE in App, we are really creating a prototype of the PE. The actual PE instances get created every time a node receives an event that is dispatched to a specific PE prototype. The first PE has Map that holds the instances by a specific key. PE instances are clones which means that we don't call the constructor. Instead we provide an init method. Also, the instance variables are cloned. This means that references are also copied. If you wanted to have a List per PE instance, you would have to create it in the init method. This design pattern, makes thing very simple and efficient but has the downside that programmers have to understand how instances are created via clone.
+ http://inbubator.apache.org/s4
-For now we identify PEs using a composite string produced by KeyFinder. We must also pass the raw data types used to create the composite key. Need to think how to do this with type safety in mind.
-
-
-Examples:
+and our wiki, at:
-* io.s4.example.counter - Generates dummy events that are counted using various keys.
-* io.s4.example.model - Training and testing a probabilistic classifier.
-
+ https://cwiki.apache.org/confluence/display/S4/S4+Wiki
+
+Currently the wiki contains the most relevant and up-to-date documentation.
+
+Source code is available here: https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git
+
+
+Requirements
+------------
+* JDK 6 or higher
+* *nix or macosx (you may build the project and develop S4 applications with
+microsoft windows though, the only limitation is that the "s4" script has not
+been ported to windows yet)
+
+
+How to build
+------------
+This only applies if you checkout from the source repository or if you download a
+released source package.
+
+
+We use gradle http://gradle.org as the build system.
+
+* From the root directory of the S4 project:
+
+./gradlew install
+
+This will build the packages and install the artefacts in the local maven repository.
+
+* Then, build the tools:
+
+./gradlew s4-tools:installApp
+
+This will build the tools so that you can drive the platform through the "s4" command.
+
+
+Directory structure
+-------------------
+* If you have a source package:
+ - root directory contains build and utility scripts, as well as informative files
+ - config/ directory contains configuration files for source code formatting
+ - doc/ directory contains the javadoc
+ - gradle/ directory contains libraries used by the gradle build tool
+ - lib/ directory contains some other gradle libraries
+ - subproject/ directory contains the plaftorm subprojects: base, comm, core, tools,
+ as well as example (example is not fully ported to 0.5.0)
+ - test-apps/ directory contains some examples (some of them very trivial are used
+ in regression tests)
+
+
+
+* If you have a binary package:
+ - root directory contains the s4 utility script as well as informative files
+ - doc/ directory contains the javadoc
+ - lib/ directory contains :
+ * the platform libraries (prefixed with "s4")
+ * the dependencies
+ - bin/ directory contains some scripts that are used by the s4 script
+ - gradle/ directory contains libraries used for building S4 projects
+
diff --git a/RELEASE_NOTES.html b/RELEASE_NOTES.html
new file mode 100644
index 0000000..4c1a67b
--- /dev/null
+++ b/RELEASE_NOTES.html
@@ -0,0 +1,118 @@
+
+<p>
+ Release Notes - Apache S4 - Version 0.5.0
+</p>
+<p>
+S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform
+that allows programmers to easily develop applications for processing continuous unbounded streams of data.
+</p>
+<p>
+Apache S4 0.5.0 is the first release of S4 as an Apache Incubator project.
+It is also a complete refactoring of the previous S4 codebase.
+</p>
+
+<h2> Sub-task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-24'>S4-24</a>] - Dynamic deployment of S4 applications from a remote repository
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-29'>S4-29</a>] - Remove file-based configurations
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-33'>S4-33</a>] - TestUtils and modules should be split across s4-comm and s4-core
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-36'>S4-36</a>] - CommTests should use ZkBasedClusterManagement
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-65'>S4-65</a>] - Create s4 status command
+</li>
+</ul>
+
+<h2> Bug
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-12'>S4-12</a>] - Dynamic Method Dispatching
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-14'>S4-14</a>] - Listener.recv() behavior when interrupted
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-18'>S4-18</a>] - gradle test fails
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-23'>S4-23</a>] - NettyEmitter -- 1. Infinitely waiting on connection, 2. numPartitions is same as numClusterNodes
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-26'>S4-26</a>] - standalone application build script fails when the source directory contains more than 1 class
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-43'>S4-43</a>] - Replace deprecated method in Gradle
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-56'>S4-56</a>] - Prevent concurrent access to published s4r files
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-61'>S4-61</a>] - Threads running the PEs don't have a Context Class Loader
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-63'>S4-63</a>] - onTime() events are not synchronized correctly
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-70'>S4-70</a>] - s4 node command ignores -zk parameter
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-71'>S4-71</a>] - s4 deploy mismanages s4r URI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-74'>S4-74</a>] - NullPointerException on TCPEmitter.onChange()
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-82'>S4-82</a>] - Fix local communication optimization
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-87'>S4-87</a>] - Checkpointing: recovery : avoid rejections upon fetching
+</li>
+</ul>
+
+<h2> Improvement
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-6'>S4-6</a>] - Netty initialization
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-17'>S4-17</a>] - Put tmp dir under build
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-21'>S4-21</a>] - Create standard PEs, and Event classes
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-22'>S4-22</a>] - Adaptor + inter app communication
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-27'>S4-27</a>] - extensions to cluster configuration through Zookeeper
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-34'>S4-34</a>] - Rename s4.comm.netty and constituents to s4.comm.tcp
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-44'>S4-44</a>] - optional backoff upon multiple consecutive failed checkpoint fetches
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-57'>S4-57</a>] - Document and improve the windowing PE
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-60'>S4-60</a>] - Development mode for rapid application development and testing
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-67'>S4-67</a>] - S4R packaging: improve packaging of dependencies
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-69'>S4-69</a>] - Update gradle to 1.0
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-72'>S4-72</a>] - Resource loading from S4R
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-75'>S4-75</a>] - Code improvements in comm layer
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-78'>S4-78</a>] - Gradle tasks don't finish as soon as possible
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-81'>S4-81</a>] - S4 tools syntax: remove curly braces + expose custom modules config
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-85'>S4-85</a>] - Improve handling of ZK connection changes
+</li>
+</ul>
+
+<h2> New Feature
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-4'>S4-4</a>] - Dynamic App Loading
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-11'>S4-11</a>] - add checkpointing mechanism to s4-piper
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-38'>S4-38</a>] - S4 command line utility
+</li>
+</ul>
+
+<h2> Task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-31'>S4-31</a>] - Facilities for packaging example and test apps
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/S4-88'>S4-88</a>] - Prepare 0.5.0 release
+</li>
+</ul>
+
\ No newline at end of file
diff --git a/build.gradle b/build.gradle
index 149fa4c..674e4a7 100644
--- a/build.gradle
+++ b/build.gradle
@@ -1,11 +1,13 @@
-/*
- * Copyright 2010 the original author or authors.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
*
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
@@ -14,6 +16,7 @@
* limitations under the License.
*/
+
/**
* The Apache Incubator S4 Project Build File
*
@@ -21,13 +24,13 @@
description = 'Apache S4'
defaultTasks 'assemble'
-project.ext["archivesBaseName"] = 's4'
+project.ext["archivesBaseName"] = 'apache-s4'
project.ext["distRootFolder"] = "$archivesBaseName-${-> version}"
allprojects {
// Append the suffix 'SNAPSHOT' when the build is not for release.
- version = new Version(major: 0, minor: 5, bugfix: 0, isRelease: false)
+ version = new Version(major: 0, minor: 5, bugfix: 0, isRelease: true)
group = 'org.apache.s4'
/* Search these repos to find artifacts. Gradle will download and cache. */
@@ -44,37 +47,25 @@
/* All project libraries must be defined here. */
project.ext["libraries"] = [
- json: 'org.json:json:20090211',
- guava: 'com.google.guava:guava:10.0.1',
+ guava: 'com.google.guava:guava:12.0.1',
gson: 'com.google.code.gson:gson:1.6',
guice: 'com.google.inject:guice:3.0',
guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
- guice_grapher: 'com.google.inject:guice-grapher:3.0',
- flexjson: 'net.sf.flexjson:flexjson:2.1',
- bcel: 'org.apache.bcel:bcel:5.2',
- jakarta_regexp: 'jakarta-regexp:jakarta-regexp:1.4',
kryo: 'com.googlecode:kryo:1.04',
netty: 'org.jboss.netty:netty:3.2.5.Final',
- reflectasm: 'com.esotericsoftware:reflectasm:0.8',
- minlog: 'com.esotericsoftware:minlog:1.2',
mockito_core: 'org.mockito:mockito-core:1.9.0',
- asm: 'asm:asm:3.2',
- commons_io: 'commons-io:commons-io:2.0.1',
commons_config: 'commons-configuration:commons-configuration:1.6',
commons_codec: 'commons-codec:commons-codec:1.4',
- commons_httpclient: 'commons-httpclient:commons-httpclient:3.1',
commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
slf4j: 'org.slf4j:slf4j-api:1.6.1',
logback_core: 'ch.qos.logback:logback-core:0.9.29',
logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
- ejml: 'com.googlecode.efficient-java-matrix-library:ejml:0.17',
zk: 'org.apache.zookeeper:zookeeper:3.3.1',
jcip: 'net.jcip:jcip-annotations:1.0',
junit: 'junit:junit:4.10',
zkclient: 'com.github.sgroschupf:zkclient:0.1',
diezel: 'net.ericaro:diezel-maven-plugin:1.0.0-beta-4',
jcommander: 'com.beust:jcommander:1.25',
- commons_io: 'commons-io:commons-io:2.1',
gradle_base_services: 'gradle-base-services:gradle-base-services:1.0',
gradle_core: 'gradle-core:gradle-core:1.0',
gradle_tooling_api: 'gradle-tooling-api:gradle-tooling-api:1.0',
@@ -112,7 +103,6 @@
compile( libraries.logback_classic )
/* Commons. */
- compile( libraries.commons_io )
compile( libraries.commons_config )
compile( libraries.commons_coll )
@@ -123,13 +113,22 @@
/* Testing. */
testCompile( libraries.junit )
}
+
+ jar {
+ from (rootDir) {
+ include "NOTICE.txt"
+ include "LICENSE"
+ into "META-INF"
+ }
+ }
manifest.mainAttributes(
- provider: 'gradle',
- 'Implementation-Url': 'http://incubator.apache.org/projects/s4.html',
- 'Implementation-Version': version,
- 'Implementation-Vendor': 'Apache S4',
- 'Implementation-Vendor-Id': 'io.s4'
+ provider: 'gradle',
+ 'Implementation-Url': 'http://incubator.apache.org/s4/',
+ 'Implementation-Version': version,
+ 'Implementation-Vendor': 'Apache Software Foundation',
+ 'Implementation-Vendor-Id': 'org.apache',
+ 'Implementation-Title': 'org.apache.s4.'+project.name.substring(3) // s4-comm becomes org.apache.s4.comm for instance
)
}
@@ -145,34 +144,105 @@
platformLibs platformProjects
}
-project.ext["binDistImage"] = copySpec {
- platformProjects.collect {proj ->
- into ("platform/lib") {
- from proj.sourceSets.main.resources
- from proj.configurations.runtime.getFiles()
- from proj.configurations.archives.allArtifacts.files
+
+task javadoc (type: Javadoc) {
+ destinationDir = new File(project.rootDir, 'doc/javadoc')
+ destinationDir.deleteDir()
+
+ title = "Apache S4 " + version
+ source platformProjects.collect { project ->
+ project.sourceSets.main.allJava
+ }
+ // Might need a classpath
+ classpath = files(subprojects.collect { project ->
+ project.sourceSets.main.compileClasspath
+ })
+}
+
+// TODO parameterize
+task javadocBinRelease (type: Javadoc) {
+ destinationDir = new File(project.rootDir,'subprojects/s4-tools/src/dist/doc/javadoc')
+ destinationDir.deleteDir()
+ title = "Apache S4 " + version
+ source platformProjects.collect { project ->
+ project.sourceSets.main.allJava
+ }
+ // Might need a classpath
+ classpath = files(subprojects.collect { project ->
+ project.sourceSets.main.compileClasspath
+ })
+}
+
+task copyRequiredFilesToS4ToolsDistDir (type: Copy) {
+ copy {
+ from (rootDir) {
+ include "NOTICE.txt"
+ include "README.md"
+ include "DISCLAIMER"
+ include "RELEASE_NOTES.txt"
+ include "gradlew"
+ include "gradle.bat"
+ include "s4"
+ }
+ into new File(project.rootDir.path+'/subprojects/s4-tools/src/dist/')
+ }
+
+ File s4DistFile = new File(project.rootDir.path+'/subprojects/s4-tools/src/dist/s4')
+ ant.replace(file: s4DistFile, token: "\n# NOTE: \"./gradlew s4-tools:installApp\" will prepare/update the tools subproject and related startup scripts", value: "")
+
+ ant.replace(file: s4DistFile, token: "subprojects/s4-tools/build/install/s4-tools/", value: "")
+
+ ant.chmod(file: s4DistFile, perm: "700")
+
+ copy {
+ from new File(rootDir, 'gradle')
+ into new File(project.rootDir.path+'/subprojects/s4-tools/src/dist/gradle')
+ }
+
+ copy {
+ from (new File(rootDir, 'lib')) {
+ include '*.properties'
+ }
+ into new File(project.rootDir.path+'/subprojects/s4-tools/src/dist/lib')
+ }
+}
+
+
+
+task binDist(dependsOn: ["s4-tools:clean", "javadocBinRelease", "copyRequiredFilesToS4ToolsDistDir", "s4-tools:cleanDistZip", "s4-tools:distZip"]) {
+ doLast {
+ copy {
+ from (project.rootDir.path+"/subprojects/s4-tools/build/distributions/")
+ into (project.buildDir)
+ exclude('tmp')
}
}
}
-task binImage(type: Copy) {
- description = "Create binary image"
- destinationDir = file("$buildDir/s4-image")
- with binDistImage
+task srcDist(dependsOn: ["javadoc"], type: Zip) {
+ doFirst {
+ platformProjects.collect { project ->
+ project.tasks.clean.execute()
+ }
+ }
+ classifier='src'
+ version = project.version
+ baseName=project.ext.archivesBaseName
+ destinationDir = project.buildDir
+ from (project.rootDir)
+ exclude ('build')
+ exclude ('.project')
+ exclude ('.classpath')
+ exclude ('tmp')
+ exclude ('.rat-excludes')
}
task clean << {
delete buildDir
}
-// This task lists all the jars. We can use this pattern to create file descriptors.
-task listJars << {
- description = 'List all the jars in the S4 framework.'
- configurations.platformLibs.each { File file -> println file.name }
-}
-
/* Generates the gradlew scripts.
- http://www.gradle.org/1.0-milestone-3/docs/userguide/gradle_wrapper.html */
+ http://www.gradle.org/docs/1.0/userguide/gradle_wrapper.html */
task wrapper(type: Wrapper) {
gradleVersion = '1.0'
jarFile='lib/gradle-wrapper-1.0.jar'
@@ -185,6 +255,6 @@
boolean isRelease
String toString() {
- "$major.$minor.$bugfix${isRelease ? '' : '-SNAPSHOT'}"
+ "$major.$minor.$bugfix${isRelease ? '' : '-SNAPSHOT'}-incubating"
}
}
diff --git a/etc/s4-counter-example.png b/etc/s4-counter-example.png
deleted file mode 100644
index 7d2a897..0000000
--- a/etc/s4-counter-example.png
+++ /dev/null
Binary files differ
diff --git a/etc/s4-objects-example.png b/etc/s4-objects-example.png
deleted file mode 100644
index dd14499..0000000
--- a/etc/s4-objects-example.png
+++ /dev/null
Binary files differ
diff --git a/s4 b/s4
index 3d3e0b4..6471486 100755
--- a/s4
+++ b/s4
@@ -10,15 +10,5 @@
subprojects/s4-tools/build/install/s4-tools/bin/s4-tools -s4ScriptPath=$S4_SCRIPT_PATH $@
-# EXAMPLES
-# deploy:
-# ./s4 deploy -appName=twitter-counter -buildFile=test-apps/twitter-counter/build.gradle -cluster=s4-test-cluster
-# ./s4 deploy -appName=twitter-adapter -buildFile=test-apps/twitter-adapter/build.gradle -cluster=s4-adapter-cluster
-# newCluster:
-# # examples:
-#./s4 newCluster -c=s4-test-cluster -flp=11000 -nbTasks=2 ; ./s4 newCluster -c=s4-adapter-cluster -flp=13000 -nbTasks=1
-
-# node:
-# ./s4 node -c=s4-test-cluster
diff --git a/settings.gradle b/settings.gradle
index 588d254..c2b9b67 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -1,18 +1,21 @@
-/*
- * Copyright 2010 the original author or authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
include 's4-base'
include 's4-core'
include 's4-comm'
@@ -22,7 +25,6 @@
//include 's4-example'
//include ':test-apps:simple-adapter-1'
include ':test-apps:simple-deployable-app-1'
-include ':test-apps:simple-deployable-app-2'
include ':test-apps:producer-app'
include ':test-apps:consumer-app'
diff --git a/subprojects/s4-base/s4-base.gradle b/subprojects/s4-base/s4-base.gradle
index 11fff30..fffe198 100644
--- a/subprojects/s4-base/s4-base.gradle
+++ b/subprojects/s4-base/s4-base.gradle
@@ -1,18 +1,21 @@
-/*
- * Copyright 2010 the original author or authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
description = 'Interfaces and most basic classes required by nultiple modules.'
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/Emitter.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/Emitter.java
index 54c08df..c73dd1c 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/Emitter.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/Emitter.java
@@ -1,11 +1,35 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
+/**
+ * Defines an event emitter, responsible for sending an event to a given partition of the cluster.
+ *
+ */
public interface Emitter {
- /*
- * @param partitionId - destination partition
+ /**
+ * @param partitionId
+ * - destination partition
*
- * @param message - message payload that needs to be sent
+ * @param message
+ * - message payload that needs to be sent
*
* @return - true - if message is sent across successfully - false - if send fails
*/
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/Event.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/Event.java
index 1f5d7e4..4e47723 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/Event.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/Event.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
*
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
*
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.base;
import java.util.Map;
@@ -68,10 +71,10 @@
}
/**
- * The stream id is used to identify streams uniquely in a cluster configuration. It is not required to operate in
- * local mode.
*
- * @param targetStreamId
+ * @param streamName
+ * used to identify streams uniquely in a cluster configuration. It is not required to operate in local
+ * mode.
*/
public void setStreamId(String streamName) {
this.streamName = streamName;
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/EventMessage.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/EventMessage.java
index 75cb3d3..e777687 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/EventMessage.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/EventMessage.java
@@ -1,15 +1,35 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
/**
- *
+ * <p>
* Encapsulates application-level events of type {@link Event}.
- *
+ * </p>
+ * <p>
* Indeed, events that are defined at the application level can only be handled by the classloader of the corresponding
* application.
- *
+ * </p>
+ * <p>
* Includes routing information (application name, stream name), so that this message can be dispatched at the
* communication level.
- *
+ * </p>
*
*/
public class EventMessage {
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/GenericKeyFinder.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/GenericKeyFinder.java
index 725452b..56a5d47 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/GenericKeyFinder.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/GenericKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
import java.lang.reflect.Field;
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/Hasher.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/Hasher.java
index c780ff1..18d5d48 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/Hasher.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/Hasher.java
@@ -1,5 +1,27 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
+/**
+ * Defines a transformation between a key and a hashcode.
+ *
+ */
public interface Hasher {
long hash(String hashKey);
}
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/Key.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/Key.java
index 8ce8f3e..ef959f6 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/Key.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/Key.java
@@ -1,32 +1,34 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.base;
import java.util.List;
import org.apache.commons.lang.StringUtils;
-/*
- * The Key class is used to get the value of the key on a specific type of
- * event. This is done to abstract all the complexity required to get the
- * value. The method for getting the value is implemented in a method of
- * an object of type KeyFinder<T>.
+/**
+ * The Key class is used to get the value of the key on a specific type of event. This is done to abstract all the
+ * complexity required to get the value. The method for getting the value is implemented in a method of an object of
+ * type KeyFinder<T>.
*
- * The application programmer provides the events and the corresponding
- * finders. The framework will use it to key on events.
+ * The application programmer provides the events and the corresponding finders. The framework will use it to key on
+ * events.
*/
public class Key<T extends Event> {
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/KeyFinder.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/KeyFinder.java
index 48456e1..c1aa1f2 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/KeyFinder.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/KeyFinder.java
@@ -1,29 +1,30 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.base;
import java.util.List;
-
-/*
- * The KeyFinder returns a list with one or more String values
- * (One value for single keys, and more than one value for
- * composite keys.)
+/**
+ * The KeyFinder returns a list with one or more String values (One value for single keys, and more than one value for
+ * composite keys.)
* */
-abstract public interface KeyFinder<T extends Event> {
+public interface KeyFinder<T extends Event> {
public List<String> get(T event);
}
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/Listener.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/Listener.java
index be60597..608d628 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/Listener.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/Listener.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
/**
@@ -7,11 +25,13 @@
*/
public interface Listener {
- /*
+ /**
* Perform blocking receive on the appropriate communication channel
*
- * @return <ul><li> byte[] message returned by the channel </li> <li> null if the associated blocking thread is
- * interrupted </li> </ul>
+ * @return <ul>
+ * <li>byte[] message returned by the channel</li>
+ * <li>null if the associated blocking thread is interrupted</li>
+ * </ul>
*/
byte[] recv();
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/RemoteEmitter.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/RemoteEmitter.java
index 59d9164..dcd7728 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/RemoteEmitter.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/RemoteEmitter.java
@@ -1,5 +1,27 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
+/**
+ * Marker interface for identifying emitters to remote S4 clusters.
+ *
+ */
public interface RemoteEmitter extends Emitter {
}
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/SerializerDeserializer.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/SerializerDeserializer.java
index 63e93b9..f7ef101 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/SerializerDeserializer.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/SerializerDeserializer.java
@@ -1,5 +1,27 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
+/**
+ * Defines serialization and deserialization methods used within the S4 platform, typically for events and PEs.
+ *
+ */
public interface SerializerDeserializer {
public byte[] serialize(Object message);
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/package-info.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/package-info.java
new file mode 100644
index 0000000..5829e0c
--- /dev/null
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Defines some of the basic elements of the S4 platforms.
+ */
+package org.apache.s4.base;
\ No newline at end of file
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/JarClassLoader.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/JarClassLoader.java
deleted file mode 100644
index 8301f62..0000000
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/JarClassLoader.java
+++ /dev/null
@@ -1,109 +0,0 @@
-package org.apache.s4.base.util;
-
-import java.io.Closeable;
-import java.io.File;
-import java.io.FileOutputStream;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.net.URL;
-import java.net.URLClassLoader;
-import java.security.CodeSource;
-import java.security.ProtectionDomain;
-import java.util.Enumeration;
-import java.util.jar.JarEntry;
-import java.util.jar.JarFile;
-
-/**
- * Our goal is to be able to put apps and dependent classes in a single jar that
- * we can easily read using a URL.
- *
- * With derived work from <a href=
- * "http://qdolan.blogspot.com/2008/10/embedded-jar-classloader-in-under-100.html"
- * >this blog</a>.
- *
- * @author Leo Neumeyer
- *
- */
-public class JarClassLoader extends URLClassLoader {
-
- private static void close(Closeable closeable) {
- if (closeable != null) {
- try {
- closeable.close();
- } catch (IOException e) {
- e.printStackTrace();
- }
- }
- }
-
- private static boolean isJar(String fileName) {
- return fileName != null && fileName.toLowerCase().endsWith(".jar");
- }
-
- private static File jarEntryAsFile(JarFile jarFile, JarEntry jarEntry) throws IOException {
- InputStream input = null;
- OutputStream output = null;
- try {
- String name = jarEntry.getName().replace('/', '_');
- int i = name.lastIndexOf(".");
- String extension = i > -1 ? name.substring(i) : "";
- File file = File.createTempFile(name.substring(0, name.length() - extension.length()) + ".", extension);
- file.deleteOnExit();
- input = jarFile.getInputStream(jarEntry);
- output = new FileOutputStream(file);
- int readCount;
- byte[] buffer = new byte[4096];
- while ((readCount = input.read(buffer)) != -1) {
- output.write(buffer, 0, readCount);
- }
- return file;
- } finally {
- close(input);
- close(output);
- }
- }
-
- public JarClassLoader(URL[] urls, ClassLoader parent) {
- super(urls, parent);
- try {
- ProtectionDomain protectionDomain = getClass().getProtectionDomain();
- CodeSource codeSource = protectionDomain.getCodeSource();
- URL rootJarUrl = codeSource.getLocation();
- String rootJarName = rootJarUrl.getFile();
- if (isJar(rootJarName)) {
- addJarResource(new File(rootJarUrl.getPath()));
- }
- } catch (IOException e) {
- e.printStackTrace();
- }
- }
-
- private void addJarResource(File file) throws IOException {
- JarFile jarFile = new JarFile(file);
- addURL(file.toURI().toURL());
- Enumeration<JarEntry> jarEntries = jarFile.entries();
- while (jarEntries.hasMoreElements()) {
- JarEntry jarEntry = jarEntries.nextElement();
- if (!jarEntry.isDirectory() && isJar(jarEntry.getName())) {
- addJarResource(jarEntryAsFile(jarFile, jarEntry));
- }
- }
- }
-
- @Override
- protected synchronized Class<?> loadClass(String name, boolean resolve)
- throws ClassNotFoundException {
- try {
- Class<?> clazz = findLoadedClass(name);
- if (clazz == null) {
- clazz = findClass(name);
- if (resolve)
- resolveClass(clazz);
- }
- return clazz;
- } catch (ClassNotFoundException e) {
- return super.loadClass(name, resolve);
- }
- }
-}
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/JarResources.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/JarResources.java
deleted file mode 100644
index 3f45575..0000000
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/JarResources.java
+++ /dev/null
@@ -1,182 +0,0 @@
-package org.apache.s4.base.util;
-
-import java.io.*;
-import java.util.*;
-import java.util.zip.*;
-
-/**
- * JarResources: JarResources maps all resources included in a Zip or Jar file. Additionaly, it provides a method to
- * extract one as a blob.
- *
- * <p>
- * CREDITS
- *
- * <p>
- * The source code for this class was derived from <a href=
- * "http://code.google.com/p/db4o-om/source/browse/trunk/objectmanager-api/src/com/db4o/objectmanager/configuration/MultiClassLoader.java"
- * >this project</a>.
- *
- */
-public final class JarResources {
-
- // external debug flag
- public boolean debugOn = false;
-
- // jar resource mapping tables
- private Map<String, Integer> htSizes = new HashMap<String, Integer>();
- private Map<String, Object> htJarContents = new HashMap<String, Object>();
-
- // a jar file
- private String jarFileName;
-
- /**
- * creates a JarResources. It extracts all resources from a Jar into an internal hashtable, keyed by resource names.
- *
- * @param jarFileName
- * a jar or zip file
- */
- public JarResources(String jarFileName) {
- this.jarFileName = jarFileName;
- init();
- }
-
- /**
- * Extracts a jar resource as a blob.
- *
- * @param name
- * a resource name.
- */
- public byte[] getResource(String name) {
- return (byte[]) htJarContents.get(name);
- }
-
- /** initializes internal hash tables with Jar file resources. */
- private void init() {
- try {
- // extracts just sizes only.
- ZipFile zf = new ZipFile(jarFileName);
- Enumeration<? extends ZipEntry> e = zf.entries();
- while (e.hasMoreElements()) {
- ZipEntry ze = (ZipEntry) e.nextElement();
-
- if (debugOn) {
- System.out.println(dumpZipEntry(ze));
- }
-
- htSizes.put(ze.getName(), new Integer((int) ze.getSize()));
- }
- zf.close();
-
- // extract resources and put them into the hashtable.
- FileInputStream fis = new FileInputStream(jarFileName);
- BufferedInputStream bis = new BufferedInputStream(fis);
- ZipInputStream zis = new ZipInputStream(bis);
- ZipEntry ze = null;
- while ((ze = zis.getNextEntry()) != null) {
- if (ze.isDirectory()) {
- continue;
- }
-
- if (debugOn) {
- System.out.println("ze.getName()=" + ze.getName() + "," + "getSize()=" + ze.getSize());
- }
-
- int size = (int) ze.getSize();
- // -1 means unknown size.
- if (size == -1) {
- size = ((Integer) htSizes.get(ze.getName())).intValue();
- }
-
- byte[] b = new byte[(int) size];
- int rb = 0;
- int chunk = 0;
- while (((int) size - rb) > 0) {
- chunk = zis.read(b, rb, (int) size - rb);
- if (chunk == -1) {
- break;
- }
- rb += chunk;
- }
-
- // add to internal resource hashtable
- htJarContents.put(ze.getName(), b);
-
- if (debugOn) {
- System.out.println(ze.getName() + " rb=" + rb + ",size=" + size + ",csize="
- + ze.getCompressedSize());
- }
- }
- zis.close();
- } catch (NullPointerException e) {
- System.out.println("done.");
- } catch (FileNotFoundException e) {
- e.printStackTrace();
- } catch (IOException e) {
- e.printStackTrace();
- }
- }
-
- /**
- * Dumps a zip entry into a string.
- *
- * @param ze
- * a ZipEntry
- */
- private String dumpZipEntry(ZipEntry ze) {
- StringBuffer sb = new StringBuffer();
- if (ze.isDirectory()) {
- sb.append("d ");
- } else {
- sb.append("f ");
- }
-
- if (ze.getMethod() == ZipEntry.STORED) {
- sb.append("stored ");
- } else {
- sb.append("defalted ");
- }
-
- sb.append(ze.getName());
- sb.append("\t");
- sb.append("" + ze.getSize());
- if (ze.getMethod() == ZipEntry.DEFLATED) {
- sb.append("/" + ze.getCompressedSize());
- }
-
- return (sb.toString());
- }
-
- /**
- * Is a test driver. Given a jar file and a resource name, it trys to extract the resource and then tells us whether
- * it could or not.
- *
- * <strong>Example</strong> Let's say you have a JAR file which jarred up a bunch of gif image files. Now, by using
- * JarResources, you could extract, create, and display those images on-the-fly.
- *
- * <pre>
- * ...
- * JarResources JR=new JarResources("GifBundle.jar");
- * Image image=Toolkit.createImage(JR.getResource("logo.gif");
- * Image logo=Toolkit.getDefaultToolkit().createImage(
- * JR.getResources("logo.gif")
- * );
- * ...
- * </pre>
- */
- public static void main(String[] args) throws IOException {
- if (args.length != 2) {
- System.err.println("usage: java JarResources <jar file name> <resource name>");
- System.exit(1);
- }
-
- JarResources jr = new JarResources(args[0]);
- byte[] buff = jr.getResource(args[1]);
- if (buff == null) {
- System.out.println("Could not find " + args[1] + ".");
- } else {
- System.out.println("Found " + args[1] + " (length=" + buff.length + ").");
- }
- }
-
-} // End of JarResources class.
-
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/MultiClassLoader.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/MultiClassLoader.java
deleted file mode 100644
index 0e942b6..0000000
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/MultiClassLoader.java
+++ /dev/null
@@ -1,124 +0,0 @@
-package org.apache.s4.base.util;
-
-import java.util.Map;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import com.google.common.collect.MapMaker;
-
-/**
- * Custom class loader to load classes in its own classloader.
- *
- * <p>
- * CREDITS
- *
- * <p>
- * The source code for this class was derived from <a href=
- * "http://code.google.com/p/db4o-om/source/browse/trunk/objectmanager-api/src/com/db4o/objectmanager/configuration/MultiClassLoader.java"
- * >this project</a> which was derived from this <a href=
- * "http://www.javaworld.com/javaworld/jw-10-1996/jw-10-indepth.html?page=1" >article by Chuck McManis</a>.
- *
- *
- * Thank you to the authors!
- */
-abstract public class MultiClassLoader extends ClassLoader {
-
- private static final Logger logger = LoggerFactory.getLogger(MultiClassLoader.class);
-
- private final Map<String, Class<?>> classes;
- private char classNameReplacementChar;
-
- public MultiClassLoader() {
- classes = new MapMaker().makeMap();
- }
-
- // ---------- Superclass Overrides ------------------------
- /**
- * This is a simple version for external clients since they will always want the class resolved before it is
- * returned to them.
- */
- @Override
- public Class<?> loadClass(String className) throws ClassNotFoundException {
- return (loadClass(className, true));
- }
-
- @Override
- public synchronized Class<?> loadClass(String className, boolean resolveIt) throws ClassNotFoundException {
-
- Class<?> result;
- byte[] classBytes;
- logger.trace("MultiClassLoader loadClass - className: " + className + ", resolveIt: " + resolveIt);
-
- /* Check our local cache of classes. */
- result = classes.get(className);
- if (result != null) {
- logger.trace("Returning cached result for class [{}]", className);
- return result;
- }
-
- /* Check with the primordial class loader. */
- try {
- result = super.findSystemClass(className);
- logger.trace("Returning system class (in CLASSPATH) [{}]", className);
- return result;
- } catch (ClassNotFoundException e) {
- logger.trace("Not a system class [{}]", className);
- }
-
- classBytes = loadClassBytes(className);
- if (classBytes == null) {
- throw new ClassNotFoundException();
- }
-
- /* Define it (parse the class file). */
- try {
- result = defineClass(className, classBytes, 0, classBytes.length);
- if (result == null) {
- throw new ClassFormatError();
- }
- } catch (NoClassDefFoundError ex) {
- // Dependencies were not found
- logger.warn(ex.getMessage());
- }
-
- /* Resolve if necessary. */
- if (resolveIt)
- resolveClass(result);
-
- /* Done. */
- if (result == null)
- return null;
- classes.put(className, result);
- logger.trace("Returning newly loaded class [{}]", className);
- return result;
- }
-
- /**
- * This optional call allows a class name such as "COM.test.Hello" to be changed to "COM_test_Hello", which is
- * useful for storing classes from different packages in the same retrieval directory. In the above example the char
- * would be '_'.
- */
- public void setClassNameReplacementChar(char replacement) {
- classNameReplacementChar = replacement;
- }
-
- /** Simple method to read a class file from a known location. */
- abstract protected byte[] loadClassBytes(String className);
-
- protected String formatClassName(String className) {
- if (classNameReplacementChar == '\u0000') {
- // '/' is used to map the package to the path
- return className.replace('.', '/') + ".class";
- } else {
- // Replace '.' with custom char, such as '_'
- return className.replace('.', classNameReplacementChar) + ".class";
- }
- }
-
- // --- Std
- protected static void print(String text) {
- System.out.println(text);
- }
-
-}
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoader.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoader.java
index 8d3e062..ca75722 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoader.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoader.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base.util;
import java.net.URL;
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoaderFactory.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoaderFactory.java
index c829f1c..d33c3bf 100644
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoaderFactory.java
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/S4RLoaderFactory.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base.util;
import java.io.File;
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/TestClassLoader.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/TestClassLoader.java
deleted file mode 100644
index 2dfeeb4..0000000
--- a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/TestClassLoader.java
+++ /dev/null
@@ -1,46 +0,0 @@
-package org.apache.s4.base.util;
-
-import java.io.FileInputStream;
-import java.io.IOException;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-public class TestClassLoader extends MultiClassLoader {
-
- private static final Logger logger = LoggerFactory.getLogger(TestClassLoader.class);
-
- @Override
- /** Simple method to read a class file from a known location. */
- public byte[] loadClassBytes(String className) {
-
- byte[] bytes = null;
- String filename = "/tmp/" + className + ".impl";
- logger.debug("Reading: " + filename);
-
- FileInputStream fi = null;
- try {
- fi = new FileInputStream(filename);
- bytes = new byte[fi.available()];
- fi.read(bytes);
- return bytes;
- } catch (Exception e) {
-
- /*
- * If we caught an exception, either the class wasn't found or it was unreadable by our process.
- */
- logger.error("Unable to load class: {}.", filename);
- e.printStackTrace();
- return null;
- } finally {
- if (fi != null) {
- try {
- fi.close();
- } catch (IOException e) {
- logger.warn("Exception while closing input stream", e);
- }
- }
- }
- }
-
-}
diff --git a/subprojects/s4-base/src/main/java/org/apache/s4/base/util/package-info.java b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/package-info.java
new file mode 100644
index 0000000..b1dbe71
--- /dev/null
+++ b/subprojects/s4-base/src/main/java/org/apache/s4/base/util/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Utility classes, mostly related to classloading.
+ */
+package org.apache.s4.base.util;
\ No newline at end of file
diff --git a/subprojects/s4-comm/s4-comm.gradle b/subprojects/s4-comm/s4-comm.gradle
index 7f3a628..7679236 100644
--- a/subprojects/s4-comm/s4-comm.gradle
+++ b/subprojects/s4-comm/s4-comm.gradle
@@ -1,11 +1,13 @@
-/*
- * Copyright 2010 the original author or authors.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
*
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
@@ -14,11 +16,11 @@
* limitations under the License.
*/
+
description = 'Implementation-specific components of the communication layer.'
dependencies {
compile project(":s4-base")
- compile libraries.json
compile libraries.gson
compile libraries.kryo
compile libraries.netty
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultCommModule.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultCommModule.java
index 863bc29..d7c8cee 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultCommModule.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultCommModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm;
import java.io.IOException;
@@ -80,17 +98,18 @@
try {
Class<? extends Emitter> emitterClass = (Class<? extends Emitter>) Class.forName(config
- .getString("comm.emitter.class"));
+ .getString("s4.comm.emitter.class"));
bind(Emitter.class).to(emitterClass);
// RemoteEmitter instances are created through a factory, depending on the topology. We inject the factory
Class<? extends RemoteEmitter> remoteEmitterClass = (Class<? extends RemoteEmitter>) Class.forName(config
- .getString("comm.emitter.remote.class"));
+ .getString("s4.comm.emitter.remote.class"));
install(new FactoryModuleBuilder().implement(RemoteEmitter.class, remoteEmitterClass).build(
RemoteEmitterFactory.class));
bind(RemoteEmitters.class);
- bind(Listener.class).to((Class<? extends Listener>) Class.forName(config.getString("comm.listener.class")));
+ bind(Listener.class).to(
+ (Class<? extends Listener>) Class.forName(config.getString("s4.comm.listener.class")));
} catch (ClassNotFoundException e) {
logger.error("Cannot find class implementation ", e);
@@ -110,14 +129,14 @@
Names.bindProperties(binder, ConfigurationConverter.getProperties(config));
if (clusterName != null) {
- if (config.containsKey("cluster.name")) {
+ if (config.containsKey("s4.cluster.name")) {
logger.warn(
"cluster [{}] passed as a parameter will not be used because an existing cluster.name parameter of value [{}] was found in the configuration file and will be used",
- clusterName, config.getProperty("cluster.name"));
+ clusterName, config.getProperty("s4.cluster.name"));
} else {
Names.bindProperties(binder, new HashMap<String, String>() {
{
- put("cluster.name", clusterName);
+ put("s4.cluster.name", clusterName);
}
});
}
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultHasher.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultHasher.java
index bf58c83..377647a 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultHasher.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/DefaultHasher.java
@@ -1,7 +1,29 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm;
import org.apache.s4.base.Hasher;
+/**
+ * Simple hasher implementation.
+ *
+ */
public class DefaultHasher implements Hasher {
@Override
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/QueueingEmitter.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/QueueingEmitter.java
deleted file mode 100644
index 3dd2955..0000000
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/QueueingEmitter.java
+++ /dev/null
@@ -1,103 +0,0 @@
-package org.apache.s4.comm;
-
-import java.util.concurrent.BlockingQueue;
-import java.util.concurrent.LinkedBlockingQueue;
-
-import org.apache.s4.base.Emitter;
-import org.apache.s4.base.EventMessage;
-
-import com.google.inject.Inject;
-import com.google.inject.name.Named;
-
-public class QueueingEmitter implements Emitter, Runnable {
- private Emitter emitter;
- private BlockingQueue<MessageHolder> queue;
- private long dropCount = 0;
- private volatile Thread thread;
-
- @Inject
- public QueueingEmitter(@Named("ll") Emitter emitter, @Named("comm.queue_emmiter_size") int queueSize) {
- this.emitter = emitter;
- queue = new LinkedBlockingQueue<MessageHolder>(queueSize);
- }
-
- public long getDropCount() {
- return dropCount;
- }
-
- public void start() {
- if (thread != null) {
- throw new IllegalStateException("QueueingEmitter is already started");
- }
- thread = new Thread(this, "QueueingEmitter");
- thread.start();
- }
-
- public void stop() {
- if (thread == null) {
- throw new IllegalStateException("QueueingEmitter is already stopped");
- }
- thread.interrupt();
- thread = null;
- }
-
- @Override
- public boolean send(int partitionId, EventMessage message) {
- MessageHolder mh = new MessageHolder(partitionId, message);
- if (!queue.offer(mh)) {
- dropCount++;
- return true;
- } else {
- return false;
- }
- }
-
- public void run() {
- while (!Thread.interrupted()) {
- try {
- MessageHolder mh = queue.take();
- // System.out.println("QueueingEmitter: Sending message on low-level emitter");
- emitter.send(mh.getPartitionId(), mh.getMessage());
- } catch (InterruptedException ie) {
- Thread.currentThread().interrupt();
- }
- }
- }
-
- public int getPartitionCount() {
- return emitter.getPartitionCount();
- }
-
- class MessageHolder {
- private int partitionId;
- private EventMessage message;
-
- public int getPartitionId() {
- return partitionId;
- }
-
- public void setPartitionId(int partitionId) {
- this.partitionId = partitionId;
- }
-
- public EventMessage getMessage() {
- return message;
- }
-
- public void setMessage(EventMessage message) {
- this.message = message;
- }
-
- public MessageHolder(int partitionId, EventMessage message) {
- super();
- this.partitionId = partitionId;
- this.message = message;
- }
- }
-
- @Override
- public void close() {
- // TODO Auto-generated method stub
-
- }
-}
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/QueueingListener.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/QueueingListener.java
deleted file mode 100644
index 2a9dc5c..0000000
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/QueueingListener.java
+++ /dev/null
@@ -1,75 +0,0 @@
-package org.apache.s4.comm;
-
-import java.util.concurrent.BlockingQueue;
-import java.util.concurrent.LinkedBlockingQueue;
-
-import org.apache.s4.base.Listener;
-
-import com.google.inject.Inject;
-import com.google.inject.name.Named;
-
-public class QueueingListener implements Listener, Runnable {
- private Listener listener;
- private BlockingQueue<byte[]> queue;
- private long dropCount = 0;
- private volatile Thread thread;
-
- @Inject
- public QueueingListener(@Named("ll") Listener listener, @Named("comm.queue_listener_size") int queueSize) {
- this.listener = listener;
- queue = new LinkedBlockingQueue<byte[]>(queueSize);
- }
-
- public long getDropCount() {
- return dropCount;
- }
-
- public void start() {
- if (thread != null) {
- throw new IllegalStateException("QueueingListener is already started");
- }
- thread = new Thread(this, "QueueingListener");
- thread.start();
- }
-
- public void stop() {
- if (thread == null) {
- throw new IllegalStateException("QueueingListener is already stopped");
- }
- thread.interrupt();
- thread = null;
- }
-
- @Override
- public byte[] recv() {
- try {
- // System.out.println("QueueingListener: About to take message from queue");
- return queue.take();
- } catch (InterruptedException ie) {
- Thread.currentThread().interrupt();
- return null;
- }
- }
-
- public int getPartitionId() {
- return listener.getPartitionId();
- }
-
- public void run() {
- while (!Thread.interrupted()) {
- byte[] message = listener.recv();
- if (!queue.offer(message)) {
- dropCount++;
- } else {
- // System.out.println("QueueingListener: Adding message of size "
- // + message.length + " to queue");
- }
- }
- }
-
- @Override
- public void close() {
- // TODO Auto-generated method stub
-
- }
-}
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/RemoteEmitterFactory.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/RemoteEmitterFactory.java
index 3da803d..5d79859 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/RemoteEmitterFactory.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/RemoteEmitterFactory.java
@@ -1,8 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm;
import org.apache.s4.base.RemoteEmitter;
import org.apache.s4.comm.topology.Cluster;
+/**
+ * Used for creating RemoteEmitter instances depending on the cluster configuration. Follows the "assisted injection"
+ * pattern from Guice 3.
+ *
+ */
public interface RemoteEmitterFactory {
RemoteEmitter createRemoteEmitter(Cluster topology);
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/loopback/LoopBackEmitter.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/loopback/LoopBackEmitter.java
deleted file mode 100644
index bb7aaa0..0000000
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/loopback/LoopBackEmitter.java
+++ /dev/null
@@ -1,37 +0,0 @@
-package org.apache.s4.comm.loopback;
-
-import org.apache.s4.base.Emitter;
-import org.apache.s4.base.EventMessage;
-import org.apache.s4.base.SerializerDeserializer;
-
-import com.google.inject.Inject;
-
-public class LoopBackEmitter implements Emitter {
- private LoopBackListener listener;
-
- @Inject
- SerializerDeserializer serDeser;
-
- public LoopBackEmitter(LoopBackListener listener) {
- this.listener = listener;
- }
-
- @Override
- public boolean send(int partitionId, EventMessage message) {
-
- listener.put(serDeser.serialize(message));
- return true;
- }
-
- @Override
- public int getPartitionCount() {
- return 1;
- }
-
- @Override
- public void close() {
- // TODO Auto-generated method stub
-
- }
-
-}
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/loopback/LoopBackListener.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/loopback/LoopBackListener.java
deleted file mode 100644
index 8077231..0000000
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/loopback/LoopBackListener.java
+++ /dev/null
@@ -1,42 +0,0 @@
-package org.apache.s4.comm.loopback;
-
-import java.util.concurrent.BlockingQueue;
-import java.util.concurrent.SynchronousQueue;
-
-import org.apache.s4.base.Listener;
-
-public class LoopBackListener implements Listener {
-
- private BlockingQueue<byte[]> handoffQueue = new SynchronousQueue<byte[]>();
-
- @Override
- public byte[] recv() {
- try {
- // System.out.println("LoopBackListener: Taking message from handoff queue");
- return handoffQueue.take();
- } catch (InterruptedException e) {
- throw new RuntimeException(e);
- }
- }
-
- @Override
- public int getPartitionId() {
- return 0;
- }
-
- public void put(byte[] message) {
- try {
- // System.out.println("LoopBackListener: putting message into handoffqueue");
- handoffQueue.put(message);
- } catch (InterruptedException ie) {
- Thread.currentThread().interrupt();
- }
- }
-
- @Override
- public void close() {
- // TODO Auto-generated method stub
-
- }
-
-}
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/package-info.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/package-info.java
new file mode 100644
index 0000000..a5a3096
--- /dev/null
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * This package and subpackages define mechanisms for low-level communications in the S4 platform.
+ */
+package org.apache.s4.comm;
\ No newline at end of file
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/serialize/KryoSerDeser.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/serialize/KryoSerDeser.java
index 8dd53db..b7fad75 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/serialize/KryoSerDeser.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/serialize/KryoSerDeser.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.serialize;
import java.nio.ByteBuffer;
@@ -9,6 +27,10 @@
import com.esotericsoftware.kryo.serialize.ClassSerializer;
import com.esotericsoftware.kryo.serialize.SimpleSerializer;
+/**
+ * Serializazer/deserializer based on <a href="http://code.google.com/p/kryo/">kryo</a>
+ *
+ */
public class KryoSerDeser implements SerializerDeserializer {
private Kryo kryo = new Kryo();
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/serialize/package-info.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/serialize/package-info.java
new file mode 100644
index 0000000..7d8f246
--- /dev/null
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/serialize/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Serialization utilities for messages and PEs.
+ */
+package org.apache.s4.comm.serialize;
\ No newline at end of file
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/RemoteEmitters.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/RemoteEmitters.java
index 7623d52..85dc86d 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/RemoteEmitters.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/RemoteEmitters.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tcp;
import java.util.HashMap;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPEmitter.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPEmitter.java
index d2aa64f..44c9247 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPEmitter.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPEmitter.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tcp;
import java.net.ConnectException;
@@ -74,7 +92,7 @@
SerializerDeserializer serDeser;
@Inject
- public TCPEmitter(Cluster topology, @Named("comm.timeout") int timeout) throws InterruptedException {
+ public TCPEmitter(Cluster topology, @Named("s4.comm.timeout") int timeout) throws InterruptedException {
this.nettyTimeout = timeout;
this.topology = topology;
this.lock = new ReentrantLock();
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPListener.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPListener.java
index d7d49bd..d495033 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPListener.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPListener.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tcp;
import java.net.InetSocketAddress;
@@ -45,7 +63,7 @@
private final int nettyTimeout;
@Inject
- public TCPListener(Assignment assignment, @Named("comm.timeout") int timeout) {
+ public TCPListener(Assignment assignment, @Named("s4.comm.timeout") int timeout) {
// wait for an assignment
node = assignment.assignClusterNode();
nettyTimeout = timeout;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPRemoteEmitter.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPRemoteEmitter.java
index b19d5be..a22767b 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPRemoteEmitter.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/TCPRemoteEmitter.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tcp;
import org.apache.s4.base.RemoteEmitter;
@@ -14,12 +32,11 @@
public class TCPRemoteEmitter extends TCPEmitter implements RemoteEmitter {
/**
- * Sends to remote subclusters. This is dynamically created, through an
- * injected factory, when new subclusters are discovered (as remote streams
- * outputs)
+ * Sends to remote subclusters. This is dynamically created, through an injected factory, when new subclusters are
+ * discovered (as remote streams outputs)
*/
@Inject
- public TCPRemoteEmitter(@Assisted Cluster topology, @Named("comm.timeout") int timeout)
+ public TCPRemoteEmitter(@Assisted Cluster topology, @Named("s4.comm.timeout") int timeout)
throws InterruptedException {
super(topology, timeout);
}
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/package-info.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/package-info.java
new file mode 100644
index 0000000..d747e5d
--- /dev/null
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tcp/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * TCP-based communication layer implementation.
+ */
+package org.apache.s4.comm.tcp;
\ No newline at end of file
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tools/TaskSetup.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tools/TaskSetup.java
index 010274a..b74bd19 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tools/TaskSetup.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tools/TaskSetup.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tools;
import java.util.concurrent.TimeUnit;
@@ -7,6 +25,10 @@
import org.apache.s4.comm.topology.ZNRecordSerializer;
import org.apache.s4.comm.topology.ZkClient;
+/**
+ * Used for defining and dimensioning logical clusters in Zookeeper.
+ *
+ */
public class TaskSetup {
private ZkClient zkclient;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tools/package-info.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tools/package-info.java
new file mode 100644
index 0000000..c228756
--- /dev/null
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/tools/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Utility classes.
+ */
+package org.apache.s4.comm.tools;
\ No newline at end of file
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Assignment.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Assignment.java
index 0cac238..4d9b85a 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Assignment.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Assignment.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
/**
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/AssignmentFromZK.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/AssignmentFromZK.java
index 66c2eca..0b81854 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/AssignmentFromZK.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/AssignmentFromZK.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.net.InetAddress;
@@ -24,6 +42,10 @@
import com.google.inject.Singleton;
import com.google.inject.name.Named;
+/**
+ * Handles partition assignment through Zookeeper.
+ *
+ */
@Singleton
public class AssignmentFromZK implements Assignment, IZkChildListener, IZkStateListener, IZkDataListener {
private static final Logger logger = LoggerFactory.getLogger(AssignmentFromZK.class);
@@ -68,10 +90,10 @@
// TODO we currently have a single assignment per node (i.e. a node can only belong to 1 topology)
@Inject
- public AssignmentFromZK(@Named("cluster.name") String clusterName,
- @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
+ public AssignmentFromZK(@Named("s4.cluster.name") String clusterName,
+ @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
this.clusterName = clusterName;
this.connectionTimeout = connectionTimeout;
taskPath = "/s4/clusters/" + clusterName + "/tasks";
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Cluster.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Cluster.java
index a9066a4..2ff6e97 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Cluster.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Cluster.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
/**
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterChangeListener.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterChangeListener.java
index c53798a..441837b 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterChangeListener.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterChangeListener.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
/**
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterFromZK.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterFromZK.java
index 1ec3cb5..6bda6b4 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterFromZK.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterFromZK.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.net.InetAddress;
@@ -5,7 +23,6 @@
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
-import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
@@ -21,32 +38,31 @@
import com.google.inject.Inject;
import com.google.inject.name.Named;
+/**
+ * Represents a logical cluster definition fetched from Zookeeper. Notifies listeners of runtime changes in the
+ * configuration.
+ *
+ */
public class ClusterFromZK implements Cluster, IZkChildListener, IZkDataListener, IZkStateListener {
private static Logger logger = LoggerFactory.getLogger(ClusterFromZK.class);
private final AtomicReference<PhysicalCluster> clusterRef;
private final List<ClusterChangeListener> listeners;
- private KeeperState state;
private final ZkClient zkClient;
private final String taskPath;
private final String processPath;
private final Lock lock;
- private AtomicBoolean currentlyOwningTask;
- private String machineId;
private String clusterName;
- private int connectionTimeout;
-
/**
* only the local topology
*/
@Inject
- public ClusterFromZK(@Named("cluster.name") String clusterName,
- @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
- this.connectionTimeout = connectionTimeout;
+ public ClusterFromZK(@Named("s4.cluster.name") String clusterName,
+ @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
this.clusterName = clusterName;
this.taskPath = "/s4/clusters/" + clusterName + "/tasks";
this.processPath = "/s4/clusters/" + clusterName + "/process";
@@ -59,10 +75,9 @@
throw new Exception("cannot connect to zookeeper");
}
try {
- machineId = InetAddress.getLocalHost().getCanonicalHostName();
+ InetAddress.getLocalHost().getCanonicalHostName();
} catch (UnknownHostException e) {
logger.warn("Unable to get hostname", e);
- machineId = "UNKNOWN";
}
this.clusterRef = new AtomicReference<PhysicalCluster>();
this.listeners = new ArrayList<ClusterChangeListener>();
@@ -85,7 +100,6 @@
this.processPath = "/s4/clusters/" + clusterName + "/process";
this.clusterName = clusterName;
this.lock = new ReentrantLock();
- this.machineId = machineId;
this.listeners = new ArrayList<ClusterChangeListener>();
this.clusterRef = new AtomicReference<PhysicalCluster>();
zkClient.subscribeChildChanges(taskPath, this);
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterNode.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterNode.java
index 6a52663..e622107 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterNode.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClusterNode.java
@@ -1,37 +1,60 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
+/**
+ * Represents an node.
+ *
+ */
public class ClusterNode {
private int partition;
private int port;
private String machineName;
private String taskId;
-
+
public ClusterNode(int partition, int port, String machineName, String taskId) {
this.partition = partition;
this.port = port;
this.machineName = machineName;
this.taskId = taskId;
}
-
+
public int getPartition() {
return partition;
}
+
public int getPort() {
return port;
}
+
public String getMachineName() {
return machineName;
}
+
public String getTaskId() {
return taskId;
}
-
+
public String toString() {
StringBuffer sb = new StringBuffer();
- sb.append("{").append("partition=").append(partition).
- append(",port=").append(port).
- append(",machineName=").append(machineName).
- append(",taskId=").append(taskId).append("}");
+ sb.append("{").append("partition=").append(partition).append(",port=").append(port).append(",machineName=")
+ .append(machineName).append(",taskId=").append(taskId).append("}");
return sb.toString();
}
}
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Clusters.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Clusters.java
index aa8b78f..c7ff5b3 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Clusters.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/Clusters.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
/**
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClustersFromZK.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClustersFromZK.java
index f177599..86f6bb5 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClustersFromZK.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ClustersFromZK.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.net.InetAddress;
@@ -33,10 +51,10 @@
private String clusterName;
@Inject
- public ClustersFromZK(@Named("cluster.name") String clusterName,
- @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
+ public ClustersFromZK(@Named("s4.cluster.name") String clusterName,
+ @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
this.clusterName = clusterName;
this.connectionTimeout = connectionTimeout;
lock = new ReentrantLock();
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/PhysicalCluster.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/PhysicalCluster.java
index 32348df..7c590ac 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/PhysicalCluster.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/PhysicalCluster.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.io.IOException;
@@ -73,9 +91,8 @@
}
/**
- * Number of partitions in the cluster.
*
- * @return
+ * @return Number of partitions in the cluster.
*/
public int getPartitionCount() {
return numPartitions;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteCluster.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteCluster.java
index 2b9d866..e8e298a 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteCluster.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteCluster.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
/**
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteStreams.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteStreams.java
index 587f823..b1fe6b7 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteStreams.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/RemoteStreams.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.util.Collections;
@@ -71,9 +89,9 @@
}
@Inject
- public RemoteStreams(@Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
+ public RemoteStreams(@Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
lock = new ReentrantLock();
zkClient = new ZkClient(zookeeperAddress, sessionTimeout, connectionTimeout);
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/StreamConsumer.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/StreamConsumer.java
index ede16a3..52fa2a8 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/StreamConsumer.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/StreamConsumer.java
@@ -1,8 +1,26 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
/**
* A subscriber to a published stream. Identified through its cluster name (for dispatching to the remote cluster) and
- * application ID (for dispatching within a node).
+ * application ID (for dispatching within a node (NOTE: this parameter is ignored)).
*
*/
public class StreamConsumer {
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecord.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecord.java
index 4aa7639..aee7f6f 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecord.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecord.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.util.List;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecordSerializer.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecordSerializer.java
index 8fe203b..b55f469 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecordSerializer.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZNRecordSerializer.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import org.I0Itec.zkclient.exception.ZkMarshallingError;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZkClient.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZkClient.java
index e4c4911..e5a8689 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZkClient.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/ZkClient.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.util.concurrent.Callable;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/package-info.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/package-info.java
new file mode 100644
index 0000000..e3d291a
--- /dev/null
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/topology/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Clusters configurations and partitions assignments to cluster nodes through Zookeeper.
+ */
+package org.apache.s4.comm.topology;
\ No newline at end of file
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPEmitter.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPEmitter.java
index 0f29f6b..2f156b2 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPEmitter.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPEmitter.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.udp;
import java.io.IOException;
@@ -19,6 +37,10 @@
import com.google.common.collect.HashBiMap;
import com.google.inject.Inject;
+/**
+ * UDP based emitter.
+ *
+ */
public class UDPEmitter implements Emitter, ClusterChangeListener {
private DatagramSocket socket;
private final HashBiMap<Integer, ClusterNode> nodes;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPListener.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPListener.java
index 43bb470..319c7ab 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPListener.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPListener.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.udp;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPRemoteEmitter.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPRemoteEmitter.java
index 4fa9aa0..050a6ec 100644
--- a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPRemoteEmitter.java
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/UDPRemoteEmitter.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.udp;
import org.apache.s4.comm.topology.Cluster;
@@ -5,6 +23,10 @@
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
+/**
+ * UDP-based emitter for sending events to remote clusters.
+ *
+ */
public class UDPRemoteEmitter extends UDPEmitter {
/**
diff --git a/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/package-info.java b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/package-info.java
new file mode 100644
index 0000000..d951725
--- /dev/null
+++ b/subprojects/s4-comm/src/main/java/org/apache/s4/comm/udp/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * UDP-based implementation of the communication layer.
+ */
+package org.apache.s4.comm.udp;
\ No newline at end of file
diff --git a/subprojects/s4-comm/src/main/resources/default.s4.comm.properties b/subprojects/s4-comm/src/main/resources/default.s4.comm.properties
index 0c5bbfd..36417b4 100644
--- a/subprojects/s4-comm/src/main/resources/default.s4.comm.properties
+++ b/subprojects/s4-comm/src/main/resources/default.s4.comm.properties
@@ -1,12 +1,8 @@
-comm.queue_emmiter_size = 8000
-comm.queue_listener_size = 8000
-comm.emitter.class=org.apache.s4.comm.tcp.TCPEmitter
-comm.emitter.remote.class=org.apache.s4.comm.tcp.TCPRemoteEmitter
-comm.listener.class=org.apache.s4.comm.tcp.TCPListener
-comm.retries=10
-comm.retry_delay=10
-comm.timeout=1000
-tcp.partition.queue_size=256
-cluster.zk_address = localhost:2181
-cluster.zk_session_timeout = 10000
-cluster.zk_connection_timeout = 10000
\ No newline at end of file
+s4.comm.emitter.class=org.apache.s4.comm.tcp.TCPEmitter
+s4.comm.emitter.remote.class=org.apache.s4.comm.tcp.TCPRemoteEmitter
+s4.comm.listener.class=org.apache.s4.comm.tcp.TCPListener
+# I/O channel connection timeout, when applicable (e.g. used by netty)
+s4.comm.timeout=1000
+s4.cluster.zk_address = localhost:2181
+s4.cluster.zk_session_timeout = 10000
+s4.cluster.zk_connection_timeout = 10000
\ No newline at end of file
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/DeliveryTestUtil.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/DeliveryTestUtil.java
index abbc31b..17340b8 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/DeliveryTestUtil.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/DeliveryTestUtil.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm;
import org.apache.s4.base.Emitter;
@@ -31,8 +49,8 @@
private final int messagesExpected;
@Inject
- public DeliveryTestUtil(Emitter emitter, Listener listener, @Named("emitter.send.interval") int interval,
- @Named("emitter.send.numMessages") int numMessages, @Named("listener.recv.sleepCount") int sleepCount) {
+ public DeliveryTestUtil(Emitter emitter, Listener listener, @Named("s4.emitter.send.interval") int interval,
+ @Named("s4.emitter.send.numMessages") int numMessages, @Named("s4.listener.recv.sleepCount") int sleepCount) {
this.emitter = emitter;
this.listener = listener;
this.interval = interval;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/MultiPartitionDeliveryTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/MultiPartitionDeliveryTest.java
index c0e811e..1813f7a 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/MultiPartitionDeliveryTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/MultiPartitionDeliveryTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tcp;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/SimpleDeliveryTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/SimpleDeliveryTest.java
index f5d1fc0..e46ed47 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/SimpleDeliveryTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/SimpleDeliveryTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tcp;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/TCPCommTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/TCPCommTest.java
index be020cb..c9153ce 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/TCPCommTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/tcp/TCPCommTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.tcp;
import java.io.IOException;
@@ -10,10 +28,8 @@
import org.slf4j.LoggerFactory;
import com.google.common.io.Resources;
-import com.google.inject.AbstractModule;
import com.google.inject.Guice;
import com.google.inject.Injector;
-import com.google.inject.name.Names;
public abstract class TCPCommTest extends ProtocolTestUtil {
@@ -32,26 +48,13 @@
public Injector newInjector() {
try {
return Guice.createInjector(new DefaultCommModule(Resources.getResource("default.s4.comm.properties")
- .openStream(), CLUSTER_NAME), new TCPCommTestModule());
+ .openStream(), CLUSTER_NAME));
} catch (IOException e) {
Assert.fail();
return null;
}
}
- class TCPCommTestModule extends AbstractModule {
- TCPCommTestModule() {
-
- }
-
- @Override
- protected void configure() {
- bind(Integer.class).annotatedWith(Names.named("emitter.send.interval")).toInstance(100);
- bind(Integer.class).annotatedWith(Names.named("emitter.send.numMessages")).toInstance(200);
- bind(Integer.class).annotatedWith(Names.named("listener.recv.sleepCount")).toInstance(10);
- }
- }
-
@Override
public void testDelivery() throws InterruptedException {
startThreads();
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest1.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest1.java
index 8e4c70a..e3ff947 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest1.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest1.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import static org.junit.Assert.assertEquals;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest2.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest2.java
index 8c51c39..51db713 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest2.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/AssignmentsFromZKTest2.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import org.apache.s4.comm.tools.TaskSetup;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ClustersFromZKTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ClustersFromZKTest.java
index fb81228..768ffd0 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ClustersFromZKTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ClustersFromZKTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import static org.junit.Assert.assertEquals;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZKBaseTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZKBaseTest.java
index 8107cde..2ef95f4 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZKBaseTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZKBaseTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZNRecordSerializerTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZNRecordSerializerTest.java
index 0789a6a..a2016cf 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZNRecordSerializerTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/topology/ZNRecordSerializerTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.topology;
import org.junit.Assert;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/MultiPartitionDeliveryTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/MultiPartitionDeliveryTest.java
index ad030c0..4e06233 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/MultiPartitionDeliveryTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/MultiPartitionDeliveryTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.udp;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/SimpleDeliveryTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/SimpleDeliveryTest.java
index 41d3017..b1c5d8a 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/SimpleDeliveryTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/SimpleDeliveryTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.udp;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/UDPCommTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/UDPCommTest.java
index c0ed6b4..0f6004f 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/UDPCommTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/udp/UDPCommTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.udp;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/PartitionInfo.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/PartitionInfo.java
index 6078f7e..97cc973 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/PartitionInfo.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/PartitionInfo.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.util;
import java.util.ArrayList;
@@ -12,7 +30,6 @@
import org.slf4j.LoggerFactory;
import com.google.inject.Inject;
-import com.google.inject.name.Named;
/**
* Test util for communication protocols.
@@ -33,9 +50,9 @@
public SendThread sendThread;
public ReceiveThread receiveThread;
- private final int numRetries;
- private final int retryDelayMs;
- private int numMessages;
+ private final int numRetries = 10;
+ private final int retryDelayMs = 10;
+ private int numMessages = 100;
private int partitionId;
private ProtocolTestUtil ptu;
@@ -43,17 +60,13 @@
SerializerDeserializer serDeser;
@Inject
- public PartitionInfo(Emitter emitter, Listener listener, @Named("comm.retries") int retries,
- @Named("comm.retry_delay") int retryDelay, @Named("emitter.send.numMessages") int numMessages) {
+ public PartitionInfo(Emitter emitter, Listener listener) {
this.emitter = emitter;
this.listener = listener;
this.partitionId = this.listener.getPartitionId();
logger.debug("# Partitions = {}; Current partition = {}", this.emitter.getPartitionCount(),
this.listener.getPartitionId());
- this.numRetries = retries;
- this.retryDelayMs = retryDelay;
- this.numMessages = numMessages;
// this.messagesExpected = numMessages * this.emitter.getPartitionCount();
this.sendThread = new SendThread();
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/ProtocolTestUtil.java b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/ProtocolTestUtil.java
index 6bc074e..c647611 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/ProtocolTestUtil.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/comm/util/ProtocolTestUtil.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.comm.util;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/CommTestUtils.java b/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/CommTestUtils.java
index 63e32b5..7316498 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/CommTestUtils.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/CommTestUtils.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import java.io.BufferedReader;
diff --git a/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/ZkBasedTest.java b/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/ZkBasedTest.java
index fdd607b..411c7c6 100644
--- a/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/ZkBasedTest.java
+++ b/subprojects/s4-comm/src/test/java/org/apache/s4/fixtures/ZkBasedTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import java.io.IOException;
diff --git a/subprojects/s4-comm/src/test/resources/udp.s4.comm.properties b/subprojects/s4-comm/src/test/resources/udp.s4.comm.properties
index a5367c7..7b525bd 100644
--- a/subprojects/s4-comm/src/test/resources/udp.s4.comm.properties
+++ b/subprojects/s4-comm/src/test/resources/udp.s4.comm.properties
@@ -1,12 +1,7 @@
-comm.queue_emmiter_size = 8000
-comm.queue_listener_size = 8000
-comm.emitter.class = org.apache.s4.comm.udp.UDPEmitter
-comm.emitter.remote.class = org.apache.s4.comm.udp.UDPRemoteEmitter
-comm.listener.class = org.apache.s4.comm.udp.UDPListener
-cluster.name = cluster1
-cluster.zk_address = localhost:2181
-cluster.zk_session_timeout = 10000
-cluster.zk_connection_timeout = 10000
-comm.retries=10
-comm.retry_delay=10
-comm.timeout=1000
\ No newline at end of file
+s4.comm.emitter.class = org.apache.s4.comm.udp.UDPEmitter
+s4.comm.emitter.remote.class = org.apache.s4.comm.udp.UDPRemoteEmitter
+s4.comm.listener.class = org.apache.s4.comm.udp.UDPListener
+s4.cluster.name = cluster1
+s4.cluster.zk_address = localhost:2181
+s4.cluster.zk_session_timeout = 10000
+s4.cluster.zk_connection_timeout = 10000
diff --git a/subprojects/s4-core/s4-core.gradle b/subprojects/s4-core/s4-core.gradle
index c3a3f52..a063b11 100644
--- a/subprojects/s4-core/s4-core.gradle
+++ b/subprojects/s4-core/s4-core.gradle
@@ -1,18 +1,21 @@
-/*
- * Copyright 2010 the original author or authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
description = 'The S4 core platform.'
@@ -32,4 +35,4 @@
test {
forkEvery=1;
-}
\ No newline at end of file
+}
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/App.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/App.java
index 2e816e6..936d225 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/App.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/App.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
*
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
*
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.core;
import java.util.ArrayList;
@@ -38,7 +41,7 @@
/**
* Container base class to hold all processing elements.
- *
+ *
* It is also where one defines the application graph: PE prototypes, internal streams, input and output streams.
*/
public abstract class App {
@@ -74,7 +77,7 @@
RemoteStreams remoteStreams;
@Inject
- @Named("cluster.name")
+ @Named("s4.cluster.name")
String clusterName;
// default is NoOpCheckpointingFramework
@@ -233,8 +236,8 @@
* Set the {@link ClockType}.
*
* @param clockType
- * the clockTyoe for this app must be {@link ClockType.WALL_CLOCK} (default) or
- * {@link ClockType.EVENT_CLOCK}
+ * the clockTyoe for this app must be {@link ClockType#WALL_CLOCK} (default) or
+ * {@link ClockType#EVENT_CLOCK}
*/
public void setClockType(ClockType clockType) {
this.clockType = clockType;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/DefaultCoreModule.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/DefaultCoreModule.java
index d441d59..5701640 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/DefaultCoreModule.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/DefaultCoreModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.io.IOException;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/Key.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/Key.java
index 49b3766..2a78213 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/Key.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/Key.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.core;
import java.util.List;
@@ -21,14 +24,13 @@
import org.apache.s4.base.Event;
import org.apache.s4.base.KeyFinder;
-/*
- * The Key class is used to get the value of the key on a specific type of
- * event. This is done to abstract all the complexity required to get the
- * value. The method for getting the value is implemented in a method of
- * an object of type KeyFinder<T>.
+/**
+ * The Key class is used to get the value of the key on a specific type of event. This is done to abstract all the
+ * complexity required to get the value. The method for getting the value is implemented in a method of an object of
+ * type KeyFinder<T>.
*
- * The application programmer provides the events and the corresponding
- * finders. The framework will use it to key on events.
+ * The application programmer provides the events and the corresponding finders. The framework will use it to key on
+ * events.
*/
public class Key<T extends Event> {
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/Main.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/Main.java
index ea7cfd6..74116a1 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/Main.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/Main.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.io.File;
@@ -103,7 +121,7 @@
}
if (mainArgs.zkConnectionString != null) {
- mainArgs.extraNamedParameters.add("cluster.zk_address=" + mainArgs.zkConnectionString);
+ mainArgs.extraNamedParameters.add("s4.cluster.zk_address=" + mainArgs.zkConnectionString);
}
if (!mainArgs.extraNamedParameters.isEmpty()) {
@@ -140,6 +158,10 @@
}
}
+ /**
+ * Defines command parameters.
+ *
+ */
@Parameters(separators = "=")
public static class MainArgs {
@@ -172,6 +194,10 @@
}
+ /**
+ * Parameters parsing utility.
+ *
+ */
public static class InlineConfigParameterConverter implements IStringConverter<String> {
@Override
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ProcessingElement.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ProcessingElement.java
index 6f8264a..b899403 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ProcessingElement.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ProcessingElement.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.lang.Thread.UncaughtExceptionHandler;
@@ -25,9 +43,9 @@
import org.slf4j.LoggerFactory;
import com.google.common.base.Preconditions;
-import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
+import com.google.common.cache.LoadingCache;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.MapMaker;
@@ -99,7 +117,7 @@
* This maps holds all the instances. We make it package private to prevent concrete classes from updating the
* collection.
*/
- transient Cache<String, ProcessingElement> peInstances;
+ transient LoadingCache<String, ProcessingElement> peInstances;
/* This map is initialized in the prototype and cloned to instances. */
transient Map<Class<? extends Event>, Trigger> triggers;
@@ -162,9 +180,9 @@
}
/**
- * This method is called by the PE timer. By default it is synchronized with the {@link #onEvent()} and
- * {@link #onTrigger()} methods. To execute concurrently with other methods, the {@link ProcessingElelment} subclass
- * must be annotated with {@link @ThreadSafe}.
+ * This method is called by the PE timer. By default it is synchronized with the onEvent() and onTrigger() methods.
+ * To execute concurrently with other methods, the {@link ProcessingElement} subclass must be annotated with
+ * {@link ThreadSafe}.
*
* Override this method to implement a periodic process.
*/
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/Receiver.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/Receiver.java
index bf5f9eb..6c0b19c 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/Receiver.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/Receiver.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.util.Map;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSender.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSender.java
index cbe037e..daccbaa 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSender.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSender.java
@@ -1,9 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import org.apache.s4.base.Emitter;
import org.apache.s4.base.EventMessage;
import org.apache.s4.base.Hasher;
+/**
+ * Sends events to a remote cluster.
+ *
+ */
public class RemoteSender {
final private Emitter emitter;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSenders.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSenders.java
index b05ca28..3a111d0 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSenders.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteSenders.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.util.HashMap;
@@ -17,6 +35,11 @@
import com.google.inject.Inject;
+/**
+ * Sends events to remote clusters. Target clusters are selected dynamically based on the stream name information from
+ * the event.
+ *
+ */
public class RemoteSenders {
Logger logger = LoggerFactory.getLogger(RemoteSenders.class);
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteStream.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteStream.java
index 330fd83..a588535 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteStream.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/RemoteStream.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.util.concurrent.atomic.AtomicInteger;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/Sender.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/Sender.java
index 87cf7bf..5b0b03d 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/Sender.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/Sender.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import org.apache.s4.base.Emitter;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/Server.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/Server.java
index 56d66c9..02e521a 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/Server.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/Server.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.io.File;
@@ -48,9 +66,9 @@
*/
@Inject
public Server(String commModuleName, @Named("s4.logger_level") String logLevel,
- @Named("cluster.name") String clusterName, @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) {
+ @Named("s4.cluster.name") String clusterName, @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) {
this.logLevel = logLevel;
zkClient = new ZkClient(zookeeperAddress, sessionTimeout, connectionTimeout);
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/Stream.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/Stream.java
index 85d80f7..6616d0f 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/Stream.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/Stream.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.util.Collection;
@@ -19,7 +37,7 @@
* <p>
* Once a stream is instantiated, it is immutable.
* <p>
- * To build an application create stream objects using use the {@link StreamFactory} class.
+ * To build an application, create stream objects using relevant methods in the {@link App} class.
*/
public class Stream<T extends Event> implements Runnable, Streamable {
@@ -40,7 +58,7 @@
private Class<T> eventType = null;
/**
- * Send events using a {@link KeyFinder<T>}. The key finder extracts the value of the key which is used to determine
+ * Send events using a {@link KeyFinder}. The key finder extracts the value of the key which is used to determine
* the target {@link org.apache.s4.comm.topology.ClusterNode} for an event.
*
* @param app
@@ -105,7 +123,7 @@
/**
* Define the key finder for this stream using a descriptor.
*
- * @param keyFinderString
+ * @param keyName
* a descriptor to lookup up the value of the key.
* @return the stream object
*/
@@ -135,7 +153,7 @@
/**
* Send events from this stream to various PEs.
*
- * @param pe
+ * @param pes
* a target PE array.
*
* @return the stream object
@@ -193,8 +211,7 @@
}
/**
- * Implements the {@link ReceiverListener} interface. The low level {@link Receiver} object call this method when a
- * new {@link Event} is available.
+ * The low level {@link Receiver} object call this method when a new {@link Event} is available.
*/
public void receiveEvent(EventMessage event) {
try {
@@ -220,13 +237,6 @@
}
/**
- * @return the stream id
- */
- // int getId() {
- // return id;
- // }
-
- /**
* @return the app
*/
public App getApp() {
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/Streamable.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/Streamable.java
index cfe54dd..d27fea5 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/Streamable.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/Streamable.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/adapter/AdapterApp.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/adapter/AdapterApp.java
index c23afb0..f0c45a9 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/adapter/AdapterApp.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/adapter/AdapterApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.adapter;
import org.apache.s4.base.Event;
@@ -15,7 +33,7 @@
public abstract class AdapterApp extends App {
@Inject
- @Named(value = "adapter.output.stream")
+ @Named(value = "s4.adapter.output.stream")
String outputStreamName;
private RemoteStream remoteStream;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/adapter/package-info.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/adapter/package-info.java
new file mode 100644
index 0000000..92e677d
--- /dev/null
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/adapter/package-info.java
@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * This package contains utilities for creating and configuring adapter apps.
+ */
+package org.apache.s4.core.adapter;
+
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointId.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointId.java
index 290d46a..373360c 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointId.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointId.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
import java.util.regex.Matcher;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingConfig.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingConfig.java
index eec765d..a0796c2 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingConfig.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingConfig.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingFramework.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingFramework.java
index 48b4616..83e9390 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingFramework.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingFramework.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import org.apache.s4.core.ProcessingElement;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingTask.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingTask.java
index dd6003c..316624e 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingTask.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/CheckpointingTask.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.util.Map;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/DefaultFileSystemStateStorage.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/DefaultFileSystemStateStorage.java
index 3a95da6..e5d4f9f 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/DefaultFileSystemStateStorage.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/DefaultFileSystemStateStorage.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
import java.io.File;
@@ -41,8 +42,8 @@
* file system such as NFS when running on a cluster.
* </p>
* <p>
- * Checkpoints are stored in individual files (1 file = 1 safeKeeperId) in directories according to the following
- * structure: <code>(storageRootpath)/prototypeId/safeKeeperId</code>
+ * Checkpoints are stored in individual files (1 file = 1 checkpointId) in directories according to the following
+ * structure: <code>(storageRootpath)/prototypeId/checkpointId</code>
* </p>
*
*/
@@ -68,7 +69,7 @@
@Override
public byte[] fetchState(CheckpointId key) {
- File file = safeKeeperID2File(key, storageRootPath);
+ File file = checkpointID2File(key, storageRootPath);
if (file != null && file.exists()) {
logger.debug("Fetching " + file.getAbsolutePath() + "for : " + key);
@@ -102,20 +103,20 @@
}
});
for (File file : files) {
- keys.add(file2SafeKeeperID(file));
+ keys.add(file2CheckpointID(file));
}
}
return keys;
}
// files kept as : root/<prototypeId>/encodedKeyWithFullInfo
- private static File safeKeeperID2File(CheckpointId key, String storageRootPath) {
+ private static File checkpointID2File(CheckpointId key, String storageRootPath) {
return new File(storageRootPath + File.separator + key.getPrototypeId() + File.separator
+ Base64.encodeBase64URLSafeString(key.getStringRepresentation().getBytes()));
}
- private static CheckpointId file2SafeKeeperID(File file) {
+ private static CheckpointId file2CheckpointID(File file) {
CheckpointId id = null;
id = new CheckpointId(new String(Base64.decodeBase64(file.getName())));
return id;
@@ -139,7 +140,7 @@
@Override
public void saveState(CheckpointId key, byte[] state, StorageCallback callback) {
- File f = safeKeeperID2File(key, storageRootPath);
+ File f = checkpointID2File(key, storageRootPath);
if (logger.isDebugEnabled()) {
logger.debug("Checkpointing [" + key + "] into file: [" + f.getAbsolutePath() + "]");
}
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FetchTask.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FetchTask.java
index 6994acd..348fd8e 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FetchTask.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FetchTask.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.util.concurrent.Callable;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FileSystemBackendCheckpointingModule.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FileSystemBackendCheckpointingModule.java
index dd984e6..ca23c79 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FileSystemBackendCheckpointingModule.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/FileSystemBackendCheckpointingModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import com.google.inject.AbstractModule;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/LoggingStorageCallbackFactory.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/LoggingStorageCallbackFactory.java
index 28279ba..2da4892 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/LoggingStorageCallbackFactory.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/LoggingStorageCallbackFactory.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
import org.apache.log4j.Logger;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/NoOpCheckpointingFramework.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/NoOpCheckpointingFramework.java
index 10387ce..86afc61 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/NoOpCheckpointingFramework.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/NoOpCheckpointingFramework.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import org.apache.s4.core.ProcessingElement;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SafeKeeper.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SafeKeeper.java
index 3b3ec6b..3ffc91b 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SafeKeeper.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SafeKeeper.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
import java.lang.Thread.UncaughtExceptionHandler;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SaveStateTask.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SaveStateTask.java
index 79931a5..fba488a 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SaveStateTask.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SaveStateTask.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
import java.util.concurrent.ExecutionException;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SerializeTask.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SerializeTask.java
index 6cdebb4..6966541 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SerializeTask.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/SerializeTask.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
import java.util.concurrent.Callable;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StateStorage.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StateStorage.java
index cbe2bda..c0251f1 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StateStorage.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StateStorage.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
import java.util.Set;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallback.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallback.java
index 6676dd0..8e970e3 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallback.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallback.java
@@ -15,20 +15,23 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
+import org.apache.s4.core.ft.CheckpointingFramework.StorageResultCode;
+
/**
- *
+ *
* Callback for reporting the result of an asynchronous storage operation
- *
+ *
*/
public interface StorageCallback {
/**
* Notifies the result of a storage operation
- *
+ *
* @param resultCode
- * code for the result : {@link SafeKeeper.StorageResultCode SafeKeeper.StorageResultCode}
+ * code for the result : {@link StorageResultCode}
* @param message
* whatever message object is suitable
*/
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallbackFactory.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallbackFactory.java
index c448399..e1eb323 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallbackFactory.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/StorageCallbackFactory.java
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
package org.apache.s4.core.ft;
/**
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/package-info.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/package-info.java
new file mode 100644
index 0000000..1321099
--- /dev/null
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/ft/package-info.java
@@ -0,0 +1,40 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * <p>This package contains classes for providing some fault tolerance
+ * to S4 PEs.</p>
+ * <p>The current approach is based on <b>checkpointing</b>.</p>
+ * <p>Checkpoints are taken periodically (configurable by time or
+ * frequency of application events), and when restarting an S4 node,
+ * saved checkpoints are automatically and lazily restored.</p>
+ * <p><b>Lazy restoration</b> is triggered by an application event to a PE
+ * that has not yet been restored.</p>
+ * <p>Checkpoints are stored in storage backends. Storage backends may
+ * implement eager techniques to prefetch checkpoint data to be
+ * recovered. Storage backends can be implemented for various kinds of systems,
+ * and must implement the {@link org.apache.s4.core.ft.StateStorage} interface.
+ * They are pluggable throught the module system.
+ * <p>
+ * The application programmer must take care of marking as <b>transient</b>
+ * the fields that do not have to be persisted (or cannot be persisted).
+ * <p>Storage backends are pluggable and we provide some default
+ * implementations in this package</p>
+ */
+package org.apache.s4.core.ft;
+
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcher.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcher.java
index 6cc662e..3b647bd 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcher.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcher.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.gen;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcherGenerator.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcherGenerator.java
index e225bde..c0f07a7 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcherGenerator.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/OverloadDispatcherGenerator.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.gen;
import static org.objectweb.asm.Opcodes.ACC_PUBLIC;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/package-info.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/package-info.java
new file mode 100644
index 0000000..602fa68
--- /dev/null
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/gen/package-info.java
@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Utility classes for dynamic code generation, in particular proxies to enable dispatching
+ * of events to methods of processing elements based on the runtime type of the event.
+ */
+package org.apache.s4.core.gen;
\ No newline at end of file
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/package-info.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/package-info.java
new file mode 100644
index 0000000..d10ad56
--- /dev/null
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/package-info.java
@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Key classes of the S4 platform, implementing concepts such as Stream, Processing Element.
+ */
+package org.apache.s4.core;
+
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/util/ParametersInjectionModule.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/util/ParametersInjectionModule.java
index 83914bd..a20d3e6 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/util/ParametersInjectionModule.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/util/ParametersInjectionModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.util;
import java.util.Map;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/util/package-info.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/util/package-info.java
new file mode 100644
index 0000000..830d7f2
--- /dev/null
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/util/package-info.java
@@ -0,0 +1,22 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Utility classes.
+ */
+package org.apache.s4.core.util;
\ No newline at end of file
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/AbstractSlidingWindowPE.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/AbstractSlidingWindowPE.java
index f26ede6..3c3cd67 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/AbstractSlidingWindowPE.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/AbstractSlidingWindowPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.window;
import java.util.Collection;
@@ -52,8 +70,8 @@
/**
*
- * Constructor for the event-based slot. The abstract method {@link #addPeriodicSlot()} must be called by the
- * concrete class.
+ * Constructor for the event-based slot. The abstract method {@link #addSlot()} must be called by the concrete
+ * class.
*
* @param app
* the application
@@ -65,7 +83,7 @@
}
/**
- * Constructor for time-based slots. The abstract method {@link #addPeriodicSlot()} is called periodically.
+ * Constructor for time-based slots. The abstract method {@link #addSlot()} is called periodically.
*
* @param app
* the application
@@ -130,15 +148,12 @@
* User provided function that evaluates the whole content of the window. It must iterate across all slots. Current
* slots are passed as a parameter and the PE instance is expected to be locked so that iteration over the slots is
* safe.
- *
- * @return
*/
abstract protected V evaluateWindow(Collection<T> slots);
/**
- * Add an object to the sliding window. Use it when the window is not periodic.
- *
- * @param slot
+ * Add a slot to the sliding window. Called automatically for periodic slots. Use it when the window is not
+ * periodic.
*/
protected final void addSlot() {
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/DefaultAggregatingSlot.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/DefaultAggregatingSlot.java
index 61af0e6..4d28a9a 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/DefaultAggregatingSlot.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/DefaultAggregatingSlot.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.window;
import java.util.ArrayList;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/OHCLSlot.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/OHCLSlot.java
index 17410d3..6860f2d 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/OHCLSlot.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/OHCLSlot.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.window;
public class OHCLSlot implements Slot<Double> {
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/Slot.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/Slot.java
index 839639f..2d23b44 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/Slot.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/Slot.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.window;
/**
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/SlotFactory.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/SlotFactory.java
index d40bc92..d4019a6 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/SlotFactory.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/SlotFactory.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.window;
/**
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/core/window/package-info.java b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/package-info.java
new file mode 100644
index 0000000..b6e13f0
--- /dev/null
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/core/window/package-info.java
@@ -0,0 +1,25 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Provides facilities for processing events as samples, through time or count-based windows.
+ * Currently we only provide a facility for sliding windows.
+ *
+ * NOTE: we are still working on improving the API here.
+ */
+package org.apache.s4.core.window;
\ No newline at end of file
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentFailedException.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentFailedException.java
index 877e885..86bead9 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentFailedException.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentFailedException.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
/**
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentManager.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentManager.java
index 37fb412..e92e97d 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentManager.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DeploymentManager.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
/**
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DistributedDeploymentManager.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DistributedDeploymentManager.java
index af94405..9fc5c53 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DistributedDeploymentManager.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/DistributedDeploymentManager.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import java.io.File;
@@ -61,10 +79,10 @@
boolean deployed = false;
@Inject
- public DistributedDeploymentManager(@Named("cluster.name") String clusterName,
- @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout, Server server) {
+ public DistributedDeploymentManager(@Named("s4.cluster.name") String clusterName,
+ @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout, Server server) {
this.clusterName = clusterName;
this.server = server;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/FileSystemS4RFetcher.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/FileSystemS4RFetcher.java
index 1a0847c..8947998 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/FileSystemS4RFetcher.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/FileSystemS4RFetcher.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import java.io.File;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/HttpS4RFetcher.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/HttpS4RFetcher.java
index 0dae7b6..1bb4c01 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/HttpS4RFetcher.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/HttpS4RFetcher.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import java.io.File;
@@ -41,11 +59,12 @@
* Fetches S4R archive through HTTP.
* </p>
* <p>
- * The underlying implementation uses Netty, and borrows code from the Netty snoop example:</br>
+ * The underlying implementation uses Netty, and borrows code from the Netty snoop example.</br>
*
- * {@link http://docs.jboss.org/netty/3.2/xref/org/jboss/netty/example/http/snoop/package-summary.html}
+ * @see <a href="http://docs.jboss.org/netty/3.2/xref/org/jboss/netty/example/http/snoop/package-summary.html">Netty
+ * snoop example</a>
*
- * </p>
+ * </p>
*/
public class HttpS4RFetcher implements S4RFetcher {
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/NoOpDeploymentManager.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/NoOpDeploymentManager.java
index f0b230e..27777d8 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/NoOpDeploymentManager.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/NoOpDeploymentManager.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
/**
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/S4RFetcher.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/S4RFetcher.java
index dfee06f..9c6590f 100644
--- a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/S4RFetcher.java
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/S4RFetcher.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import java.io.InputStream;
diff --git a/subprojects/s4-core/src/main/java/org/apache/s4/deploy/package-info.java b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/package-info.java
new file mode 100644
index 0000000..7afa272
--- /dev/null
+++ b/subprojects/s4-core/src/main/java/org/apache/s4/deploy/package-info.java
@@ -0,0 +1,23 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Dynamic application deployment framework, that allows identifying new available applications,
+ * downloading them through various protocols, loading them and starting them in the current S4 node.
+ */
+package org.apache.s4.deploy;
\ No newline at end of file
diff --git a/subprojects/s4-core/src/main/resources/default.s4.core.properties b/subprojects/s4-core/src/main/resources/default.s4.core.properties
index a078f75..cb5e20a 100644
--- a/subprojects/s4-core/src/main/resources/default.s4.core.properties
+++ b/subprojects/s4-core/src/main/resources/default.s4.core.properties
@@ -1,2 +1 @@
s4.logger_level = DEBUG
-appsDir=/tmp/deploy-test
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/base/EventAttributeTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/base/EventAttributeTest.java
index 51376bd..2c1b173 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/base/EventAttributeTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/base/EventAttributeTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
import java.util.HashMap;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/base/SimpleEvent.java b/subprojects/s4-core/src/test/java/org/apache/s4/base/SimpleEvent.java
index 475b754..f6e8141 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/base/SimpleEvent.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/base/SimpleEvent.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.base;
public class SimpleEvent extends Event {
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/TestCircularFifoBuffer.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/TestCircularFifoBuffer.java
index 88fb157..2714933 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/TestCircularFifoBuffer.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/TestCircularFifoBuffer.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import org.apache.commons.collections15.buffer.CircularFifoBuffer;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/TriggerTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/TriggerTest.java
index b3ce9ed..97b025c 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/TriggerTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/TriggerTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core;
import java.io.IOException;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingModuleWithUnrespondingFetchingStorageBackend.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingModuleWithUnrespondingFetchingStorageBackend.java
index 431e31e..9c95694 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingModuleWithUnrespondingFetchingStorageBackend.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingModuleWithUnrespondingFetchingStorageBackend.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import org.apache.s4.core.ft.FileSystemBasedBackendWithZKStorageCallbackCheckpointingModule.DummyZKStorageCallbackFactory;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingTest.java
index a9649e5..6001595 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/CheckpointingTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.io.File;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountApp.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountApp.java
index 770b219..50a5a8c 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountApp.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import org.apache.s4.core.ft.CheckpointingConfig.CheckpointingMode;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountTest.java
index 05a38be..c3d30b7 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FTWordCountTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.io.IOException;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FileSystemBasedBackendWithZKStorageCallbackCheckpointingModule.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FileSystemBasedBackendWithZKStorageCallbackCheckpointingModule.java
index 3787ef7..6b8dc4a 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FileSystemBasedBackendWithZKStorageCallbackCheckpointingModule.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/FileSystemBasedBackendWithZKStorageCallbackCheckpointingModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import org.apache.s4.core.ft.CheckpointingFramework.StorageResultCode;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/RecoveryTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/RecoveryTest.java
index 0566b3b..f2b0297 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/RecoveryTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/RecoveryTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.io.IOException;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithCountBasedCheckpointing.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithCountBasedCheckpointing.java
index 0ad4c0d..32bac03 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithCountBasedCheckpointing.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithCountBasedCheckpointing.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import org.apache.s4.core.App;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithManualCheckpointing.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithManualCheckpointing.java
index 2971c57..e62e893 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithManualCheckpointing.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithManualCheckpointing.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import org.apache.s4.core.App;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithTimeBasedCheckpointing.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithTimeBasedCheckpointing.java
index 910f882..843b78b 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithTimeBasedCheckpointing.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/S4AppWithTimeBasedCheckpointing.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StatefulTestPE.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StatefulTestPE.java
index 70e321d..f68289f 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StatefulTestPE.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StatefulTestPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.io.IOException;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StorageWithUnrespondingFetching.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StorageWithUnrespondingFetching.java
index 2906fe8..3c7c34c 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StorageWithUnrespondingFetching.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/ft/StorageWithUnrespondingFetching.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.ft;
import java.util.Set;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/A.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/A.java
index 7ae257d..7d7aa7e 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/A.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/A.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/B.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/B.java
index 1d62b4a..b1feec3 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/B.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/B.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
import org.apache.s4.core.ProcessingElement;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/C.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/C.java
index da16bf7..ed2d60f 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/C.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/C.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
import org.apache.s4.core.ProcessingElement;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/D.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/D.java
index 93167ed..2abc128 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/D.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/D.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1.java
index ef441ad..2c62be1 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1a.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1a.java
index 29392f2..9988b37 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1a.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event1a.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event2.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event2.java
index 98cfee0..eae5ecd 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event2.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/Event2.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/OverloadDispatcherTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/OverloadDispatcherTest.java
index 627df67..050afb3 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/OverloadDispatcherTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/overloadgen/OverloadDispatcherTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.overloadgen;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/timers/MultithreadingTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/timers/MultithreadingTest.java
index e1c0c11..903f288 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/timers/MultithreadingTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/timers/MultithreadingTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.timers;
import static org.junit.Assert.assertFalse;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/CountTriggerTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/CountTriggerTest.java
index c6f840a..372bf99 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/CountTriggerTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/CountTriggerTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.triggers;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/NoTriggerTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/NoTriggerTest.java
index f5a5a45..493713e 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/NoTriggerTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/NoTriggerTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.triggers;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TimeTriggerTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TimeTriggerTest.java
index e2d0f87..9b0f00d 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TimeTriggerTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TimeTriggerTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.triggers;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggerablePE.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggerablePE.java
index 6f1cc9f..e4a4812 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggerablePE.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggerablePE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.triggers;
import java.io.IOException;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggeredApp.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggeredApp.java
index e2a1b38..747ec7e 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggeredApp.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/triggers/TriggeredApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.triggers;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPE1.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPE1.java
index 76e60b8..b1676dc 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPE1.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPE1.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.windowing;
import java.util.ArrayList;
@@ -11,6 +29,7 @@
import org.apache.s4.core.window.AbstractSlidingWindowPE;
import org.apache.s4.core.window.DefaultAggregatingSlot;
import org.apache.s4.core.window.SlotFactory;
+import org.apache.s4.core.windowing.WindowingPETest;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPETest.java b/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPETest.java
index 80ba837..78d814a 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPETest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/core/windowing/WindowingPETest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.core.windowing;
import java.util.ArrayList;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/AppConstants.java b/subprojects/s4-core/src/test/java/org/apache/s4/deploy/AppConstants.java
index a12b6ea..b727f1e 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/AppConstants.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/deploy/AppConstants.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
public class AppConstants {
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/TestAutomaticDeployment.java b/subprojects/s4-core/src/test/java/org/apache/s4/deploy/TestAutomaticDeployment.java
index f129dae..1fbd37a 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/TestAutomaticDeployment.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/deploy/TestAutomaticDeployment.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import java.io.File;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/TestModule.java b/subprojects/s4-core/src/test/java/org/apache/s4/deploy/TestModule.java
deleted file mode 100644
index ff3fa02..0000000
--- a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/TestModule.java
+++ /dev/null
@@ -1,67 +0,0 @@
-package org.apache.s4.deploy;
-
-import java.io.InputStream;
-
-import org.apache.commons.configuration.ConfigurationConverter;
-import org.apache.commons.configuration.ConfigurationException;
-import org.apache.commons.configuration.ConfigurationUtils;
-import org.apache.commons.configuration.PropertiesConfiguration;
-import org.apache.s4.base.Emitter;
-import org.apache.s4.base.Hasher;
-import org.apache.s4.base.Listener;
-import org.apache.s4.base.SerializerDeserializer;
-import org.apache.s4.comm.DefaultHasher;
-import org.apache.s4.comm.serialize.KryoSerDeser;
-import org.apache.s4.comm.tcp.TCPEmitter;
-import org.apache.s4.comm.tcp.TCPListener;
-import org.apache.s4.comm.topology.Assignment;
-import org.apache.s4.comm.topology.AssignmentFromZK;
-import org.apache.s4.comm.topology.Cluster;
-import org.apache.s4.comm.topology.ClusterFromZK;
-
-import com.google.inject.AbstractModule;
-import com.google.inject.Binder;
-import com.google.inject.name.Names;
-
-public class TestModule extends AbstractModule {
-
- private PropertiesConfiguration config;
-
- private void loadProperties(Binder binder) {
-
- try {
- InputStream is = this.getClass().getResourceAsStream("/org.apache.s4.deploy.s4.properties");
- config = new PropertiesConfiguration();
- config.load(is);
- System.out.println(ConfigurationUtils.toString(config));
- // TODO - validate properties.
-
- /* Make all properties injectable. Do we need this? */
- Names.bindProperties(binder, ConfigurationConverter.getProperties(config));
- } catch (ConfigurationException e) {
- binder.addError(e);
- e.printStackTrace();
- }
- }
-
- @Override
- protected void configure() {
- if (config == null) {
- loadProperties(binder());
- }
- bind(Cluster.class);
- bind(Hasher.class).to(DefaultHasher.class);
- bind(SerializerDeserializer.class).to(KryoSerDeser.class);
- bind(Assignment.class).to(AssignmentFromZK.class);
- bind(Cluster.class).to(ClusterFromZK.class);
- bind(Emitter.class).to(TCPEmitter.class);
- bind(Listener.class).to(TCPListener.class);
-
- bind(Integer.class).annotatedWith(Names.named("comm.retries")).toInstance(10);
- bind(Integer.class).annotatedWith(Names.named("comm.retry_delay")).toInstance(10);
- bind(Integer.class).annotatedWith(Names.named("comm.timeout")).toInstance(1000);
-
- bind(Integer.class).annotatedWith(Names.named("tcp.partition.queue_size")).toInstance(256);
- }
-
-}
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/prodcon/TestProducerConsumer.java b/subprojects/s4-core/src/test/java/org/apache/s4/deploy/prodcon/TestProducerConsumer.java
index 2e05b7a..442bbcd 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/deploy/prodcon/TestProducerConsumer.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/deploy/prodcon/TestProducerConsumer.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy.prodcon;
import java.io.File;
@@ -28,7 +46,6 @@
import org.junit.BeforeClass;
import org.junit.Test;
-import com.google.common.io.ByteStreams;
import com.google.common.io.Files;
import com.google.common.io.Resources;
import com.sun.net.httpserver.HttpServer;
@@ -61,20 +78,6 @@
}
@Before
- public void cleanLocalAppsDir() throws ConfigurationException, IOException {
- PropertiesConfiguration config = loadConfig();
-
- if (!new File(config.getString("appsDir")).exists()) {
- Assert.assertTrue(new File(config.getString("appsDir")).mkdirs());
- } else {
- if (!config.getString("appsDir").startsWith("/tmp")) {
- Assert.fail("apps dir should a subdir of /tmp for safety");
- }
- CommTestUtils.deleteDirectoryContents(new File(config.getString("appsDir")));
- }
- }
-
- @Before
public void prepare() throws Exception {
CommTestUtils.cleanupTmpDirs();
zookeeperServerConnectionFactory = CommTestUtils.startZookeeperServer();
@@ -104,18 +107,10 @@
@Test
public void testInitialDeploymentFromFileSystem() throws Exception {
- File producerS4R = new File(loadConfig().getString("appsDir") + File.separator + "producer"
- + System.currentTimeMillis() + ".s4r");
- System.out.println(tmpAppsDir.getAbsolutePath());
- Assert.assertTrue(ByteStreams.copy(Files.newInputStreamSupplier(new File(tmpAppsDir.getAbsolutePath()
- + "/producer-app-0.0.0-SNAPSHOT.s4r")), Files.newOutputStreamSupplier(producerS4R)) > 0);
+ File producerS4R = new File(tmpAppsDir, "producer-app-0.0.0-SNAPSHOT.s4r");
String uriProducer = producerS4R.toURI().toString();
- File consumerS4R = new File(loadConfig().getString("appsDir") + File.separator + "consumer"
- + System.currentTimeMillis() + ".s4r");
- Assert.assertTrue(ByteStreams.copy(Files.newInputStreamSupplier(new File(tmpAppsDir.getAbsolutePath()
- + "/consumer-app-0.0.0-SNAPSHOT.s4r")), Files.newOutputStreamSupplier(consumerS4R)) > 0);
-
+ File consumerS4R = new File(tmpAppsDir, "consumer-app-0.0.0-SNAPSHOT.s4r");
String uriConsumer = consumerS4R.toURI().toString();
initializeS4Node();
@@ -160,10 +155,6 @@
}
private void initializeS4Node() throws ConfigurationException, IOException, InterruptedException, KeeperException {
- // 0. package s4 app
- // TODO this is currently done offline, and the app contains the TestApp class copied from the one in the
- // current package .
-
// 1. start s4 nodes. Check that no app is deployed.
TaskSetup taskSetup = new TaskSetup("localhost:" + CommTestUtils.ZK_PORT);
taskSetup.clean("s4");
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/AssignmentFromZKNoFailFast.java b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/AssignmentFromZKNoFailFast.java
index 4744e03..e8b7e87 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/AssignmentFromZKNoFailFast.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/AssignmentFromZKNoFailFast.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import org.apache.s4.comm.topology.AssignmentFromZK;
@@ -9,10 +27,10 @@
public class AssignmentFromZKNoFailFast extends AssignmentFromZK {
@Inject
- public AssignmentFromZKNoFailFast(@Named("cluster.name") String clusterName,
- @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
+ public AssignmentFromZKNoFailFast(@Named("s4.cluster.name") String clusterName,
+ @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
super(clusterName, zookeeperAddress, sessionTimeout, connectionTimeout);
}
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClusterFromZKNoFailFast.java b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClusterFromZKNoFailFast.java
index f600c2f..025fb75 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClusterFromZKNoFailFast.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClusterFromZKNoFailFast.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import org.apache.s4.comm.topology.ClusterFromZK;
@@ -9,10 +27,10 @@
public class ClusterFromZKNoFailFast extends ClusterFromZK {
@Inject
- public ClusterFromZKNoFailFast(@Named("cluster.name") String clusterName,
- @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
+ public ClusterFromZKNoFailFast(@Named("s4.cluster.name") String clusterName,
+ @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
super(clusterName, zookeeperAddress, sessionTimeout, connectionTimeout);
}
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClustersFromZKNoFailFast.java b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClustersFromZKNoFailFast.java
index 3a72050..4ea5644 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClustersFromZKNoFailFast.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/ClustersFromZKNoFailFast.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import org.apache.s4.comm.topology.ClustersFromZK;
@@ -9,10 +27,10 @@
public class ClustersFromZKNoFailFast extends ClustersFromZK {
@Inject
- public ClustersFromZKNoFailFast(@Named("cluster.name") String clusterName,
- @Named("cluster.zk_address") String zookeeperAddress,
- @Named("cluster.zk_session_timeout") int sessionTimeout,
- @Named("cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
+ public ClustersFromZKNoFailFast(@Named("s4.cluster.name") String clusterName,
+ @Named("s4.cluster.zk_address") String zookeeperAddress,
+ @Named("s4.cluster.zk_session_timeout") int sessionTimeout,
+ @Named("s4.cluster.zk_connection_timeout") int connectionTimeout) throws Exception {
super(clusterName, zookeeperAddress, sessionTimeout, connectionTimeout);
}
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/CoreTestUtils.java b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/CoreTestUtils.java
index 91cd799..7e88dd8 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/CoreTestUtils.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/CoreTestUtils.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import java.io.File;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCommModule.java b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCommModule.java
index a9a72ce..75143c1 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCommModule.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCommModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import org.apache.s4.base.Hasher;
@@ -19,7 +37,7 @@
/**
* Mock module for the comm layer. Mocks comm layer basic functionalities, and uses some default when required.
- *
+ *
*/
public class MockCommModule extends AbstractModule {
@@ -41,7 +59,7 @@
Assignment mockedAssignment = Mockito.mock(Assignment.class);
Mockito.when(mockedAssignment.assignClusterNode()).thenReturn(new ClusterNode(0, 0, "machine", "Task-0"));
bind(Assignment.class).toInstance(mockedAssignment);
- Names.bindProperties(binder(), ImmutableMap.of("cluster.name", "testCluster"));
+ Names.bindProperties(binder(), ImmutableMap.of("s4.cluster.name", "testCluster"));
}
}
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCoreModule.java b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCoreModule.java
index 7c52c94..71bae7a 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCoreModule.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/MockCoreModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import org.apache.s4.base.Emitter;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/NonFailFastZookeeperClientsModule.java b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/NonFailFastZookeeperClientsModule.java
index 790d591..b272a35 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/NonFailFastZookeeperClientsModule.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/fixtures/NonFailFastZookeeperClientsModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.fixtures;
import org.apache.s4.comm.topology.Assignment;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueEvent.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueEvent.java
index 8bd4ddf..513aea1 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueEvent.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueEvent.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import org.apache.s4.wordcount.StringEvent;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueKeyFinder.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueKeyFinder.java
index 28ee8d5..a383a7d 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueKeyFinder.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/KeyValueKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/SentenceKeyFinder.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/SentenceKeyFinder.java
index b234ff3..667afa6 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/SentenceKeyFinder.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/SentenceKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import java.util.List;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/StringEvent.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/StringEvent.java
index 13db453..dd69366 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/StringEvent.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/StringEvent.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/Word.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/Word.java
index eacb80a..384265c 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/Word.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/Word.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordClassifierPE.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordClassifierPE.java
index 107f271..b69c6fe 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordClassifierPE.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordClassifierPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import java.io.File;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountApp.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountApp.java
index cda0ab4..094680b 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountApp.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountEvent.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountEvent.java
index fa288d3..f432cd4 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountEvent.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountEvent.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountKeyFinder.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountKeyFinder.java
index 695a915..4ff80d0 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountKeyFinder.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountModule.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountModule.java
index 9907bee..6cdeeea 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountModule.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountModule.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import com.google.inject.AbstractModule;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountTest.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountTest.java
index 0484b57..db45fcb 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountTest.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountTest.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import java.io.IOException;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCounterPE.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCounterPE.java
index 2bb3a82..c9d8635 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCounterPE.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCounterPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import org.apache.s4.core.App;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenEvent.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenEvent.java
index 932f559..b784cde 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenEvent.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenEvent.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenKeyFinder.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenKeyFinder.java
index b089dac..741ba77 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenKeyFinder.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSeenKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
diff --git a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSplitterPE.java b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSplitterPE.java
index b2a43a9..7423b13 100644
--- a/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSplitterPE.java
+++ b/subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordSplitterPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.wordcount;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-edsl/s4-edsl.gradle b/subprojects/s4-edsl/s4-edsl.gradle
index 71f232e..cb96e79 100644
--- a/subprojects/s4-edsl/s4-edsl.gradle
+++ b/subprojects/s4-edsl/s4-edsl.gradle
@@ -1,6 +1,21 @@
-// Use the Diezel Maven plugin to build the project.
-// Gradle doesn't support Maven plugin as of version 1.0
-// Useful article: http://forums.gradle.org/gradle/topics/how_to_download_and_evoke_a_maven_plugin
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
def generatedSourceDir = "${buildDir}/generated-src/java"
def diezelSrcDir = "${projectDir}/src/main/diezel";
diff --git a/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/AppBuilder.java b/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/AppBuilder.java
index e823101..fb40b43 100644
--- a/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/AppBuilder.java
+++ b/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/AppBuilder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import java.lang.reflect.Array;
@@ -65,7 +83,6 @@
* <p>
* See the <a href="http://code.google.com/p/diezel">Diezel</a> project for details.
*
- * @author Leo Neumeyer (@leoneu)
*/
public class AppBuilder extends App {
diff --git a/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/StreamBuilder.java b/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/StreamBuilder.java
index 82976e0..c68ef89 100644
--- a/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/StreamBuilder.java
+++ b/subprojects/s4-edsl/src/main/java/org/apache/s4/edsl/StreamBuilder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import java.util.Set;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/DurationKeyFinder.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/DurationKeyFinder.java
index 78bddb0..767a553 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/DurationKeyFinder.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/DurationKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import java.util.ArrayList;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventA.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventA.java
index c881ea5..bf03acd 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventA.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventA.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventB.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventB.java
index 0ebe91e..236751a 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventB.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/EventB.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/HeightKeyFinder.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/HeightKeyFinder.java
index 4929344..86f3180 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/HeightKeyFinder.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/HeightKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import java.util.ArrayList;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/MyApp.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/MyApp.java
index ad40e86..45e4316 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/MyApp.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/MyApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEX.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEX.java
index c142e03..f976f7d 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEX.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEX.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import org.apache.s4.core.App;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEY.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEY.java
index ce9dbc7..29680d8 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEY.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEY.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import org.apache.s4.core.App;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEZ.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEZ.java
index d494ae2..0183801 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEZ.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/PEZ.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import org.apache.s4.core.App;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/QueryKeyFinder.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/QueryKeyFinder.java
index 1338432..306944a 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/QueryKeyFinder.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/QueryKeyFinder.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import java.util.ArrayList;
diff --git a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/TestEDSL.java b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/TestEDSL.java
index 8225ee4..1b9f310 100644
--- a/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/TestEDSL.java
+++ b/subprojects/s4-edsl/src/test/java/org/apache/s4/edsl/TestEDSL.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.edsl;
import java.lang.reflect.Field;
diff --git a/subprojects/s4-example/s4-example.gradle b/subprojects/s4-example/s4-example.gradle
index 95f3a97..f1c663c 100644
--- a/subprojects/s4-example/s4-example.gradle
+++ b/subprojects/s4-example/s4-example.gradle
@@ -1,26 +1,38 @@
-/*
- * Copyright 2010 the original author or authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-description = 'Examples of S4 applications.'
-
-dependencies {
- compile project( ":s4-base" )
- compile project( ":s4-core" )
- compile project( ":s4-comm" )
- compile project( ":s4-edsl" )
- compile libraries.ejml
- compile libraries.junit
-}
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+
+description = 'Examples of S4 applications.'
+
+
+
+dependencies {
+
+ compile project( ":s4-base" )
+
+ compile project( ":s4-core" )
+
+ compile project( ":s4-comm" )
+
+ compile project( ":s4-edsl" )
+
+ compile libraries.junit
+
+}
+
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/AgeKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/AgeKeyFinder.java
index b03ca5f..7ebb5f9 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/AgeKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/AgeKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountEvent.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountEvent.java
index b80b863..0d62f58 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountEvent.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountEvent.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountKeyFinder.java
index a3aa733..831f8ed 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CountKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CounterPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CounterPE.java
index 35c0583..a5e895b 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CounterPE.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/CounterPE.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenderKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenderKeyFinder.java
index 361d760..7f50e33 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenderKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenderKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenerateUserEventPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenerateUserEventPE.java
index 6016433..70238e9 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenerateUserEventPE.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/GenerateUserEventPE.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/Module.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/Module.java
index a4ec6d6..14d3483 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/Module.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/Module.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
*
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
*
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import java.io.InputStream;
@@ -29,9 +32,7 @@
/**
* Configures the controller.
*
- * Reads a properties file, provides a {@link Communicator} singleton, and configures Guice bindings.
*
- * @author Leo Neumeyer
*/
public class Module extends AbstractModule {
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/MyApp.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/MyApp.java
index d68a41a..0209933 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/MyApp.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/MyApp.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
*
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
*
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import java.util.concurrent.TimeUnit;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/PrintPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/PrintPE.java
index 29837d4..e6e9d2b 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/PrintPE.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/PrintPE.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import javax.annotation.concurrent.ThreadSafe;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md
index 72e0a3d..ebe6228 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/README.md
@@ -1,4 +1,4 @@
-S4 Counter Example
+S4 Counter Example (NOT updated for 0.5.0)
==================
In this example we do the following:
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserEvent.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserEvent.java
index 73ea8ac..95f524e 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserEvent.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserEvent.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserIDKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserIDKeyFinder.java
index 0cbef15..8e7f3d8 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserIDKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/counter/UserIDKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/AgeKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/AgeKeyFinder.java
index f4d894c..6950cff 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/AgeKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/AgeKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountEvent.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountEvent.java
index ac8e8ac..93adb60 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountEvent.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountEvent.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountKeyFinder.java
index 4fd5eb8..cc2c2aa 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CountKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterApp.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterApp.java
index ef5cd94..aff5f49 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterApp.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterApp.java
@@ -1,4 +1,21 @@
-// NOTE: this is commented until we fix the dependency to the classes generated by the edsl subproject
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
//package org.apache.s4.example.edsl.counter;
//
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterPE.java
index 33e383b..8aa43a1 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterPE.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/CounterPE.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenderKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenderKeyFinder.java
index 708f862..7071fdf 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenderKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenderKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenerateUserEventPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenerateUserEventPE.java
index a68a048..7367ef9 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenerateUserEventPE.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/GenerateUserEventPE.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/PrintPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/PrintPE.java
index bf841f4..bf2ae35 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/PrintPE.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/PrintPE.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserEvent.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserEvent.java
index f2e3bc6..4560d1e 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserEvent.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserEvent.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserIDKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserIDKeyFinder.java
index 3fded2a..3009dc7 100644
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserIDKeyFinder.java
+++ b/subprojects/s4-example/src/main/java/org/apache/s4/example/edsl/counter/UserIDKeyFinder.java
@@ -1,18 +1,21 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
*/
+
package org.apache.s4.example.edsl.counter;
import java.util.ArrayList;
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ClassIDKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ClassIDKeyFinder.java
deleted file mode 100644
index ea3bf77..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ClassIDKeyFinder.java
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.s4.base.KeyFinder;
-
-public class ClassIDKeyFinder implements KeyFinder<ObsEvent> {
-
- @Override
- public List<String> get(ObsEvent event) {
-
- List<String> results = new ArrayList<String>();
-
- /* Retrieve the user ID and add it to the list. */
- results.add(Integer.toString(event.getClassId()));
-
- return results;
- }
-
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Controller.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Controller.java
deleted file mode 100644
index 4571157..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Controller.java
+++ /dev/null
@@ -1,244 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import java.io.FileNotFoundException;
-import java.io.IOException;
-import java.util.HashMap;
-import java.util.Map;
-import java.util.concurrent.TimeUnit;
-
-import org.apache.s4.core.Receiver;
-import org.apache.s4.core.Sender;
-import org.apache.s4.model.Model;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import ch.qos.logback.classic.Level;
-
-import com.google.inject.Inject;
-import com.google.inject.name.Named;
-
-/*
- * Train a classifier, run a test, compute the accuracy of the classifier.
- */
-public class Controller {
-
- private static final Logger logger = LoggerFactory.getLogger(Controller.class);
-
- final private String trainFilename;
- final private String testFilename;
- final private long numTrainVectors;
- final private long numTestVectors;
- final private int numIterations;
- final private int outputInterval;
- final private Model model;
- final private int vectorSize;
- private int numClasses;
- final private Sender sender;
- final private Receiver receiver;
-
- @Inject
- private Controller(@Named("model.train_data") String trainFilename, @Named("model.test_data") String testFilename,
- Model model, @Named("model.vector_size") int vectorSize, @Named("model.num_iterations") int numIterations,
- @Named("model.output_interval_in_seconds") int outputInterval,
- @Named("model.logger.level") String logLevel, Sender sender, Receiver receiver) {
-
- this.trainFilename = trainFilename;
- this.testFilename = testFilename;
- this.numTrainVectors = getNumLines(trainFilename);
- this.numTestVectors = getNumLines(testFilename);
- this.numIterations = numIterations;
- this.vectorSize = vectorSize;
- this.outputInterval = outputInterval;
- this.model = model;
- this.sender = sender;
- this.receiver = receiver;
-
- logger.info("Number of test vectors is " + numTestVectors);
- logger.info("Number of train vectors is " + numTrainVectors);
- ch.qos.logback.classic.Logger root = (ch.qos.logback.classic.Logger) LoggerFactory
- .getLogger(Logger.ROOT_LOGGER_NAME);
- root.setLevel(Level.toLevel(logLevel));
- }
-
- public void start() {
-
- logger.info("Processing file: " + trainFilename);
- try {
-
- /* Get vector size and number of classes from data set. */
- getDataSetInfo(trainFilename);
-
- MyApp app = new MyApp(numClasses, numTrainVectors, model, outputInterval, TimeUnit.SECONDS);
-
- // app.setCommLayer(sender, receiver);
-
- logger.info("Init app.");
- app.initApp();
-
- /* Initialize modelPEs by injecting one dummy events per class. */
- for (int i = 0; i < numClasses; i++) {
- ObsEvent obsEvent = new ObsEvent(-1, new float[vectorSize], -Float.MAX_VALUE, i, -1, true);
- app.injectByKey(obsEvent);
- }
-
- /* Wait until the app is initialized. */
- while (!app.isInited()) {
- Thread.sleep(1);
- }
-
- long start = System.nanoTime();
- for (int i = 0; i < numIterations; i++) {
- logger.info("Starting iteration {}.", i);
- injectData(app, true, trainFilename);
-
- /*
- * Make sure all the data has been processed.
- */
- while (!app.isTrained(i)) {
- Thread.sleep(5);
- }
- }
- long stop = System.nanoTime();
- long trainTime = stop - start;
-
- /* Start testing. */
- logger.info("Start testing.");
- start = System.nanoTime();
- injectData(app, false, testFilename);
- stop = System.nanoTime();
- long testTime = stop - start;
-
- while (!app.isTested(numTestVectors)) {
- Thread.sleep(5);
- }
-
- /* Print final report. */
- logger.info(app.getReport());
-
- /* Print timing info. */
- logger.info("Total training time was {} seconds.", trainTime / 1000000000);
- logger.info("Training time per observation was {} microseconds.", trainTime / numTrainVectors / 1000);
- logger.info("Training time per observation per iteration was {} microseconds.", trainTime / numTrainVectors
- / numIterations / 1000);
- logger.info("Total testing time was {} seconds.", testTime / 1000000000);
- logger.info("Testing time per observation was {} microseconds.", testTime / numTrainVectors / 1000);
-
- /* Done. */
- app.closeApp();
-
- } catch (FileNotFoundException e) {
- e.printStackTrace();
- logger.error(e.getMessage());
- } catch (IOException e) {
- e.printStackTrace();
- logger.error(e.getMessage());
- } catch (InterruptedException e) {
- e.printStackTrace();
- logger.error(e.getMessage());
- }
- }
-
- private void injectData(MyApp app, boolean isTraining, String filename) throws FileNotFoundException, IOException {
-
- DataFile data = new DataFile(filename);
- int count = 0;
- for (String line : data) {
-
- String[] result = line.split("\\s");
-
- /* Class ID range starts in 1, shift to start in zero. */
- int classID = Integer.parseInt(result[0]) - 1;
-
- float[] vector = new float[vectorSize];
- for (int j = 0; j < vectorSize; j++) {
-
- vector[j] = Float.parseFloat(result[j + 1]);
- }
- ObsEvent obsEvent = new ObsEvent(count++, vector, -Float.MAX_VALUE, classID, -1, isTraining);
- app.injectToAll(obsEvent);
- }
- data.close();
- }
-
- private void getDataSetInfo(String filename) throws FileNotFoundException, IOException {
-
- Map<Integer, Long> countsPerClass = new HashMap<Integer, Long>();
-
- DataFile data = new DataFile(filename);
-
- for (String line : data) {
-
- String[] result = line.split("\\s");
-
- /* Format is: label val1 val2 ... valN */
- if (vectorSize != result.length - 1) {
- throw new IllegalArgumentException("vectorSize: (" + vectorSize
- + ") does not match number of columns in data file (" + (result.length - 1) + ").");
- }
-
- /* Class ID range starts in 1, shift to start in zero. */
- int classID = Integer.parseInt(result[0]) - 1;
-
- /* Count num vectors per class. */
- if (!countsPerClass.containsKey(classID)) {
- countsPerClass.put(classID, 1L);
- } else {
- long count = countsPerClass.get(classID) + 1;
- countsPerClass.put(classID, count);
- }
- }
- data.close();
-
- /* Summary. */
- numClasses = countsPerClass.size();
- logger.info("Number of classes is " + numClasses);
- logger.info("Vector size is " + vectorSize);
-
- for (Map.Entry<Integer, Long> entry : countsPerClass.entrySet()) {
-
- int key = entry.getKey();
- long val = entry.getValue();
-
- logger.info("Num vectors for class ID: " + key + " is " + val);
- }
- }
-
- /*
- * @return Returns the number of lines in a text file.
- */
- private long getNumLines(String filename) {
-
- long count = 0;
- try {
- DataFile data = new DataFile(filename);
-
- for (@SuppressWarnings("unused")
- String line : data) {
- count++;
- }
- data.close();
-
- } catch (FileNotFoundException e) {
- e.printStackTrace();
- } catch (IOException e) {
- e.printStackTrace();
- }
- return count;
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/DataFile.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/DataFile.java
deleted file mode 100644
index 2fbe29a..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/DataFile.java
+++ /dev/null
@@ -1,63 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import java.util.*;
-import java.util.zip.GZIPInputStream;
-import java.io.*;
-
-public class DataFile implements Iterable<String> {
- private BufferedReader reader;
-
- public DataFile(String filePath) throws FileNotFoundException, IOException {
-
- InputStream is = this.getClass().getResourceAsStream(filePath);
-
- GZIPInputStream gzip = new GZIPInputStream(is);
-
- this.reader = new BufferedReader(new InputStreamReader(gzip));
- }
-
- public void close() throws IOException {
- reader.close();
- }
-
- public Iterator<String> iterator() {
- return new FileIterator();
- }
-
- private class FileIterator implements Iterator<String> {
- private String currentLine;
-
- public boolean hasNext() {
- try {
- currentLine = reader.readLine();
- } catch (Exception ex) {
- currentLine = null;
- }
-
- return currentLine != null;
- }
-
- public String next() {
- return currentLine;
- }
-
- public void remove() {
- }
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/HypIDKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/HypIDKeyFinder.java
deleted file mode 100644
index bf37a3a..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/HypIDKeyFinder.java
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.s4.base.KeyFinder;
-
-public class HypIDKeyFinder implements KeyFinder<ObsEvent> {
-
- @Override
- public List<String> get(ObsEvent event) {
-
- List<String> results = new ArrayList<String>();
-
- /* Retrieve the user ID and add it to the list. */
- results.add(Integer.toString(event.getHypId()));
-
- return results;
- }
-
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Main.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Main.java
deleted file mode 100644
index 067c551..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Main.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-
-public class Main {
-
- /**
- * @param args
- */
- public static void main(String[] args) {
-
- // Injector injector = Guice.createInjector(new DefaultModule());
- //
- // Controller controller = injector.getInstance(Controller.class);
- // controller.start();
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MaximizerPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MaximizerPE.java
deleted file mode 100644
index a980eb4..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MaximizerPE.java
+++ /dev/null
@@ -1,104 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import org.apache.s4.base.Event;
-import org.apache.s4.core.App;
-import org.apache.s4.core.ProcessingElement;
-import org.apache.s4.core.Stream;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-
-final public class MaximizerPE extends ProcessingElement {
-
- private static final Logger logger = LoggerFactory
- .getLogger(MaximizerPE.class);
-
- private int numClasses;
- private Stream<ObsEvent> assignmentStream;
- private int numEventsReceived = 0;
- private float maxLogProb = -Float.MAX_VALUE;
- private int hypID;
-
- public MaximizerPE(App app) {
- super(app);
- }
-
- /**
- * @param countStream
- * the countStream to set
- */
- public void setAssignmentStream(Stream<ObsEvent> stream) {
- assignmentStream = stream;
- }
-
- /**
- * @return the numClasses
- */
- public int getNumClasses() {
- return numClasses;
- }
-
- /**
- * @param numClasses
- * the numClasses to set
- */
- public void setNumClasses(int numClasses) {
- this.numClasses = numClasses;
- }
-
- public void onEvent(Event event) {
-
- ObsEvent inEvent = (ObsEvent) event;
- float[] obs = inEvent.getObsVector();
-
- if (inEvent.getProb() > maxLogProb) {
- maxLogProb = inEvent.getProb();
- hypID = inEvent.getHypId();
- }
-
- if (++numEventsReceived == numClasses) {
-
- /* Got all the distances. Send class id with minimum distance. */
- ObsEvent outEvent = new ObsEvent(inEvent.getIndex(), obs,
- maxLogProb, inEvent.getClassId(), hypID, false);
-
- logger.trace("IN: " + inEvent.toString());
- logger.trace("OUT: " + outEvent.toString());
-
- assignmentStream.put(outEvent);
-
- /* This PE instance is no longer needed. */
- close();
- }
-
- }
-
- @Override
- protected void onRemove() {
- // TODO Auto-generated method stub
-
- }
-
- @Override
- protected void onCreate() {
- // TODO Auto-generated method stub
-
- }
-
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MetricsPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MetricsPE.java
deleted file mode 100644
index eece75c..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MetricsPE.java
+++ /dev/null
@@ -1,148 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import java.util.Map;
-import java.util.HashMap;
-
-import org.apache.s4.base.Event;
-import org.apache.s4.core.App;
-import org.apache.s4.core.ProcessingElement;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-
-final public class MetricsPE extends ProcessingElement {
-
- private static final Logger logger = LoggerFactory
- .getLogger(MetricsPE.class);
-
- private Map<Integer, HashMap<Integer, MutableInt>> counts;
- private long totalCount = 0;
- private int numClasses;
-
- public MetricsPE(App app) {
- super(app);
- }
-
- public void onEvent(Event event) {
-
- ResultEvent resultEvent = (ResultEvent) event;
- int classID = resultEvent.getClassId();
- int hypID = resultEvent.getHypId();
- totalCount += 1;
-
- /* Increment counter. */
- if (!counts.containsKey(classID)) {
- counts.put(classID, new HashMap<Integer, MutableInt>());
- numClasses++;
- }
- MutableInt value = counts.get(classID).get(hypID);
- if (value == null) {
- value = new MutableInt();
- counts.get(classID).put(hypID, value);
- }
- value.inc();
- }
-
- public void onTrigger(Event event) {
- logger.info(this.toString());
- }
-
- @Override
- protected void onCreate() {
- counts = new HashMap<Integer, HashMap<Integer, MutableInt>>();
- }
-
- @Override
- protected void onRemove() {
- }
-
- /** @return number of data vectors processed. */
- public long getCount() {
- return totalCount;
- }
-
- @Override
- public String toString() {
- StringBuilder report = new StringBuilder();
- report.append("\n\nConfusion Matrix [%]:\n");
- report.append("\n ");
- for (int i = 0; i < numClasses; i++) {
- report.append(String.format("%6d", i));
- }
- report.append("\n ----------------------------------------\n");
- long truePositives = 0;
- for (Map.Entry<Integer, HashMap<Integer, MutableInt>> entry : counts
- .entrySet()) {
-
- int classID = entry.getKey();
- report.append(String.format("%5d:", classID));
- HashMap<Integer, MutableInt> hypCounts = entry.getValue();
-
- long totalCountForClass = getTotalCountForClass(hypCounts);
- float[] sortedCounts = new float[numClasses];
- for (Map.Entry<Integer, MutableInt> hypEntry : hypCounts.entrySet()) {
- int hypID = hypEntry.getKey();
- long count = hypEntry.getValue().get();
-
- /*
- * Because of timing, it is possible to have a hypId that was
- * not counted in numClasses yet. In this case we bail out and
- * without producing a report.
- */
- if (hypID > (numClasses - 1))
- return "Insufficient data.";
-
- sortedCounts[hypID] = (float) count / totalCountForClass * 100f;
- if (classID == hypID)
- truePositives += count;
- }
- for (int i = 0; i < numClasses; i++) {
- report.append(String.format("%6.1f", sortedCounts[i]));
- }
-
- report.append("\n");
- }
- report.append(String.format(
- "\nAccuracy: %6.1f%% - Num Observations: %6d\n",
- (float) truePositives / totalCount * 100f, totalCount));
-
- return report.toString();
- }
-
- private long getTotalCountForClass(HashMap<Integer, MutableInt> counts) {
-
- long count = 0;
- for (Map.Entry<Integer, MutableInt> hypEntry : counts.entrySet()) {
- count += hypEntry.getValue().get();
- }
- return count;
- }
-
- private class MutableInt {
- private int value = 0;
-
- private void inc() {
- ++value;
- }
-
- private int get() {
- return value;
- }
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ModelPE.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ModelPE.java
deleted file mode 100644
index 7be3530..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ModelPE.java
+++ /dev/null
@@ -1,223 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import net.jcip.annotations.ThreadSafe;
-
-import org.apache.s4.base.Event;
-import org.apache.s4.core.App;
-import org.apache.s4.core.ProcessingElement;
-import org.apache.s4.core.Stream;
-import org.apache.s4.model.Model;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-
-@ThreadSafe
-final public class ModelPE extends ProcessingElement {
-
- private static final Logger logger = LoggerFactory.getLogger(ModelPE.class);
-
- private long numVectors;
- private Model model;
- private Stream<ObsEvent> distanceStream;
- private Stream<ResultEvent> resultStream;
- private int modelId;
- private double logPriorProb;
- private long obsCount = 0;
- private long totalCount = 0;
- private int iteration = 0;
-
- public ModelPE(App app) {
- super(app);
- }
-
- /**
- * @param numVectors the numVectors to set
- */
- public void setNumVectors(long numVectors) {
- this.numVectors = numVectors;
- }
-
- /**
- * @return the number of training vectors.
- */
- public long getNumVectors() {
- return numVectors;
- }
-
- /**
- * @param model
- * the model to set
- */
- public void setModel(Model model) {
- this.model = model;
- }
-
- /**
- * @return the model
- */
- public Model getModel() {
- return model;
- }
-
- /**
- * Set the output streams.
- *
- * @param distanceStream
- * sends an {@link ObsEvent} to the {@link MaximizerPE}.
- * @param resultStream
- * sends a {@link ResultEvent} to the {@link MetricsPE}.
- */
- public void setStream(Stream<ObsEvent> distanceStream,
- Stream<ResultEvent> resultStream) {
-
- /* Init prototype. */
- this.distanceStream = distanceStream;
- this.resultStream = resultStream;
- }
-
- /**
- * @return number of observation vectors used in training iteration.
- */
- public long getObsCount() {
- return obsCount;
- }
-
- /**
- * @return current iteration.
- */
- public int getIteration() {
- return iteration;
- }
-
- private void updateStats(ObsEvent event) {
-
- logger.trace("TRAINING: ModelID: {}, {}", modelId, event.toString());
- model.update(event.getObsVector());
-
- obsCount++;
-
- /* Log info. */
- if (obsCount % 10000 == 0) {
- logger.info("Trained model using {} events with class id {}",
- obsCount, modelId);
- }
- }
-
- private void estimateModel() {
-
- model.estimate();
-
- double prob = (double) obsCount / numVectors;
- logPriorProb = Math.log(prob);
- logger.info("Prior prob: {}", prob);
- logger.info("Update params for model {} is: {}", modelId,
- model.toString());
-
- obsCount = 0;
- totalCount = 0;
- model.clearStatistics();
-
- /* Ready to start next iteration. */
- iteration++;
- }
-
- public void onEvent(Event event) {
-
- ObsEvent inEvent = (ObsEvent) event;
- float[] obs = inEvent.getObsVector();
-
- /* Estimate model parameters using the training data. */
- if (inEvent.isTraining()) {
-
- /*
- * Ignore events with negative index. They are just used to create
- * the PE.
- */
- if (inEvent.getIndex() < 0) {
- return;
- }
-
- if (++totalCount == numVectors) {
-
- /* End of training stream. */
- estimateModel();
-
- /* Could send ack here. */
-
- return;
- }
-
- /* Check if the event belongs to this class. */
- if (inEvent.getClassId() == modelId) {
-
- updateStats(inEvent);
-
- } else {
-
- /* Not needed to compute the mean vector. */
- return;
- }
-
- } else { // scoring
-
- if (inEvent.getHypId() < 0) {
- /* Score observed vector and send it to the maximizer. */
- float dist = (float) (model.logProb(obs) + logPriorProb);
- ObsEvent outEvent = new ObsEvent(inEvent.getIndex(), obs, dist,
- inEvent.getClassId(), modelId, false);
-
- logger.trace(inEvent.getIndex() + " " + inEvent.getClassId()
- + " " + modelId + " " + model.logProb(obs) + " "
- + logPriorProb + " " + dist);
-
- distanceStream.put(outEvent);
-
- } else {
-
- /* Send out result. */
- if (resultStream != null) {
- ResultEvent resultEvent = new ResultEvent(
- inEvent.getIndex(), inEvent.getClassId(),
- inEvent.getHypId());
-
- resultStream.put(resultEvent);
- }
- }
- }
- }
-
- @Override
- protected void onCreate() {
-
- this.modelId = Integer.parseInt(getId());
-
- /*
- * Initialize model. When a new PE instance is created we use the
- * reference to the model in the PE prototype (initial value in variable
- * model) to create a new model for this PE instance (final value in
- * variable model).
- */
- model = model.create();
- }
-
- @Override
- protected void onRemove() {
-
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Module.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Module.java
deleted file mode 100644
index 018e2b7..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/Module.java
+++ /dev/null
@@ -1,84 +0,0 @@
-/*
- * Copyright (c) 2011 Yahoo! Inc. All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-
-import java.io.InputStream;
-
-import org.apache.commons.configuration.ConfigurationConverter;
-import org.apache.commons.configuration.ConfigurationException;
-import org.apache.commons.configuration.ConfigurationUtils;
-import org.apache.commons.configuration.PropertiesConfiguration;
-import org.apache.s4.model.Model;
-
-import com.google.inject.AbstractModule;
-import com.google.inject.Binder;
-import com.google.inject.name.Names;
-
-/**
- * Configures the controller.
- *
- * Reads a properties file, provides a {@link Communicator} singleton, and
- * configures Guice bindings.
- *
- * @author Leo Neumeyer
- */
-public class Module extends AbstractModule {
-
- protected PropertiesConfiguration config = null;
-
- private void loadProperties(Binder binder) {
-
- try {
- InputStream is = this.getClass().getResourceAsStream(
- "/model.properties");
- config = new PropertiesConfiguration();
- config.load(is);
-
- System.out.println(ConfigurationUtils.toString(config));
- // TODO - validate properties.
-
- /* Make all properties injectable. Do we need this? */
- Names.bindProperties(binder,
- ConfigurationConverter.getProperties(config));
- } catch (ConfigurationException e) {
- binder.addError(e);
- e.printStackTrace();
- }
- }
-
- @Override
- protected void configure() {
- if (config == null)
- loadProperties(binder());
-
- int vectorSize = config.getInt("model.vector_size");
- int numGaussians = config.getInt("model.num_gaussians");
-
- /*
- * Here we bind a specific Model implementation. This one is the
- * GaussianMixtureModel
- */
- bind(Model.class).toInstance(
- new org.apache.s4.model.GaussianMixtureModel(vectorSize, numGaussians,
- org.apache.s4.model.GaussianMixtureModel.TrainMethod.STEP));
-
- /* Alternatively, we could bind to the GaussianModel. */
- // bind(Model.class).toInstance(
- // new org.apache.s4.model.GaussianModel(vectorSize, true));
-
- }
-}
\ No newline at end of file
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MyApp.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MyApp.java
deleted file mode 100644
index 2985c63..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/MyApp.java
+++ /dev/null
@@ -1,174 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import java.util.concurrent.TimeUnit;
-
-import org.apache.s4.core.App;
-import org.apache.s4.core.ProcessingElement;
-import org.apache.s4.core.Stream;
-import org.apache.s4.model.Model;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-public class MyApp extends App {
-
- private static final Logger logger = LoggerFactory.getLogger(MyApp.class);
-
- final private int numClasses;
- final private long numVectors;
- final private int outputInterval;
- final private TimeUnit timeUnit;
- final private Model model;
- private Stream<ObsEvent> obsStream;
-
- private ModelPE modelPE;
- private MetricsPE metricsPE;
- Stream<ObsEvent> assignmentStream;
- private String report;
-
- MyApp(int numClasses, long numVectors, Model model, int outputInterval, TimeUnit timeUnit) {
- super();
- this.numClasses = numClasses;
- this.numVectors = numVectors;
- this.model = model;
- this.outputInterval = outputInterval;
- this.timeUnit = timeUnit;
- }
-
- public void injectToAll(ObsEvent obsEvent) {
- logger.trace("Inject: " + obsEvent.toString());
- obsStream.put(obsEvent);
- }
-
- public void injectByKey(ObsEvent obsEvent) {
- logger.trace("Inject: " + obsEvent.toString());
- assignmentStream.put(obsEvent);
- }
-
- @Override
- protected void onStart() {
-
- }
-
- /* init() visibility is protected. Normally the Server class only should call init. */
- public void initApp() {
- init();
- }
-
- @Override
- protected void onInit() {
-
- metricsPE = createPE(MetricsPE.class);
-
- Stream<ResultEvent> resultStream = createStream("Result Stream", new ResultKeyFinder(), metricsPE);
-
- modelPE = createPE(ModelPE.class);
- modelPE.setModel(model);
- modelPE.setNumVectors(numVectors);
-
- assignmentStream = createStream("Assignment Stream", new ClassIDKeyFinder(), modelPE);
-
- MaximizerPE maximizerPE = createPE(MaximizerPE.class);
- maximizerPE.setNumClasses(numClasses);
- maximizerPE.setAssignmentStream(assignmentStream);
-
- Stream<ObsEvent> distanceStream = createStream("Distance Stream", new ObsIndexKeyFinder(), maximizerPE);
-
- /*
- * There is a loop in this graph so we need to set the stream at the end. Is there a cleaner way to do this?
- */
- modelPE.setStream(distanceStream, resultStream);
- // modelPE.setOutputIntervalInEvents(10); // output every 10 events
- metricsPE.setTimerInterval(outputInterval, timeUnit); // output every 5
- // seconds
- // obsStream = new Stream<ObsEvent>(this, "Observation Stream", new
- // ClassIDKeyFinder(), modelPE);
- obsStream = createStream("Observation Stream", modelPE);
- }
-
- /** @return true if modelPE is initialized. */
- public boolean isInited() {
- if (modelPE.getRemoteInstances().size() == numClasses)
- return true;
- else
- return false;
- }
-
- /**
- * Check if a training iteration is completed.
- *
- * @param iteration
- * @return true when a training iteration is completed.
- */
- public boolean isTrained(int iteration) {
-
- int sum = 0;
- for (ProcessingElement pe : modelPE.getRemoteInstances().values()) {
- sum += ((ModelPE) pe).getIteration();
- }
- if (sum < iteration * numClasses)
- return false;
- else
- return true;
- }
-
- /**
- * Check if a test is complete.
- *
- * @param number
- * of test vectors.
- * @return true when testing is complete.
- */
- public boolean isTested(long numTestVectors) {
-
- long count = 0;
- MetricsPE mpe = null;
- for (ProcessingElement pe : metricsPE.getRemoteInstances().values()) {
- mpe = (MetricsPE) pe;
- count = mpe.getCount();
- }
- if (count < numTestVectors) {
- return false;
- } else {
- report = mpe.toString();
- return true;
- }
- }
-
- /**
- * @return a metrics report.
- */
- public String getReport() {
-
- return report;
- }
-
- /* close() visibility is protected. Normally the Server class only should call init. */
- public void closeApp() {
- close();
- }
-
- public long getObsCount() {
-
- return modelPE.getObsCount();
- }
-
- @Override
- protected void onClose() {
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ObsEvent.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ObsEvent.java
deleted file mode 100644
index b2c6ae9..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ObsEvent.java
+++ /dev/null
@@ -1,100 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import org.apache.s4.base.Event;
-
-import net.jcip.annotations.Immutable;
-
-@Immutable
-final public class ObsEvent extends Event {
-
- private float[] obsVector;
- private float prob;
- private long index;
- private int classId;
- private int hypId;
- private boolean isTraining;
-
- public ObsEvent() {
-
- }
-
- public ObsEvent(long index, float[] obsVector, float prob, int classId,
- int hypId, boolean isTraining) {
- this.obsVector = obsVector;
- this.prob = prob;
- this.index = index;
- this.classId = classId;
- this.hypId = hypId;
- this.isTraining = isTraining;
- }
-
- /**
- * @return the observed data vector.
- */
- public float[] getObsVector() {
- return obsVector;
- }
-
- /**
- * @return the probability of this event.
- */
- public float getProb() {
- return prob;
- }
-
- /**
- * @return the index of the data vector.
- */
- public long getIndex() {
- return index;
- }
-
- /**
- * @return the true class of the vector.
- */
- public int getClassId() {
- return classId;
- }
-
- /**
- * @return the hypothesized class of the vector. Use -1 when unknown.
- */
- public int getHypId() {
- return hypId;
- }
-
- /**
- * @return true if this is training data.
- */
- public boolean isTraining() {
- return isTraining;
- }
-
- public String toString() {
-
- StringBuilder vector = new StringBuilder();
- for (int i = 0; i < obsVector.length; i++) {
- vector.append(obsVector[i] + " ");
- }
- return "Idx: " + index + ", Class: " + classId + ", Hyp:" + hypId
- + ", Prob: " + prob + ", isTraining: " + isTraining
- + ", Obs: " + vector.toString();
- }
-
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ObsIndexKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ObsIndexKeyFinder.java
deleted file mode 100644
index dc45975..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ObsIndexKeyFinder.java
+++ /dev/null
@@ -1,21 +0,0 @@
-package org.apache.s4.example.model;
-
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.s4.base.KeyFinder;
-
-public class ObsIndexKeyFinder implements KeyFinder<ObsEvent> {
-
- @Override
- public List<String> get(ObsEvent event) {
-
- List<String> results = new ArrayList<String>();
-
- /* Retrieve the user ID and add it to the list. */
- results.add(Long.toString(event.getIndex()));
-
- return results;
- }
-
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/README.md b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/README.md
deleted file mode 100644
index 0991e74..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/README.md
+++ /dev/null
@@ -1,390 +0,0 @@
-# Modeling Framework for S4
-
-## The Pattern Recognition Problem
-
-In this example we show how to design a pluggable probabilistic modeling framework in S4. The task is to classify incoming objects into
-well defined categories. Each object is represented by an observation vector that represents the object, also called
-features of the object. The variability and consistency of the features within a category and between categories will
-determine the accuracy of the classifier and the complexity of the models. Because it is impossible to achieve perfect accuracy in real-world systems,
-we use probabilistic models to classify the objects. So instead of just assigning a category to each object, the model will
-provide the probability that the object belongs to a category. The final decision may depend on several factors, for example,
-the cost of wrongly assign a category to an object may not be the same for all categories. In this example, we assume that
-the cost is the same for all categories and simply select the category whose model has the highest probability.
-
-Probabilistic models are widely used in many application areas. A distributed systems may be needed when decisions need to be made
-with low delay and by processing a large number of incoming events. S4 is designed to scale to an unlimited number of nodes providing
-the flexibility to run complex algorithms, process at high data rates, and deliver results with low latency. Typical applications include
-processing user events in web applications, analyzing financial market data, monitoring network traffic, and many more.
-
-To learn more, get a copy of this classic book: "Pattern Classification" by R. Duda, P. Hart, and D. Stork.
-
-## The Approach
-
-In this example we implemented an application that uses a train data set to estimate model parameters. Because most estimation
-algorithms learn iteratively, we inject the train data set several times until a stop condition is achieved. To train the models,
-we run S4 in batch mode. That is, we push the data at the highest possible rate and when a queue fills up, we let the system block
-until more space in the queues become available. In other words, no event data is ever lost in this process. To achieve this, we remove all
-latency constraints and let the process run for as long as needed until all the data is processed. This approach is quite similar
-to MapReduce except that the data is injected sequentially from a single source.
-
-Once the model parameters are estimated we are ready to run a test. In real-time applications, we would have no control over the speed of the
-incoming data vectors. If we didn't have sufficient computing resources to process all the data within the time constraints, we would be
-forced to either lose some of the data (load shedding) or switch to a less complex classification algorithm. In this example, we simply assume
-that there is no data loss.
-
-## The Forest Cover Data Set
-
-To evaluate the application we use a [publicly available labeled data set to predict
-forest cover type](http://kdd.ics.uci.edu/databases/covertype/covertype.html).
-For details about the data set please download the paper published by the author of this work.
-([PDF](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.128.2475&rep=rep1&type=pdf))
-
-Here is a description of the data. There are 7 types of forest cover. Each observation vector represents an area of
-forest with a type of cover and ten measurements (Elevation, Aspect, Slope, etc.) There is a total of 581,012 observation
-vectors in the data set.
-
-<pre>
-Name Data Type Measurement Description
-
-Cover_Type (7 types) integer 1 to 7 Forest Cover Type designation
-Elevation quantitative meters Elevation in meters
-Aspect quantitative azimuth Aspect in degrees azimuth
-Slope quantitative degrees Slope in degrees
-Horizontal_Distance_To_Hydrology quantitative meters Horz Dist to nearest surface water features
-Vertical_Distance_To_Hydrology quantitative meters Vert Dist to nearest surface water features
-Horizontal_Distance_To_Roadways quantitative meters Horz Dist to nearest roadway
-Hillshade_9am quantitative 0 to 255 index Hillshade index at 9am, summer solstice
-Hillshade_Noon quantitative 0 to 255 index Hillshade index at noon, summer soltice
-Hillshade_3pm quantitative 0 to 255 index Hillshade index at 3pm, summer solstice
-Horizontal_Distance_To_Fire_Points quantitative meters Horz Dist to nearest wildfire ignition points
-
-
-Forest Cover Types:
- 1 -- Spruce/Fir
- 2 -- Lodgepole Pine
- 3 -- Ponderosa Pine
- 4 -- Cottonwood/Willow
- 5 -- Aspen
- 6 -- Douglas-fir
- 7 -- Krummholz
-
-Class Distribution
-
-Number of records of Spruce-Fir 211840
-Number of records of Lodgepole Pine 283301
-Number of records of Ponderosa Pine 35754
-Number of records of Cottonwood/Willow 2747
-Number of records of Aspen 9493
-Number of records of Douglas-fir 17367
-Number of records of Krummholz 20510
-
-Total records 581012
-</pre>
-
-Here are the steps I used to download and prepare the data files. The files are located in the project under
-[src/main/resources/](https://github.com/leoneu/s4-piper/tree/master/src/main/resources).
-
- # Download data set and uncoompress.
- wget http://kdd.ics.uci.edu/databases/covertype/covtype.data.gz
- gunzip covtype.data.gz
-
- # Remove some columns and put the class label in the first column.
- gawk -F "," '{print $55, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10}' covtype.data > covtype-modified.data
-
- # Randomize data set.
- sort -R covtype-modified.data > covtype-random.data
-
- # Check number of data points.
- wc -l covtype-*
- # 581012 covtype-modified.data
- # 581012 covtype-random.data
-
- # Create a train and a test set.
- tail -100000 covtype-random.data > covtype-test.data
- head -481012 covtype-random.data > covtype-train.data
-
- wc -l covtype-{train,test}.data
- # 481012 covtype-train.data
- # 100000 covtype-test.data
- # 581012 total
-
-## Implementation of the Probabilistic Models
-
-The modeling package [org.apache.s4.model](https://github.com/leoneu/s4-piper/tree/master/src/main/java/io/s4/model) provides
-a basic interface for implementing probabilistic models. The main methods are:
-
-* **update()** - updates sufficient statistics given an observation vector.
-* **estimate()** - estimates model parameters using the sufficient statistics.
-* **prob()** - computes the probability.
-
-The abstract class [org.apache.s4.model.Model](https://github.com/leoneu/s4-piper/tree/master/src/main/java/io/s4/model/Model.java)
-is used to build the models. There are no dependencies with any specific implementation of a
-model because the code only uses Model to estimate and run the classifier.
-
-There are two concrete implementation of Model:
-
- * [Gaussian Model](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/model/GaussianModel.java) a single
- Gaussian distribution with parameters _mean_ and _variance_.
- * [Gaussian Mixture Model](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/model/GaussianMixtureModel.java)
- A mixture of Gaussian distributions with parameters _mixture weights_ and each Gaussian component with parameters _mean_ and
- _variance_.
-
-More model can be implemented and thanks to object oriented design, swapping models is as easy as
-editing one line of code in the [Guice](http://code.google.com/p/google-guice/)
-[Module](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/Module.java).
-
-When deployed in a cluster the same code will be run on many physical servers without changing any application code.
-As an application developer you don't have to worry about how distributed processing works. Scientist can focus on writing
-model code and dropping it in the right place. Moreover, the same code can be used to run experiments in batch mode and to
-deploy in a real-time production environment.
-
-## Training
-
-Here is the basic structure of the program:
-
-* Determine number of train vectors, and number of categories from the
- train data set. [org.apache.s4.example.model.Controller](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/Controller.java)
-* Create events of type ObsEvent and inject them into ModelPE with key = classId (classId refers to the category)
-* There is a [ModelPE](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/ModelPE.java)
- instance for each cover type (a total of seven cover types in this example with classId: 0-6).
- For each observation vector that matches the cover type, call the update() method in the Model.
-* Once all the train events are received, estimate the parameters for each model.
-
-
-We choose to use events of type ObsEvent to communicate between Processing Elements and ResultEvent to send the results to the MetricsPE.
-The events are immutable and can only be created using the constructor. The ObsEvent fields are:
-
-* _obsVector_ is the observation vector. The size of the float array should be the same for all the vectors.
-* _prob_ is the probability of the vector given the model.
-* _index_ is a unique identifier for the event.
-* _classId_ is the true category for the vector as it was labeled in the original data set.
-* _hypId_ is the hypothesized category for the vector returned by classification algorithm.
-* _isTraining_ is a boolean to differentiate between train and test modes.
-
-Here is a snippet of ObsEvent.java:
-
- public class ObsEvent extends Event {
-
- final private float[] obsVector;
- final private float prob;
- final private long index;
- final private int classId;
- final private int hypId;
-
- public ObsEvent(long index, float[] obsVector, float prob, int classId,
- int hypId) {
- this.obsVector = obsVector;
- this.prob = prob;
- this.index = index;
- this.classId = classId;
- this.hypId = hypId;
- }
-
-
-The application graph is defined in [MyApp](https://github.com/leoneu/s4-piper/blob/master/src/main/java/io/s4/example/model/MyApp.java)
-To define a graph we simply connect processing elements to streams. When teh objects in the graph are created, they are simply a representation
-of the application. The real work is done by the processing element instances that may reside anywhere in the cluster.
-
-Notice that the application graph has a cycle (events go from ModelPE to MaximizerPE and back). This creates a minor challenge to create the
-application graph. To solve this problem we added a setter method to set the distanceStream in ClusterPE.
-
-## Testing
-
-For testing we follow the following steps:
-
-* Compute the posterior probability of the observations for each model.
-* For an observation, select model id with the highest posterior probability. This is done by MaximizerPE: for each observation
- get all seven probabilities (one from each model), select the category with the highest probability. Self destroy the instance of MaximizerPE
- once the work is done.
-* MaximizerPE sends ObsEvent with HypID back to the ModelPE instance using ClassID as key.
-* Send results to MetricsPE.
-* MetricsPE accumulate counts to compute the confusion matrix and the overall accuracy. In the confusion matrix rows correspond
- to true categories and columns to hypotheses. The diagonal
- shows the percentage of observations that were correctly categorized and the off-diagonal numbers are the errors.
-
-## Experiments
-
-We first run the classifier using the Gaussian model, that is, we model each class using a Gaussian probability density
-function for which we need to estimate its parameters (mean and variance).
-
-To run the experiment, we bind the Model type to the GaussianModel class in Module.java as follows:
-
-<pre>
- protected void configure() {
- if (config == null)
- loadProperties(binder());
-
- int vectorSize = config.getInt("model.vector_size");
- int numGaussians = config.getInt("model.num_gaussians");
-
- bind(Model.class).toInstance(
- new org.apache.s4.model.GaussianModel(vectorSize, true));
-
- }
-</pre>
-
-With this binding the GaussianModel instance will be injected in the Controller constructor.
-
-Next, we edit the properties file as follows:
-
- model.train_data = /covtype-train.data.gz
- model.test_data = /covtype-test.data.gz
- model.logger.level = DEBUG
- model.num_iterations = 1
- model.vector_size = 10
- model.output_interval_in_seconds = 2
-
-In the properties file we configure the data sets for training and testing, the logger level, the number of iterations
-(we only need one iteration to estimate the mean and variance), the vector size which is 10 for this data set and
-how often we want to print partial results, we choose 2-second intervals. A final result will be printed by the
-controller at the end of the experiment.
-
-To run using Gradle, make sure you set the Main class in build.gradle to:
-
- mainClassName = "org.apache.s4.example.model.Main"
-
-To run the experiment type:
-
- gradlew run
-
-
- and after a few seconds we get the result:
-
- Confusion Matrix [%]:
-
- 0 1 2 3 4 5 6
- ----------------------------------------
- 0: 67.4 25.2 0.7 0.0 1.0 0.3 5.5
- 1: 24.1 65.8 4.6 0.0 2.3 1.9 1.4
- 2: 0.0 19.6 64.3 3.7 0.3 12.0 0.0
- 3: 0.0 0.4 38.6 48.8 0.0 12.2 0.0
- 4: 0.0 69.0 4.8 0.0 24.0 2.2 0.0
- 5: 0.0 18.3 47.3 2.3 0.5 31.5 0.0
- 6: 70.5 0.6 0.7 0.0 0.0 0.0 28.2
-
- Accuracy: 63.1% - Num Observations: 100000
-
-The observation vectors are correctly categorized in an independent data set at a rate of 63.1%. Note that 85% of the observations
-are in categories 0 and 1. The classifier learned this fact and relied on the prior probabilities to optimize the overall accuracy
-of the classifier. That's why accuracy is higher for these categories. For example, only 3.5% of the observations are in category 6
-so the low accuracy of 28.2% has little impact on the overall accuracy. Depending on the application, this may or may not be the right
-optimization approach.
-
-Next we, want to try the more sophisticated GaussianMixtureModel. We changed the properties file as follows:
-
- model.train_data = /covtype-train.data.gz
- model.test_data = /covtype-test.data.gz
- model.logger.level = DEBUG
- model.num_gaussians = 1
- model.num_iterations = 2
- model.vector_size = 10
- model.output_interval_in_seconds = 2
-
-Note that we are only using one Gaussian per mixture which is equivalent to using the GaussianModel so we expect the results to be identical.
-We need 2 iterations because the model uses the first pass to estimate the mean and variance of the data. This is only useful when using
-more than one mixture component.
-
-We changed the Module class as follows:
-
-<pre>
- protected void configure() {
- if (config == null)
- loadProperties(binder());
-
- int vectorSize = config.getInt("model.vector_size");
- int numGaussians = config.getInt("model.num_gaussians");
-
- bind(Model.class).toInstance(
- new org.apache.s4.model.GaussianMixtureModel(vectorSize, numGaussians,
- org.apache.s4.model.GaussianMixtureModel.TrainMethod.STEP));
- }
-</pre>
-
-and we run the experiment again:
-
- gradlew run
-
-The result is identical as expected:
-
- Confusion Matrix [%]:
-
- 0 1 2 3 4 5 6
- ----------------------------------------
- 0: 67.4 25.2 0.7 0.0 1.0 0.3 5.5
- 1: 24.1 65.8 4.6 0.0 2.3 1.9 1.4
- 2: 0.0 19.6 64.3 3.7 0.3 12.0 0.0
- 3: 0.0 0.4 38.6 48.8 0.0 12.2 0.0
- 4: 0.0 69.0 4.8 0.0 24.0 2.2 0.0
- 5: 0.0 18.3 47.3 2.3 0.5 31.5 0.0
- 6: 70.5 0.6 0.7 0.0 0.0 0.0 28.2
-
- Accuracy: 63.1% - Num Observations: 100000
-
-
-Now let's increase the number of mixture components to two Gaussian distributions per category:
-
-
- model.train_data = /covtype-train.data.gz
- model.test_data = /covtype-test.data.gz
- model.logger.level = DEBUG
- model.num_gaussians = 2
- model.num_iterations = 6
- model.vector_size = 10
- model.output_interval_in_seconds = 2
-
-
- Confusion Matrix [%]:
-
- 0 1 2 3 4 5 6
- ----------------------------------------
- 0: 66.4 27.8 0.1 0.0 1.5 0.4 3.8
- 1: 24.9 63.3 3.6 0.1 4.5 3.0 0.6
- 2: 0.0 13.1 65.7 8.2 1.0 12.0 0.0
- 3: 0.0 0.4 17.0 80.9 0.0 1.7 0.0
- 4: 5.1 50.0 3.0 0.0 37.2 4.7 0.0
- 5: 0.0 15.8 39.4 7.5 1.0 36.3 0.0
- 6: 71.4 1.1 0.0 0.0 0.0 0.0 27.6
-
- Accuracy: 62.2% - Num Observations: 100000
-
-The overall accuracy went down from 63.1% to 62.2%. However, we can see a dramatic improvement in category 3
-(from 48.8% to 80.9%) at the cost of a slight degradation in categories 0 and 1. Clearly, using two Gaussians
-per category helped category three.
-
-To improve the accuracy of the classifier, one could do some additional analysis and come up with an improved
-model until the accuracy is acceptable for the target application. For example, why are so many category 6 observations
-classified as category 0? Maybe we need a different number of mixtures per category to allocate more parameters to the
-categories with more training data and fewer to the other ones. Give it a try and let me know. I will add any
-models that get better overall accuracy than this one.
-
-## Performance
-
-I tested the execution speed on a single node configuration in a MacBook air with a Core i5 CPU and 4GB of RAM. The data
-is read from the solid state disk and uncompressed using gzip in every iteration. Initialization time is
-excluded from the measurements. Because I used only one node and all the data is local, there is no network overhead
-involved.
-
-The following results are for the GaussianMixture Model with 2 components per mixture and 6 iterations.
-
- Total training time was 33 seconds.
- Training time per observation was 69 microseconds.
- Training time per observation per iteration was 11 microseconds.
- Total testing time was 4 seconds.
- Testing time per observation was 8 microseconds.
-
-Based on this number, the data rate at which we injected data for training was (1/11 microseconds) or 90,000
-observations per second. If we look at the ObsEvent class, the effective number of bits transmitted per event is:
-
- 10 x float + 1 x float + 2 x int + 1 x long = 11 x 32 + 2 x 32 + 1 x 64 = 480 bits / observation
-
-This results in an injected data rate of 90,000 x 480 bits/sec = 43 mbps
-
-On an Ubuntu machine with an Intel Core i7-860 processor the average time per observation-iteration was
-8 microseconds or about 30% faster running at a data rate of 56 mbps.
-
-This is just to get an idea of the execution speed before even thinking about how to optimize. The throughput will vary
-greatly depending on the complexity of the algorithm and the hardware configuration.
-
-
-Please share your feedback at:
-http://groups.google.com/group/s4-project/topics
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultEvent.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultEvent.java
deleted file mode 100644
index 0d406d6..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultEvent.java
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import org.apache.s4.base.Event;
-
-final public class ResultEvent extends Event {
-
- private long index;
- private int classId;
- private int hypId;
-
- public ResultEvent() {
-
- }
-
- public ResultEvent(long index, int classId,
- int hypId) {
- this.index = index;
- this.classId = classId;
- this.hypId = hypId;
- }
-
- /**
- * @return the index of the data vector.
- */
- public long getIndex() {
- return index;
- }
-
- /**
- * @return the true class of the vector.
- */
- public int getClassId() {
- return classId;
- }
-
- /**
- * @return the hypothesized class of the vector.
- */
- public int getHypId() {
- return hypId;
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultKeyFinder.java b/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultKeyFinder.java
deleted file mode 100644
index 387d96c..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/example/model/ResultKeyFinder.java
+++ /dev/null
@@ -1,37 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.example.model;
-
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.s4.base.KeyFinder;
-
-public class ResultKeyFinder implements KeyFinder<ResultEvent> {
-
- @Override
- public List<String> get(ResultEvent event) {
-
- List<String> results = new ArrayList<String>();
-
- /* Retrieve the user ID and add it to the list. */
- results.add("1234");
-
- return results;
- }
-
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/Gaussian.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/Gaussian.java
deleted file mode 100644
index 51689c2..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/Gaussian.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.model;
-
-import java.lang.annotation.Retention;
-import java.lang.annotation.Target;
-import static java.lang.annotation.RetentionPolicy.RUNTIME;
-import static java.lang.annotation.ElementType.PARAMETER;
-import static java.lang.annotation.ElementType.FIELD;
-import static java.lang.annotation.ElementType.METHOD;
-
-import com.google.inject.BindingAnnotation;
-
-
- @BindingAnnotation @Target({ FIELD, PARAMETER, METHOD }) @Retention(RUNTIME)
- public @interface Gaussian {}
-
-
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixture.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixture.java
deleted file mode 100644
index 95568c0..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixture.java
+++ /dev/null
@@ -1,32 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.model;
-
-import java.lang.annotation.Retention;
-import java.lang.annotation.Target;
-import static java.lang.annotation.RetentionPolicy.RUNTIME;
-import static java.lang.annotation.ElementType.PARAMETER;
-import static java.lang.annotation.ElementType.FIELD;
-import static java.lang.annotation.ElementType.METHOD;
-
-import com.google.inject.BindingAnnotation;
-
-
- @BindingAnnotation @Target({ FIELD, PARAMETER, METHOD }) @Retention(RUNTIME)
- public @interface GaussianMixture {}
-
-
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixtureModel.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixtureModel.java
deleted file mode 100644
index 13df6c0..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianMixtureModel.java
+++ /dev/null
@@ -1,319 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.model;
-
-import org.apache.commons.lang.NotImplementedException;
-import org.apache.s4.util.MatrixOps;
-import org.ejml.data.D1Matrix64F;
-import org.ejml.data.DenseMatrix64F;
-import org.ejml.ops.CommonOps;
-
-
-/**
- * A multivariate Gaussian mixture model. Only diagonal covariance matrices are
- * supported.
- *
- * @author Leo Neumeyer
- *
- */
-public class GaussianMixtureModel extends Model {
-
- /** Supported algorithms for training this model. */
- public enum TrainMethod {
-
- /**
- * Estimate mean and variance of Gaussian distribution in the first
- * iteration. Create the target number of Gaussian components
- * (numComponents) in the mixture at the end of the first iteration
- * using the estimated mean and variance.
- */
- STEP,
-
- /**
- * Double the number of Gaussian components at the end of each
- * iteration.
- */
- DOUBLE,
-
- /** Do not allocate structures for training. */
- NONE
- }
-
- final private int numElements;
- final private TrainMethod trainMethod;
- private int numComponents;
- private double numSamples;
- private D1Matrix64F posteriorSum;
- private D1Matrix64F weights;
- private D1Matrix64F logWeights;
- private D1Matrix64F tmpProbs1;
- private D1Matrix64F tmpProbs2;
- private double totalLikelihood;
- private GaussianModel[] components;
- private int iteration = 0;
-
- public GaussianMixtureModel(int numElements, int numComponents,
- TrainMethod trainMethod) {
- super();
- this.numComponents = numComponents;
- this.numElements = numElements;
- this.trainMethod = trainMethod;
-
- if (trainMethod == TrainMethod.DOUBLE)
- throw new NotImplementedException(
- "Want this? Join as a contributor at http://s4.io");
-
- /* Allocate arrays needed for estimation. */
- isTrain = false;
- if (trainMethod != TrainMethod.NONE) {
- setTrain(true);
-
- if (trainMethod == TrainMethod.STEP) {
-
- /* Set up for first iteration using a single Gaussian. */
- allocateTrainDataStructures(1);
- }
- }
- }
-
- /*
- * TODO: we use this method when the pattern is: create a prototype of the
- * model and use it to create instances. Notice that we are allocating data
- * structures in the prototype that we will not use in this case. Not a big
- * deal but we may want to optimize this somehow later.
- */
- public Model create() {
-
- return new GaussianMixtureModel(numElements, numComponents, trainMethod);
- }
-
- private void allocateTrainDataStructures(int numComp) {
-
- components = new GaussianModel[numComp];
- for (int i = 0; i < numComp; i++) {
- this.components[i] = new GaussianModel(numElements, true);
-
- this.weights = new DenseMatrix64F(numComp, 1);
- CommonOps.set(this.weights, 1.0 / numComp);
- this.logWeights = new DenseMatrix64F(numComp, 1);
- CommonOps.set(this.logWeights, Math.log(1.0 / numComp));
- posteriorSum = new DenseMatrix64F(numComp, 1);
- tmpProbs1 = new DenseMatrix64F(numComp, 1);
- tmpProbs2 = new DenseMatrix64F(numComp, 1);
- }
- }
-
- /*
- * This method is used in {@link TrainMethod#STEP} Convert to a mixture with
- * N components. This method guarantees that the data structures are created
- * and that all the variables are set for starting a new training iteration.
- */
- private void increaseNumComponents(int newNumComponents) {
-
- /*
- * We use the Gaussian distribution of the parent GMM to create the
- * children.
- */
-
- /* Get mean and variance from parent before we allocate resized data structures. */
- D1Matrix64F mean = MatrixOps.doubleArrayToMatrix(this.components[0]
- .getMean());
- D1Matrix64F variance = MatrixOps.doubleArrayToMatrix(this.components[0]
- .getVariance());
-
- /* Throw away all previous data structures. */
- allocateTrainDataStructures(newNumComponents);
-
- /*
- * Create new mixture components. Abandon the old ones. We already got
- * the mean and variance in the previous step.
- */
- for (int i = 0; i < newNumComponents; i++) {
- components[i].setMean(MatrixOps.createRandom(i, mean, variance));
- components[i].setVariance(new DenseMatrix64F(variance));
- }
- }
-
- /* Thread safe internal logProb method. Must pass temp array. */
- private double logProbInternal(D1Matrix64F obs, D1Matrix64F probs) {
-
- /* Compute log probabilities for this observation. */
- for (int i = 0; i < components.length; i++) {
- probs.set(i, components[i].logProb(obs) + logWeights.get(i));
- }
-
- /*
- * To simplify computation, use the max prob in the denominator instead
- * of the sum.
- */
- return CommonOps.elementMax(probs);
- }
-
- public double logProb(D1Matrix64F obs) {
-
- return logProbInternal(obs, tmpProbs1);
- }
-
- public double logProb(double[] obs) {
-
- return logProb(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- public double logProb(float[] obs) {
-
- return logProb(MatrixOps.floatArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(D1Matrix64F obs) {
-
- return Math.exp(logProb(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(double[] obs) {
-
- return prob(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(float[] obs) {
-
- return prob(MatrixOps.floatArrayToMatrix(obs));
-
- }
-
- /** Update using Matrix array. */
- public void update(D1Matrix64F obs) {
-
- if (isTrain() == true) {
-
- /* Compute log probabilities for this observation. */
- double maxProb = logProbInternal(obs, tmpProbs2);
- totalLikelihood += maxProb;
- CommonOps.add(tmpProbs2, -maxProb);
-
- /* Compute posterior probabilities. */
- MatrixOps.elementExp(tmpProbs2);
-
- /* Update posterior sum, needed to compute mixture weights. */
- CommonOps.addEquals(posteriorSum, tmpProbs2);
-
- for (int i = 0; i < components.length; i++) {
- components[i].update(obs, tmpProbs2.get(i));
- }
-
- /* Count number of observations. */
- numSamples++;
- }
- }
-
- public void update(double[] obs) {
- update(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /** Update using float array. */
- public void update(float[] obs) {
- update(MatrixOps.floatArrayToMatrix(obs));
- }
-
- @Override
- public void estimate() {
-
- if (isTrain() == true) {
-
- /* Estimate mixture weights. */
- // double sum = CommonOps.elementSum(posteriorSum);
- // CommonOps.scale(1.0/sum, posteriorSum, weights);
- CommonOps.scale(1.0 / numSamples, posteriorSum, weights);
- MatrixOps.elementLog(weights, logWeights);
-
- /* Estimate component density. */
- for (int i = 0; i < components.length; i++) {
- components[i].estimate();
- }
-
- /*
- * After the first iteration, we can estimate the target number of
- * mixture components.
- */
- if (iteration == 0 && trainMethod == TrainMethod.STEP) {
-
- increaseNumComponents(numComponents);
- }
- iteration++;
- }
- }
-
- @Override
- public void clearStatistics() {
- if (isTrain() == true) {
-
- for (int i = 0; i < components.length; i++) {
- components[i].clearStatistics();
- }
- CommonOps.set(posteriorSum, 0.0);
- numSamples = 0;
- totalLikelihood = 0;
- }
-
- }
-
- /** Number of Gaussian components in the mixture. */
- public int getNumComponents() {
- return this.numComponents;
- }
-
- /** Data vector size. */
- public int getNumElements() {
- return this.numElements;
- }
-
- /**
- * @return the value of the parameters and sufficient statistics of this
- * model in a printable format.
- */
- public String toString() {
-
- StringBuilder sb = new StringBuilder("");
- sb.append("Gaussian Mixture Model\n");
- sb.append("num samples: " + numSamples + "\n");
- sb.append("num components: " + components.length + "\n");
- sb.append("weights: " + weights.toString() + "\n");
- sb.append("log weights: " + logWeights.toString() + "\n");
- sb.append("total log likelihood: " + totalLikelihood + "\n");
-
- for (int i = 0; i < components.length; i++) {
- sb.append(components[i].toString());
- }
-
- return sb.toString();
- }
-
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianModel.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianModel.java
deleted file mode 100644
index 6d20301..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/GaussianModel.java
+++ /dev/null
@@ -1,382 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.model;
-
-
-import org.apache.s4.util.MatrixOps;
-import org.ejml.data.D1Matrix64F;
-import org.ejml.data.DenseMatrix64F;
-import org.ejml.ops.CommonOps;
-
-/**
- * A multivariate Gaussian model with parameters mean (mu) and variance (sigma
- * squared). Only diagonal covariance matrices are supported.
- *
- * @author Leo Neumeyer
- *
- */
-public class GaussianModel extends Model {
-
- public final static double SMALL_VARIANCE = 0.01f;
- public final static double minNumSamples = 0.01f;
-
- private boolean isDiagonal = true; // Full covariance not yet supported.
- private D1Matrix64F sumx;
- private D1Matrix64F sumxsq;
- private D1Matrix64F tmpArray;
- private double numSamples;
- private D1Matrix64F mean;
- private D1Matrix64F variance; // ==> sigma squared
- private int numElements;
-
- private double const1; // -(N/2)log(2PI) Depends only on numElements.
- private double const2; // const1 - sum(log sigma_i) Also depends on
- // variance.
-
- /**
- * @param numElements
- * the model dimension.
- * @param train
- * allocate training arrays when true.
- */
- public GaussianModel(int numElements, boolean train) {
- this(numElements, null, null, train);
- }
-
- /**
- * Initialize model, no allocation of training arrays.
- *
- * @param numElements
- * the model dimension.
- * @param mean
- * model parameter.
- * @param variance
- * model parameter.
- */
- public GaussianModel(int numElements, D1Matrix64F mean, D1Matrix64F variance) {
- this(numElements, mean, variance, false);
- }
-
- /**
- * Initialize model, no allocation of training arrays.
- *
- * @param numElements
- * the model dimension.
- * @param mean
- * model parameter.
- * @param variance
- * model parameter.
- * @param train
- * allocate training arrays when true.
- */
- public GaussianModel(int numElements, D1Matrix64F mean,
- D1Matrix64F variance, boolean train) {
- super();
- this.numElements = numElements;
- tmpArray = new DenseMatrix64F(numElements, 1);
-
- if (mean == null) {
- this.mean = new DenseMatrix64F(numElements, 1);
- } else {
- this.mean = mean;
- }
-
- if (variance == null) {
- this.variance = new DenseMatrix64F(numElements, 1);
- CommonOps.set(this.variance, SMALL_VARIANCE);
- } else {
- this.variance = variance;
- }
-
- const1 = -numElements * (float) Math.log(2 * Math.PI) / 2;
- MatrixOps.elementLog(this.variance, tmpArray);
- const2 = const1 - CommonOps.elementSum(tmpArray) / 2.0;
-
- /* Allocate arrays needed for estimation. */
- if (train == true) {
- setTrain(true);
- sumx = new DenseMatrix64F(numElements, 1);
- sumxsq = new DenseMatrix64F(numElements, 1);
- clearStatistics();
- } else {
- setTrain(false);
- }
-
- }
-
- public Model create() {
-
- return new GaussianModel(numElements, isTrain);
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the log probability.
- */
- public double logProb(D1Matrix64F obs) {
-
- CommonOps.sub(mean, obs, tmpArray);
- MatrixOps.elementSquare(tmpArray);
- CommonOps.elementDiv(tmpArray, variance);
- return const2 - CommonOps.elementSum(tmpArray) / 2.0;
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the log probability.
- */
- public double logProb(float[] obs) {
-
- return logProb(MatrixOps.floatArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the log probability.
- */
- public double logProb(double[] obs) {
-
- return logProb(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#evaluate(double[])
- */
- public double prob(double[] obs) {
-
- return prob(MatrixOps.doubleArrayToMatrix(obs));
- }
-
- /** Evaluate using float array. */
- public double prob(float[] obs) {
-
- return prob(MatrixOps.floatArrayToMatrix(obs));
- }
-
- /**
- * @param obs
- * the observed data vector.
- * @return the probability.
- */
- public double prob(D1Matrix64F obs) {
-
- return Math.exp(logProb(obs));
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#update(double[])
- */
- public void update(double[] obs) {
-
- update(MatrixOps.doubleArrayToMatrix(obs));
-
- }
-
- /** Update using float array. */
- public void update(float[] obs) {
-
- update(MatrixOps.floatArrayToMatrix(obs));
-
- }
-
- /**
- * Update sufficient statistics.
- *
- * @param obs
- * the observed data vector.
- */
- public void update(D1Matrix64F obs) {
-
- if (isTrain() == true) {
-
- /* Update sufficient statistics. */
- CommonOps.add(obs, sumx, sumx);
- MatrixOps.elementSquare(obs, tmpArray);
- CommonOps.add(tmpArray, sumxsq, sumxsq);
- numSamples++;
- }
- }
-
- /**
- * Update sufficient statistics.
- *
- * @param obs
- * the observed data vector.
- * @param weight
- * the weight assigned to this observation.
- */
- public void update(D1Matrix64F obs, double weight) {
-
- if (isTrain() == true) {
-
- /* Update sufficient statistics. */
- CommonOps.scale(weight, obs, tmpArray);
- CommonOps.add(tmpArray, sumx, sumx);
-
- MatrixOps.elementSquare(obs, tmpArray);
- CommonOps.scale(weight, tmpArray);
- CommonOps.add(tmpArray, sumxsq, sumxsq);
-
- numSamples += weight;
- }
-
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#estimate()
- */
- public void estimate() {
-
- if (numSamples > minNumSamples) {
-
- /* Estimate the mean. */
- CommonOps.scale(1.0 / numSamples, sumx, mean);
-
- /*
- * Estimate the variance. sigma_sq = 1/n (sumxsq - 1/n sumx^2) or
- * 1/n sumxsq - mean^2.
- */
- D1Matrix64F tmp = variance; // borrow as an intermediate array.
-
- MatrixOps.elementSquare(mean, tmpArray);
- CommonOps.scale(1.0 / numSamples, sumxsq, tmp);
-
- CommonOps.sub(tmp, tmpArray, variance);
-
- MatrixOps.elementFloor(SMALL_VARIANCE, variance, variance);
-
- } else {
-
- /* Not enough training sample. */
- CommonOps.set(variance, SMALL_VARIANCE);
- CommonOps.set(mean, 0.0);
- }
-
- /* Update log Gaussian constant. */
- MatrixOps.elementLog(this.variance, tmpArray);
- const2 = const1 - CommonOps.elementSum(tmpArray) / 2.0;
- }
-
- /*
- * (non-Javadoc)
- *
- * @see org.apache.s4.model.Model#clearStatistics()
- */
- public void clearStatistics() {
-
- if (isTrain() == true) {
- CommonOps.set(sumx, 0.0);
- CommonOps.set(sumxsq, 0.0);
- numSamples = 0;
- }
- }
-
- /** @return the mean (mu) of the Gaussian density. */
- public double[] getMean() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(mean);
- return tmp.getData();
- }
-
- /** @return the variance (sigma squared) of the Gaussian density. */
- public double[] getVariance() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(variance);
- return tmp.getData();
- }
-
- public void setMean(D1Matrix64F mean) {
- this.mean = mean;
- }
-
- public void setVariance(D1Matrix64F variance) {
- this.variance = variance;
-
- /* Update log Gaussian constant. */
- MatrixOps.elementLog(this.variance, tmpArray);
- const2 = const1 - CommonOps.elementSum(tmpArray) / 2.0;
- }
-
- /** @return the standard deviation (sigma) of the Gaussian density. */
- public double[] getStd() {
-
- DenseMatrix64F std = new DenseMatrix64F(numElements, 1);
- MatrixOps.elementSquareRoot(variance, std);
- return std.getData();
- }
-
- /** @return the sum of the observed vectors. */
- public double[] getSumX() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(sumx);
- return tmp.getData();
- }
-
- /** @return the sum of the observed vectors squared. */
- public double[] getSumXSq() {
-
- DenseMatrix64F tmp = new DenseMatrix64F(sumxsq);
- return tmp.getData();
- }
-
- /** @return the number of observations. */
- public double getNumSamples() {
-
- return numSamples;
- }
-
- /** @return the dimensionality. */
- public int getNumElements() {
-
- return numElements;
- }
-
- /** @return true if the covariance matrix is diagonal. */
- public boolean isDiagonal() {
-
- return isDiagonal;
- }
-
- /**
- * @return the value of the parameters and sufficient statistics of this
- * model in a printable format.
- */
- public String toString() {
-
- StringBuilder sb = new StringBuilder("");
- sb.append("Gaussian Model\n");
- sb.append("const: " + const2 + "\n");
-
- sb.append("num samp: " + numSamples + "\n");
-
- sb.append("mean: " + mean.toString() + "\n");
- sb.append("var: " + variance.toString() + "\n");
- sb.append("sumx: " + sumx.toString() + "\n");
- sb.append("sunxsq: " + sumxsq.toString() + "\n");
-
- return sb.toString();
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/model/Model.java b/subprojects/s4-example/src/main/java/org/apache/s4/model/Model.java
deleted file mode 100644
index e757742..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/model/Model.java
+++ /dev/null
@@ -1,100 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.model;
-
-/**
- *
- * Base class for statistical models.
- *
- * @author Leo Neumeyer
- *
- */
-abstract public class Model {
-
- protected boolean isTrain;
- protected String name;
-
- public Model() {
- }
-
- public Model(String name, boolean isTrain) {
- this.name = name;
- this.isTrain = isTrain;
- }
-
- /**
- * Return an instance of this model initialized with the same parameters as its parent.
- *
- * @return a copy of the parent model.
- */
- abstract public Model create();
-
- /**
- * Compute the probability of the observed vector.
- *
- * @param obs
- * An observed data vector.
- * @return the probability.
- */
- abstract public double prob(float[] obs);
-
- /**
- * Compute the log probability of the observed vector.
- *
- * @param obs
- * An observed data vector.
- * @return the log probability.
- */
- abstract public double logProb(float[] obs);
-
- /**
- * Update sufficient statistics/
- *
- * @param obs
- * An observed data vector.
- */
- abstract public void update(float[] obs);
-
- /** Estimate model parameters. */
- abstract public void estimate();
-
- /** Clear sufficient statistics. */
- abstract public void clearStatistics();
-
- /** @return true if training resources have been initialized. */
- public boolean isTrain() {
- return isTrain;
- }
-
- /**
- * @param train
- * set train.
- */
- public void setTrain(boolean isTrain) {
- this.isTrain = isTrain;
- }
-
- /** @return model name. */
- public String getName() {
- return name;
- }
-
- /** Set model name. */
- public void setName(String name) {
- this.name = name;
- }
-}
diff --git a/subprojects/s4-example/src/main/java/org/apache/s4/util/MatrixOps.java b/subprojects/s4-example/src/main/java/org/apache/s4/util/MatrixOps.java
deleted file mode 100644
index e0ffd77..0000000
--- a/subprojects/s4-example/src/main/java/org/apache/s4/util/MatrixOps.java
+++ /dev/null
@@ -1,284 +0,0 @@
-/*
- * Copyright (c) 2011 The S4 Project, http://s4.io.
- * All rights reserved.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
- * either express or implied. See the License for the specific
- * language governing permissions and limitations under the
- * License. See accompanying LICENSE file.
- */
-package org.apache.s4.util;
-
-import java.util.Random;
-
-import org.ejml.data.D1Matrix64F;
-import org.ejml.data.DenseMatrix64F;
-
-/**
- * Extensions to the EJML library.
- *
- * @author Leo Neumeyer
- */
-public class MatrixOps {
-
- /**
- * <p>
- * Performs an in-place element by element natural logarithm operation.<br>
- * <br>
- * a<sub>ij</sub> = log(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the log. Modified.
- */
- public static void elementLog(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, Math.log(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an element by element natural logarithm operation.<br>
- * <br>
- * b<sub>ij</sub> = log(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the log. Not Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementLog(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, Math.log(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an in-place element by element exponential function.<br>
- * <br>
- * a<sub>ij</sub> = exp(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the exponentiation. Modified.
- */
- public static void elementExp(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, Math.exp(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an element by element natural exponential function.<br>
- * <br>
- * b<sub>ij</sub> = exp(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the exponentiation. Not
- * Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementExp(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, Math.exp(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an in-place element by element square operation.<br>
- * <br>
- * a<sub>ij</sub> = a<sub>ij</sub>^2
- * </p>
- *
- * @param a
- * The matrix on which we perform the square. Modified.
- */
- public static void elementSquare(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, a.get(i) * a.get(i));
- }
- }
-
- /**
- * <p>
- * Performs an element by element square.<br>
- * <br>
- * b<sub>ij</sub> = a<sub>ij</sub>^2
- * </p>
- *
- * @param a
- * The matrix on which we perform the square. Not Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementSquare(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, a.get(i) * a.get(i));
- }
- }
-
- /**
- * <p>
- * Performs an in-place element by element square root operation.<br>
- * <br>
- * a<sub>ij</sub> = sqrt(a<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the square root. Modified.
- */
- public static void elementSquareRoot(D1Matrix64F a) {
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- a.set(i, Math.sqrt(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Performs an element by element square root.<br>
- * <br>
- * b<sub>ij</sub> = sqrt(<sub>ij</sub>)
- * </p>
- *
- * @param a
- * The matrix on which we perform the square root. Not Modified.
- * @param b
- * Where the results of the operation are stored. Modified.
- */
- public static void elementSquareRoot(D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- b.set(i, Math.sqrt(a.get(i)));
- }
- }
-
- /**
- * <p>
- * Set a floor value for the matrix elements.<br>
- *
- * @param floor
- * The minimum element value.
- * @param a
- * The input matrix. Not Modified.
- * @param b
- * Matrix whose elements with values less than floor are set to
- * floor. Modified.
- */
- public static void elementFloor(double floor, D1Matrix64F a, D1Matrix64F b) {
- if (a.numRows != b.numRows || a.numCols != b.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = a.getNumElements();
-
- for (int i = 0; i < size; i++) {
- if (a.get(i) < floor)
- b.set(i, floor);
- else
- b.set(i, a.get(i));
- }
- }
-
- /** Convert an array of doubles to a matrix. A new matrix is created. */
- public static D1Matrix64F doubleArrayToMatrix(double[] data) {
-
- DenseMatrix64F tmp = new DenseMatrix64F(data.length, 1);
- for (int i = 0; i < data.length; i++) {
- tmp.set(i, data[i]);
- }
- return tmp;
- }
-
- /** Convert an array of doubles to a matrix passed as a parameter. */
- public static D1Matrix64F doubleArrayToMatrix(double[] data, D1Matrix64F mat) {
-
- for (int i = 0; i < data.length; i++) {
- mat.set(i, data[i]);
- }
- return mat;
- }
-
- /** Convert an array of floats to a matrix. A new matrix is created. */
- public static D1Matrix64F floatArrayToMatrix(float[] data) {
-
- DenseMatrix64F tmp = new DenseMatrix64F(data.length, 1);
- for (int i = 0; i < data.length; i++) {
- tmp.set(i, data[i]);
- }
- return tmp;
- }
-
- /** Convert an array of doubles to a matrix passed as a parameter. */
- public static D1Matrix64F floatArrayToMatrix(float[] data, D1Matrix64F mat) {
-
- for (int i = 0; i < data.length; i++) {
- mat.set(i, data[i]);
- }
- return mat;
- }
-
- /**
- * Create a random vector using the mean and variance of a Gaussian
- * distribution.
- */
- public static D1Matrix64F createRandom(long seed, D1Matrix64F mean, D1Matrix64F variance) {
- if (mean.numRows != variance.numRows || mean.numCols != variance.numCols)
- throw new IllegalArgumentException(
- "Matrices must have the same shape");
-
- final int size = mean.getNumElements();
- DenseMatrix64F mat = new DenseMatrix64F(size, 1);
- Random random = new Random(seed);
-
- for (int i = 0; i < size; i++) {
- mat.set(i, mean.get(i) + random.nextGaussian() * Math.sqrt(variance.get(i)));
- }
-
- return mat;
- }
-}
diff --git a/subprojects/s4-example/src/main/resources/covtype-test.data.gz b/subprojects/s4-example/src/main/resources/covtype-test.data.gz
deleted file mode 100644
index ec7462c..0000000
--- a/subprojects/s4-example/src/main/resources/covtype-test.data.gz
+++ /dev/null
Binary files differ
diff --git a/subprojects/s4-example/src/main/resources/covtype-train-1000.data.gz b/subprojects/s4-example/src/main/resources/covtype-train-1000.data.gz
deleted file mode 100644
index 5be2362..0000000
--- a/subprojects/s4-example/src/main/resources/covtype-train-1000.data.gz
+++ /dev/null
Binary files differ
diff --git a/subprojects/s4-example/src/main/resources/covtype-train.data.gz b/subprojects/s4-example/src/main/resources/covtype-train.data.gz
deleted file mode 100644
index 44ccd84..0000000
--- a/subprojects/s4-example/src/main/resources/covtype-train.data.gz
+++ /dev/null
Binary files differ
diff --git a/subprojects/s4-example/src/main/resources/model.properties b/subprojects/s4-example/src/main/resources/model.properties
deleted file mode 100644
index bf627a5..0000000
--- a/subprojects/s4-example/src/main/resources/model.properties
+++ /dev/null
@@ -1,14 +0,0 @@
-# Use the training set with 1000 data point for development.
-# model.train_data = /covtype-train.data.gz
-# model.train_data = /covtype-train-1000.data.gz
-#
-model.train_data = /covtype-train-1000.data.gz
-model.test_data = /covtype-test.data.gz
-model.logger.level = DEBUG
-model.num_gaussians = 2
-model.num_iterations = 6
-model.vector_size = 10
-# model.output_interval_in_events = 10
-model.output_interval_in_seconds = 2
-
-
diff --git a/subprojects/s4-example/src/main/resources/s4-piper-example.properties b/subprojects/s4-example/src/main/resources/s4-piper-example.properties
deleted file mode 100644
index b60f40a..0000000
--- a/subprojects/s4-example/src/main/resources/s4-piper-example.properties
+++ /dev/null
@@ -1,7 +0,0 @@
-comm.queue_emmiter_size = 8000
-comm.queue_listener_size = 8000
-cluster.hosts = localhost
-cluster.ports = 5077
-cluster.lock_dir = /tmp
-
-
diff --git a/subprojects/s4-example/src/test/io/s4/model/TestGaussianModel.java b/subprojects/s4-example/src/test/io/s4/model/TestGaussianModel.java
index e6acf76..6c358dd 100644
--- a/subprojects/s4-example/src/test/io/s4/model/TestGaussianModel.java
+++ b/subprojects/s4-example/src/test/io/s4/model/TestGaussianModel.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.model;
import java.util.Random;
diff --git a/subprojects/s4-tools/s4-tools.gradle b/subprojects/s4-tools/s4-tools.gradle
index e8edd14..8f47441 100644
--- a/subprojects/s4-tools/s4-tools.gradle
+++ b/subprojects/s4-tools/s4-tools.gradle
@@ -1,20 +1,23 @@
-/*
- * Copyright 2010 the original author or authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
-description = 'The S4 core platform.'
+description = 'The S4 tools platform.'
apply plugin: 'java'
@@ -29,7 +32,6 @@
compile project(":s4-core")
compile libraries.jcommander
compile libraries.zkclient
- compile libraries.commons_io
compile libraries.gradle_base_services
compile libraries.gradle_core
compile libraries.gradle_tooling_api
@@ -47,26 +49,20 @@
}
}
-applicationDistribution.from(configurations.compile) {
- // this is supposed to be done by default but is not anymore in 1.0-rc-3
- include "*.jar"
- into("lib")
-}
-
-startScripts {
- classpath = files('$APP_HOME/lib/*')
-}
-
-
-
-
run {
// run doesn't yet directly accept command line parameters...
if ( project.hasProperty('args') ) {
args project.args.split('\\s+')
print args
}
- }
+ }
+
+distZip {
+ // what we build here is really a binary release of S4 with required subprojects and dependencies (hence refs to rootProject)
+ classifier='bin'
+ version = rootProject.version
+ baseName=rootProject.ext.archivesBaseName
+}
test {
forkEvery=1
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/CreateApp.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/CreateApp.java
index beaee41..ca75cf8 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/CreateApp.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/CreateApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.io.File;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/DefineCluster.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/DefineCluster.java
index 646ae92..bd0deec 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/DefineCluster.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/DefineCluster.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import org.apache.s4.comm.tools.TaskSetup;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Deploy.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Deploy.java
index 8baa038..94e359c 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Deploy.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Deploy.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.io.File;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/FileExistsValidator.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/FileExistsValidator.java
index 9e21600..820ac91 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/FileExistsValidator.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/FileExistsValidator.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.io.File;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Package.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Package.java
index 5861031..fe8761a 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Package.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Package.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.io.File;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/S4ArgsBase.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/S4ArgsBase.java
index 52f1800..fb9a0f4 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/S4ArgsBase.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/S4ArgsBase.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.util.ArrayList;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Status.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Status.java
index a856f69..824aa03 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Status.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Status.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.util.ArrayList;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Tools.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Tools.java
index 8815a33..dbd63b9 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Tools.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/Tools.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.lang.reflect.Method;
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/ZKServer.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/ZKServer.java
index d473065..de56788 100644
--- a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/ZKServer.java
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/ZKServer.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.tools;
import java.io.File;
@@ -10,7 +28,6 @@
import org.I0Itec.zkclient.IDefaultNameSpace;
import org.I0Itec.zkclient.ZkClient;
import org.I0Itec.zkclient.ZkServer;
-import org.apache.commons.io.FileUtils;
import org.apache.s4.comm.tools.TaskSetup;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -35,8 +52,8 @@
if (zkArgs.clean) {
logger.info("cleaning existing data in [{}] and [{}]", zkArgs.dataDir, zkArgs.logDir);
- FileUtils.deleteDirectory(new File(zkArgs.dataDir));
- FileUtils.deleteDirectory(new File(zkArgs.logDir));
+ deleteDirectory(new File(zkArgs.dataDir));
+ deleteDirectory(new File(zkArgs.logDir));
}
IDefaultNameSpace defaultNameSpace = new IDefaultNameSpace() {
@@ -79,6 +96,24 @@
}
}
+ static private void deleteDirectory(File path) {
+ if (path.exists()) {
+ File[] files = path.listFiles();
+ for (int i = 0; i < files.length; i++) {
+ if (files[i].isDirectory()) {
+ deleteDirectory(files[i]);
+ } else {
+ if (!(files[i].delete())) {
+ logger.error("Could not delete file {}", files[i].getAbsolutePath());
+ }
+ }
+ }
+ if (!path.delete()) {
+ logger.error("Could not delete directory {}", path.getAbsolutePath());
+ }
+ }
+ }
+
@Parameters(commandNames = "s4 zkServer", separators = "=", commandDescription = "Start Zookeeper server")
static class ZKServerArgs extends S4ArgsBase {
diff --git a/subprojects/s4-tools/src/main/java/org/apache/s4/tools/package-info.java b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/package-info.java
new file mode 100644
index 0000000..940025e
--- /dev/null
+++ b/subprojects/s4-tools/src/main/java/org/apache/s4/tools/package-info.java
@@ -0,0 +1,4 @@
+/**
+ * Implementations of S4-related commands. Includes parsing, validation and interpretation of the various commands and options available.
+ */
+package org.apache.s4.tools;
\ No newline at end of file
diff --git a/subprojects/s4-tools/src/main/resources/templates/HelloApp.java.txt b/subprojects/s4-tools/src/main/resources/templates/HelloApp.java.txt
index 076b0a1..deee191 100644
--- a/subprojects/s4-tools/src/main/resources/templates/HelloApp.java.txt
+++ b/subprojects/s4-tools/src/main/resources/templates/HelloApp.java.txt
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package hello;
import java.util.Arrays;
diff --git a/subprojects/s4-tools/src/main/resources/templates/HelloInputAdapter.java.txt b/subprojects/s4-tools/src/main/resources/templates/HelloInputAdapter.java.txt
index dff4d8e..281f6b6 100644
--- a/subprojects/s4-tools/src/main/resources/templates/HelloInputAdapter.java.txt
+++ b/subprojects/s4-tools/src/main/resources/templates/HelloInputAdapter.java.txt
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package hello;
import java.io.BufferedReader;
diff --git a/subprojects/s4-tools/src/main/resources/templates/HelloPE.java.txt b/subprojects/s4-tools/src/main/resources/templates/HelloPE.java.txt
index f1c6014..b3e0b3f 100644
--- a/subprojects/s4-tools/src/main/resources/templates/HelloPE.java.txt
+++ b/subprojects/s4-tools/src/main/resources/templates/HelloPE.java.txt
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package hello;
import org.apache.s4.base.Event;
diff --git a/subprojects/s4-tools/src/main/resources/templates/build.gradle b/subprojects/s4-tools/src/main/resources/templates/build.gradle
index a55a207..bedbdef 100644
--- a/subprojects/s4-tools/src/main/resources/templates/build.gradle
+++ b/subprojects/s4-tools/src/main/resources/templates/build.gradle
@@ -1,23 +1,25 @@
/**
-* Apache S4 Application Build File
-*
-* Use this script to build and package S4 apps.
-*
-* Run 'gradle install' on the s4 project to publish to your local maven repo.
-*
-* TODO: This should probably be distributed as an s4 plugin for Gradle.
-* TODO: There seem to be to be similarities with the war and jetty plugins. (war -> s4r, jetty -> s4Run).
-* We should make it easy to test the app from this script by a running a test task that starts and stops
-* an s4 server. See: http://www.gradle.org/releases/1.0-milestone-3/docs/userguide/userguide_single.html#war_plugin
-*
-* This is an interesting discussion:
-* http://gradle.1045684.n5.nabble.com/Exclude-properties-file-from-war-td3365147.html
-*
-*/
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
project.ext["s4AppInstallDir"] = hasProperty('appsDir') ? "$appsDir" : "/tmp/appsDir"
-project.ext["s4Version"] = '0.5.0-SNAPSHOT'
+project.ext["s4Version"] = '0.5.0-incubating'
description = 'Apache S4 App'
//defaultTasks 'installS4R'
project.ext["archivesBaseName"] = "$project.name"
diff --git a/subprojects/s4-tools/src/main/resources/templates/newApp.README b/subprojects/s4-tools/src/main/resources/templates/newApp.README
index a174e31..2ce7c36 100644
--- a/subprojects/s4-tools/src/main/resources/templates/newApp.README
+++ b/subprojects/s4-tools/src/main/resources/templates/newApp.README
@@ -20,7 +20,7 @@
To execute the application in a new S4 cluster:
- 1. start a ZooKeeper instance "./s4 zkServer"
+ 1. start a ZooKeeper instance "./s4 zkServer" (./s4 zkServer -help provides a list of options)
2. define a logical cluster for your application "./s4 newCluster -cluster=<nameOfTheCluster> -nbTasks=<number of partitions> -flp=<a port number for the first node, other nodes use an increment on this initial port>"
3. start a node and attach it to the cluster "./s4 node -cluster=<nameOfTheCluster>"
4. deploy the application "./s4 deploy -cluster=<nameOfTheCluster>"
@@ -31,5 +31,5 @@
If you want to use a simple adapter process, listening to an external source, converting incoming data into S4 events, and sending that to S4 apps, you can define
your own app that extends the AdapterApp class. Then to start it (for instance):
- - ./s4 adapter -cluster=c1 -appClass=org.apache.s4.example.twitter.TwitterInputAdapter -p={adapter.output.stream=stream1}
+ - ./s4 adapter -cluster=c1 -appClass=org.apache.s4.example.twitter.TwitterInputAdapter -p=s4.adapter.output.stream=stream1
diff --git a/subprojects/s4-tools/src/main/resources/templates/s4 b/subprojects/s4-tools/src/main/resources/templates/s4
index fd54444..45c5efd 100644
--- a/subprojects/s4-tools/src/main/resources/templates/s4
+++ b/subprojects/s4-tools/src/main/resources/templates/s4
@@ -5,7 +5,7 @@
# we need something different in order to pass the classpath of the current project
# you must specify: appClassName (extends AdapterApp class) , cluster name, output stream name
# current syntax is not yet straightforward:
- # example : ./s4 adapter -cluster=c1 -appClass=org.apache.s4.example.twitter.TwitterInputAdapter -namedStringParameters=adapter.output.stream:s1
+ # example : ./s4 adapter -cluster=c1 -appClass=org.apache.s4.example.twitter.TwitterInputAdapter -p=s4.adapter.output.stream:s1
shift 1
<s4_install_dir>/gradlew cp
java -cp `cat classpath.txt` org.apache.s4.core.Main $@
diff --git a/subprojects/s4-tools/src/main/resources/templates/settings.gradle b/subprojects/s4-tools/src/main/resources/templates/settings.gradle
index e0a06c9..c895f18 100644
--- a/subprojects/s4-tools/src/main/resources/templates/settings.gradle
+++ b/subprojects/s4-tools/src/main/resources/templates/settings.gradle
@@ -1 +1,19 @@
-rootProject.name=<project-name>
\ No newline at end of file
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+rootProject.name=<project-name>
diff --git a/test-apps/consumer-app/build.gradle b/test-apps/consumer-app/build.gradle
index e2fec49..d8d56c4 100644
--- a/test-apps/consumer-app/build.gradle
+++ b/test-apps/consumer-app/build.gradle
@@ -1,18 +1,21 @@
-/*
-* Copyright 2010 the original author or authors.
-*
-* Licensed under the Apache License, Version 2.0 (the "License");
-* you may not use this file except in compliance with the License.
-* You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-*/
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/**
* Apache S4 Application Build File
@@ -61,31 +64,29 @@
/* All project libraries must be defined here. */
project.ext["libraries"] = [
- json: 'org.json:json:20090211',
- guava: 'com.google.guava:guava:10.0.1',
- gson: 'com.google.code.gson:gson:1.6',
- guice: 'com.google.inject:guice:3.0',
- guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
- guice_grapher: 'com.google.inject:guice-grapher:3.0',
- flexjson: 'net.sf.flexjson:flexjson:2.1',
- bcel: 'org.apache.bcel:bcel:5.2',
- jakarta_regexp: 'jakarta-regexp:jakarta-regexp:1.4',
- kryo: 'com.googlecode:kryo:1.04',
- netty: 'org.jboss.netty:netty:3.2.5.Final',
- reflectasm: 'com.esotericsoftware:reflectasm:0.8',
- minlog: 'com.esotericsoftware:minlog:1.2',
- asm: 'asm:asm:3.2',
- commons_io: 'commons-io:commons-io:2.0.1',
- commons_config: 'commons-configuration:commons-configuration:1.6',
- commons_codec: 'commons-codec:commons-codec:1.4',
- commons_httpclient: 'commons-httpclient:commons-httpclient:3.1',
- commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
- slf4j: 'org.slf4j:slf4j-api:1.6.1',
- logback_core: 'ch.qos.logback:logback-core:0.9.29',
- logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
- zk: 'org.apache.zookeeper:zookeeper:3.3.1',
- jcip: 'net.jcip:jcip-annotations:1.0',
- junit: 'junit:junit:4.10',
+ guava: 'com.google.guava:guava:12.0.1',
+ gson: 'com.google.code.gson:gson:1.6',
+ guice: 'com.google.inject:guice:3.0',
+ guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
+ kryo: 'com.googlecode:kryo:1.04',
+ netty: 'org.jboss.netty:netty:3.2.5.Final',
+ mockito_core: 'org.mockito:mockito-core:1.9.0',
+ commons_config: 'commons-configuration:commons-configuration:1.6',
+ commons_codec: 'commons-codec:commons-codec:1.4',
+ commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
+ slf4j: 'org.slf4j:slf4j-api:1.6.1',
+ logback_core: 'ch.qos.logback:logback-core:0.9.29',
+ logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
+ zk: 'org.apache.zookeeper:zookeeper:3.3.1',
+ jcip: 'net.jcip:jcip-annotations:1.0',
+ junit: 'junit:junit:4.10',
+ zkclient: 'com.github.sgroschupf:zkclient:0.1',
+ diezel: 'net.ericaro:diezel-maven-plugin:1.0.0-beta-4',
+ jcommander: 'com.beust:jcommander:1.25',
+ gradle_base_services: 'gradle-base-services:gradle-base-services:1.0',
+ gradle_core: 'gradle-core:gradle-core:1.0',
+ gradle_tooling_api: 'gradle-tooling-api:gradle-tooling-api:1.0',
+ gradle_wrapper: 'gradle-wrapper:gradle-wrapper:1.0'
]
@@ -107,7 +108,6 @@
compile( libraries.logback_classic )
/* Commons. */
- compile( libraries.commons_io )
compile( libraries.commons_config )
compile( libraries.commons_coll )
diff --git a/test-apps/consumer-app/src/main/java/s4app/ConsumerApp.java b/test-apps/consumer-app/src/main/java/s4app/ConsumerApp.java
index c13c53e..5b3ae04 100644
--- a/test-apps/consumer-app/src/main/java/s4app/ConsumerApp.java
+++ b/test-apps/consumer-app/src/main/java/s4app/ConsumerApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package s4app;
import org.apache.s4.core.App;
diff --git a/test-apps/consumer-app/src/main/java/s4app/ConsumerPE.java b/test-apps/consumer-app/src/main/java/s4app/ConsumerPE.java
index 7c6f123..d47bbec 100644
--- a/test-apps/consumer-app/src/main/java/s4app/ConsumerPE.java
+++ b/test-apps/consumer-app/src/main/java/s4app/ConsumerPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package s4app;
import org.I0Itec.zkclient.ZkClient;
diff --git a/test-apps/producer-app/build.gradle b/test-apps/producer-app/build.gradle
index e95434d..8f79377 100644
--- a/test-apps/producer-app/build.gradle
+++ b/test-apps/producer-app/build.gradle
@@ -1,18 +1,21 @@
-/*
-* Copyright 2010 the original author or authors.
-*
-* Licensed under the Apache License, Version 2.0 (the "License");
-* you may not use this file except in compliance with the License.
-* You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-*/
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/**
* Apache S4 Application Build File
@@ -61,31 +64,25 @@
/* All project libraries must be defined here. */
project.ext["libraries"] = [
- json: 'org.json:json:20090211',
- guava: 'com.google.guava:guava:10.0.1',
- gson: 'com.google.code.gson:gson:1.6',
- guice: 'com.google.inject:guice:3.0',
- guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
- guice_grapher: 'com.google.inject:guice-grapher:3.0',
- flexjson: 'net.sf.flexjson:flexjson:2.1',
- bcel: 'org.apache.bcel:bcel:5.2',
- jakarta_regexp: 'jakarta-regexp:jakarta-regexp:1.4',
- kryo: 'com.googlecode:kryo:1.04',
- netty: 'org.jboss.netty:netty:3.2.5.Final',
- reflectasm: 'com.esotericsoftware:reflectasm:0.8',
- minlog: 'com.esotericsoftware:minlog:1.2',
- asm: 'asm:asm:3.2',
- commons_io: 'commons-io:commons-io:2.0.1',
- commons_config: 'commons-configuration:commons-configuration:1.6',
- commons_codec: 'commons-codec:commons-codec:1.4',
- commons_httpclient: 'commons-httpclient:commons-httpclient:3.1',
- commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
- slf4j: 'org.slf4j:slf4j-api:1.6.1',
- logback_core: 'ch.qos.logback:logback-core:0.9.29',
- logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
- zk: 'org.apache.zookeeper:zookeeper:3.3.1',
- jcip: 'net.jcip:jcip-annotations:1.0',
- junit: 'junit:junit:4.10',
+ guava: 'com.google.guava:guava:12.0.1',
+ gson: 'com.google.code.gson:gson:1.6',
+ guice: 'com.google.inject:guice:3.0',
+ guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
+ kryo: 'com.googlecode:kryo:1.04',
+ netty: 'org.jboss.netty:netty:3.2.5.Final',
+ mockito_core: 'org.mockito:mockito-core:1.9.0',
+ commons_config: 'commons-configuration:commons-configuration:1.6',
+ commons_codec: 'commons-codec:commons-codec:1.4',
+ commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
+ slf4j: 'org.slf4j:slf4j-api:1.6.1',
+ logback_core: 'ch.qos.logback:logback-core:0.9.29',
+ logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
+ zk: 'org.apache.zookeeper:zookeeper:3.3.1',
+ jcip: 'net.jcip:jcip-annotations:1.0',
+ junit: 'junit:junit:4.10',
+ zkclient: 'com.github.sgroschupf:zkclient:0.1',
+ diezel: 'net.ericaro:diezel-maven-plugin:1.0.0-beta-4',
+ jcommander: 'com.beust:jcommander:1.25'
]
@@ -107,7 +104,6 @@
compile( libraries.logback_classic )
/* Commons. */
- compile( libraries.commons_io )
compile( libraries.commons_config )
compile( libraries.commons_coll )
diff --git a/test-apps/producer-app/src/main/java/s4app/ProducerApp.java b/test-apps/producer-app/src/main/java/s4app/ProducerApp.java
index 66ed4dc..ac7f251 100644
--- a/test-apps/producer-app/src/main/java/s4app/ProducerApp.java
+++ b/test-apps/producer-app/src/main/java/s4app/ProducerApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package s4app;
import org.apache.s4.core.App;
diff --git a/test-apps/producer-app/src/main/java/s4app/ProducerPE.java b/test-apps/producer-app/src/main/java/s4app/ProducerPE.java
index c8a7597..91dfac3 100644
--- a/test-apps/producer-app/src/main/java/s4app/ProducerPE.java
+++ b/test-apps/producer-app/src/main/java/s4app/ProducerPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package s4app;
import org.apache.s4.base.Event;
@@ -19,7 +37,7 @@
/**
* @param targetStreams
- * the {@link UserEvent} streams.
+ * the {@link Streamable} streams.
*/
public void setStreams(Streamable... targetStreams) {
this.targetStreams = targetStreams;
diff --git a/test-apps/simple-deployable-app-1/build.gradle b/test-apps/simple-deployable-app-1/build.gradle
index 9ad3970..a2b216f 100644
--- a/test-apps/simple-deployable-app-1/build.gradle
+++ b/test-apps/simple-deployable-app-1/build.gradle
@@ -1,18 +1,21 @@
-/*
-* Copyright 2010 the original author or authors.
-*
-* Licensed under the Apache License, Version 2.0 (the "License");
-* you may not use this file except in compliance with the License.
-* You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-*/
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/**
* Apache S4 Application Build File
@@ -61,31 +64,24 @@
/* All project libraries must be defined here. */
project.ext["libraries"] = [
- json: 'org.json:json:20090211',
- guava: 'com.google.guava:guava:10.0.1',
- gson: 'com.google.code.gson:gson:1.6',
- guice: 'com.google.inject:guice:3.0',
- guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
- guice_grapher: 'com.google.inject:guice-grapher:3.0',
- flexjson: 'net.sf.flexjson:flexjson:2.1',
- bcel: 'org.apache.bcel:bcel:5.2',
- jakarta_regexp: 'jakarta-regexp:jakarta-regexp:1.4',
- kryo: 'com.googlecode:kryo:1.04',
- netty: 'org.jboss.netty:netty:3.2.5.Final',
- reflectasm: 'com.esotericsoftware:reflectasm:0.8',
- minlog: 'com.esotericsoftware:minlog:1.2',
- asm: 'asm:asm:3.2',
- commons_io: 'commons-io:commons-io:2.0.1',
- commons_config: 'commons-configuration:commons-configuration:1.6',
- commons_codec: 'commons-codec:commons-codec:1.4',
- commons_httpclient: 'commons-httpclient:commons-httpclient:3.1',
- commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
- slf4j: 'org.slf4j:slf4j-api:1.6.1',
- logback_core: 'ch.qos.logback:logback-core:0.9.29',
- logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
- zk: 'org.apache.zookeeper:zookeeper:3.3.1',
- jcip: 'net.jcip:jcip-annotations:1.0',
- junit: 'junit:junit:4.10',
+ guava: 'com.google.guava:guava:12.0.1',
+ gson: 'com.google.code.gson:gson:1.6',
+ guice: 'com.google.inject:guice:3.0',
+ guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
+ kryo: 'com.googlecode:kryo:1.04',
+ netty: 'org.jboss.netty:netty:3.2.5.Final',
+ mockito_core: 'org.mockito:mockito-core:1.9.0',
+ commons_config: 'commons-configuration:commons-configuration:1.6',
+ commons_codec: 'commons-codec:commons-codec:1.4',
+ commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
+ slf4j: 'org.slf4j:slf4j-api:1.6.1',
+ logback_core: 'ch.qos.logback:logback-core:0.9.29',
+ logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
+ zk: 'org.apache.zookeeper:zookeeper:3.3.1',
+ jcip: 'net.jcip:jcip-annotations:1.0',
+ junit: 'junit:junit:4.10',
+ zkclient: 'com.github.sgroschupf:zkclient:0.1',
+ diezel: 'net.ericaro:diezel-maven-plugin:1.0.0-beta-4'
]
@@ -107,7 +103,6 @@
compile( libraries.logback_classic )
/* Commons. */
- compile( libraries.commons_io )
compile( libraries.commons_config )
compile( libraries.commons_coll )
diff --git a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/A.java b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/A.java
index 45bc37b..a70971c 100644
--- a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/A.java
+++ b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/A.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import org.I0Itec.zkclient.ZkClient;
diff --git a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/AppConstants.java b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/AppConstants.java
index a12b6ea..b727f1e 100644
--- a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/AppConstants.java
+++ b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/AppConstants.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
public class AppConstants {
diff --git a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/SimplePE.java b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/SimplePE.java
index 23fb9cb..40d07aa 100644
--- a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/SimplePE.java
+++ b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/SimplePE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import java.io.IOException;
diff --git a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/TestApp.java b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/TestApp.java
index 389562b..e405393 100644
--- a/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/TestApp.java
+++ b/test-apps/simple-deployable-app-1/src/main/java/org/apache/s4/deploy/TestApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.deploy;
import org.I0Itec.zkclient.ZkClient;
diff --git a/test-apps/simple-deployable-app-2/build.gradle b/test-apps/simple-deployable-app-2/build.gradle
deleted file mode 100644
index 38e8eb7..0000000
--- a/test-apps/simple-deployable-app-2/build.gradle
+++ /dev/null
@@ -1,242 +0,0 @@
-/*
-* Copyright 2010 the original author or authors.
-*
-* Licensed under the Apache License, Version 2.0 (the "License");
-* you may not use this file except in compliance with the License.
-* You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-*/
-
-/**
-* Apache S4 Application Build File
-*
-* Use this script to buils and package S4 apps.
-*
-* Run 'gradle install' on the s4 project to publish to your local maven repo.
-*
-* TODO: This should probably be distributed as an s4 plugin for Gradle.
-* TODO: There seem to be to be similarities with the war and jetty plugins. (war -> s4r, jetty -> s4Run).
-* We should make it easy to test the app from this script by a running a test task that starts and stops
-* an s4 server. See: http://www.gradle.org/releases/1.0-milestone-3/docs/userguide/userguide_single.html#war_plugin
-*
-* This is an interesting discussion:
-* http://gradle.1045684.n5.nabble.com/Exclude-properties-file-from-war-td3365147.html
-*
-*/
-
-/* Set the destination where we want to install the apps. */
-//s4AppInstallDir = "/tmp/s4Apps" // TODO: decide how to standarize dirs, use env var?
-
-project.ext["s4AppInstallDir"] = hasProperty('appsDir') ? "$appsDir" : "/tmp/appsDir"
-
-project.ext["s4Version"] = '0.5.0-SNAPSHOT'
-description = 'Apache S4 App'
-//defaultTasks 'installS4R'
-project.ext["archivesBaseName"] = "$project.name"
-project.ext["distRootFolder"] = "$archivesBaseName-${-> version}"
-
-
-// Append the suffix 'SNAPSHOT' when the build is not for release.
-version = new Version(major: 0, minor: 0, bugfix: 0, isRelease: false)
-group = 'org.apache.s4'
-
-apply plugin: 'java'
-apply plugin: 'eclipse'
-
-/* The app classname is set automatically from the source files. */
-def appClassname = ''
-
-/* Set Java version. */
-sourceCompatibility = 1.6
-targetCompatibility = 1.6
-
-
-
-/* All project libraries must be defined here. */
-project.ext["libraries"] = [
- json: 'org.json:json:20090211',
- guava: 'com.google.guava:guava:10.0.1',
- gson: 'com.google.code.gson:gson:1.6',
- guice: 'com.google.inject:guice:3.0',
- guice_assist: 'com.google.inject.extensions:guice-assistedinject:3.0',
- guice_grapher: 'com.google.inject:guice-grapher:3.0',
- flexjson: 'net.sf.flexjson:flexjson:2.1',
- bcel: 'org.apache.bcel:bcel:5.2',
- jakarta_regexp: 'jakarta-regexp:jakarta-regexp:1.4',
- kryo: 'com.googlecode:kryo:1.04',
- netty: 'org.jboss.netty:netty:3.2.5.Final',
- reflectasm: 'com.esotericsoftware:reflectasm:0.8',
- minlog: 'com.esotericsoftware:minlog:1.2',
- asm: 'asm:asm:3.2',
- commons_io: 'commons-io:commons-io:2.0.1',
- commons_config: 'commons-configuration:commons-configuration:1.6',
- commons_codec: 'commons-codec:commons-codec:1.4',
- commons_httpclient: 'commons-httpclient:commons-httpclient:3.1',
- commons_coll: 'net.sourceforge.collections:collections-generic:4.01', // Use this lib until the commons collection with Generics is released.
- slf4j: 'org.slf4j:slf4j-api:1.6.1',
- logback_core: 'ch.qos.logback:logback-core:0.9.29',
- logback_classic: 'ch.qos.logback:logback-classic:0.9.29',
- zk: 'org.apache.zookeeper:zookeeper:3.3.1',
- jcip: 'net.jcip:jcip-annotations:1.0',
- junit: 'junit:junit:4.10',
- ]
-
-
-dependencies {
-
- /* S4 Platform. We only need the API, not the transitive dependencies. */
-// s4Libs.each { module ->
-// compile( module ) //{ transitive = false }
-// s4API( module )
-// }
-
- compile project(":s4-base")
- compile project(":s4-comm")
- compile project(":s4-core")
-
- /* Logging. */
- compile( libraries.slf4j )
- compile( libraries.logback_core )
- compile( libraries.logback_classic )
-
- /* Commons. */
- compile( libraries.commons_io )
- compile( libraries.commons_config )
- compile( libraries.commons_coll )
-
- /* Misc. */
- compile( libraries.jcip )
-
- /* Testing. */
- testCompile( libraries.junit )
-}
-
-/* Set the manifest attributes for the S4 archive here.
-* TODO: separate custom properties from std ones and set custom properties at the top of the build script.
-*/
-manifest.mainAttributes(
- provider: 'gradle',
- 'Implementation-Url': 'http://incubator.apache.org/projects/s4.html',
- 'Implementation-Version': version,
- 'Implementation-Vendor': 'Apache S4',
- 'Implementation-Vendor-Id': 's4app',
- 'S4-App-Class': appClassname, // gets set by the s4r task.
- 'S4-Version': s4Version
- )
-
-project.ext["appDependencies"] = ( configurations.compile )
-
-
-
-
-
-
-task copyDependenciesToLib(type: Copy) {
- into project.libsDir.path+"/lib"
- from configurations.runtime
-}
-
-task buildProjectJar() {
- dependsOn jar {
- destinationDir file(project.libsDir.path + "/app")
- from sourceSets.main.output
- }
-}
-
-
-
-/* This task will extract all the class files and create a fat jar. We set the manifest and the extension to make it an S4 archive file. */
-// TODO: exclude schenma files as needed (not critical) see: http://forums.gradle.org/gradle/topics/using_gradle_to_fat_jar_a_spring_project
-task s4r(type: Jar) {
- dependsOn cleanCopyDependenciesToLib, copyDependenciesToLib, cleanBuildProjectJar, buildProjectJar
- from { project.libsDir }
- manifest = project.manifest
- extension = 's4r'
-
- /* Set class name in manifest. Parse source files until we find a class that extends App.
- * Get fully qualified Java class name and set attribute in Manifest.
- */
- sourceSets.main.allSource.files.each { File file ->
- if (appClassname =="" || appClassname == "UNKNOWN") {
- // only execute the closure for this file if we haven't already found the app class name
- appClassname = getAppClassname(file)
- if(appClassname != "") {
- manifest.mainAttributes('S4-App-Class': appClassname)
- }
- }
- }
-
- if (appClassname == "UNKNOWN") {
-
- println "Couldn't find App class in source files...aborting."
- exit(1)
- }
-}
-
-/* List the artifacts that will br added to the s4 archive (and explode if needed). */
-s4r << {
- appDependencies.each { File file -> println 'Adding to s4 archive: ' + file.name }
- configurations.archives.allArtifacts.files.each { File file -> println 'Adding to s4 archive: ' + file.name }
-
- /* This is for debugging. */
- //configurations.s4All.each { File file -> println 's4All: ' + file.name }
- //deployableDependencies.each { File file -> println 'Deploy: ' + file.name }
-
- // more debugging statements.
- //sourceSets.main.compileClasspath.each { File file -> println 'compileClasspath: ' + file.name }
-
-}
-
-/* Install the S4 archive to the install directory. */
-task installS4R (type: Copy) {
- dependsOn s4r
- from s4r.archivePath
- into s4AppInstallDir
-}
-
-/* Generates the gradlew scripts.
-http://www.gradle.org/1.0-milestone-3/docs/userguide/gradle_wrapper.html */
-task wrapper(type: Wrapper) { gradleVersion = '1.0-milestone-3' }
-
-
-/* Parse source file to get the app classname so we can use it in the manifest.
-* TODO: Use a real Java parser. (This is not skippong comments for example.)
-*/
-def getAppClassname(file) {
- def classname = "UNKNOWN"
- def lines= file.readLines()
- def packageName = ""
- for(line in lines) {
-
- def pn = line =~ /.*package\s+([\w\.]+)\s*;.*/
- if(pn) {
- packageName = pn[0][1] + "."
- }
-
- def an = line =~ /.*public\s+class\s+(\w+)\s+extends.+App.*\{/
- if (an) {
- classname = packageName + an[0][1]
- println "Found app class name: " + classname
- break
- }
- }
- classname
-}
-
-class Version {
- int major
- int minor
- int bugfix
- boolean isRelease
-
- String toString() {
- "$major.$minor.$bugfix${isRelease ? '' : '-SNAPSHOT'}"
- }
-}
diff --git a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/A.java b/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/A.java
deleted file mode 100644
index 732be13..0000000
--- a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/A.java
+++ /dev/null
@@ -1,16 +0,0 @@
-package org.apache.s4.deploy;
-
-import org.I0Itec.zkclient.ZkClient;
-
-public class A {
-
- public A(ZkClient zkClient) {
- try {
- zkClient.createEphemeral(AppConstants.STARTED_ZNODE_2, null);
- } catch (Exception e) {
- System.exit(-1);
- }
-
- }
-
-}
diff --git a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/AppConstants.java b/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/AppConstants.java
deleted file mode 100644
index a12b6ea..0000000
--- a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/AppConstants.java
+++ /dev/null
@@ -1,10 +0,0 @@
-package org.apache.s4.deploy;
-
-public class AppConstants {
- public static final String STARTED_ZNODE_1 = "/s4-test/test-app-1-started";
- public static final String INITIALIZED_ZNODE_1 = "/s4-test/test-app-1-initialized";
-
- public static final String STARTED_ZNODE_2 = "/s4-test/test-app-2-started";
- public static final String INITIALIZED_ZNODE_2 = "/s4-test/test-app-2-initialized";
-
-}
diff --git a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/TestApp.java b/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/TestApp.java
deleted file mode 100644
index 8eb345b..0000000
--- a/test-apps/simple-deployable-app-2/src/main/java/org/apache/s4/deploy/TestApp.java
+++ /dev/null
@@ -1,38 +0,0 @@
-package org.apache.s4.deploy;
-
-import org.I0Itec.zkclient.ZkClient;
-import org.apache.s4.core.App;
-import org.apache.zookeeper.CreateMode;
-
-public class TestApp extends App {
-
- private ZkClient zkClient;
-
- @Override
- protected void onClose() {
- // TODO Auto-generated method stub
-
- }
-
- @Override
- protected void onInit() {
- try {
- zkClient = new ZkClient("localhost:" + 2181);
- if (!zkClient.exists("/s4-test")) {
- zkClient.create("/s4-test", null, CreateMode.PERSISTENT);
- }
- zkClient.createEphemeral(AppConstants.INITIALIZED_ZNODE_2, null);
- } catch (Exception e) {
- System.exit(-1);
- }
- }
-
- @Override
- protected void onStart() {
- try {
- Class.forName("org.apache.s4.deploy.A").getConstructor(ZkClient.class).newInstance(zkClient);
- } catch (Exception e) {
- System.exit(-1);
- }
- }
-}
diff --git a/test-apps/twitter-adapter/build.gradle b/test-apps/twitter-adapter/build.gradle
index 7157e6f..ab71961 100644
--- a/test-apps/twitter-adapter/build.gradle
+++ b/test-apps/twitter-adapter/build.gradle
@@ -1,18 +1,21 @@
-/*
-* Copyright 2010 the original author or authors.
-*
-* Licensed under the Apache License, Version 2.0 (the "License");
-* you may not use this file except in compliance with the License.
-* You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-*/
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/**
* Apache S4 Application Build File
diff --git a/test-apps/twitter-adapter/src/main/java/org/apache/s4/example/twitter/TwitterInputAdapter.java b/test-apps/twitter-adapter/src/main/java/org/apache/s4/example/twitter/TwitterInputAdapter.java
index 4eb3f2b..9afb04e 100644
--- a/test-apps/twitter-adapter/src/main/java/org/apache/s4/example/twitter/TwitterInputAdapter.java
+++ b/test-apps/twitter-adapter/src/main/java/org/apache/s4/example/twitter/TwitterInputAdapter.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.example.twitter;
import java.io.File;
diff --git a/test-apps/twitter-adapter/src/main/resources/s4.properties b/test-apps/twitter-adapter/src/main/resources/s4.properties
deleted file mode 100644
index fdd1bf6..0000000
--- a/test-apps/twitter-adapter/src/main/resources/s4.properties
+++ /dev/null
@@ -1,18 +0,0 @@
-comm.queue_emmiter_size = 8000
-comm.queue_listener_size = 8000
-cluster.hosts = localhost
-cluster.ports = 5077
-cluster.name = s4-adapter-cluster
-cluster.zk_address = localhost:2181
-cluster.zk_session_timeout = 10000
-cluster.zk_connection_timeout = 10000
-comm.module = org.apache.s4.core.adapter.AdapterModule
-s4.logger_level = DEBUG
-appsDir=/tmp/deploy-test
-tcp.partition.queue_size=1000
-comm.timeout=100
-comm.retry_delay=100
-comm.retries=10
-
-# specify the name of the remote cluster (there is currently only 1 remote cluster max)
-cluster.remote.name=s4-test-cluster
diff --git a/test-apps/twitter-counter/README.txt b/test-apps/twitter-counter/README.txt
index d46b491..17eafb9 100644
--- a/test-apps/twitter-counter/README.txt
+++ b/test-apps/twitter-counter/README.txt
@@ -2,7 +2,7 @@
It was ported and adapted from S4 0.3
Architecture:
-- twitter-adapter app in adapter node connects to the twitter stream, extracts the twitted text and passes that to the application cluster
+- twitter-adapter app in adapter node connects to the twitter stream, extracts the tweeted text and passes that to the application cluster
- twitter-counter app in the application cluster receives the text of the tweets, extracts the topics, counts topic occurences and periodically displays the top 10 topics on the console
How to configure:
@@ -12,22 +12,6 @@
password=<the matching password>
How to run:
-0/ make sure tools are compiled by running ./gradlew s4-tools:installApp
-1/ start zookeeper server
-./s4 zkServer
-
-2/ create adapter cluster configuration
-./s4 newCluster -name=s4-test-cluster -firstListeningPort=10000 -nbTasks=1
-
-3/ create application cluster configuration
-./s4 newCluster -name=s4-adapter-cluster -firstListeningPort=11000 -nbTasks=<number of nodes>
-NOTE: - the name of the downstream cluster is currently hardcoded in <s4-dir>/test-apps/twitter-adapter/src/main/resources/s4.properties Make sure you use the same name
-
-4/ start application nodes (as many as defined in the cluster configuration, more for failover capabilities)
-./s4 appNode <s4-dir>/subprojects/s4-core/src/test/resources/default.s4.properties
-
-5/ start adapter node
-./s4 adapterNode -s4Properties=<s4-dir>/test-apps/twitter-adapter/src/main/resources/s4.properties
-
-6/ observe the topic count output (on 1 of the application nodes)
\ No newline at end of file
+Please follow the instructions in the S4 piper walkthrough at the following place:
+https://cwiki.apache.org/confluence/display/S4/S4+piper+walkthrough
\ No newline at end of file
diff --git a/test-apps/twitter-counter/build.gradle b/test-apps/twitter-counter/build.gradle
index c6b7469..981b508 100644
--- a/test-apps/twitter-counter/build.gradle
+++ b/test-apps/twitter-counter/build.gradle
@@ -1,18 +1,21 @@
-/*
-* Copyright 2010 the original author or authors.
-*
-* Licensed under the Apache License, Version 2.0 (the "License");
-* you may not use this file except in compliance with the License.
-* You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing, software
-* distributed under the License is distributed on an "AS IS" BASIS,
-* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-* See the License for the specific language governing permissions and
-* limitations under the License.
-*/
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/**
* Apache S4 Application Build File
diff --git a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopNTopicPE.java b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopNTopicPE.java
index 2218ca5..d3e68c3 100644
--- a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopNTopicPE.java
+++ b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopNTopicPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.example.twitter;
import java.io.File;
diff --git a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicCountAndReportPE.java b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicCountAndReportPE.java
index 90c3729..01bc5f3 100644
--- a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicCountAndReportPE.java
+++ b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicCountAndReportPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.example.twitter;
import org.apache.s4.core.App;
diff --git a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicEvent.java b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicEvent.java
index 34e936c..7c12c8b 100644
--- a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicEvent.java
+++ b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicEvent.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.example.twitter;
import org.apache.s4.base.Event;
diff --git a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicExtractorPE.java b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicExtractorPE.java
index 9211583..e01a733 100644
--- a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicExtractorPE.java
+++ b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TopicExtractorPE.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.example.twitter;
import java.net.ServerSocket;
diff --git a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TwitterCounterApp.java b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TwitterCounterApp.java
index fc4e3ae..5d7855f 100644
--- a/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TwitterCounterApp.java
+++ b/test-apps/twitter-counter/src/main/java/org/apache/s4/example/twitter/TwitterCounterApp.java
@@ -1,3 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
package org.apache.s4.example.twitter;
import java.util.List;
diff --git a/test-apps/twitter-counter/src/main/resources/default.s4.properties b/test-apps/twitter-counter/src/main/resources/default.s4.properties
deleted file mode 100644
index 16fc4ba..0000000
--- a/test-apps/twitter-counter/src/main/resources/default.s4.properties
+++ /dev/null
@@ -1,19 +0,0 @@
-comm.queue_emmiter_size = 8000
-comm.queue_listener_size = 8000
-comm.retries=10
-comm.retry_delay=10
-comm.timeout=1000
-tcp.partition.queue_size=256
-cluster.hosts = localhost
-cluster.ports = 5077
-cluster.name = s4-adapter-cluster
-cluster.zk_address = localhost:21810
-cluster.zk_session_timeout = 10000
-cluster.zk_connection_timeout = 10000
-comm.module = org.apache.s4.core.CustomModule
-s4.logger_level = DEBUG
-appsDir=/tmp/deploy-test
-tcp.partition.queue_size=1000
-comm.timeout=100
-comm.retry_delay=100
-comm.retries=10
