First commit.
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..9df12b2
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,4 @@
+*.pyc
+.tox
+*.egg-info
+.vagrant
diff --git a/3rdparty/README.md b/3rdparty/README.md
new file mode 100644
index 0000000..d650b96
--- /dev/null
+++ b/3rdparty/README.md
@@ -0,0 +1,4 @@
+Place any bleeding edge versions of packages here so that they may be picked up by tox during
+testing.
+
+This is also the place for the required `mesos.native` packages because they are platform specific.
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..e06d208
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,202 @@
+Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "{}"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright {yyyy} {name of copyright owner}
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..5cb76f4
--- /dev/null
+++ b/README.md
@@ -0,0 +1,54 @@
+# Mysos
+Mysos is an Apache Mesos framework for running MySQL instances. It dramatically simplifies the management of a MySQL cluster and is designed to offer:
+
+ * Efficient hardware utilization through multi-tenancy (in performance-isolated containers)
+ * High reliability through preserving the MySQL state during failure and automatic backing up to/restoring from HDFS
+ * An automated self-service option for bringing up new MySQL clusters
+ * High availability through automatic MySQL master failover
+ * An elastic solution that allows users to easily scale up and down a MySQL cluster by changing the number of slave instances
+
+Mysos is also being [proposed as a project in the Apache Incubator](https://wiki.apache.org/incubator/MysosProposal).
+
+## Documentation
+A [user guide](docs/user-guide.md) is available. Documentation improvements are always welcome, so please send patches our way.
+
+## Getting Involved
+The project maintains an IRC channel, `#mysos` on irc.freenode.net
+
+## License
+Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0
+
+## Requirements
+ * Python 2.7
+ * Mesos Python bindings
+
+## Building and Testing
+### Build/Download Mesos Python Bindings
+Mysos uses Mesos Python bindings which consist of two Python packages. `mesos.interface` is on PyPI
+and gets automatically installed but `mesos.native` is platform dependent. You need to either build
+the package on your machine ([instructions](http://mesos.apache.org/gettingstarted/)) or download a
+compiled one for your platform (e.g. Mesosphere hosts
+[the eggs for some Linux platforms](https://mesosphere.com/downloads/)).
+
+Since `pip` doesn't support eggs, you need to convert eggs into wheels using `wheel convert`, then
+drop them into the `3rdparty` folder. See the [README file](3rdparty/README.md) for more
+information.
+
+### Unit Tests
+Make sure [tox](https://tox.readthedocs.org/en/latest/) is installed and a `mesos.native` wheel for
+your platform is placed in `3rdparty/`. Then run:
+
+ tox
+
+Tox also builds the Mysos source package and drops it in `.tox/dist`.
+
+### End-to-end test in the Vagrant VM
+The Vagrant test uses the `sdist` Mysos package in `.tox/dist` so be sure to run `tox` first. Then:
+
+ vagrant up
+
+ # Wait for the VM and Mysos API endpoint to come up (http://192.168.33.17:55001 becomes available).
+
+ ./vagrant/test.sh
+
+`test.sh` verifies that Mysos successfully creates a MySQL cluster and then deletes it.
diff --git a/Vagrantfile b/Vagrantfile
new file mode 100644
index 0000000..a825670
--- /dev/null
+++ b/Vagrantfile
@@ -0,0 +1,21 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
+VAGRANTFILE_API_VERSION = "2"
+
+# 1.5.0 is required to use vagrant cloud images.
+# https://www.vagrantup.com/blog/vagrant-1-5-and-vagrant-cloud.html
+Vagrant.require_version ">= 1.5.0"
+
+Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+ config.vm.box = "ubuntu/trusty64"
+
+ config.vm.define "devcluster" do |dev|
+ dev.vm.network :private_network, ip: "192.168.33.17"
+ dev.vm.provider :virtualbox do |vb|
+ vb.customize ["modifyvm", :id, "--memory", "2048"]
+ end
+ dev.vm.provision "shell", path: "vagrant/provision-dev-cluster.sh"
+ end
+end
diff --git a/docs/user-guide.md b/docs/user-guide.md
new file mode 100644
index 0000000..682cc90
--- /dev/null
+++ b/docs/user-guide.md
@@ -0,0 +1,88 @@
+# Using Mysos
+
+Mysos provides a REST API for creating and managing MySQL clusters on Mesos.
+
+## Dashboard
+- HTTP Method: `GET`
+- Path: `/`
+
+The root of API endpoint is a web page that lists the managed MySQL clusters.
+
+## Creating a MySQL cluster
+- HTTP Method: `POST`
+- Path: `/clusters/<cluster_name>`
+
+
+### Parameters
+- `cluster_name`: Name of the cluster.
+- `cluster_user`: The user account for all MySQL instances in the cluster which as full admin
+privileges.
+- `num_nodes`: Number of nodes in the cluster. [default: 3]
+- `backup_id`: An ID for the MySQL backup to restore from when the MySQL instance starts. If not
+specified, Mysos will start an empty MySQL instance. The format and meaning of `backup_id` is
+specific to the implementation of `BackupStore` that the Mysos cluster uses.
+
+`cluster_name` is part of the path and the rest of the parameters are specified as form fields.
+
+
+### Response
+A JSON object with the following fields:
+
+- `cluster_password`: The password for accessing the MySQL instance (associated with `cluster_user`).
+- `cluster_url`: A URL to the ZooKeeper group for discovering the MySQL instances of this cluster.
+See the *Service Discovery* section below.
+
+
+### Example
+```
+# Create a cluster named 'test_cluster3' and restore from the backup 'foo/bar:201503122000'.
+curl -X POST 192.168.33.7/clusters/test_cluster3 --form "cluster_user=mysos" --form "num_nodes=2" --form "num_nodes=2" --form "backup_id=foo/bar:201503122000"
+# Response
+{"cluster_password": "w9gMCkecsMh6sWsRdxNTa", "cluster_url": "zk://mysos:mysos@192.168.33.7:2181/mysos/discover/test_cluster3"}
+```
+
+### Notes
+- Cluster creation is asynchronous. The API call returns (with status 200) as soon as the Mysos
+scheduler has accepted the request. The same goes for cluster deletion.
+- ZooKeeper `<cluster_url>/master` sub-group has at most one ZNode which is the master of the MySQL
+cluster.
+- ZooKeeper `<cluster_url>/slaves` sub-group can have multiple ZNodes which are the slaves of the
+MySQL cluster.
+- A ZNode is added to the ZooKeeper group when the instance becomes available and ready to serve
+traffic.
+
+## Removing a MySQL cluster
+- HTTP Method: `DELETE`
+- Path: `/clusters/<cluster_name>`
+
+### Parameters
+- `cluster_name`: Name of the cluster.
+- `password`: The password for the cluster returned by cluster creation call.
+
+### Response
+A JSON object with:
+
+- `cluster_url`: A URL to the ZooKeeper group to watch for the termination of the cluster. The group
+ ZNode is removed from ZooKeeper when the MySQL cluster is removed/terminated.
+
+### Example
+```
+# Remove a cluster named 'test_cluster3'
+curl -X DELETE 192.168.33.7/cluster/test_cluster3 --form "password=w9gMCkecsMh6sWsRdxNTa"
+# Response
+{"cluster_url": "zk://mysos:mysos@192.168.33.7:2181/mysos/discover/test_cluster3"}
+```
+
+## Service Discovery
+Mysos' service discovery with ZooKeeper conforms to the ServerSet protocol. Each MySQL instance is
+represented by a ZNode with its data being a
+[ServiceInstance](https://github.com/twitter/commons/blob/master/src/thrift/com/twitter/thrift/endpoint.thrift)
+serialized into JSON.
+
+- The `Endpoint serviceEndpoint` field in ServiceInstance has the `host` and `port` that MySQL
+client can connect to.
+- Some utilities for watching ZooKeeper and parsing the `ServiceInstance`s:
+[Java src](https://github.com/twitter/commons/tree/master/src/java/com/twitter/common/zookeeper) |
+[Maven](http://maven.twttr.com/com/twitter/zookeeper-client/LATEST/),
+[Python src](https://github.com/twitter/commons/tree/master/src/python/twitter/common/zookeeper) |
+[PyPI](https://pypi.python.org/pypi/twitter.common.zookeeper/).
diff --git a/setup.py b/setup.py
new file mode 100644
index 0000000..b201d8a
--- /dev/null
+++ b/setup.py
@@ -0,0 +1,76 @@
+import os
+from setuptools import setup, find_packages
+
+
+COMMONS_VERSION = '==0.3.2'
+MESOS_VERSION = '==0.20.1'
+
+
+here = os.path.abspath(os.path.dirname(__file__))
+
+
+def make_commons_requirement(name):
+ return 'twitter.common.{0}{1}'.format(name, COMMONS_VERSION)
+
+
+def list_package_data_files(package_root, data_folder):
+ """List the data files in the data_folder under the given package_root."""
+ paths = []
+ for root, _, files in os.walk(os.path.join(package_root, data_folder)):
+ for filename in files:
+ paths.append(os.path.relpath(os.path.join(root, filename), package_root))
+
+ return paths
+
+
+setup(
+ name='mysos',
+ version='0.1.0-dev0',
+ description='Mysos (MySQL on Mesos)',
+ url='https://github.com/twitter/mysos',
+ license='Apache License, Version 2.0',
+ classifiers=[
+ 'Development Status :: 3 - Alpha',
+ 'Programming Language :: Python :: 2',
+ 'Programming Language :: Python :: 2.7',
+ ],
+ keywords='mesos mysql',
+ packages=find_packages(exclude=['tests*']),
+ namespace_packages=['twitter'],
+ package_data={
+ '': (list_package_data_files('twitter/mysos/executor', 'files') +
+ list_package_data_files('twitter/mysos/scheduler', 'assets'))
+ },
+ install_requires=[
+ 'cherrypy==3.2.2',
+ 'kazoo==1.3.1',
+ 'mako==0.4.0',
+ 'mesos.interface{0}'.format(MESOS_VERSION),
+ 'mesos.native{0}'.format(MESOS_VERSION),
+ 'mysql-python',
+ 'pyyaml==3.10',
+ 'sqlalchemy',
+ 'zake==0.2.1',
+ make_commons_requirement('app'),
+ make_commons_requirement('collections'),
+ make_commons_requirement('concurrent'),
+ make_commons_requirement('exceptions'),
+ make_commons_requirement('http'),
+ make_commons_requirement('lang'),
+ make_commons_requirement('log'),
+ make_commons_requirement('quantity'),
+ make_commons_requirement('zookeeper'),
+ ],
+ extras_require={
+ 'test': ['webtest',],
+ },
+ entry_points={
+ 'console_scripts': [
+ 'mysos_scheduler=twitter.mysos.scheduler.mysos_scheduler:proxy_main',
+ 'mysos_executor=twitter.mysos.executor.mysos_executor:proxy_main',
+ 'vagrant_mysos_executor=twitter.mysos.executor.testing.vagrant_mysos_executor:proxy_main',
+ 'fake_mysos_executor=twitter.mysos.executor.testing.fake_mysos_executor:proxy_main',
+ 'mysos_test_client=twitter.mysos.testing.mysos_test_client:proxy_main',
+ ],
+ },
+)
diff --git a/tests/mysos/common/test_cluster.py b/tests/mysos/common/test_cluster.py
new file mode 100644
index 0000000..2624b5c
--- /dev/null
+++ b/tests/mysos/common/test_cluster.py
@@ -0,0 +1,209 @@
+import Queue
+import threading
+import unittest
+
+from twitter.common.zookeeper.serverset.endpoint import Endpoint, ServiceInstance
+from twitter.mysos.common.cluster import ClusterListener, ClusterManager
+
+from kazoo.handlers.threading import SequentialThreadingHandler
+import pytest
+from zake.fake_client import FakeClient
+from zake.fake_storage import FakeStorage
+
+
+class CallbackHandler(object):
+ """Utility for testing callbacks."""
+
+ def __init__(self):
+ self.promoted = threading.Event()
+ self.demoted = threading.Event()
+ self.detected = Queue.Queue()
+ self.terminated = threading.Event()
+
+ def promotion_callback(self):
+ self.promoted.set()
+
+ def demotion_callback(self):
+ self.demoted.set()
+
+ def master_callback(self, master):
+ self.detected.put(master)
+
+ def termination_callback(self):
+ self.terminated.set()
+
+
+class TestCluster(unittest.TestCase):
+ def setUp(self):
+ self.storage = FakeStorage(SequentialThreadingHandler())
+ self.client = FakeClient(storage=self.storage)
+ self.client.start()
+
+ def tearDown(self):
+ self.client.stop()
+
+ def test_add_member(self):
+ manager = ClusterManager(self.client, "/home/my_cluster")
+
+ instance1 = ServiceInstance(Endpoint("host1", 10000))
+ member1 = manager.add_member(instance1)
+ assert member1 == manager.add_member(instance1) # Second insertion is ignored.
+
+ instance2 = ServiceInstance(Endpoint("host2", 10000))
+ manager.add_member(instance2)
+
+ assert len(manager._cluster.members) == 2
+
+ assert (self.storage.paths["/home/my_cluster/slaves/member_0000000000"]["data"] ==
+ ServiceInstance.pack(instance1))
+ assert (self.storage.paths["/home/my_cluster/slaves/member_0000000001"]["data"] ==
+ ServiceInstance.pack(instance2))
+
+ def test_promote_member(self):
+ manager = ClusterManager(self.client, "/home/my_cluster")
+ instance = ServiceInstance(Endpoint("host", 10000))
+ member = manager.add_member(instance)
+
+ assert manager.promote_member(member)
+ assert not manager.promote_member(member) # The 2nd promotion is a no-op.
+
+ assert (self.storage.paths["/home/my_cluster/master/member_0000000000"]["data"] ==
+ ServiceInstance.pack(instance))
+
+ def test_remove_member(self):
+ manager = ClusterManager(self.client, "/home/my_cluster")
+ instance = ServiceInstance(Endpoint("host", 10000))
+ member = manager.add_member(instance)
+
+ assert manager.remove_member(member)
+ assert not manager.remove_member(member) # The second deletion is ignored.
+
+ assert "/home/my_cluster/master/member_0000000000" not in self.storage.paths
+
+ def test_callbacks(self):
+ manager = ClusterManager(self.client, "/home/my_cluster")
+
+ # Set up 2 listeners.
+ instance1 = ServiceInstance(Endpoint("host1", 10000))
+ handler1 = CallbackHandler()
+ listener1 = ClusterListener(
+ self.client,
+ "/home/my_cluster",
+ instance1,
+ handler1.promotion_callback,
+ handler1.demotion_callback,
+ handler1.master_callback,
+ handler1.termination_callback)
+ listener1.start()
+ member1 = manager.add_member(instance1)
+
+ instance2 = ServiceInstance(Endpoint("host2", 10000))
+ handler2 = CallbackHandler()
+ listener2 = ClusterListener(
+ self.client,
+ "/home/my_cluster",
+ instance2,
+ handler2.promotion_callback,
+ handler2.demotion_callback,
+ handler2.master_callback)
+ listener2.start()
+ member2 = manager.add_member(instance2)
+
+ # Test promotion.
+ manager.promote_member(member1)
+
+ assert handler1.promoted.wait(1)
+ assert handler2.detected.get(True, 1) == instance1
+
+ assert (self.storage.paths["/home/my_cluster/master/member_0000000000"]["data"] ==
+ ServiceInstance.pack(instance1))
+ assert (self.storage.paths["/home/my_cluster/slaves/member_0000000001"]["data"] ==
+ ServiceInstance.pack(instance2))
+
+ manager.promote_member(member2)
+
+ assert handler1.demoted.wait(1)
+ assert handler2.promoted.wait(1)
+
+ assert (self.storage.paths["/home/my_cluster/master/member_0000000001"]["data"] ==
+ ServiceInstance.pack(instance2))
+ assert "/home/my_cluster/master/member_0000000000" not in self.storage.paths
+
+ manager.remove_member(member2)
+ assert handler2.demoted.wait(1)
+
+ # Test removing cluster.
+ manager.remove_member(member1)
+ manager.delete_cluster()
+ assert handler1.terminated.wait(1)
+
+ def test_invalid_arguments(self):
+ client = FakeClient()
+ client.start()
+ manager = ClusterManager(client, "/home/my_cluster")
+
+ with pytest.raises(ValueError) as e:
+ manager.promote_member("123")
+ assert e.value.message == 'Invalid member_id: 123'
+
+ def test_invalid_znode(self):
+ instance1 = ServiceInstance(Endpoint("host1", 10000))
+ handler1 = CallbackHandler()
+ listener1 = ClusterListener(
+ self.client,
+ "/home/my_cluster",
+ instance1,
+ handler1.promotion_callback,
+ handler1.demotion_callback,
+ handler1.master_callback)
+ listener1.start()
+
+ self.client.ensure_path("/home/my_cluster/master")
+ self.client.create("/home/my_cluster/master/member_", "Invalid Data", sequence=True)
+
+ # Invalid ZNode data translates into a 'None' return.
+ assert handler1.detected.get(True, 1) is None
+
+ def test_existing_zk(self):
+ """
+ ClusterManager needs to be able to recover from an existing ZK group for scheduler failover.
+ """
+ manager = ClusterManager(self.client, "/home/my_cluster")
+
+ instance1 = ServiceInstance(Endpoint("host1", 10000))
+ member1 = manager.add_member(instance1)
+ instance2 = ServiceInstance(Endpoint("host2", 10000))
+ member2 = manager.add_member(instance2)
+
+ assert (self.storage.paths["/home/my_cluster/slaves/member_0000000000"]["data"] ==
+ ServiceInstance.pack(instance1))
+ assert (self.storage.paths["/home/my_cluster/slaves/member_0000000001"]["data"] ==
+ ServiceInstance.pack(instance2))
+
+ manager.promote_member(member1)
+
+ # Test the new ClusterManager.
+ manager2 = ClusterManager(self.client, "/home/my_cluster")
+ assert len(manager2._cluster.members) == 2
+ assert member1 in manager2._cluster.members
+ assert member2 in manager2._cluster.members
+ assert manager2._cluster.members[member1] == ServiceInstance.pack(instance1)
+
+ def test_remove_cluster(self):
+ manager = ClusterManager(self.client, "/home/my_cluster")
+
+ instance1 = ServiceInstance(Endpoint("host1", 10000))
+ member1 = manager.add_member(instance1)
+ instance2 = ServiceInstance(Endpoint("host2", 10000))
+ member2 = manager.add_member(instance2)
+
+ manager.promote_member(member1)
+
+ with pytest.raises(ClusterManager.Error) as e:
+ manager.delete_cluster()
+
+ manager.remove_member(member1)
+ manager.remove_member(member2)
+ manager.delete_cluster()
+
+ assert "/home/my_cluster" not in self.storage.paths
diff --git a/tests/mysos/common/test_zookeeper.py b/tests/mysos/common/test_zookeeper.py
new file mode 100644
index 0000000..433e2a8
--- /dev/null
+++ b/tests/mysos/common/test_zookeeper.py
@@ -0,0 +1,26 @@
+from twitter.mysos.common.zookeeper import parse
+
+import pytest
+
+
+def test_parse():
+ assert parse("zk://host1:port1") == (None, "host1:port1", "/")
+ assert parse("zk://jake:1@host1:port1") == ("jake:1", "host1:port1", "/")
+ assert parse("zk://jake:1@host1:port1/") == ("jake:1", "host1:port1", "/")
+ assert (parse("zk://jake:1@host1:port1,host2:port2") ==
+ ("jake:1", "host1:port1,host2:port2", "/"))
+ assert (parse("zk://jake:1@host1:port1,host2:port2/") ==
+ ("jake:1", "host1:port1,host2:port2", "/"))
+ assert (parse("zk://jake:1@host1:port1,host2:port2/path/to/znode") ==
+ ("jake:1", "host1:port1,host2:port2", "/path/to/znode"))
+
+
+def test_parse_errors():
+ with pytest.raises(ValueError) as e:
+ parse("host1:port1")
+ assert e.value.message == "Expecting 'zk://' at the beginning of the URL"
+
+ # This method doesn't validate the values in the tuple.
+ assert parse("zk://") == (None, "", "/")
+ assert parse("zk://host_no_port") == (None, "host_no_port", "/")
+ assert parse("zk://jake@host") == ("jake", "host", "/")
diff --git a/tests/mysos/executor/test_mysos_task_runner.py b/tests/mysos/executor/test_mysos_task_runner.py
new file mode 100644
index 0000000..48c16ed
--- /dev/null
+++ b/tests/mysos/executor/test_mysos_task_runner.py
@@ -0,0 +1,207 @@
+import os
+import signal
+import unittest
+
+from twitter.common.concurrent import deadline
+from twitter.common.quantity import Amount, Time
+from twitter.common.zookeeper.serverset.endpoint import Endpoint, ServiceInstance
+from twitter.mysos.common.cluster import ClusterManager
+from twitter.mysos.common.testing import Fake
+from twitter.mysos.executor.noop_installer import NoopPackageInstaller
+from twitter.mysos.executor.mysos_task_runner import MysosTaskRunner
+from twitter.mysos.executor.task_runner import TaskError
+from twitter.mysos.executor.testing.fake import FakeTaskControl
+
+from kazoo.handlers.threading import SequentialThreadingHandler
+import pytest
+from zake.fake_client import FakeClient
+from zake.fake_storage import FakeStorage
+
+
+if 'MYSOS_DEBUG' in os.environ:
+ from twitter.common import log
+ from twitter.common.log.options import LogOptions
+ LogOptions.set_stderr_log_level('google:DEBUG')
+ LogOptions.set_simple(True)
+ log.init('mysos_tests')
+
+
+class FakeStateManager(Fake): pass
+
+
+class TestTaskRunner(unittest.TestCase):
+ def setUp(self):
+ self._storage = FakeStorage(SequentialThreadingHandler())
+ self._client = FakeClient(storage=self._storage)
+ self._client.start()
+ self._self_instance = ServiceInstance(Endpoint("host", 10000))
+ self._state_manager = FakeStateManager()
+
+ def tearDown(self):
+ self._client.stop()
+
+ def test_stop(self):
+ task_control = FakeTaskControl()
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+ runner.start()
+ assert runner.stop()
+
+ # Killed by SIGTERM.
+ assert deadline(runner.join, Amount(1, Time.SECONDS)) == -signal.SIGTERM
+
+ def test_demote(self):
+ task_control = FakeTaskControl()
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ manager = ClusterManager(self._client, "/home/test/my_cluster")
+ runner.start()
+
+ self_member = manager.add_member(self._self_instance)
+
+ # 'self_instance' becomes the master.
+ manager.promote_member(self_member)
+
+ runner.promoted.wait(1)
+
+ another_member = manager.add_member(ServiceInstance(Endpoint("another_host", 10000)))
+
+ # This demotes 'self_instance', which should cause runner to stop.
+ manager.promote_member(another_member)
+
+ assert deadline(runner.join, Amount(1, Time.SECONDS))
+
+ def test_reparent(self):
+ task_control = FakeTaskControl()
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ manager = ClusterManager(self._client, "/home/test/my_cluster")
+ runner.start()
+
+ # Promote another instance.
+ master = ServiceInstance(Endpoint("another_host", 10000))
+ another_member = manager.add_member(master)
+ manager.promote_member(another_member)
+
+ assert runner.master.get(True, 1) == master
+
+ assert runner.stop()
+ assert deadline(runner.join, Amount(1, Time.SECONDS))
+
+ def test_mysqld_error(self):
+ task_control = FakeTaskControl(mysqld="exit 123")
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ runner.start()
+ assert deadline(runner.join, Amount(1, Time.SECONDS)) == 123
+
+ def test_start_command_error(self):
+ task_control = FakeTaskControl(start_cmd="exit 1")
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ with pytest.raises(TaskError) as e:
+ runner.start()
+ assert e.value.message.startswith("Failed to start MySQL task")
+
+ def test_promote_command_error(self):
+ task_control = FakeTaskControl(promote_cmd="exit 1")
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ manager = ClusterManager(self._client, "/home/test/my_cluster")
+ runner.start()
+
+ self_member = manager.add_member(self._self_instance)
+
+ # 'self_instance' becomes the master.
+ manager.promote_member(self_member)
+
+ runner.promoted.wait(1)
+
+ with pytest.raises(TaskError) as e:
+ runner.join()
+ assert e.value.message.startswith("Failed to promote the slave")
+
+ def test_get_log_position(self):
+ task_control = FakeTaskControl(position=1)
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ runner.start()
+ assert runner.get_log_position() == 1
+
+ def test_get_log_position_error(self):
+ task_control = FakeTaskControl(get_log_position_cmd="exit 1")
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ with pytest.raises(TaskError) as e:
+ runner.get_log_position()
+ assert (e.value.message ==
+ "Unable to get the slave's log position: " +
+ "Command 'exit 1' returned non-zero exit status 1")
+
+ def test_stop_interminable(self):
+ cmd = """trap "echo Trapped SIGTERM!" SIGTERM
+while :
+do
+ sleep 60
+done
+"""
+ task_control = FakeTaskControl(mysqld=cmd)
+ runner = MysosTaskRunner(
+ self._self_instance,
+ self._client,
+ "/home/test/my_cluster",
+ NoopPackageInstaller(),
+ task_control,
+ self._state_manager)
+
+ task_control._mysqld = cmd
+ runner.start()
+ assert runner.stop(timeout=1)
+ assert deadline(runner.join, Amount(1, Time.SECONDS)) == -signal.SIGKILL
diff --git a/tests/mysos/scheduler/test_elector.py b/tests/mysos/scheduler/test_elector.py
new file mode 100644
index 0000000..f817314
--- /dev/null
+++ b/tests/mysos/scheduler/test_elector.py
@@ -0,0 +1,135 @@
+import Queue
+import os
+import unittest
+
+from twitter.common.quantity import Amount, Time
+from twitter.mysos.common.testing import Fake
+from twitter.mysos.scheduler.elector import MySQLMasterElector
+
+
+if 'MYSOS_DEBUG' in os.environ:
+ from twitter.common import log
+ from twitter.common.log.options import LogOptions
+ LogOptions.set_stderr_log_level('google:DEBUG')
+ LogOptions.set_simple(True)
+ log.init('mysos_tests')
+
+
+class FakeDriver(Fake): pass
+
+
+class TestElector(unittest.TestCase):
+ def setUp(self):
+ self._driver = FakeDriver()
+ self._epoch = 1
+ self._result = Queue.Queue()
+ self._elector = MySQLMasterElector(
+ self._driver,
+ "cluster0",
+ self._epoch,
+ lambda x: self._result.put(x),
+ Amount(1, Time.SECONDS),
+ Amount(150, Time.MILLISECONDS))
+ self._elector.start()
+
+ def tearDown(self):
+ if self._elector: # Terminate the elector if it's not used in the test.
+ self._elector.abort()
+ self._elector.join()
+
+ def test_single_slave(self):
+ slave1 = ("task_id1", "slave_id1")
+ self._elector.add_slave(*slave1)
+ self._elector.update_position(self._epoch, slave1[0], 1)
+
+ assert self._result.get(True, 1) == slave1[0]
+
+ def test_two_slaves(self):
+ slave1 = ("task_id1", "slave_id1")
+ slave2 = ("task_id2", "slave_id2")
+ self._elector.add_slave(*slave1)
+ self._elector.add_slave(*slave2)
+
+ self._elector.update_position(self._epoch, slave1[0], 1)
+ self._elector.update_position(self._epoch, slave2[0], 2)
+
+ assert self._result.get(True, 1) == slave2[0]
+
+ def test_two_slaves_complex_position(self):
+ slave1 = ("task_id1", "slave_id1")
+ slave2 = ("task_id2", "slave_id2")
+ self._elector.add_slave(*slave1)
+ self._elector.add_slave(*slave2)
+
+ # The positions are sequences of numeric strings.
+ self._elector.update_position(self._epoch, slave1[0], ["1", "2"])
+ self._elector.update_position(self._epoch, slave2[0], ["2", "1"])
+
+ assert self._result.get(True, 1) == slave2[0]
+
+ def test_delayed_update(self):
+ slave1 = ("task_id1", "slave_id1")
+ slave2 = ("task_id2", "slave_id2")
+ self._elector.add_slave(*slave1)
+ self._elector.add_slave(*slave2)
+ self._elector.update_position(self._epoch, slave2[0], 2)
+
+ # Force an election (after timing out) and test that slave2 is elected because it's the only
+ # slave that responded.
+ self._elector._elect(timedout=True)
+ assert self._result.get(True, 1) == slave2[0]
+
+ def test_position_for_invalid_slave(self):
+ slave1 = ("task_id1", "slave_id1")
+ slave2 = ("task_id2", "slave_id2")
+ self._elector.update_position(self._epoch, slave1[0], 100) # This update is ignored.
+ self._elector.add_slave(*slave1)
+ self._elector.add_slave(*slave2)
+ self._elector.update_position(self._epoch, slave2[0], 1)
+
+ # Timeout is 1 second. Testing the organic 'timeout' of the thread.
+ assert self._result.get(True, 2) == slave2[0]
+
+ def test_position_for_previous_epoch(self):
+ slave1 = ("task_id1", "slave_id1")
+ slave2 = ("task_id2", "slave_id2")
+ self._elector.add_slave(*slave1)
+ self._elector.add_slave(*slave2)
+
+ self._elector.update_position(self._epoch - 1, slave1[0], 100) # Update from a previous epoch.
+ self._elector.update_position(self._epoch, slave2[0], 1)
+
+ # Induce an election after it timed out.
+ self._elector._elect(timedout=True)
+ assert self._result.get(True, 1) == slave2[0]
+
+ def test_remove_slave_after_election(self):
+ slave1 = ("task_id1", "slave_id1")
+ slave2 = ("task_id2", "slave_id2")
+ self._elector.add_slave(*slave1)
+ self._elector.add_slave(*slave2)
+
+ self._elector.update_position(self._epoch, slave1[0], 1)
+ self._elector.update_position(self._epoch, slave2[0], 2)
+
+ assert self._result.get(True, 1) == slave2[0]
+
+ # At this point a master is already elected. Slave removal is ignored.
+ self._elector.remove_slave(slave2[0])
+
+ assert len(self._elector._positions) == 2
+
+ def test_remove_slave_during_election(self):
+ slave1 = ("task_id1", "slave_id1")
+ slave2 = ("task_id2", "slave_id2")
+ self._elector.add_slave(*slave1)
+ self._elector.add_slave(*slave2)
+
+ self._elector.update_position(self._epoch, slave2[0], 2)
+
+ # Election still ongoing. Removing slave2 allows slave1 to be elected.
+ self._elector.remove_slave(slave2[0])
+ self._elector.update_position(self._epoch, slave1[0], 1)
+
+ self._elector._elect()
+ assert self._result.get(True, 1) == slave1[0]
diff --git a/tests/mysos/scheduler/test_http.py b/tests/mysos/scheduler/test_http.py
new file mode 100644
index 0000000..55eae33
--- /dev/null
+++ b/tests/mysos/scheduler/test_http.py
@@ -0,0 +1,73 @@
+import json
+import shutil
+import tempfile
+import unittest
+
+from twitter.mysos.common import pkgutil
+from twitter.mysos.scheduler.http import MysosServer
+from twitter.mysos.scheduler.scheduler import MysosScheduler
+
+import pytest
+from webtest import AppError, TestApp
+
+
+MYSOS_MODULE = 'twitter.mysos.scheduler'
+ASSET_RELPATH = 'assets'
+
+
+class FakeScheduler(object):
+ def __init__(self):
+ self._exception = None
+ self._response = None
+
+ def set_exception(self, exception):
+ self._exception = exception
+
+ def set_response(self, response):
+ self._response = response
+
+ def create_cluster(self, cluster_name, cluster_user, num_nodes, backup_id=None):
+ if self._exception:
+ raise self._exception
+ return self._response
+
+
+class TestHTTP(unittest.TestCase):
+ @classmethod
+ def setUpClass(cls):
+ cls.web_assets_dir = tempfile.mkdtemp()
+ pkgutil.unpack_assets(cls.web_assets_dir, MYSOS_MODULE, ASSET_RELPATH)
+
+ @classmethod
+ def tearDownClass(cls):
+ shutil.rmtree(cls.web_assets_dir)
+
+ def setUp(self):
+ self._scheduler = FakeScheduler()
+ self._app = TestApp(MysosServer(self._scheduler, self.web_assets_dir).app)
+
+ def test_create_cluster_successful(self):
+ response = ('test_cluster_url', 'passwordfortestcluster')
+ self._scheduler.set_response(response)
+ body = self._app.post(
+ '/clusters/test_cluster', {'num_nodes': 3, 'cluster_user': 'mysos'}).normal_body
+ assert json.loads(body) == dict(cluster_url=response[0], cluster_password=response[1])
+
+ def test_create_cluster_exists(self):
+ self._scheduler.set_exception(MysosScheduler.ClusterExists())
+
+ with pytest.raises(AppError) as e:
+ assert self._app.post('/clusters/test_cluster', {'num_nodes': 3, 'cluster_user': 'mysos'})
+ assert e.value.message.startswith('Bad response: 409')
+
+ def test_create_cluster_value_error(self):
+ self._scheduler.set_exception(ValueError())
+ with pytest.raises(AppError) as e:
+ self._app.post('/clusters/test_cluster', {'num_nodes': 3, 'cluster_user': 'mysos'})
+ assert e.value.message.startswith('Bad response: 400')
+
+ def test_create_cluster_invalid_user(self):
+ self._scheduler.set_exception(MysosScheduler.InvalidUser())
+ with pytest.raises(AppError) as e:
+ self._app.post('/clusters/test_cluster', {'num_nodes': 3, 'cluster_user': 'mysos'})
+ assert e.value.message.startswith('Bad response: 400')
diff --git a/tests/mysos/scheduler/test_launcher.py b/tests/mysos/scheduler/test_launcher.py
new file mode 100644
index 0000000..325038a
--- /dev/null
+++ b/tests/mysos/scheduler/test_launcher.py
@@ -0,0 +1,549 @@
+import json
+import os
+import shutil
+import tempfile
+import threading
+
+from twitter.common import log
+from twitter.common.concurrent import deadline
+from twitter.common.quantity import Amount, Time
+from twitter.mysos.common.cluster import get_cluster_path, wait_for_master
+from twitter.mysos.common.testing import Fake
+from twitter.mysos.scheduler.launcher import create_resources, MySQLClusterLauncher
+from twitter.mysos.scheduler.state import LocalStateProvider, MySQLCluster
+from twitter.mysos.scheduler.zk_state import ZooKeeperStateProvider
+
+from kazoo.handlers.threading import SequentialThreadingHandler
+import mesos.interface.mesos_pb2 as mesos_pb2
+from zake.fake_client import FakeClient
+from zake.fake_storage import FakeStorage
+
+import pytest
+
+
+if 'MYSOS_DEBUG' in os.environ:
+ from twitter.common.log.options import LogOptions
+ LogOptions.set_stderr_log_level('google:DEBUG')
+ LogOptions.set_simple(True)
+ log.init('mysos_tests')
+
+
+class FakeDriver(Fake): pass
+
+
+class TestLauncher(object):
+ @pytest.fixture(params=[LocalStateProvider, ZooKeeperStateProvider], autouse=True)
+ def setup(self, request):
+ self._driver = FakeDriver()
+ self._storage = FakeStorage(SequentialThreadingHandler())
+ self._zk_client = FakeClient(storage=self._storage)
+ self._zk_client.start()
+
+ self._offer = mesos_pb2.Offer()
+ self._offer.id.value = "offer_id_0"
+ self._offer.framework_id.value = "framework_id_0"
+ self._offer.slave_id.value = "slave_id_0"
+ self._offer.hostname = "localhost"
+
+ # Enough memory and ports to fit three tasks.
+ resources = create_resources(cpus=4, mem=512 * 3, ports=set([10000, 10001, 10002]))
+ self._offer.resources.extend(resources)
+
+ self._framework_user = "framework_user"
+
+ # Some tests use the default launcher; some don't.
+ self._zk_url = "zk://host/mysos/test"
+ self._cluster = MySQLCluster("cluster0", "user", "pass", 3)
+
+ # Construct the state provider based on the test parameter.
+ if request.param == LocalStateProvider:
+ tmpdir = tempfile.mkdtemp()
+ self._state_provider = LocalStateProvider(tmpdir)
+ request.addfinalizer(lambda: shutil.rmtree(tmpdir, True)) # Clean up after ourselves.
+ elif request.param == ZooKeeperStateProvider:
+ self._state_provider = ZooKeeperStateProvider(self._zk_client, "/mysos/test")
+
+ self._launcher = MySQLClusterLauncher(
+ self._driver,
+ self._cluster,
+ self._state_provider,
+ self._zk_url,
+ self._zk_client,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml",
+ query_interval=Amount(150, Time.MILLISECONDS)) # Short interval.
+
+ self._elected = threading.Event()
+ self._launchers = [self._launcher] # See teardown().
+
+ request.addfinalizer(self.teardown)
+
+ def teardown(self):
+ for launcher in self._launchers:
+ if launcher._elector:
+ launcher._elector.abort() # Abort the thread even if the election is pending.
+ launcher._elector.join()
+
+ def test_launch_cluster_all_nodes_successful(self):
+ for i in range(self._cluster.num_nodes):
+ task_id, remaining = self._launcher.launch(self._offer)
+ del self._offer.resources[:]
+ self._offer.resources.extend(remaining)
+ assert task_id == "mysos-cluster0-%s" % i
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == self._cluster.num_nodes
+
+ # No new tasks are launched.
+ assert self._launcher.launch(self._offer)[0] is None
+ assert len(self._driver.method_calls["launchTasks"]) == self._cluster.num_nodes
+
+ # All 3 nodes have successfully started.
+ status = mesos_pb2.TaskStatus()
+ status.state = mesos_pb2.TASK_RUNNING # Valid state.
+ status.slave_id.value = self._offer.slave_id.value
+ for i in range(self._cluster.num_nodes):
+ status.task_id.value = "mysos-cluster0-%s" % i
+ self._launcher.status_update(status)
+
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The first slave is elected.
+ assert "/mysos/test/cluster0/master/member_0000000000" in self._storage.paths
+ # Two slaves.
+ assert len([x for x in self._storage.paths.keys() if x.startswith(
+ "/mysos/test/cluster0/slaves/member_")]) == 2
+
+ def test_launch_cluster_insufficient_resources(self):
+ """All but one slave in the slave are launched successfully."""
+ del self._offer.resources[:]
+ resources = create_resources(cpus=4, mem=512 * 3, ports=set([10000, 10001]))
+ self._offer.resources.extend(resources)
+
+ # There is one fewer port than required to launch the entire cluster.
+ for i in range(self._cluster.num_nodes - 1):
+ task_id, remaining = self._launcher.launch(self._offer)
+ del self._offer.resources[:]
+ self._offer.resources.extend(remaining)
+ assert task_id == "mysos-cluster0-%s" % i
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == self._cluster.num_nodes - 1
+
+ # The final task cannot get launched.
+ assert self._launcher.launch(self._offer)[0] is None
+ assert len(self._driver.method_calls["launchTasks"]) == self._cluster.num_nodes - 1
+
+ # The two nodes have successfully started.
+ status = mesos_pb2.TaskStatus()
+ status.state = mesos_pb2.TASK_RUNNING # Valid state.
+ status.slave_id.value = self._offer.slave_id.value
+ for i in range(self._cluster.num_nodes - 1):
+ status.task_id.value = "mysos-cluster0-%s" % i
+ self._launcher.status_update(status)
+
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The first slave is elected.
+ assert "/mysos/test/cluster0/master/member_0000000000" in self._storage.paths
+ # One slave.
+ assert len([x for x in self._storage.paths.keys() if x.startswith(
+ "/mysos/test/cluster0/slaves/member_")]) == 1
+
+ def test_two_launchers(self):
+ """Two launchers share resources and launch their clusters successfully."""
+ launchers = [
+ MySQLClusterLauncher(
+ self._driver,
+ MySQLCluster("cluster0", "user0", "pass0", 1),
+ self._state_provider,
+ self._zk_url,
+ self._zk_client,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml"),
+ MySQLClusterLauncher(
+ self._driver,
+ MySQLCluster("cluster1", "user1", "pass1", 2),
+ self._state_provider,
+ self._zk_url,
+ self._zk_client,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml")]
+ self._launchers.extend(launchers)
+
+ resources = create_resources(cpus=4, mem=512 * 3, ports=set([10000, 10001, 10002]))
+ self._offer.resources.extend(resources)
+
+ # Three nodes in total across two clusters.
+ # Simulate the scheduler.
+ for i in range(3):
+ for launcher in launchers:
+ task_id, remaining = launcher.launch(self._offer)
+ if task_id:
+ # Update the offer so other launchers will use its remaining resources.
+ del self._offer.resources[:]
+ self._offer.resources.extend(remaining)
+ break
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == 3
+
+ def test_invalid_status_update(self):
+ """Launcher raises an exception when an invalid status is received."""
+ self._cluster.num_nodes = 1
+ launcher = MySQLClusterLauncher(
+ self._driver,
+ self._cluster,
+ self._state_provider,
+ self._zk_url,
+ self._zk_client,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml")
+ self._launchers.append(launcher)
+
+ resources = create_resources(cpus=4, mem=512 * 3, ports=set([10000]))
+ self._offer.resources.extend(resources)
+
+ task_id, _ = launcher.launch(self._offer)
+ assert task_id == "mysos-cluster0-0"
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == self._cluster.num_nodes
+
+ status = mesos_pb2.TaskStatus()
+ status.task_id.value = task_id
+ status.state = mesos_pb2.TASK_RUNNING # Valid state.
+ launcher.status_update(status)
+
+ status.state = mesos_pb2.TASK_FINISHED # An invalid state.
+
+ with pytest.raises(MySQLClusterLauncher.Error):
+ launcher.status_update(status)
+
+ def test_terminal_status_update(self):
+ """Launcher reacts to terminated task by launching a new one."""
+ self._cluster.num_nodes = 1
+ launcher = MySQLClusterLauncher(
+ self._driver,
+ self._cluster,
+ self._state_provider,
+ self._zk_url,
+ self._zk_client,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ Amount(1, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml")
+ self._launchers.append(launcher)
+
+ resources = create_resources(cpus=4, mem=512 * 3, ports=set([10000]))
+ self._offer.resources.extend(resources)
+
+ task_id, _ = launcher.launch(self._offer)
+ assert task_id == "mysos-cluster0-0"
+
+ launched = self._driver.method_calls["launchTasks"]
+ assert len(launched) == self._cluster.num_nodes
+
+ status = mesos_pb2.TaskStatus()
+ status.task_id.value = task_id
+ status.state = mesos_pb2.TASK_RUNNING
+ launcher.status_update(status)
+
+ assert len(launcher._cluster.running_tasks) == 1
+
+ status.state = mesos_pb2.TASK_LOST
+ launcher.status_update(status)
+
+ assert len(launcher._cluster.running_tasks) == 0
+
+ task_id, _ = launcher.launch(self._offer)
+ assert task_id == "mysos-cluster0-1"
+
+ launched = self._driver.method_calls["launchTasks"]
+
+ # One task is relaunched to make up for the lost one.
+ assert len(launched) == self._cluster.num_nodes + 1
+
+ def test_master_failover(self):
+ for i in range(self._cluster.num_nodes):
+ task_id, remaining = self._launcher.launch(self._offer)
+ del self._offer.resources[:]
+ self._offer.resources.extend(remaining)
+ assert task_id == "mysos-cluster0-%s" % i
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == self._cluster.num_nodes
+
+ # All 3 nodes have successfully started.
+ status = mesos_pb2.TaskStatus()
+ status.state = mesos_pb2.TASK_RUNNING
+ status.slave_id.value = self._offer.slave_id.value
+
+ for i in range(self._cluster.num_nodes):
+ status.task_id.value = "mysos-cluster0-%s" % i
+ self._launcher.status_update(status)
+
+ # No log positions queries are sent for the first epoch.
+ assert "sendFrameworkMessage" not in self._driver.method_calls
+
+ # Wait for the election to complete.
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The first slave is elected.
+ assert "/mysos/test/cluster0/master/member_0000000000" in self._storage.paths
+
+ # Now fail the master task.
+ status.task_id.value = "mysos-cluster0-0"
+ status.state = mesos_pb2.TASK_FAILED
+ self._launcher.status_update(status)
+
+ assert len(self._launcher._cluster.running_tasks) == 2
+
+ # Log positions queries are sent.
+ self._launcher._elector._elect()
+ assert len(self._driver.method_calls["sendFrameworkMessage"]) >= 2
+
+ for i in range(1, self._cluster.num_nodes):
+ self._launcher.framework_message(
+ "mysos-cluster0-%s" % i,
+ self._offer.slave_id.value,
+ json.dumps(dict(epoch=1, position=str(i))))
+
+ # Wait for the election to complete.
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The slave with the highest position is elected.
+ assert "/mysos/test/cluster0/master/member_0000000002" in self._storage.paths
+
+ assert len(self._launcher._cluster.running_tasks) == 2
+
+ # When a new offer comes in, a new task is launched.
+ del self._offer.resources[:]
+ resources = create_resources(cpus=1, mem=512, ports=set([10000]))
+ self._offer.resources.extend(resources)
+ task_id, _ = self._launcher.launch(self._offer)
+ assert task_id == "mysos-cluster0-3"
+
+ launched = self._driver.method_calls["launchTasks"]
+ # One task is relaunched to make up for the failed one.
+ assert len(launched) == self._cluster.num_nodes + 1
+
+ def test_launcher_recovery_after_election_completed(self):
+ # 1. Launch a cluster on the running launcher.
+ for i in range(self._cluster.num_nodes):
+ task_id, remaining = self._launcher.launch(self._offer)
+ del self._offer.resources[:]
+ self._offer.resources.extend(remaining)
+ assert task_id == "mysos-cluster0-%s" % i
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == self._cluster.num_nodes
+
+ # No new tasks are launched.
+ assert self._launcher.launch(self._offer)[0] is None
+ assert len(self._driver.method_calls["launchTasks"]) == self._cluster.num_nodes
+
+ # All 3 nodes have successfully started.
+ status = mesos_pb2.TaskStatus()
+ status.state = mesos_pb2.TASK_RUNNING
+ status.slave_id.value = self._offer.slave_id.value
+ for i in range(self._cluster.num_nodes):
+ status.task_id.value = "mysos-cluster0-%s" % i
+ self._launcher.status_update(status)
+
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The first slave is elected.
+ assert "/mysos/test/cluster0/master/member_0000000000" in self._storage.paths
+ # Two slaves.
+ assert len([x for x in self._storage.paths.keys() if x.startswith(
+ "/mysos/test/cluster0/slaves/member_")]) == 2
+
+ # 2. Recover the launcher.
+ self._cluster = self._state_provider.load_cluster_state(self._cluster.name)
+ self._launcher = MySQLClusterLauncher(
+ self._driver,
+ self._cluster,
+ self._state_provider,
+ self._zk_url,
+ self._zk_client,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml",
+ query_interval=Amount(150, Time.MILLISECONDS))
+
+ # Now fail the master task.
+ status.task_id.value = "mysos-cluster0-0"
+ status.state = mesos_pb2.TASK_FAILED
+ self._launcher.status_update(status)
+
+ for i in range(1, self._cluster.num_nodes):
+ self._launcher.framework_message(
+ "mysos-cluster0-%s" % i,
+ self._offer.slave_id.value,
+ json.dumps(dict(epoch=1, position=str(i))))
+
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The second slave has the larger position and is elected.
+ assert "/mysos/test/cluster0/master/member_0000000002" in self._storage.paths
+
+ def test_launcher_recovery_before_election_completed(self):
+ # 1. Launch a cluster on the running launcher.
+ for i in range(self._cluster.num_nodes):
+ task_id, remaining = self._launcher.launch(self._offer)
+ del self._offer.resources[:]
+ self._offer.resources.extend(remaining)
+ assert task_id == "mysos-cluster0-%s" % i
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == self._cluster.num_nodes
+
+ # No new tasks are launched.
+ assert self._launcher.launch(self._offer)[0] is None
+ assert len(self._driver.method_calls["launchTasks"]) == self._cluster.num_nodes
+
+ # All 3 nodes have successfully started.
+ status = mesos_pb2.TaskStatus()
+ status.state = mesos_pb2.TASK_RUNNING
+ status.slave_id.value = self._offer.slave_id.value
+ for i in range(self._cluster.num_nodes):
+ status.task_id.value = "mysos-cluster0-%s" % i
+ self._launcher.status_update(status)
+
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The first slave is elected.
+ assert "/mysos/test/cluster0/master/member_0000000000" in self._storage.paths
+ # Two slaves.
+ assert len([x for x in self._storage.paths.keys() if x.startswith(
+ "/mysos/test/cluster0/slaves/member_")]) == 2
+
+ # Now fail the master task which leads to re-election.
+ status.task_id.value = "mysos-cluster0-0"
+ status.state = mesos_pb2.TASK_FAILED
+ self._launcher.status_update(status)
+
+ # 2. Recover the launcher.
+ self._cluster = self._state_provider.load_cluster_state(self._cluster.name)
+ self._launcher = MySQLClusterLauncher(
+ self._driver,
+ self._cluster,
+ self._state_provider,
+ self._zk_url,
+ self._zk_client,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml",
+ query_interval=Amount(150, Time.MILLISECONDS))
+
+ for i in range(1, self._cluster.num_nodes):
+ self._launcher.framework_message(
+ "mysos-cluster0-%s" % i,
+ self._offer.slave_id.value,
+ json.dumps(dict(epoch=2, position=str(i))))
+
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The second slave has the larger position and is elected.
+ assert "/mysos/test/cluster0/master/member_0000000002" in self._storage.paths
+
+ def test_launcher_kill(self):
+ for i in range(self._cluster.num_nodes):
+ task_id, remaining = self._launcher.launch(self._offer)
+ del self._offer.resources[:]
+ self._offer.resources.extend(remaining)
+ assert task_id == "mysos-cluster0-%s" % i
+
+ tasks = self._driver.method_calls["launchTasks"]
+ assert len(tasks) == self._cluster.num_nodes
+
+ # No new tasks are launched.
+ assert self._launcher.launch(self._offer)[0] is None
+ assert len(self._driver.method_calls["launchTasks"]) == self._cluster.num_nodes
+
+ # All 3 nodes have successfully started.
+ status = mesos_pb2.TaskStatus()
+ status.state = mesos_pb2.TASK_RUNNING # Valid state.
+ status.slave_id.value = self._offer.slave_id.value
+ for i in range(self._cluster.num_nodes):
+ status.task_id.value = "mysos-cluster0-%s" % i
+ self._launcher.status_update(status)
+
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(self._zk_url, self._cluster.name),
+ self._zk_client),
+ Amount(5, Time.SECONDS))
+
+ # The first slave is elected.
+ assert "/mysos/test/cluster0/master/member_0000000000" in self._storage.paths
+ # Two slaves.
+ assert len([x for x in self._storage.paths.keys() if x.startswith(
+ "/mysos/test/cluster0/slaves/member_")]) == 2
+
+ # Kill the cluster.
+ with pytest.raises(MySQLClusterLauncher.PermissionError):
+ self._launcher.kill("wrong_password")
+
+ self._launcher.kill(self._cluster.password) # Correct password.
+
+ # All 3 nodes are successfully killed.
+ status = mesos_pb2.TaskStatus()
+ status.state = mesos_pb2.TASK_KILLED
+ status.slave_id.value = self._offer.slave_id.value
+ for i in range(self._cluster.num_nodes):
+ status.task_id.value = "mysos-cluster0-%s" % i
+ self._launcher.status_update(status)
+
+ assert "/mysos/test/cluster0" not in self._storage.paths # ServerSets removed.
+ assert not self._state_provider.load_cluster_state("cluster0") # State removed.
diff --git a/tests/mysos/scheduler/test_mysos_scheduler.py b/tests/mysos/scheduler/test_mysos_scheduler.py
new file mode 100644
index 0000000..464b6dd
--- /dev/null
+++ b/tests/mysos/scheduler/test_mysos_scheduler.py
@@ -0,0 +1,86 @@
+import getpass
+import os
+import posixpath
+import subprocess
+
+from twitter.common import log
+from twitter.common.concurrent import deadline
+from twitter.common.dirutil import safe_mkdtemp
+from twitter.common.quantity import Amount, Time
+from twitter.mysos.common.cluster import get_cluster_path, wait_for_master
+from twitter.mysos.scheduler.scheduler import MysosScheduler
+from twitter.mysos.scheduler.state import LocalStateProvider, Scheduler
+
+from kazoo.handlers.threading import SequentialThreadingHandler
+import mesos.interface
+from mesos.interface.mesos_pb2 import DRIVER_STOPPED, FrameworkInfo
+import mesos.native
+from zake.fake_client import FakeClient
+from zake.fake_storage import FakeStorage
+
+
+if 'MYSOS_DEBUG' in os.environ:
+ from twitter.common.log.options import LogOptions
+ LogOptions.set_stderr_log_level('google:DEBUG')
+ LogOptions.set_simple(True)
+ log.init('mysos_tests')
+
+
+def test_scheduler_runs():
+ """
+ Verifies that the scheduler successfully launches 3 "no-op" MySQL tasks.
+ NOTE: Due to the limitation of zake the scheduler's ZK operations are not propagated to
+ executors in separate processes but they are unit-tested separately.
+ """
+
+ # Make sure fake_mysos_executor.pex is available to be fetched by Mesos slave.
+ # TODO(xujyan): Fix this as we don't use pex here anymore.
+ assert os.path.isfile('dist/fake_mysos_executor.pex')
+
+ storage = FakeStorage(SequentialThreadingHandler())
+ zk_client = FakeClient(storage=storage)
+ zk_client.start()
+
+ zk_url = "zk://fake_host/home/mysos/clusters"
+ cluster_name = "test_cluster"
+ num_nodes = 3
+
+ state_provider = LocalStateProvider(safe_mkdtemp())
+
+ framework_info = FrameworkInfo(
+ user=getpass.getuser(),
+ name="mysos",
+ checkpoint=False)
+
+ state = Scheduler(framework_info)
+
+ scheduler = MysosScheduler(
+ state,
+ state_provider,
+ getpass.getuser(),
+ os.path.abspath("dist/fake_mysos_executor.pex"),
+ "./fake_mysos_executor.pex",
+ zk_client,
+ zk_url,
+ Amount(40, Time.SECONDS),
+ "/fakepath")
+
+ scheduler_driver = mesos.native.MesosSchedulerDriver(
+ scheduler,
+ framework_info,
+ "local")
+ scheduler_driver.start()
+
+ # Wait until the scheduler is connected and becomes available.
+ assert scheduler.connected.wait(30)
+
+ scheduler.create_cluster(cluster_name, "mysql_user", num_nodes)
+
+ # A slave is promoted to be the master.
+ deadline(
+ lambda: wait_for_master(
+ get_cluster_path(posixpath.join(zk_url, 'discover'), cluster_name),
+ zk_client),
+ Amount(40, Time.SECONDS))
+
+ assert scheduler_driver.stop() == DRIVER_STOPPED
diff --git a/tests/mysos/scheduler/test_scheduler.py b/tests/mysos/scheduler/test_scheduler.py
new file mode 100644
index 0000000..2ed313c
--- /dev/null
+++ b/tests/mysos/scheduler/test_scheduler.py
@@ -0,0 +1,182 @@
+import getpass
+import os
+import shutil
+import tempfile
+import unittest
+
+from twitter.common import log
+from twitter.common.quantity import Amount, Time
+from twitter.mysos.common.testing import Fake
+from twitter.mysos.scheduler.scheduler import (
+ INCOMPATIBLE_ROLE_OFFER_REFUSE_DURATION,
+ MysosScheduler)
+from twitter.mysos.scheduler.launcher import create_resources
+from twitter.mysos.scheduler.state import LocalStateProvider, MySQLCluster, Scheduler
+
+from kazoo.handlers.threading import SequentialThreadingHandler
+import mesos.interface.mesos_pb2 as mesos_pb2
+from zake.fake_client import FakeClient
+from zake.fake_storage import FakeStorage
+
+import pytest
+
+
+if 'MYSOS_DEBUG' in os.environ:
+ from twitter.common.log.options import LogOptions
+ LogOptions.set_stderr_log_level('google:DEBUG')
+ LogOptions.set_simple(True)
+ log.init('mysos_tests')
+
+
+class FakeDriver(Fake): pass
+
+
+class TestScheduler(unittest.TestCase):
+ def setUp(self):
+ self._driver = FakeDriver()
+ self._storage = FakeStorage(SequentialThreadingHandler())
+ self._zk_client = FakeClient(storage=self._storage)
+ self._zk_client.start()
+
+ self._framework_id = mesos_pb2.FrameworkID()
+ self._framework_id.value = "framework_id_0"
+
+ self._offer = mesos_pb2.Offer()
+ self._offer.id.value = "offer_id_0"
+ self._offer.framework_id.value = self._framework_id.value
+ self._offer.slave_id.value = "slave_id_0"
+ self._offer.hostname = "localhost"
+
+ resources = create_resources(cpus=4, mem=512 * 3, ports=set([10000, 10001, 10002]))
+ self._offer.resources.extend(resources)
+
+ self._framework_user = "framework_user"
+
+ self._zk_url = "zk://host/mysos/test"
+ self._cluster = MySQLCluster("cluster0", "user", "pass", 3)
+
+ self._tmpdir = tempfile.mkdtemp()
+ self._state_provider = LocalStateProvider(self._tmpdir)
+
+ framework_info = mesos_pb2.FrameworkInfo(
+ user=getpass.getuser(),
+ name="mysos",
+ checkpoint=False)
+ self._state = Scheduler(framework_info)
+
+ def tearDown(self):
+ shutil.rmtree(self._tmpdir, True) # Clean up after ourselves.
+
+ def test_scheduler_recovery(self):
+ scheduler1 = MysosScheduler(
+ self._state,
+ self._state_provider,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ self._zk_client,
+ self._zk_url,
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml")
+ scheduler1.registered(self._driver, self._framework_id, object())
+ scheduler1.create_cluster("cluster1", "mysql_user", 3)
+ scheduler1.resourceOffers(self._driver, [self._offer])
+
+ # One task is launched for one offer.
+ assert len(scheduler1._launchers["cluster1"]._cluster.tasks) == 1
+
+ with pytest.raises(MysosScheduler.ClusterExists):
+ scheduler1.create_cluster("cluster1", "mysql_user", 3)
+
+ # FrameworkID should have been persisted.
+ self._state = self._state_provider.load_scheduler_state()
+ assert self._state.framework_info.id.value == self._framework_id.value
+
+ # Simulate restart.
+ scheduler2 = MysosScheduler(
+ self._state,
+ self._state_provider,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ self._zk_client,
+ self._zk_url,
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml")
+
+ # Scheduler always receives registered() with the same FrameworkID after failover.
+ scheduler2.registered(self._driver, self._framework_id, object())
+
+ assert len(scheduler2._launchers) == 1
+ assert scheduler2._launchers["cluster1"].cluster_name == "cluster1"
+
+ # Scheduler has recovered the cluster so it doesn't accept another of the same name.
+ with pytest.raises(MysosScheduler.ClusterExists):
+ scheduler2.create_cluster("cluster1", "mysql_user", 3)
+
+ def test_scheduler_recovery_failure_before_launch(self):
+ scheduler1 = MysosScheduler(
+ self._state,
+ self._state_provider,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ self._zk_client,
+ self._zk_url,
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml")
+ scheduler1.registered(self._driver, self._framework_id, object())
+ scheduler1.create_cluster("cluster1", "mysql_user", 3)
+
+ # Simulate restart before the task is successfully launched.
+ scheduler2 = MysosScheduler(
+ self._state,
+ self._state_provider,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ self._zk_client,
+ self._zk_url,
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml")
+
+ assert len(scheduler2._launchers) == 0 # No launchers are recovered.
+
+ # Scheduler always receives registered() with the same FrameworkID after failover.
+ scheduler2.registered(self._driver, self._framework_id, object())
+
+ assert len(scheduler2._launchers) == 1
+ assert scheduler2._launchers["cluster1"].cluster_name == "cluster1"
+
+ # Now offer the resources for this task.
+ scheduler2.resourceOffers(self._driver, [self._offer])
+
+ # One task is launched for the offer.
+ assert len(scheduler2._launchers["cluster1"]._cluster.active_tasks) == 1
+
+ # Scheduler has recovered the cluster so it doesn't accept another of the same name.
+ with pytest.raises(MysosScheduler.ClusterExists):
+ scheduler2.create_cluster("cluster1", "mysql_user", 3)
+
+ def test_incompatible_resource_role(self):
+ scheduler1 = MysosScheduler(
+ self._state,
+ self._state_provider,
+ self._framework_user,
+ "./executor.pex",
+ "cmd.sh",
+ self._zk_client,
+ self._zk_url,
+ Amount(5, Time.SECONDS),
+ "/etc/mysos/admin_keyfile.yml",
+ framework_role='mysos') # Require 'mysos' but the resources are in '*'.
+ scheduler1.registered(self._driver, self._framework_id, object())
+ scheduler1.create_cluster("cluster1", "mysql_user", 3)
+ scheduler1.resourceOffers(self._driver, [self._offer])
+
+ assert "declineOffer" in self._driver.method_calls
+ assert len(self._driver.method_calls["declineOffer"]) == 1
+ # [0][0][1]: [First declineOffer call][The positional args][The first positional arg], which is
+ # a 'Filters' object.
+ assert (self._driver.method_calls["declineOffer"][0][0][1].refuse_seconds ==
+ INCOMPATIBLE_ROLE_OFFER_REFUSE_DURATION.as_(Time.SECONDS))
diff --git a/tests/mysos/scheduler/test_state.py b/tests/mysos/scheduler/test_state.py
new file mode 100644
index 0000000..69b783e
--- /dev/null
+++ b/tests/mysos/scheduler/test_state.py
@@ -0,0 +1,59 @@
+import os
+import shutil
+import tempfile
+import unittest
+
+from twitter.mysos.scheduler.state import (
+ LocalStateProvider,
+ MySQLCluster,
+ MySQLTask,
+ Scheduler
+)
+
+from mesos.interface.mesos_pb2 import FrameworkInfo
+
+
+if 'MYSOS_DEBUG' in os.environ:
+ from twitter.common import log
+ from twitter.common.log.options import LogOptions
+ LogOptions.set_stderr_log_level('google:DEBUG')
+ LogOptions.set_simple(True)
+ log.init('mysos_tests')
+
+
+class TestState(unittest.TestCase):
+ def setUp(self):
+ self._tmpdir = tempfile.mkdtemp()
+ self._state_provider = LocalStateProvider(self._tmpdir)
+
+ def tearDown(self):
+ shutil.rmtree(self._tmpdir, True)
+
+ def test_scheduler_state(self):
+ expected = Scheduler(FrameworkInfo(
+ user='test_user',
+ name='test_fw_name',
+ checkpoint=True))
+ expected.clusters.add('cluster2')
+ expected.clusters.add('cluster1')
+
+ self._state_provider.dump_scheduler_state(expected)
+ actual = self._state_provider.load_scheduler_state()
+
+ assert expected.framework_info == actual.framework_info
+ assert expected.clusters == actual.clusters
+
+ def test_cluster_state(self):
+ expected = MySQLCluster('cluster1', 'cluster_user', 'cluster_password', 3)
+
+ expected.tasks['task1'] = MySQLTask(
+ 'cluster1', 'task1', 'slave1', 'host1', 10000)
+
+ self._state_provider.dump_cluster_state(expected)
+ actual = self._state_provider.load_cluster_state('cluster1')
+
+ assert expected.user == actual.user
+ assert isinstance(actual.num_nodes, int)
+ assert expected.num_nodes == actual.num_nodes
+ assert len(expected.tasks) == len(actual.tasks)
+ assert expected.tasks['task1'].port == actual.tasks['task1'].port
diff --git a/tests/mysos/scheduler/test_zk_state.py b/tests/mysos/scheduler/test_zk_state.py
new file mode 100644
index 0000000..27c51be
--- /dev/null
+++ b/tests/mysos/scheduler/test_zk_state.py
@@ -0,0 +1,81 @@
+import cPickle
+import os
+import unittest
+
+from twitter.mysos.scheduler.state import (
+ MySQLCluster,
+ MySQLTask,
+ Scheduler,
+ StateProvider
+)
+from twitter.mysos.scheduler.zk_state import ZooKeeperStateProvider
+
+from kazoo.handlers.threading import SequentialThreadingHandler
+from mesos.interface.mesos_pb2 import FrameworkInfo
+import pytest
+from zake.fake_client import FakeClient
+from zake.fake_storage import FakeStorage
+
+
+if 'MYSOS_DEBUG' in os.environ:
+ from twitter.common import log
+ from twitter.common.log.options import LogOptions
+ LogOptions.set_stderr_log_level('google:DEBUG')
+ LogOptions.set_simple(True)
+ log.init('mysos_tests')
+
+
+class TestZooKeeperStateProvider(unittest.TestCase):
+ def setUp(self):
+ self._storage = FakeStorage(SequentialThreadingHandler())
+ self._client = FakeClient(storage=self._storage)
+ self._client.start()
+ self._state_provider = ZooKeeperStateProvider(self._client, '/mysos')
+
+ def tearDown(self):
+ self._client.stop()
+
+ def test_scheduler_state(self):
+ expected = Scheduler(FrameworkInfo(
+ user='test_user',
+ name='test_fw_name',
+ checkpoint=True))
+ expected.tasks = dict(taks1='cluster1', task2='cluster2')
+
+ self._state_provider.dump_scheduler_state(expected)
+ actual = self._state_provider.load_scheduler_state()
+
+ assert expected.framework_info == actual.framework_info
+ assert expected.tasks == actual.tasks
+
+ def test_scheduler_state_errors(self):
+ assert not self._state_provider.load_scheduler_state() # Not an error for scheduler state to be
+ # not found.
+
+ self._client.ensure_path("/mysos/state")
+ self._client.create("/mysos/state/scheduler", cPickle.dumps(object()))
+ with pytest.raises(StateProvider.Error):
+ self._state_provider.load_scheduler_state()
+
+ def test_cluster_state(self):
+ expected = MySQLCluster('cluster1', 'cluster_user', 'cluster_password', 3)
+
+ expected.tasks['task1'] = MySQLTask(
+ 'cluster1', 'task1', 'slave1', 'host1', 10000)
+
+ self._state_provider.dump_cluster_state(expected)
+ actual = self._state_provider.load_cluster_state('cluster1')
+
+ assert expected.user == actual.user
+ assert isinstance(actual.num_nodes, int)
+ assert expected.num_nodes == actual.num_nodes
+ assert len(expected.tasks) == len(actual.tasks)
+ assert expected.tasks['task1'].port == actual.tasks['task1'].port
+
+ def test_cluster_state_errors(self):
+ assert not self._state_provider.load_cluster_state('nonexistent')
+
+ self._client.ensure_path("/mysos/state/clusters")
+ self._client.create("/mysos/state/clusters/cluster1", cPickle.dumps(object()))
+ with pytest.raises(StateProvider.Error):
+ self._state_provider.load_cluster_state('cluster1')
diff --git a/tox.ini b/tox.ini
new file mode 100644
index 0000000..14ad2d2
--- /dev/null
+++ b/tox.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py27
+
+[testenv]
+deps = pytest
+install_command = pip install -e .[test] --find-links {toxinidir}/3rdparty {opts} {packages}
+# TODO(xujyan): Fix up 'test_scheduler_runs'.
+commands = py.test -v -k 'not test_scheduler_runs' tests {posargs:}
diff --git a/twitter/__init__.py b/twitter/__init__.py
new file mode 100644
index 0000000..de40ea7
--- /dev/null
+++ b/twitter/__init__.py
@@ -0,0 +1 @@
+__import__('pkg_resources').declare_namespace(__name__)
diff --git a/twitter/mysos/__init__.py b/twitter/mysos/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/twitter/mysos/__init__.py
diff --git a/twitter/mysos/common/__init__.py b/twitter/mysos/common/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/twitter/mysos/common/__init__.py
diff --git a/twitter/mysos/common/cluster.py b/twitter/mysos/common/cluster.py
new file mode 100644
index 0000000..d67de2e
--- /dev/null
+++ b/twitter/mysos/common/cluster.py
@@ -0,0 +1,339 @@
+import Queue
+import functools
+import posixpath
+import sys
+import threading
+
+from twitter.common import log
+from twitter.common.zookeeper.serverset.endpoint import ServiceInstance
+from twitter.mysos.common import zookeeper
+
+from kazoo.client import KazooClient
+from kazoo.exceptions import NoNodeError
+from kazoo.protocol.states import EventType
+from kazoo.recipe.watchers import ChildrenWatch, DataWatch
+
+
+def get_cluster_path(zk_root, cluster_name):
+ """
+ :param zk_root: the root path for the mysos scheduler.
+ :param cluster_name: Name of the the cluster.
+ :return: The path for the cluster.
+ """
+ return posixpath.join(zk_root, cluster_name)
+
+
+class Cluster(object):
+ """
+ A class that represents all members of the MySQL cluster.
+
+ A newly added cluster member becomes a read-only slave until it's promoted to a master.
+ Only the master can write.
+
+ The members of the cluster are maintained in two ZooKeeper groups: a slaves group and a master
+ group under the same 'directory'. Slaves have unique member IDs backed by ZooKeeper's sequential
+ ZNodes. When a slave is promoted to a master, it is moved (its ID preserved) from the slave
+ group to the master group.
+
+ There is at most one member in the master group.
+ """
+ SLAVES_GROUP = 'slaves'
+ MASTER_GROUP = 'master'
+ MEMBER_PREFIX = "member_" # Use the prefix so the path conforms to the ServerSet convention.
+
+ def __init__(self, cluster_path):
+ self.cluster_path = cluster_path
+ self.members = {} # {ID : (serialized) ServiceInstance} mappings for members of the cluster.
+ self.master = None # The master's member ID.
+ self.slaves_group = posixpath.join(cluster_path, self.SLAVES_GROUP)
+ self.master_group = posixpath.join(cluster_path, self.MASTER_GROUP)
+
+
+# TODO(jyx): Handle errors e.g. sessions expirations and recoverable failures.
+class ClusterManager(object):
+ """
+ Kazoo wrapper used by the scheduler to inform executors about cluster change.
+ NOTE: ClusterManager is thread safe, i.e., it can be accessed from multiple threads at once.
+ """
+
+ class Error(Exception): pass
+
+ def __init__(self, client, cluster_path):
+ """
+ :param client: Kazoo client.
+ :param cluster_path: The path for this cluster on ZooKeeper.
+ """
+ self._client = client
+ self._cluster = Cluster(cluster_path)
+ self._lock = threading.Lock()
+ self._populate()
+
+ def _read_child_content(self, group, member_id):
+ try:
+ return self._client.get(posixpath.join(group, member_id))[0]
+ except NoNodeError:
+ return None
+
+ def _populate(self):
+ self._client.ensure_path(self._cluster.slaves_group)
+ self._client.ensure_path(self._cluster.master_group)
+
+ # Populate slaves.
+ for child in self._client.get_children(self._cluster.slaves_group):
+ child_content = self._read_child_content(self._cluster.slaves_group, child)
+ if child_content:
+ self._cluster.members[child] = child_content
+
+ # Populate the master.
+ master_group = self._client.get_children(self._cluster.master_group)
+ assert len(master_group) <= 1
+ if len(master_group) == 1:
+ child = master_group[0]
+ child_content = self._read_child_content(self._cluster.master_group, child)
+ if child_content:
+ self._cluster.members[child] = child_content
+ self._cluster.master = child
+
+ def add_member(self, service_instance):
+ """
+ Add the member to the ZooKeeper group.
+ NOTE:
+ - New members are slaves until being promoted.
+ - A new member is not added if the specified service_instance already exists in the group.
+ :return: The member ID for the ServiceInstance generated by ZooKeeper.
+ """
+ if not isinstance(service_instance, ServiceInstance):
+ raise TypeError("'service_instance' should be a ServiceInstance")
+
+ content = ServiceInstance.pack(service_instance)
+
+ for k, v in self._cluster.members.items():
+ if content == v:
+ log.info("%s not added because it already exists in the group" % service_instance)
+ return k
+
+ znode_path = self._client.create(
+ posixpath.join(self._cluster.slaves_group, self._cluster.MEMBER_PREFIX),
+ content,
+ sequence=True)
+ _, member_id = posixpath.split(znode_path)
+ with self._lock:
+ self._cluster.members[member_id] = content
+ return member_id
+
+ def remove_member(self, member_id):
+ """
+ Remove the member if it is in the group.
+
+ :return: True if the member is deleted. False if the member cannot be found.
+ """
+ with self._lock:
+ if member_id not in self._cluster.members:
+ log.info("Member %s is not in the ZK group" % member_id)
+ return False
+
+ self._cluster.members.pop(member_id, None)
+
+ if member_id == self._cluster.master:
+ self._cluster.master = None
+ self._client.delete(posixpath.join(self._cluster.master_group, member_id))
+ else:
+ self._client.delete(posixpath.join(self._cluster.slaves_group, member_id))
+
+ return True
+
+ def promote_member(self, member_id):
+ """
+ Promote the member with the given ID to be the master of the cluster if it's not already the
+ master.
+
+ :return: True if the member is promoted. False if the member is already the master.
+ """
+ with self._lock:
+ if member_id not in self._cluster.members:
+ raise ValueError("Invalid member_id: %s" % member_id)
+
+ # Do nothing if the member is already the master.
+ if self._cluster.master and self._cluster.master == member_id:
+ log.info("Not promoting %s because is already the master" % member_id)
+ return False
+
+ tx = self._client.transaction()
+ if self._cluster.master:
+ tx.delete(posixpath.join(self._cluster.master_group, self._cluster.master))
+ self._cluster.members.pop(self._cluster.master)
+
+ # "Move" the ZNode, i.e., create a ZNode of the same ID in the master group.
+ tx.delete(posixpath.join(self._cluster.slaves_group, member_id))
+ tx.create(
+ posixpath.join(self._cluster.master_group, member_id),
+ self._cluster.members[member_id])
+
+ tx.commit()
+
+ self._cluster.master = member_id
+
+ return True
+
+ def delete_cluster(self):
+ with self._lock:
+ if self._cluster.members:
+ raise self.Error("Cannot remove a cluster that is not empty")
+
+ # Need to delete master/slave sub-dirs.
+ self._client.delete(self._cluster.cluster_path, recursive=True)
+
+
+# TODO(wickman): Implement kazoo connection acquiescence.
+class ClusterListener(object):
+ """Kazoo wrapper used by the executor to listen to cluster change."""
+
+ def __init__(self,
+ client,
+ cluster_path,
+ self_instance=None,
+ promotion_callback=None,
+ demotion_callback=None,
+ master_callback=None,
+ termination_callback=None):
+ """
+ :param client: Kazoo client.
+ :param cluster_path: The path for this cluster on ZooKeeper.
+ :param self_instance: The local ServiceInstance associated with this listener.
+ :param promotion_callback: Invoked when 'self_instance' is promoted.
+ :param demotion_callback: Invoked when 'self_instance' is demoted.
+ :param master_callback: Invoked when there is a master change otherwise.
+ :param termination_callback: Invoked when the cluster is terminated.
+ NOTE: Callbacks are executed synchronously in Kazoo's completion thread to ensure the delivery
+ order of events. Blocking the callback method means no future callbacks will be invoked.
+ """
+ self._client = client
+ self._cluster = Cluster(cluster_path)
+ self._self_content = ServiceInstance.pack(self_instance) if self_instance else None
+ self._master = None
+ self._master_content = None
+ self._promotion_callback = promotion_callback or (lambda: True)
+ self._demotion_callback = demotion_callback or (lambda: True)
+ self._master_callback = master_callback or (lambda x: True)
+ self._termination_callback = termination_callback or (lambda: True)
+
+ self._children_watch = None # Set when the watcher detects that the master group exists.
+
+ def start(self):
+ """
+ Start the listener to watch the master group.
+
+ NOTE: The listener only starts watching master after the base ZNode for the group is created.
+ """
+ DataWatch(self._client, self._cluster.cluster_path, func=self._cluster_path_callback)
+ DataWatch(self._client, self._cluster.master_group, func=self._master_group_callback)
+
+ def _swap(self, master, master_content):
+ i_was_master = self._self_content and self._master_content == self._self_content
+ self._master, self._master_content = master, master_content
+ i_am_master = self._self_content and self._master_content == self._self_content
+
+ # Invoke callbacks accordingly.
+ # NOTE: No callbacks are invoked if there is currently no master and 'self_instance' wasn't the
+ # master.
+ if i_was_master and not i_am_master:
+ self._demotion_callback()
+ elif not i_was_master and i_am_master:
+ self._promotion_callback()
+ elif not i_was_master and not i_am_master and master:
+ assert master_content
+ self._master_callback(ServiceInstance.unpack(master_content))
+
+ def _data_callback(self, master_id, master_completion):
+ try:
+ master_content, _ = master_completion.get()
+ except NoNodeError:
+ # ZNode could be gone after we detected it but before we read it.
+ master_id, master_content = None, None
+ self._swap(master_id, master_content)
+
+ def _child_callback(self, masters):
+ assert len(masters) <= 1, "There should be at most one master"
+
+ if len(masters) == 1 and self._master != masters[0]:
+ self._client.get_async(posixpath.join(self._cluster.master_group, masters[0])).rawlink(
+ functools.partial(self._data_callback, masters[0]))
+ elif len(masters) == 0:
+ self._swap(None, None)
+
+ def _cluster_path_callback(self, data, stat, event):
+ if event and event.type == EventType.DELETED:
+ self._termination_callback()
+
+ def _master_group_callback(self, data, stat, event):
+ if stat and not self._children_watch:
+ log.info("Master group %s exists. Starting to watch for election result" %
+ self._cluster.master_group)
+ self._children_watch = ChildrenWatch(
+ self._client, self._cluster.master_group, func=self._child_callback)
+
+
+def resolve_master(
+ cluster_url, master_callback=lambda: True, termination_callback=lambda: True, zk_client=None):
+ """
+ Resolve the MySQL cluster master's endpoint from the given URL for this cluster.
+ :param cluster_url: The ZooKeeper URL for this cluster.
+ :param master_callback: A callback method with one argument: the ServiceInstance for the elected
+ master.
+ :param termination_callback: A callback method with no argument. Invoked when the cluster
+ terminates.
+ :param zk_client: Use a custom ZK client instead of Kazoo if specified.
+ """
+ try:
+ _, zk_servers, cluster_path = zookeeper.parse(cluster_url)
+ except Exception as e:
+ raise ValueError("Invalid cluster_url: %s" % e.message)
+
+ if not zk_client:
+ zk_client = KazooClient(zk_servers)
+ zk_client.start()
+
+ listener = ClusterListener(
+ zk_client,
+ cluster_path,
+ None,
+ master_callback=master_callback,
+ termination_callback=termination_callback)
+ listener.start()
+
+
+def wait_for_master(cluster_url, zk_client=None):
+ """
+ Convenience function to wait for the master to be elected and return the master.
+ :param cluster_url: The ZooKeeper URL for this cluster.
+ :param zk_client: Use a custom ZK client instead of Kazoo if specified.
+ :return: The ServiceInstance for the elected master.
+ """
+ master = Queue.Queue()
+ resolve_master(
+ cluster_url,
+ master_callback=lambda x: master.put(x),
+ termination_callback=lambda: True,
+ zk_client=zk_client)
+ # Block forever but using sys.maxint makes the wait interruptable by Ctrl-C. See
+ # http://bugs.python.org/issue1360.
+ return master.get(True, sys.maxint)
+
+
+def wait_for_termination(cluster_url, zk_client=None):
+ """
+ Convenience function to wait for the cluster to terminate. The corresponding ZNode is removed
+ when the cluster terminates.
+ :param cluster_url: The ZooKeeper URL for this cluster.
+ :param zk_client: Use a custom ZK client instead of Kazoo if specified.
+ """
+ terminated = threading.Event()
+ resolve_master(
+ cluster_url,
+ master_callback=lambda x: True,
+ termination_callback=lambda: terminated.set(),
+ zk_client=zk_client)
+
+ # Block forever but using sys.maxint makes the wait interruptable by Ctrl-C. See
+ # http://bugs.python.org/issue1360.
+ terminated.wait(sys.maxint)
diff --git a/twitter/mysos/common/decorators.py b/twitter/mysos/common/decorators.py
new file mode 100644
index 0000000..4c54a01
--- /dev/null
+++ b/twitter/mysos/common/decorators.py
@@ -0,0 +1,24 @@
+import functools
+import inspect
+
+from twitter.common import log
+
+
+def logged(func):
+ arg_names = inspect.getargspec(func).args[1:]
+ @functools.wraps(func)
+ def wrapped_func(self, *args):
+ log.debug('%s(%s)' % (
+ func.__name__,
+ ', '.join('%s=%s' % (name, arg) for (name, arg) in zip(arg_names, args))))
+ return func(self, *args)
+ return wrapped_func
+
+
+def synchronized(func):
+ @functools.wraps(func)
+ def synchronizer(self, *args, **kwargs):
+ assert hasattr(self, '_lock'), "Need to define a _lock to use this decorator"
+ with self._lock:
+ return func(self, *args, **kwargs)
+ return synchronizer
diff --git a/twitter/mysos/common/fetcher.py b/twitter/mysos/common/fetcher.py
new file mode 100644
index 0000000..e73ca52
--- /dev/null
+++ b/twitter/mysos/common/fetcher.py
@@ -0,0 +1,34 @@
+from abc import abstractmethod
+
+from twitter.common.lang import Interface
+
+
+class Fetcher(Interface):
+ class Error(Exception): pass
+
+ @abstractmethod
+ def fetch(self, uri, directory):
+ pass
+
+
+class FetcherFactory(object):
+ """A singleton factory for Fetchers."""
+
+ _FETCHERS = {}
+
+ @classmethod
+ def register_fetcher(cls, scheme, fetcher):
+ cls._FETCHERS[scheme.rstrip('://')] = fetcher
+
+ @classmethod
+ def get_fetcher(cls, uri):
+ """
+ :return: A Fetcher instance that matches this URI. None if no fetcher is registered with the
+ URI's scheme.
+ """
+ scheme = uri.split('://')[0]
+ fetcher = cls._FETCHERS.get(scheme)
+ if not fetcher:
+ raise ValueError("No Fetcher is registered for URI scheme '%s'" % scheme)
+
+ return fetcher
diff --git a/twitter/mysos/common/hdfs.py b/twitter/mysos/common/hdfs.py
new file mode 100644
index 0000000..327128c
--- /dev/null
+++ b/twitter/mysos/common/hdfs.py
@@ -0,0 +1,38 @@
+import os
+
+from twitter.common import log
+from twitter.common.concurrent import deadline, Timeout
+from twitter.common.fs import HDFSHelper
+from twitter.common.quantity import Amount, Data, Time
+
+from .fetcher import Fetcher
+
+
+HADOOP_CONF_DIR = '/etc/hadoop/conf'
+
+
+class HDFSFetcher(Fetcher):
+ """
+ NOTE: Specify custom config directory using the environment variable 'HADOOP_CONF_DIR'.
+ """
+
+ def __init__(self, timeout=Amount(5, Time.MINUTES)):
+ if not isinstance(timeout, Amount) or not isinstance(timeout.unit(), Time):
+ raise ValueError("'timeout' must be an Amount of Time")
+ self._timeout = timeout
+
+ def fetch(self, uri, directory):
+ log.info("Fetching %s from HDFS" % uri)
+
+ if "JAVA_HOME" in os.environ:
+ log.info("Using JAVA_HOME '%s' for HDFS commands" % os.environ["JAVA_HOME"])
+
+ config = os.environ.get("HADOOP_CONF_DIR", HADOOP_CONF_DIR)
+ h = HDFSHelper(config, heap_limit=Amount(256, Data.MB))
+ try:
+ f = lambda: h.copy_to_local(uri, directory)
+ deadline(f, timeout=self._timeout, propagate=True, daemon=True)
+ except HDFSHelper.InternalError as e:
+ raise self.Error('Unable to fetch HDFS package: %s' % e)
+ except Timeout as e:
+ raise self.Error("Failed to fetch package from HDFS within : %s" % (self._timeout, e))
diff --git a/twitter/mysos/common/pkgutil.py b/twitter/mysos/common/pkgutil.py
new file mode 100644
index 0000000..69ba56f
--- /dev/null
+++ b/twitter/mysos/common/pkgutil.py
@@ -0,0 +1,41 @@
+import os
+
+from twitter.common.dirutil import safe_mkdir
+
+import pkg_resources
+
+
+def unpack_assets(output_dir, module, asset_path, execute=lambda x: None):
+ """
+ Extract files from a module in a package into 'output_dir'.
+
+ :param output_dir: The directory to copy the assets *root* to.
+ :param module: The module in the package to find the assets in. e.g., twitter.mysos.executor.
+ :param asset_path: The path of the asset relative to the module root to unpack.
+ :param execute: A function that expects a single argument as the path to an asset file. It can
+ be used to process this file.
+
+ NOTE: If the specified 'asset_path' is a directory, its contents are copied but it itself is not
+ recreated in the output directory. e.g., <module_dir>/files/bin/file.sh is copied to
+ <output_dir>/bin/file.sh. An analogy is that its like `cp -R asset_path/* output_dir` and
+ not like `cp -R asset_path output_dir`.
+ """
+ _unpack_assets(output_dir, module, asset_path, execute, asset_path)
+
+
+def _unpack_assets(output_dir, module, asset_root, execute, current_path):
+ """
+ The internal helper function for unpack_assets(...) recursion.
+ :param current_path: Records the current
+ """
+ for asset in pkg_resources.resource_listdir(module, current_path):
+ asset_target = os.path.join(os.path.relpath(current_path, asset_root), asset)
+ if pkg_resources.resource_isdir(module, os.path.join(current_path, asset)):
+ safe_mkdir(os.path.join(output_dir, asset_target))
+ _unpack_assets(output_dir, module, asset_root, execute, os.path.join(current_path, asset))
+ else:
+ output_file = os.path.join(output_dir, asset_target)
+ with open(output_file, 'wb') as fp:
+ fp.write(pkg_resources.resource_string(
+ module, os.path.join(asset_root, asset_target)))
+ execute(output_file)
diff --git a/twitter/mysos/common/testing.py b/twitter/mysos/common/testing.py
new file mode 100644
index 0000000..a4aa4cf
--- /dev/null
+++ b/twitter/mysos/common/testing.py
@@ -0,0 +1,29 @@
+from collections import defaultdict
+import subprocess
+
+
+class Fake(object):
+ """
+ A fake object that does nothing but recording the method calls and arguments.
+ """
+
+ def __init__(self):
+ self.method_calls = defaultdict(list)
+
+ def __getattr__(self, attr):
+ def enqueue_arguments(*args, **kw):
+ self.method_calls[attr].append((args, kw))
+ return enqueue_arguments
+
+
+def build_and_execute_pex_target(target, binary):
+ """
+ :param target: The pants target.
+ :param binary: The path to the pex binary relative to the root of the repository.
+ """
+ assert subprocess.call(["./pants", "goal", "binary", target]) == 0
+
+ p = subprocess.Popen([binary, "--help"], stderr=subprocess.STDOUT, stdout=subprocess.PIPE)
+ out, err = p.communicate()
+ assert p.returncode == 1
+ assert out.startswith('Options'), 'Unexpected build output: %s' % out
diff --git a/twitter/mysos/common/zookeeper.py b/twitter/mysos/common/zookeeper.py
new file mode 100644
index 0000000..44ba7d8
--- /dev/null
+++ b/twitter/mysos/common/zookeeper.py
@@ -0,0 +1,34 @@
+import string
+
+
+def parse(url):
+ """
+ Parse ZooKeeper URL.
+ :param url: The URL in the form of "zk://username:password@servers/path".
+ :return: Tuple (credential, servers, path).
+ credential: Credential for authentication with "digest" scheme. Optional and default to
+ None.
+ servers: Compatible with Kazoo's 'hosts' argument.
+ path: Optional and default to '/'.
+ NOTE: This method doesn't validate the values in the returned tuple.
+ """
+ index = string.find(url, "zk://")
+ if index != 0:
+ raise ValueError("Expecting 'zk://' at the beginning of the URL")
+
+ url = string.lstrip(url, "zk://")
+
+ try:
+ servers, path = string.split(url, '/', 1)
+ except ValueError:
+ servers = url
+ path = ''
+
+ path = '/' + path
+
+ try:
+ credential, servers = string.split(servers, '@', 1)
+ except ValueError:
+ credential = None
+
+ return credential, servers, path
diff --git a/twitter/mysos/executor/__init__.py b/twitter/mysos/executor/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/twitter/mysos/executor/__init__.py
diff --git a/twitter/mysos/executor/backup.py b/twitter/mysos/executor/backup.py
new file mode 100644
index 0000000..c3cf221
--- /dev/null
+++ b/twitter/mysos/executor/backup.py
@@ -0,0 +1,55 @@
+from abc import abstractmethod
+
+from twitter.common.lang import Interface
+
+
+class BackupInfo(object):
+ def __init__(self, backup_file, cold_backup):
+ self.backup_file = backup_file
+ self.cold_backup = cold_backup
+
+
+class BackupStore(Interface):
+ """
+ The storage for Mysos executor state backup (i.e. MySQL data, etc.).
+
+ Thread-safety: The BackupStore implementation is not expected to be thread-safe and the caller
+ should be responsible for it.
+ """
+
+ class Error(Exception): pass
+ class BackupNotFoundError(Error): pass
+
+ @abstractmethod
+ def restore(self):
+ """
+ Restore the backup.
+
+ :return: The BackupInfo object. None if no state is restored.
+ """
+ pass
+
+
+class BackupStoreProvider(Interface):
+ @abstractmethod
+ def from_task(self, task, sandbox):
+ """
+ Factory method that creates a BackupStore instance from 'task' (TaskInfo).
+
+ :return: The BackupStore instance.
+ """
+ pass
+
+
+class NoopBackupStore(BackupStore):
+ """Used when no restore is requested."""
+
+ def restore(self):
+ return None # Returning 'None' because no state is restored.
+
+
+class NoopBackupStoreProvider(BackupStoreProvider):
+ """Used when no restore is requested."""
+
+ def from_task(self, task, sandbox):
+ return NoopBackupStore()
diff --git a/twitter/mysos/executor/executor.py b/twitter/mysos/executor/executor.py
new file mode 100644
index 0000000..3d7dff6
--- /dev/null
+++ b/twitter/mysos/executor/executor.py
@@ -0,0 +1,169 @@
+import json
+
+from twitter.common import log
+from twitter.common.concurrent import defer
+from twitter.mysos.common.decorators import logged
+
+from .task_runner import TaskError
+
+from mesos.interface import Executor
+import mesos.interface.mesos_pb2 as mesos_pb2
+
+
+class MysosExecutor(Executor):
+ """
+ MysosExecutor is a fine-grained executor, i.e., one executor executes a single task.
+ """
+
+ def __init__(self, runner_provider, sandbox):
+ """
+ :param runner_provider: An implementation of TaskRunnerProvider.
+ :param sandbox: The path to the sandbox where all files the executor reads/writes are located.
+ """
+ self._runner_provider = runner_provider
+ self._runner = None # A singleton task runner created by launchTask().
+ self._driver = None # Assigned in registered().
+ self._killed = False # True if the executor's singleton task is killed by the scheduler.
+ self._sandbox = sandbox
+
+ # --- Mesos methods. ---
+ @logged
+ def registered(self, driver, executorInfo, frameworkInfo, slaveInfo):
+ log.info('Registered with slave: %s' % slaveInfo)
+ self._driver = driver # Cache the driver to kill later.
+
+ @logged
+ def reregistered(self, driver, slaveInfo):
+ log.info('Reregistered with slave: %s' % slaveInfo)
+
+ @logged
+ def disconnected(self, driver):
+ log.info("ExecutorDriver disconnected from Mesos slave")
+
+ @logged
+ def launchTask(self, driver, task):
+ if self._runner:
+ log.error("Executor allows only one task")
+ update = mesos_pb2.TaskStatus()
+ update.state = mesos_pb2.TASK_FAILED
+ driver.sendStatusUpdate(update)
+ return
+
+ # Create the runner here in the driver thread so subsequent task launches are rejected.
+ try:
+ self._runner = self._runner_provider.from_task(task, self._sandbox)
+ except (TaskError, ValueError) as e:
+ # TODO(jyx): These should really all be 'ValueError's from all providers because they are
+ # simply factory methods.
+ log.error("Failed to create TaskRunner: %s" % e.message)
+ self._send_update(task.task_id.value, mesos_pb2.TASK_FAILED, e.message)
+ self._kill()
+ return
+
+ # Run the task in a separate daemon thread.
+ defer(lambda: self._run_task(task))
+
+ def _run_task(self, task):
+ assert self._runner, "_runner should be created before this method is called"
+
+ try:
+ self._runner.start()
+ log.info("Task runner for task %s started" % task.task_id)
+
+ self._send_update(task.task_id.value, mesos_pb2.TASK_RUNNING)
+ except TaskError as e:
+ log.error("Task runner for task %s failed to start: %s" % (task.task_id, str(e)))
+ # Send TASK_FAILED if the task failed to start.
+ self._send_update(task.task_id.value, mesos_pb2.TASK_FAILED)
+ except Exception as e:
+ log.error("Error occurred while executing the task: %s" % e)
+ # Send TASK_LOST for unknown errors.
+ self._send_update(task.task_id.value, mesos_pb2.TASK_LOST)
+
+ # Wait for the task's return code (when it terminates).
+ try:
+ returncode = self._runner.join()
+ # Regardless of the return code, if '_runner' terminates, it failed!
+ log.error("Task process terminated with return code %s" % returncode)
+ except TaskError as e:
+ log.error("Task terminated: %s" % e)
+
+ if self._killed:
+ self._send_update(task.task_id.value, mesos_pb2.TASK_KILLED)
+ else:
+ self._send_update(task.task_id.value, mesos_pb2.TASK_FAILED)
+
+ self._kill()
+
+ @logged
+ def frameworkMessage(self, driver, message):
+ if not self._runner:
+ log.info('Ignoring framework message because no task is running yet')
+ return
+
+ defer(lambda: self._framework_message(message))
+
+ def _framework_message(self, message):
+ master_epoch = message # The log position request is for electing the master of this 'epoch'.
+ try:
+ position = self._runner.get_log_position()
+ log.info('Obtained log position %s for epoch %s' % (position, master_epoch))
+
+ assert self._driver
+
+ # TODO(jyx): Define the message in ProtoBuf or Thrift.
+ self._driver.sendFrameworkMessage(json.dumps({
+ 'epoch': master_epoch, # Send the epoch back without parsing it.
+ 'position': position
+ }))
+ except TaskError as e:
+ # Log the error and do not reply to the framework.
+ log.error("Committing suicide due to failure to process framework message: %s" % e)
+ self._kill()
+
+ @logged
+ def killTask(self, driver, taskId):
+ # Killing the task also kills the executor because there is one task per executor.
+ log.info("Asked to kill task %s" % taskId)
+ self._killed = True
+ self._kill()
+
+ def _kill(self):
+ if self._runner:
+ self._runner.stop() # It could be already stopped. If so, self._runner.stop() is a no-op.
+
+ assert self._driver
+
+ # TODO(jyx): Fix https://issues.apache.org/jira/browse/MESOS-243.
+ self._driver.stop()
+
+ @logged
+ def shutdown(self, driver):
+ log.info("Asked to shut down")
+ self._killed = True
+ self._kill()
+
+ @logged
+ def error(self, driver, message):
+ log.error("Shutting down due to error: %s" % message)
+ self._killed = True
+ self._kill()
+
+ def _send_update(self, task_id, state, message=None):
+ update = mesos_pb2.TaskStatus()
+ if not isinstance(state, int):
+ raise TypeError('Invalid state type %s, should be int.' % type(state))
+ if state not in [
+ mesos_pb2.TASK_STARTING,
+ mesos_pb2.TASK_RUNNING,
+ mesos_pb2.TASK_FINISHED,
+ mesos_pb2.TASK_KILLED,
+ mesos_pb2.TASK_FAILED,
+ mesos_pb2.TASK_LOST]:
+ raise ValueError('Invalid state: %s' % state)
+ update.state = state
+ update.task_id.value = task_id
+ if message:
+ update.message = str(message)
+ log.info('Updating %s => %s. Reason: %s' % (task_id, mesos_pb2.TaskState.Name(state), message))
+ self._driver.sendStatusUpdate(update)
diff --git a/twitter/mysos/executor/files/bin/mysql/scripts/mysos_install_db.sh b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_install_db.sh
new file mode 100755
index 0000000..bf4dbee
--- /dev/null
+++ b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_install_db.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+#
+# Install the MySQL server files before starting 'mysqld'.
+#
+
+set -uex
+
+framework_user=$1
+data_dir=$2
+conf_file=$3
+
+# Expecting mysqld to be under $mysql_basedir/bin/
+mysql_basedir=$(dirname $(dirname $(which mysqld)))
+
+# Remove the datadir if any since we are creating a new instance.
+rm -rf $data_dir
+
+# Initialize the DB.
+mysql_install_db \
+ --defaults-file=$conf_file \
+ --datadir=$data_dir \
+ --user=$framework_user \
+ --bind-address=0.0.0.0 \
+ --basedir=$mysql_basedir
\ No newline at end of file
diff --git a/twitter/mysos/executor/files/bin/mysql/scripts/mysos_launch_mysqld.sh b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_launch_mysqld.sh
new file mode 100755
index 0000000..5a2ba90
--- /dev/null
+++ b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_launch_mysqld.sh
@@ -0,0 +1,40 @@
+#!/bin/sh
+
+set -uex
+
+framework_user=$1
+host=$2
+port=$3
+server_id=$4
+data_dir=$5
+log_dir=$6
+tmp_dir=$7
+conf_file=$8
+buffer_pool_size=$9
+
+# Expecting mysqld to be under $mysql_basedir/bin/
+mysql_basedir=$(dirname $(dirname $(which mysqld)))
+
+# Need a temp directory under /tmp because the sandbox path is too long for '--socket'. We also need
+# this path to be unique on the host.
+socket_tmp=`mktemp -d`
+
+# Start the server in read only mode.
+mysqld \
+ --defaults-file=$conf_file \
+ --user=$framework_user \
+ --port=$port \
+ --server-id=$server_id \
+ --socket=$socket_tmp/mysql.sock \
+ --pid-file=$log_dir/mysqld.pid \
+ --basedir=$mysql_basedir \
+ --datadir=$data_dir \
+ --tmpdir=$tmp_dir \
+ --innodb_data_home_dir=$data_dir \
+ --innodb_log_group_home_dir=$log_dir \
+ --log-error=$data_dir/mysql-error.log \
+ --log-bin=$log_dir/mysql-bin \
+ --relay-log=$log_dir/mysql-relay-bin \
+ --relay-log-index=$log_dir/mysql-relay-bin.index \
+ --innodb_buffer_pool_size=$buffer_pool_size \
+ --skip-grant-tables
diff --git a/twitter/mysos/executor/files/bin/mysql/scripts/mysos_log_position.sh b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_log_position.sh
new file mode 100755
index 0000000..a205021
--- /dev/null
+++ b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_log_position.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+#
+# Get the log position of the MySQL slave.
+#
+
+set -uxe
+
+slave_host=$1
+slave_port=$2
+
+# Get the relay log file.
+relay_master_log_file=`mysql -u root -P $slave_port -h $slave_host -e "show slave status\G;" | \
+ grep Relay_Master_Log_File | awk '{print $2}'`
+
+# Get the relay log position.
+exec_master_log_pos=`mysql -u root -P $slave_port -h $slave_host -e "show slave status\G;" | \
+ grep Exec_Master_Log_Pos | awk '{print $2}'`
+
+# The output can be empty if the slave has not set up replication.
+echo $relay_master_log_file,$exec_master_log_pos
\ No newline at end of file
diff --git a/twitter/mysos/executor/files/bin/mysql/scripts/mysos_promote_master.sh b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_promote_master.sh
new file mode 100755
index 0000000..722f902
--- /dev/null
+++ b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_promote_master.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+#
+# Promote the MySQL slave to be a master.
+#
+
+set -uex
+
+host=$1
+port=$2
+user=$3
+password=$4
+admin_user=$5
+admin_passwd=$6
+
+# Stop and reset the slave.
+mysql -u root -P $port -h $host -e "STOP SLAVE; RESET SLAVE ALL;
+
+# Put the master in read-write mode.
+SET GLOBAL read_only = 0; UNLOCK TABLES;
+
+# This reloads the grant tables so the following account-management statements can work.
+FLUSH PRIVILEGES;
+
+# Grant all permissions to the admin user (and create it if not exists).
+GRANT ALL ON *.* to '$admin_user'@'%' IDENTIFIED BY '$admin_passwd' WITH GRANT OPTION;
+
+# Grant all permissions to the user (and create it if not exists).
+GRANT ALL ON *.* to '$user'@'%' IDENTIFIED BY '$password' WITH GRANT OPTION;
+
+# Drop anonymous user.
+DELETE FROM mysql.user WHERE user=''; FLUSH PRIVILEGES;"
diff --git a/twitter/mysos/executor/files/bin/mysql/scripts/mysos_reparent.sh b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_reparent.sh
new file mode 100755
index 0000000..c609925
--- /dev/null
+++ b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_reparent.sh
@@ -0,0 +1,49 @@
+#!/bin/sh
+#
+# Reparent the slave to a new master.
+#
+
+set -uxe
+
+master_host=$1
+master_port=$2
+slave_host=$3
+slave_port=$4
+admin_user=$5
+admin_passwd=$6
+
+# Stop the replication (if any).
+mysql -u root -P $slave_port -h $slave_host -e "STOP SLAVE;"
+
+# Get the relay log file.
+relay_master_log_file=`mysql -u root -P $slave_port -h $slave_host -e "show slave status\G;" | \
+ grep Relay_Master_Log_File | awk '{print $2}'`
+
+# Get the relay log position.
+exec_master_log_pos=`mysql -u root -P $slave_port -h $slave_host -e "show slave status\G;" | \
+ grep Exec_Master_Log_Pos | awk '{print $2}'`
+
+# TODO(vinod): Make sure the slave processed the relay log.
+
+# Remove the relay logs.
+mysql -u root -P $slave_port -h $slave_host -e "RESET SLAVE;"
+
+# Point to a master.
+if [ "x$relay_master_log_file" != "x" ]; then
+ mysql -u root -P $slave_port -h $slave_host -e "CHANGE MASTER TO
+ MASTER_HOST='$master_host',
+ MASTER_PORT=$master_port,
+ MASTER_USER='$admin_user',
+ MASTER_PASSWORD='$admin_passwd',
+ MASTER_LOG_FILE='$relay_master_log_file',
+ MASTER_LOG_POS=$exec_master_log_pos;"
+else
+ mysql -u root -P $slave_port -h $slave_host -e "CHANGE MASTER TO
+ MASTER_HOST='$master_host',
+ MASTER_PORT=$master_port,
+ MASTER_USER='$admin_user',
+ MASTER_PASSWORD='$admin_passwd';"
+fi
+
+# Start replication.
+mysql -u root -P $slave_port -h $slave_host -e "START SLAVE;"
diff --git a/twitter/mysos/executor/files/bin/mysql/scripts/mysos_wait_for_mysqld.sh b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_wait_for_mysqld.sh
new file mode 100755
index 0000000..110d3d5
--- /dev/null
+++ b/twitter/mysos/executor/files/bin/mysql/scripts/mysos_wait_for_mysqld.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+#
+# Wait for the mysqld instance to start up and be ready for traffic.
+#
+
+set -ue
+
+path=$1
+port=$2
+timeout=$3
+
+waited=1
+while true
+do
+ if [ -f $path ]; then
+ listen=`netstat -lnt | grep $port | wc -l`
+ if [ "x$listen" = "x1" ]; then
+ exit 0
+ fi
+ fi
+ sleep 1
+ waited=`expr $waited + 1`
+ if [ $waited -ge $timeout ]; then
+ exit 1
+ fi
+done
diff --git a/twitter/mysos/executor/installer.py b/twitter/mysos/executor/installer.py
new file mode 100644
index 0000000..a8b98aa
--- /dev/null
+++ b/twitter/mysos/executor/installer.py
@@ -0,0 +1,32 @@
+from abc import abstractmethod
+
+from twitter.common.lang import Interface
+
+
+class PackageInstaller(Interface):
+ """
+ Responsible for installing MySQL and dependent packages.
+
+ NOTE: It is not responsible for installing DB instances.
+ """
+
+ class Error(Exception): pass
+
+ @abstractmethod
+ def install(self):
+ """
+ :return: The environment variables necessary for MySQL to execute.
+ :rtype: dict
+ """
+ pass
+
+
+class PackageInstallerProvider(Interface):
+ @abstractmethod
+ def from_task(self, task, sandbox):
+ """
+ Factory method that creates a PackageInstaller instance from 'task' (TaskInfo).
+
+ :return: The PackageInstaller instance.
+ """
+ pass
diff --git a/twitter/mysos/executor/mysos_task_runner.py b/twitter/mysos/executor/mysos_task_runner.py
new file mode 100644
index 0000000..97e3ba1
--- /dev/null
+++ b/twitter/mysos/executor/mysos_task_runner.py
@@ -0,0 +1,264 @@
+import Queue
+import json
+import os
+import signal
+from subprocess import CalledProcessError
+import sys
+import threading
+
+from twitter.common import log
+from twitter.common.concurrent import defer
+from twitter.common.zookeeper.serverset.endpoint import Endpoint, ServiceInstance
+from twitter.mysos.common.cluster import ClusterListener, get_cluster_path
+from twitter.mysos.common.zookeeper import parse
+
+from .installer import PackageInstaller
+from .state import StateManager
+from .task_runner import TaskError, TaskRunner, TaskRunnerProvider
+from .task_control import TaskControl
+
+from kazoo.client import KazooClient
+
+
+class MysosTaskRunnerProvider(TaskRunnerProvider):
+ def __init__(self, task_control_provider, installer_provider, backup_store_provider):
+ self._task_control_provider = task_control_provider
+ self._installer_provider = installer_provider
+ self._backup_store_provider = backup_store_provider
+
+ def from_task(self, task, sandbox):
+ data = json.loads(task.data)
+ cluster_name, host, port, zk_url = data['cluster'], data['host'], data['port'], data['zk_url']
+ _, servers, path = parse(zk_url)
+ kazoo = KazooClient(servers)
+ kazoo.start()
+ self_instance = ServiceInstance(Endpoint(host, port))
+
+ try:
+ task_control = self._task_control_provider.from_task(task, sandbox)
+ installer = self._installer_provider.from_task(task, sandbox)
+ backup_store = self._backup_store_provider.from_task(task, sandbox)
+ except (TaskControl.Error, PackageInstaller.Error) as e:
+ raise TaskError(e.message)
+
+ state_manager = StateManager(sandbox, backup_store)
+
+ return MysosTaskRunner(
+ self_instance,
+ kazoo,
+ get_cluster_path(path, cluster_name),
+ installer,
+ task_control,
+ state_manager)
+
+
+class MysosTaskRunner(TaskRunner):
+ """
+ A runner that manages the lifecycle of a MySQL task (through the provided 'task_control').
+
+ The task is executed as a long-running process its return code can be obtained using 'join()'.
+
+ Thread-safety:
+ This class is accessed from the MysosExecutor thread (not the ExecutorDriver thread because
+ MysosExecutor invokes operations asynchronously) and the ClusterListener thread and is
+ thread-safe.
+
+ TODO(jyx): Push the knowledge of the underlying subprocess down to the task control and stop the
+ the subprocess using the task control.
+ """
+
+ def __init__(self, self_instance, kazoo, cluster_root, installer, task_control, state_manager):
+ """
+ :param self_instance: The local ServiceInstance associated with this task runner.
+ :param kazoo: Kazoo client, it should be started before being passed in.
+ :param cluster_root: The ZooKeeper root path for *this cluster*.
+ :param installer: The PackageInstaller for MySQL.
+ :param task_control: The TaskControl that interacts with the task process.
+ :param state_manager: The StateManager for managing the executor state.
+ """
+ self._installer = installer
+ self._env = None # The environment variables for the 'task_control' commands. Set by the
+ # installer.
+
+ self._task_control = task_control
+ self._state_manager = state_manager
+
+ self._lock = threading.Lock()
+ self._popen = None # The singleton task process started by '_task_control'.
+
+ self._started = False # Indicates whether start() has already been called.
+ self._stopping = False # Indicates whether stop() has already been called.
+ self._exited = threading.Event() # Set when the task process has exited.
+ self._result = Queue.Queue() # The returncode returned by the task process or an exception.
+
+ # Public events and queue.
+ self.promoted = threading.Event()
+ self.demoted = threading.Event()
+ self.master = Queue.Queue() # Set when a master change is detected.
+
+ self._listener = ClusterListener(
+ kazoo,
+ cluster_root,
+ self_instance,
+ promotion_callback=self._on_promote,
+ demotion_callback=self._on_demote,
+ master_callback=self._on_master_change) # Listener started by start().
+
+ # --- Public interface. ---
+ def start(self):
+ """
+ Start the runner in a separate thread and wait for the task process to be forked.
+ """
+ with self._lock:
+ if self._started:
+ raise TaskError("Runner already started")
+ self._started = True
+
+ # Can potentially hold the lock for a long time but it's OK since the runner is not accessed
+ # by multiple threads until after it's started; can be a noop as well, depending on the
+ # installer implementation.
+ try:
+ # 1. Install the application.
+ self._env = self._installer.install()
+ log.info("Package installation completed. Resulting environment variables: %s" % self._env)
+
+ # 2. Restore/initialize the application state.
+ self._state_manager.bootstrap(self._task_control, self._env)
+ log.info("Executor state fully bootstrapped")
+
+ # 3. Start the task subprocess.
+ # Store the process so we can kill it if necessary.
+ self._popen = self._task_control.start(env=self._env)
+ log.info("Task started in subprocess %s" % self._popen.pid)
+ defer(self._wait)
+
+ # 4. Start monitoring.
+ # Only start listening to ZK events after the task subprocess has been successfully started.
+ self._listener.start()
+ except (PackageInstaller.Error, StateManager.Error, CalledProcessError) as e:
+ raise TaskError("Failed to start MySQL task: %s" % e)
+
+ def _wait(self):
+ # Block until the subprocess exits and delivers the return code.
+ self._result.put(self._popen.wait())
+
+ # Notify stop() if it is waiting.
+ self._exited.set()
+
+ def stop(self, timeout=10):
+ """
+ Stop the runner and wait for its thread (and the sub-processes) to exit.
+
+ :param timeout: The timeout that the process should die before a hard SIGKILL is issued
+ (SIGTERM is used initially).
+ :return: True if an active runner is stopped, False if the runner is not started or already
+ stopping/stopped.
+ """
+ if not self._started:
+ log.warn("Cannot stop the runner because it's not started")
+ return False
+
+ if self._stopping:
+ log.warn("The runner is already stopping/stopped")
+ return False
+
+ with self._lock:
+ log.info("Stopping runner")
+ self._stopping = True
+
+ if not self._popen:
+ log.info("The runner task did not start successfully so no need to kill it")
+ return False
+
+ try:
+ log.info("Terminating process group: %s" % self._popen.pid)
+ os.killpg(self._popen.pid, signal.SIGTERM)
+ except OSError as e:
+ log.info("The sub-processes are already terminated: %s" % e)
+ return False
+
+ log.info("Waiting for process to terminate due to SIGTERM")
+
+ # Escalate to SIGKILL if SIGTERM is not sufficient.
+ if not self._exited.wait(timeout=timeout):
+ with self._lock:
+ try:
+ log.warn("Killing process group %s which failed to terminate cleanly within %s secs" %
+ (self._popen.pid, timeout))
+ os.killpg(self._popen.pid, signal.SIGKILL)
+ except OSError as e:
+ log.info("The sub-processes are already terminated: %s" % e)
+ return False
+
+ log.info("Waiting for process to terminate due to SIGKILL")
+ if not self._exited.wait(timeout=timeout):
+ raise TaskError("Failed to kill process group %s" % self._popen.pid)
+
+ return True
+
+ def get_log_position(self):
+ """
+ Get the log position of the MySQL slave. Return None if it cannot be obtained.
+ """
+ try:
+ log_position = self._task_control.get_log_position(env=self._env)
+ return log_position
+ except CalledProcessError as e:
+ raise TaskError("Unable to get the slave's log position: %s" % e)
+
+ def join(self):
+ """
+ Wait for the runner to terminate.
+
+ :return: The return code of the subprocess. NOTE: A negative value -N indicates that the
+ child was terminated by signal N (on Unix).
+
+ :exception: The TaskError exception due to an error in task control operations.
+ """
+ # Using 'sys.maxint' makes this forever wait interruptible.
+ result = self._result.get(True, sys.maxint)
+ if isinstance(result, Exception):
+ raise result
+ else:
+ return result
+
+ # --- ClusterListener handlers. ---
+ def _on_promote(self):
+ self.promoted.set()
+ if not self._exited.is_set():
+ defer(self._promote)
+
+ def _promote(self):
+ try:
+ self._task_control.promote(env=self._env)
+ except CalledProcessError as e:
+ self._result.put(TaskError("Failed to promote the slave: %s" % e))
+ self.stop()
+
+ def _on_demote(self):
+ """
+ Executor shuts itself down when demoted.
+ """
+ self.demoted.set()
+
+ # Stop the runner asynchronously.
+ if not self._exited.is_set():
+ log.info("Shutting down runner because it is demoted.")
+ # Call stop() asynchronously because this callback is invoked from the Kazoo thread which we
+ # don't want to block.
+ defer(self.stop)
+
+ def _on_master_change(self, master):
+ self.master.put(master)
+ if not self._exited.is_set():
+ defer(lambda: self._reparent(master))
+
+ def _reparent(self, master):
+ try:
+ self._task_control.reparent(
+ master.service_endpoint.host,
+ master.service_endpoint.port,
+ env=self._env)
+ except CalledProcessError as e:
+ self._result.put(TaskError("Failed to reparent the slave: %s" % e))
+ self.stop()
diff --git a/twitter/mysos/executor/mysql_task_control.py b/twitter/mysos/executor/mysql_task_control.py
new file mode 100644
index 0000000..b4979a2
--- /dev/null
+++ b/twitter/mysos/executor/mysql_task_control.py
@@ -0,0 +1,212 @@
+import json
+import os
+import subprocess
+import threading
+
+from twitter.common import log
+from twitter.common.quantity import Amount, Data
+from twitter.mysos.common.decorators import synchronized
+
+from .sandbox import Sandbox
+from .task_control import TaskControl, TaskControlProvider
+
+import yaml
+
+
+MEM_FRACTION_FOR_BUFFER_POOL = 0.75
+
+
+class MySQLTaskControlProvider(TaskControlProvider):
+ """
+ The default implementation of MySQLTaskControlProvider.
+ There exist other implementations for testing purposes.
+ """
+
+ def from_task(self, task, sandbox):
+ data = json.loads(task.data)
+ task_mem = None
+ for resource in task.resources:
+ if resource.name == 'mem':
+ task_mem = resource.scalar.value
+ break
+
+ assert task_mem, "Task resources should always include 'mem'"
+
+ buffer_pool_size = int(
+ Amount(int(task_mem), Data.MB).as_(Data.BYTES) * MEM_FRACTION_FOR_BUFFER_POOL)
+ log.info("Allocating %s bytes of memory to MySQL buffer pool" % buffer_pool_size)
+
+ # TODO(jyx): Use an ephemeral sandbox for now. Will change when Mesos adds persistent resources
+ # support: MESOS-1554.
+ return MySQLTaskControl(
+ sandbox,
+ data['framework_user'],
+ data['host'],
+ data['port'],
+ data['cluster'],
+ data['cluster_user'],
+ data['cluster_password'],
+ data['server_id'],
+ data['admin_keypath'],
+ buffer_pool_size)
+
+
+class MySQLTaskControl(TaskControl):
+ def __init__(
+ self,
+ sandbox,
+ framework_user,
+ host,
+ port,
+ cluster_name,
+ cluster_user,
+ password,
+ server_id,
+ admin_keypath,
+ buffer_pool_size):
+ """
+ :param sandbox: The sandbox where all files of this Mysos executor instance reside.
+ :param framework_user: The Unix user this framework runs as.
+ :param host: The hostname of the host that runs the MySQL instance.
+ :param port: The port of the MySQL instance.
+ :param cluster_name: The name of the cluster.
+ :param cluster_user: The Unix account that mysqld will run as and also the MySQL username.
+ :param password: The MySQL password associated with 'cluster_user' in MySQL.
+ :param server_id: The ID that identifies the MySQL instance.
+ :param buffer_pool_size: For the 'innodb_buffer_pool_size' variable in MySQL options.
+ """
+ if not isinstance(sandbox, Sandbox):
+ raise TypeError("'sandbox' should be an instance of Sandbox")
+ self._sandbox = sandbox
+
+ self._framework_user = framework_user
+ self._host = host
+ self._port = port
+ self._cluster_name = cluster_name
+ self._cluster_user = cluster_user
+ self._password = password
+ self._server_id = server_id
+
+ if not isinstance(buffer_pool_size, int):
+ raise ValueError("'buffer_pool_size' should be an instance of int")
+
+ self._buffer_pool_size = buffer_pool_size
+
+ try:
+ with open(admin_keypath, "r") as f:
+ cred = yaml.load(f)
+ self._admin_username = cred["username"]
+ self._admin_password = cred["password"]
+ log.info("Loaded credentials for admin account %s" % self._admin_username)
+ except IOError as e:
+ raise ValueError("Unable to obtain admin credentials: %s" % e)
+ except (KeyError, yaml.YAMLError) as e:
+ raise ValueError("Invalid key file format %s" % e)
+
+ self._lock = threading.Lock()
+ self._process = None # The singleton task process that launches mysqld.
+
+ self._scripts_dir = os.path.join(self._sandbox.bin, "mysql", "scripts")
+ if not os.path.isdir(self._scripts_dir):
+ raise TaskControl.Error("Scripts directory %s does not exist" % self._scripts_dir)
+
+ custom_conf_file = os.environ.get('MYSOS_DEFAULTS_FILE', None)
+ if custom_conf_file:
+ log.info("Using 'MYSOS_DEFAULTS_FILE': %s" % custom_conf_file)
+ self._conf_file = custom_conf_file.strip()
+ else:
+ self._conf_file = os.path.join(self._sandbox.bin, "mysql", "conf", "my.cnf")
+
+ if not os.path.isfile(self._conf_file):
+ raise TaskControl.Error("Option file %s does not exist" % self._conf_file)
+
+ @synchronized
+ def initialize(self, env):
+ command = "%(cmd)s %(framework_user)s %(data_dir)s %(conf_file)s" % dict(
+ cmd=os.path.join(self._scripts_dir, "mysos_install_db.sh"),
+ framework_user=self._framework_user,
+ data_dir=self._sandbox.mysql_data_dir,
+ conf_file=self._conf_file)
+ log.info("Executing command: %s" % command)
+ subprocess.check_call(command, shell=True, env=env)
+
+ @synchronized
+ def start(self, env=None):
+ if self._process:
+ log.warn("start() called when a running task subprocess already exists")
+ return
+
+ command = (
+ "%(cmd)s %(framework_user)s %(host)s %(port)s %(server_id)s %(data_dir)s %(log_dir)s "
+ "%(tmp_dir)s %(conf_file)s %(buffer_pool_size)s" % dict(
+ cmd=os.path.join(self._scripts_dir, "mysos_launch_mysqld.sh"),
+ framework_user=self._framework_user,
+ host=self._host,
+ port=self._port,
+ server_id=self._server_id,
+ data_dir=self._sandbox.mysql_data_dir,
+ log_dir=self._sandbox.mysql_log_dir,
+ tmp_dir=self._sandbox.mysql_tmp_dir,
+ conf_file=self._conf_file,
+ buffer_pool_size=self._buffer_pool_size))
+ log.info("Executing command: %s" % command)
+ self._process = subprocess.Popen(command, shell=True, env=env)
+
+ # There is a delay before mysqld becomes available to accept requests. Wait for it.
+ command = "%(cmd)s %(pid_file)s %(port)s %(timeout)s" % dict(
+ cmd=os.path.join(self._scripts_dir, "mysos_wait_for_mysqld.sh"),
+ pid_file=os.path.join(self._sandbox.mysql_log_dir, "mysqld.pid"),
+ port=self._port,
+ timeout=60)
+ log.info("Executing command: %s" % command)
+ subprocess.check_call(command, shell=True, env=env)
+
+ return self._process
+
+ @synchronized
+ def reparent(self, master_host, master_port, env=None):
+ command = ("%(cmd)s %(master_host)s %(master_port)s %(slave_host)s %(slave_port)s "
+ "%(admin_user)s %(admin_password)s" % dict(
+ cmd=os.path.join(self._scripts_dir, "mysos_reparent.sh"),
+ master_host=master_host,
+ master_port=master_port,
+ slave_host=self._host,
+ slave_port=self._port,
+ admin_user=self._admin_username,
+ admin_password=self._admin_password))
+
+ log.info("Executing command: %s" % command)
+ subprocess.check_call(command, shell=True, env=env)
+
+ @synchronized
+ def promote(self, env=None):
+ command = ("%(cmd)s %(host)s %(port)s %(cluster_user)s %(password)s %(admin_user)s "
+ "%(admin_password)s" % dict(
+ cmd=os.path.join(self._scripts_dir, "mysos_promote_master.sh"),
+ host=self._host,
+ port=self._port,
+ cluster_user=self._cluster_user,
+ password=self._password,
+ admin_user=self._admin_username,
+ admin_password=self._admin_password))
+
+ # TODO(jyx): Scrub the command log line to hide the password.
+ log.info("Executing command: %s" % command)
+ subprocess.check_call(command, shell=True, env=env)
+
+ @synchronized
+ def get_log_position(self, env=None):
+ command = '%(cmd)s %(host)s %(port)s' % dict(
+ cmd=os.path.join(self._scripts_dir, "mysos_log_position.sh"),
+ host=self._host,
+ port=self._port)
+
+ log.info("Executing command: %s" % command)
+ output = subprocess.check_output(command, shell=True, env=env).strip()
+
+ if len(output.split(',')) == 2:
+ log_file, log_position = output.split(',') # log_file may be empty.
+ log.info('Obtained log position: %s ' % str((log_file, log_position)))
+ return log_file, log_position
+ else:
+ return None
diff --git a/twitter/mysos/executor/noop_installer.py b/twitter/mysos/executor/noop_installer.py
new file mode 100644
index 0000000..57bbae7
--- /dev/null
+++ b/twitter/mysos/executor/noop_installer.py
@@ -0,0 +1,18 @@
+from twitter.mysos.executor.installer import PackageInstaller, PackageInstallerProvider
+
+
+class NoopPackageInstaller(PackageInstaller):
+ """
+ An installer that doesn't actually install the package.
+
+ It can be used when the host has dependent packages (i.e. MySQL) pre-installed and its upgrades
+ managed externally.
+ """
+
+ def install(self):
+ return None # No environment variables to pass along to the task sub-processes.
+
+
+class NoopPackageInstallerProvider(PackageInstallerProvider):
+ def from_task(self, task, sandbox):
+ return NoopPackageInstaller()
diff --git a/twitter/mysos/executor/sandbox.py b/twitter/mysos/executor/sandbox.py
new file mode 100644
index 0000000..60f1111
--- /dev/null
+++ b/twitter/mysos/executor/sandbox.py
@@ -0,0 +1,82 @@
+from os.path import isabs, join
+
+from twitter.common.dirutil import safe_mkdir
+
+
+class Sandbox(object):
+ """
+ Represents the structure of the Mysos executor sandbox:
+
+ sandbox_root/ # Sandbox for this Mysos instance.
+ bin/ # Binaries, executables.
+ mysql/scripts/ # Mysos scripts (come with the executor).
+ mysos_install_db.sh
+ ...
+ lib/ # Libraries installed by the installer.
+ var/ # For Mysos (and MySQL) to save its state ('datadir', 'tmpdir', etc).
+ mysql/ # The sandbox maintains and exposes some standard mysql directories.
+ data/
+ tmp/
+ logs/
+ """
+
+ def __init__(self, root):
+ """
+ Initializes the sandbox.
+
+ :param root: Root path of the sandbox.
+
+ The sandbox makes sure that the folder paths it exposes as properties are created.
+ """
+ if not isabs(root):
+ raise ValueError("Only an absolute path is allowed for 'root")
+
+ self._root = root
+
+ safe_mkdir(self.bin)
+ safe_mkdir(self.lib)
+ safe_mkdir(self.var)
+ safe_mkdir(self.mysql_var)
+ safe_mkdir(self.mysql_data_dir)
+ safe_mkdir(self.mysql_tmp_dir)
+ safe_mkdir(self.mysql_log_dir)
+
+ @property
+ def root(self):
+ """Root path of the sandbox."""
+ return self._root
+
+ @property
+ def bin(self):
+ """For Mysos binaries."""
+ return join(self._root, "bin")
+
+ @property
+ def lib(self):
+ """For libraries that the Mysos executor replies on."""
+ return join(self._root, "lib")
+
+ @property
+ def var(self):
+ """For the Mysos executor's application data."""
+ return join(self._root, "var")
+
+ @property
+ def mysql_var(self):
+ """For mysql related data files."""
+ return join(self.var, 'mysql')
+
+ @property
+ def mysql_data_dir(self):
+ return join(self.mysql_var, 'data')
+
+ @property
+ def mysql_tmp_dir(self):
+ return join(self.mysql_var, 'tmp')
+
+ @property
+ def mysql_log_dir(self):
+ return join(self.mysql_var, 'logs')
+
+ def __str__(self):
+ return self._root
diff --git a/twitter/mysos/executor/shell_utils.py b/twitter/mysos/executor/shell_utils.py
new file mode 100644
index 0000000..bed9ddc
--- /dev/null
+++ b/twitter/mysos/executor/shell_utils.py
@@ -0,0 +1,122 @@
+"""Set of utility functions for working with OS commands.
+
+Functions in this module return the command string. These commands are composed but not executed.
+"""
+
+import os
+from subprocess import call
+
+
+HADOOP_CONF_DIR = '/etc/hadoop/conf'
+
+
+def encrypt(key_file):
+ """
+ Encrypt the data from stdin and write output to stdout.
+
+ :param key_file: The key file used to encrypt the stream.
+ """
+ if not os.path.isfile(key_file):
+ raise ValueError("Cannot find key_file: %" % key_file)
+
+ return "openssl aes-256-cbc -salt -pass file:%s" % key_file
+
+
+def decrypt(key_file):
+ """
+ Decrypt the data from stdin and write output to stdout.
+
+ :param key_file: The key file used to decrypt the stream.
+ """
+ if not os.path.isfile(key_file):
+ raise ValueError("Cannot find key_file: %" % key_file)
+
+ return "openssl aes-256-cbc -d -pass file:%s" % key_file
+
+
+def compress(extension):
+ """
+ Compress the data from stdin and write output to stdout.
+ :param extension: The compression format identified by the file extension. Allowed values are:
+ 'gz' for gzip, 'bz' or 'bz2' for bzip.
+ """
+ if extension == "gz":
+ cmd = "pigz" if exists("pigz") else "gzip"
+ elif extension == "bz" or extension == "bz2":
+ cmd = "bzip2"
+ elif extension == 'lzo':
+ cmd = "lzop"
+ else:
+ raise ValueError("Unknown compression format/file extension")
+
+ return cmd
+
+
+def decompress(extension):
+ """
+ Decompress the data from stdin and write output to stdout.
+
+ :param extension: The compression format identified by the file extension. Allowed values are:
+ 'gz' for gzip, 'bz' or 'bz2' for bzip.
+ """
+ if extension == "gz":
+ cmd = "pigz -d" if exists("pigz") else "gzip -d"
+ elif extension == "bz" or extension == "bz2":
+ cmd = "bzip2 -d"
+ elif extension == 'lzo':
+ cmd = "lzop -d"
+ else:
+ raise ValueError("Unknown compression format/file extension")
+
+ return cmd
+
+
+def hdfs_cat(uri, conf=HADOOP_CONF_DIR):
+ """
+ Fetch the data from the specified uri and write output to stdout.
+
+ :param uri: The HDFS URI.
+ :param conf: The hadoop config directory.
+ """
+ return "hadoop --config %s dfs -cat %s" % (conf, uri)
+
+
+def pv(size):
+ """
+ Monitor the progress of data through a pipe. If 'pv' is not available, simply 'cat' it.
+
+ :param size: The size of the data, to calculate percentage.
+ """
+ if exists('pv'):
+ return "pv --wait --size %s" % size
+ else:
+ return "cat"
+
+
+def untar(directory):
+ """
+ Untar the data from stdin into the specified directory.
+
+ :param directory: The directory to write files to.
+ """
+ return "tar -C %s -x" % directory
+
+
+def tar(path):
+ """
+ Tar the path and write output to stdout.
+
+ :param path: All contents under path are 'tar'ed.
+ """
+ if not os.path.exists(path):
+ raise ValueError("Invalid argument: 'path' doesn't exist")
+
+ path = path.rstrip(os.sep)
+ parent, base = os.path.split(path)
+ return "tar -C %s %s" % (parent, base)
+
+
+def exists(cmd):
+ """Return true if 'cmd' exists in $PATH."""
+ with open(os.devnull, "w") as f:
+ return call(['which', cmd], stdout=f) == 0 # No stdout.
diff --git a/twitter/mysos/executor/state.py b/twitter/mysos/executor/state.py
new file mode 100644
index 0000000..eff4ffe
--- /dev/null
+++ b/twitter/mysos/executor/state.py
@@ -0,0 +1,67 @@
+import os
+import json
+import subprocess
+
+from twitter.common import log
+
+from .backup import BackupStore
+
+
+BACKUP_INFO_FILENAME = "backup_info.json"
+
+
+class StateManager(object):
+ """
+ Responsible for managing (restoring, initializing) the executor state.
+
+ TODO(jyx): Will also periodically back up the state.
+ """
+
+ class Error(Exception): pass
+
+ def __init__(self, sandbox, backup_store):
+ """
+ :param sandbox: The sandbox.
+ :param backup_store: The BackupStore implementation.
+ """
+ self._sandbox = sandbox
+ self._backup_store = backup_store
+
+ def bootstrap(self, task_control, env):
+ """
+ Bootstraps the executor state.
+
+ :param task_control: Task control for carrying out state bootstrapping commands.
+ :param env: The environment variables for task_control methods.
+ """
+
+ # 1. Directly return if the data folder is not empty.
+ if os.listdir(self._sandbox.mysql_data_dir):
+ # TODO(jyx): This will be expected when we use persistent volumes. Validate the state in that
+ # case.
+ log.warn("MySQL state already exists unexpectedly. Finishing bootstrap without restoration "
+ "or initialization")
+ return
+
+ # 2. If the data folder is clean, restore state from the backup store. This can be a noop if the
+ # user doesn't want to restore any state.
+ try:
+ backup_info = self._backup_store.restore()
+ except BackupStore.Error as e:
+ raise self.Error("Failed to restore MySQL state: %s" % e)
+
+ if backup_info:
+ # If some backup is restored, persist the backup info.
+ log.info("Finished restoring the backup")
+ backup_info_file = os.path.join(self._sandbox.var, BACKUP_INFO_FILENAME)
+ # Useful for the user to check the result of backup restore.
+ with open(backup_info_file, 'w') as f:
+ json.dump(backup_info.__dict__, f)
+ log.info("Persisted backup info '%s' to file %s" % (backup_info.__dict__, backup_info_file))
+ else:
+ # If no recovery necessary, initialize the data dirs.
+ log.info("No MySQL backup is restored. Initializing a new MySQL instance")
+ try:
+ task_control.initialize(env)
+ except subprocess.CalledProcessError as e:
+ raise self.Error("Unable to initialize MySQL state: %s" % e)
diff --git a/twitter/mysos/executor/task_control.py b/twitter/mysos/executor/task_control.py
new file mode 100644
index 0000000..73e42ef
--- /dev/null
+++ b/twitter/mysos/executor/task_control.py
@@ -0,0 +1,72 @@
+from abc import abstractmethod
+
+from twitter.common.lang import Interface
+
+
+class TaskControlProvider(Interface):
+ @abstractmethod
+ def from_task(self, task, sandbox):
+ pass
+
+
+class TaskControl(Interface):
+ """
+ This class encapsulates commands that control the MySQL task process.
+
+ Implementation NOTEs:
+ 1. These commands (methods) should run serially when accessed from multiple
+ threads.
+ 2. This class doesn't capture subprocess.CalledProcessErrors thrown when underlying MySQL
+ commands exit with a non-zero return code. The caller should handle it.
+ """
+
+ class Error(Exception):
+ pass
+
+ @abstractmethod
+ def initialize(self, env):
+ """
+ Initialize a new DB instance.
+
+ :param env: The 'env' necessary for 'subprocess' to run the command.
+ """
+ pass
+
+ @abstractmethod
+ def start(self, env=None):
+ """
+ Start the task in a subprocess.
+
+ :param env: The 'env' necessary for 'subprocess' to run the command.
+ :return: A subprocess.Popen object that represents the leader of the process group that executes
+ the task.
+ """
+ pass
+
+ @abstractmethod
+ def reparent(self, master_host, master_port, env=None):
+ """
+ Reparent the MySQL slave to the new master.
+
+ :param env: The 'env' necessary for 'subprocess' to run the command.
+ """
+ pass
+
+ @abstractmethod
+ def promote(self, env=None):
+ """
+ Promote a slave to mastership.
+
+ :param env: The 'env' necessary for 'subprocess' to run the command.
+ """
+ pass
+
+ @abstractmethod
+ def get_log_position(self, env=None):
+ """
+ Retrieve the log position from mysqld.
+
+ :param env: The 'env' necessary for 'subprocess' to run the command.
+ :return: The log position, None if it cannot be obtained.
+ """
+ pass
diff --git a/twitter/mysos/executor/task_runner.py b/twitter/mysos/executor/task_runner.py
new file mode 100644
index 0000000..d834e6c
--- /dev/null
+++ b/twitter/mysos/executor/task_runner.py
@@ -0,0 +1,20 @@
+from abc import abstractmethod
+
+from twitter.common.lang import Interface
+
+
+class TaskError(Exception):
+ pass
+
+
+class TaskRunner(Interface):
+ pass
+
+
+class TaskRunnerProvider(Interface):
+ @abstractmethod
+ def from_task(self, task, sandbox):
+ """
+ Factory method that creates a TaskRunner instance from 'task' (TaskInfo).
+ """
+ pass
diff --git a/twitter/mysos/executor/testing/__init__.py b/twitter/mysos/executor/testing/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/twitter/mysos/executor/testing/__init__.py
diff --git a/twitter/mysos/executor/testing/fake.py b/twitter/mysos/executor/testing/fake.py
new file mode 100644
index 0000000..863cbaf
--- /dev/null
+++ b/twitter/mysos/executor/testing/fake.py
@@ -0,0 +1,70 @@
+"""This module is for all fake implementations of things in Mysos executor for testing."""
+
+import os
+import random
+import subprocess
+import threading
+
+from twitter.mysos.common.decorators import synchronized
+from twitter.mysos.executor.task_control import TaskControl, TaskControlProvider
+
+
+class FakeTaskControlProvider(TaskControlProvider):
+ """
+ An implementation of TaskControlProvider for testing.
+ """
+
+ def from_task(self, task, sandbox):
+ return FakeTaskControl(position=random.randint(0, 100))
+
+
+class FakeTaskControl(TaskControl):
+ def __init__(
+ self,
+ mysqld="tail -f /dev/null",
+ initialize_cmd=":",
+ start_cmd=":",
+ reparent_cmd=":",
+ promote_cmd=":",
+ get_log_position_cmd=":",
+ position=1):
+ """
+ :param mysqld: The command that 'simulates' mysqld (and does nothing).
+ :param *_cmd: The commands that are executed for the respective FakeTaskControl operations.
+ :param position: The 'mysqld' log position to return as the result of get_log_position().
+ """
+ self._lock = threading.Lock()
+ self._mysqld = mysqld
+ self._initialize_cmd = initialize_cmd
+ self._start_cmd = start_cmd,
+ self._reparent_cmd = reparent_cmd,
+ self._promote_cmd = promote_cmd,
+ self._get_log_position_cmd = get_log_position_cmd
+ self._position = position
+ self._process = None
+
+ @synchronized
+ def initialize(self, env):
+ subprocess.check_call(self._initialize_cmd, shell=True)
+
+ @synchronized
+ def start(self, env=None):
+ if self._process:
+ return
+
+ self._process = subprocess.Popen(self._mysqld, shell=True, preexec_fn=os.setsid)
+ subprocess.check_call(self._start_cmd, shell=True)
+ return self._process
+
+ @synchronized
+ def reparent(self, master_host, master_port, env=None):
+ subprocess.check_call(self._reparent_cmd, shell=True)
+
+ @synchronized
+ def promote(self, env=None):
+ subprocess.check_call(self._promote_cmd, shell=True)
+
+ @synchronized
+ def get_log_position(self, env=None):
+ subprocess.check_call(self._get_log_position_cmd, shell=True)
+ return self._position
diff --git a/twitter/mysos/executor/testing/fake_mysos_executor.py b/twitter/mysos/executor/testing/fake_mysos_executor.py
new file mode 100644
index 0000000..53e1ed7
--- /dev/null
+++ b/twitter/mysos/executor/testing/fake_mysos_executor.py
@@ -0,0 +1,71 @@
+import json
+import os
+import posixpath
+import socket
+
+from twitter.common import app, log
+from twitter.common.log.options import LogOptions
+from twitter.common.zookeeper.serverset.endpoint import Endpoint, ServiceInstance
+from twitter.mysos.common import zookeeper
+from twitter.mysos.common.testing import Fake
+from twitter.mysos.executor.executor import MysosExecutor
+from twitter.mysos.executor.mysos_task_runner import MysosTaskRunner, TaskRunnerProvider
+from twitter.mysos.executor.noop_installer import NoopPackageInstaller
+from twitter.mysos.executor.sandbox import Sandbox
+
+from .fake import FakeTaskControlProvider
+
+import mesos.native
+from zake.fake_client import FakeClient
+
+
+SANDBOX_ROOT = os.path.join(os.path.realpath('.'), "sandbox")
+
+
+class FakeTaskRunnerProvider(TaskRunnerProvider):
+ """
+ Creates a MysosTaskRunner with FakeTaskControl for testing purposes.
+ NOTE: zake is used so a running ZK server is not required.
+ """
+
+ def __init__(self, task_control_provider):
+ self._task_control_provider = task_control_provider
+
+ def from_task(self, task, sandbox):
+ data = json.loads(task.data)
+ cluster_name, port, zk_url = data['cluster'], data['port'], data['zk_url']
+
+ _, servers, path = zookeeper.parse(zk_url)
+
+ zk_client = FakeClient()
+ zk_client.start()
+ self_instance = ServiceInstance(Endpoint(socket.gethostbyname(socket.gethostname()), port))
+ task_control = self._task_control_provider.from_task(task, sandbox)
+
+ return MysosTaskRunner(
+ self_instance,
+ zk_client,
+ posixpath.join(path, cluster_name),
+ NoopPackageInstaller(),
+ task_control,
+ Fake())
+
+
+# This is a testing executor. We log more verbosely.
+LogOptions.disable_disk_logging()
+LogOptions.set_stderr_log_level('google:DEBUG')
+
+
+def proxy_main():
+ def main(args, options):
+ log.info('Starting testing mysos executor')
+
+ executor = MysosExecutor(
+ FakeTaskRunnerProvider(FakeTaskControlProvider()), Sandbox(SANDBOX_ROOT))
+
+ driver = mesos.native.MesosExecutorDriver(executor)
+ driver.run()
+
+ log.info('Exiting executor main')
+
+ app.main()
diff --git a/twitter/mysos/executor/testing/vagrant_mysos_executor.py b/twitter/mysos/executor/testing/vagrant_mysos_executor.py
new file mode 100644
index 0000000..97f307b
--- /dev/null
+++ b/twitter/mysos/executor/testing/vagrant_mysos_executor.py
@@ -0,0 +1,58 @@
+"""This 'testing' executor is built to be run in the vagrant VM.
+
+It is basically the same as the normal Mysos executor except that it doesn't rely on HDFS.
+"""
+
+import os
+import stat
+
+from twitter.common import app, log
+from twitter.common.log.options import LogOptions
+from twitter.mysos.common.pkgutil import unpack_assets
+from twitter.mysos.executor.mysql_task_control import MySQLTaskControlProvider
+from twitter.mysos.executor.executor import MysosExecutor
+from twitter.mysos.executor.mysos_task_runner import MysosTaskRunnerProvider
+from twitter.mysos.executor.noop_installer import NoopPackageInstallerProvider
+from twitter.mysos.executor.sandbox import Sandbox
+from twitter.mysos.executor.backup import NoopBackupStoreProvider
+
+import mesos.native
+
+
+MYSOS_MODULE = 'twitter.mysos.executor'
+ASSET_RELPATH = 'files'
+
+
+def chmod_scripts(path):
+ """Make scripts executable."""
+ if path.endswith('.sh'):
+ st = os.stat(path)
+ os.chmod(path, st.st_mode | stat.S_IEXEC)
+
+
+LogOptions.disable_disk_logging()
+LogOptions.set_stderr_log_level('google:INFO')
+
+
+def proxy_main():
+ def main(args, options):
+ # 'sandbox' directory resides under the working directory assigned by the Mesos slave.
+ sandbox_root = os.path.join(os.path.realpath('.'), "sandbox")
+
+ unpack_assets(sandbox_root, MYSOS_MODULE, ASSET_RELPATH, execute=chmod_scripts)
+
+ log.info("Starting Vagrant Mysos executor within sandbox %s" % sandbox_root)
+
+ sandbox = Sandbox(sandbox_root)
+ executor = MysosExecutor(
+ MysosTaskRunnerProvider(
+ MySQLTaskControlProvider(),
+ NoopPackageInstallerProvider(), # Do not install any package.
+ NoopBackupStoreProvider()), # Do not recover any state.
+ sandbox)
+ driver = mesos.native.MesosExecutorDriver(executor)
+ driver.run()
+
+ log.info('Exiting executor main')
+
+ app.main()
\ No newline at end of file
diff --git a/twitter/mysos/scheduler/__init__.py b/twitter/mysos/scheduler/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/twitter/mysos/scheduler/__init__.py
diff --git a/twitter/mysos/scheduler/assets/static/bootstrap.min.css b/twitter/mysos/scheduler/assets/static/bootstrap.min.css
new file mode 100644
index 0000000..28f154d
--- /dev/null
+++ b/twitter/mysos/scheduler/assets/static/bootstrap.min.css
@@ -0,0 +1,5 @@
+/*!
+ * Bootstrap v3.3.2 (http://getbootstrap.com)
+ * Copyright 2011-2015 Twitter, Inc.
+ * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
+ *//*! normalize.css v3.0.2 | MIT License | git.io/normalize */html{font-family:sans-serif;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}body{margin:0}article,aside,details,figcaption,figure,footer,header,hgroup,main,menu,nav,section,summary{display:block}audio,canvas,progress,video{display:inline-block;vertical-align:baseline}audio:not([controls]){display:none;height:0}[hidden],template{display:none}a{background-color:transparent}a:active,a:hover{outline:0}abbr[title]{border-bottom:1px dotted}b,strong{font-weight:700}dfn{font-style:italic}h1{margin:.67em 0;font-size:2em}mark{color:#000;background:#ff0}small{font-size:80%}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sup{top:-.5em}sub{bottom:-.25em}img{border:0}svg:not(:root){overflow:hidden}figure{margin:1em 40px}hr{height:0;-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box}pre{overflow:auto}code,kbd,pre,samp{font-family:monospace,monospace;font-size:1em}button,input,optgroup,select,textarea{margin:0;font:inherit;color:inherit}button{overflow:visible}button,select{text-transform:none}button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}button[disabled],html input[disabled]{cursor:default}button::-moz-focus-inner,input::-moz-focus-inner{padding:0;border:0}input{line-height:normal}input[type=checkbox],input[type=radio]{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box;padding:0}input[type=number]::-webkit-inner-spin-button,input[type=number]::-webkit-outer-spin-button{height:auto}input[type=search]{-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;-webkit-appearance:textfield}input[type=search]::-webkit-search-cancel-button,input[type=search]::-webkit-search-decoration{-webkit-appearance:none}fieldset{padding:.35em .625em .75em;margin:0 2px;border:1px solid silver}legend{padding:0;border:0}textarea{overflow:auto}optgroup{font-weight:700}table{border-spacing:0;border-collapse:collapse}td,th{padding:0}/*! Source: https://github.com/h5bp/html5-boilerplate/blob/master/src/css/main.css */@media print{*,:after,:before{color:#000!important;text-shadow:none!important;background:0 0!important;-webkit-box-shadow:none!important;box-shadow:none!important}a,a:visited{text-decoration:underline}a[href]:after{content:" (" attr(href) ")"}abbr[title]:after{content:" (" attr(title) ")"}a[href^="javascript:"]:after,a[href^="#"]:after{content:""}blockquote,pre{border:1px solid #999;page-break-inside:avoid}thead{display:table-header-group}img,tr{page-break-inside:avoid}img{max-width:100%!important}h2,h3,p{orphans:3;widows:3}h2,h3{page-break-after:avoid}select{background:#fff!important}.navbar{display:none}.btn>.caret,.dropup>.btn>.caret{border-top-color:#000!important}.label{border:1px solid #000}.table{border-collapse:collapse!important}.table td,.table th{background-color:#fff!important}.table-bordered td,.table-bordered th{border:1px solid #ddd!important}}@font-face{font-family:'Glyphicons Halflings';src:url(../fonts/glyphicons-halflings-regular.eot);src:url(../fonts/glyphicons-halflings-regular.eot?#iefix) format('embedded-opentype'),url(../fonts/glyphicons-halflings-regular.woff2) format('woff2'),url(../fonts/glyphicons-halflings-regular.woff) format('woff'),url(../fonts/glyphicons-halflings-regular.ttf) format('truetype'),url(../fonts/glyphicons-halflings-regular.svg#glyphicons_halflingsregular) format('svg')}.glyphicon{position:relative;top:1px;display:inline-block;font-family:'Glyphicons Halflings';font-style:normal;font-weight:400;line-height:1;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}.glyphicon-asterisk:before{content:"\2a"}.glyphicon-plus:before{content:"\2b"}.glyphicon-eur:before,.glyphicon-euro:before{content:"\20ac"}.glyphicon-minus:before{content:"\2212"}.glyphicon-cloud:before{content:"\2601"}.glyphicon-envelope:before{content:"\2709"}.glyphicon-pencil:before{content:"\270f"}.glyphicon-glass:before{content:"\e001"}.glyphicon-music:before{content:"\e002"}.glyphicon-search:before{content:"\e003"}.glyphicon-heart:before{content:"\e005"}.glyphicon-star:before{content:"\e006"}.glyphicon-star-empty:before{content:"\e007"}.glyphicon-user:before{content:"\e008"}.glyphicon-film:before{content:"\e009"}.glyphicon-th-large:before{content:"\e010"}.glyphicon-th:before{content:"\e011"}.glyphicon-th-list:before{content:"\e012"}.glyphicon-ok:before{content:"\e013"}.glyphicon-remove:before{content:"\e014"}.glyphicon-zoom-in:before{content:"\e015"}.glyphicon-zoom-out:before{content:"\e016"}.glyphicon-off:before{content:"\e017"}.glyphicon-signal:before{content:"\e018"}.glyphicon-cog:before{content:"\e019"}.glyphicon-trash:before{content:"\e020"}.glyphicon-home:before{content:"\e021"}.glyphicon-file:before{content:"\e022"}.glyphicon-time:before{content:"\e023"}.glyphicon-road:before{content:"\e024"}.glyphicon-download-alt:before{content:"\e025"}.glyphicon-download:before{content:"\e026"}.glyphicon-upload:before{content:"\e027"}.glyphicon-inbox:before{content:"\e028"}.glyphicon-play-circle:before{content:"\e029"}.glyphicon-repeat:before{content:"\e030"}.glyphicon-refresh:before{content:"\e031"}.glyphicon-list-alt:before{content:"\e032"}.glyphicon-lock:before{content:"\e033"}.glyphicon-flag:before{content:"\e034"}.glyphicon-headphones:before{content:"\e035"}.glyphicon-volume-off:before{content:"\e036"}.glyphicon-volume-down:before{content:"\e037"}.glyphicon-volume-up:before{content:"\e038"}.glyphicon-qrcode:before{content:"\e039"}.glyphicon-barcode:before{content:"\e040"}.glyphicon-tag:before{content:"\e041"}.glyphicon-tags:before{content:"\e042"}.glyphicon-book:before{content:"\e043"}.glyphicon-bookmark:before{content:"\e044"}.glyphicon-print:before{content:"\e045"}.glyphicon-camera:before{content:"\e046"}.glyphicon-font:before{content:"\e047"}.glyphicon-bold:before{content:"\e048"}.glyphicon-italic:before{content:"\e049"}.glyphicon-text-height:before{content:"\e050"}.glyphicon-text-width:before{content:"\e051"}.glyphicon-align-left:before{content:"\e052"}.glyphicon-align-center:before{content:"\e053"}.glyphicon-align-right:before{content:"\e054"}.glyphicon-align-justify:before{content:"\e055"}.glyphicon-list:before{content:"\e056"}.glyphicon-indent-left:before{content:"\e057"}.glyphicon-indent-right:before{content:"\e058"}.glyphicon-facetime-video:before{content:"\e059"}.glyphicon-picture:before{content:"\e060"}.glyphicon-map-marker:before{content:"\e062"}.glyphicon-adjust:before{content:"\e063"}.glyphicon-tint:before{content:"\e064"}.glyphicon-edit:before{content:"\e065"}.glyphicon-share:before{content:"\e066"}.glyphicon-check:before{content:"\e067"}.glyphicon-move:before{content:"\e068"}.glyphicon-step-backward:before{content:"\e069"}.glyphicon-fast-backward:before{content:"\e070"}.glyphicon-backward:before{content:"\e071"}.glyphicon-play:before{content:"\e072"}.glyphicon-pause:before{content:"\e073"}.glyphicon-stop:before{content:"\e074"}.glyphicon-forward:before{content:"\e075"}.glyphicon-fast-forward:before{content:"\e076"}.glyphicon-step-forward:before{content:"\e077"}.glyphicon-eject:before{content:"\e078"}.glyphicon-chevron-left:before{content:"\e079"}.glyphicon-chevron-right:before{content:"\e080"}.glyphicon-plus-sign:before{content:"\e081"}.glyphicon-minus-sign:before{content:"\e082"}.glyphicon-remove-sign:before{content:"\e083"}.glyphicon-ok-sign:before{content:"\e084"}.glyphicon-question-sign:before{content:"\e085"}.glyphicon-info-sign:before{content:"\e086"}.glyphicon-screenshot:before{content:"\e087"}.glyphicon-remove-circle:before{content:"\e088"}.glyphicon-ok-circle:before{content:"\e089"}.glyphicon-ban-circle:before{content:"\e090"}.glyphicon-arrow-left:before{content:"\e091"}.glyphicon-arrow-right:before{content:"\e092"}.glyphicon-arrow-up:before{content:"\e093"}.glyphicon-arrow-down:before{content:"\e094"}.glyphicon-share-alt:before{content:"\e095"}.glyphicon-resize-full:before{content:"\e096"}.glyphicon-resize-small:before{content:"\e097"}.glyphicon-exclamation-sign:before{content:"\e101"}.glyphicon-gift:before{content:"\e102"}.glyphicon-leaf:before{content:"\e103"}.glyphicon-fire:before{content:"\e104"}.glyphicon-eye-open:before{content:"\e105"}.glyphicon-eye-close:before{content:"\e106"}.glyphicon-warning-sign:before{content:"\e107"}.glyphicon-plane:before{content:"\e108"}.glyphicon-calendar:before{content:"\e109"}.glyphicon-random:before{content:"\e110"}.glyphicon-comment:before{content:"\e111"}.glyphicon-magnet:before{content:"\e112"}.glyphicon-chevron-up:before{content:"\e113"}.glyphicon-chevron-down:before{content:"\e114"}.glyphicon-retweet:before{content:"\e115"}.glyphicon-shopping-cart:before{content:"\e116"}.glyphicon-folder-close:before{content:"\e117"}.glyphicon-folder-open:before{content:"\e118"}.glyphicon-resize-vertical:before{content:"\e119"}.glyphicon-resize-horizontal:before{content:"\e120"}.glyphicon-hdd:before{content:"\e121"}.glyphicon-bullhorn:before{content:"\e122"}.glyphicon-bell:before{content:"\e123"}.glyphicon-certificate:before{content:"\e124"}.glyphicon-thumbs-up:before{content:"\e125"}.glyphicon-thumbs-down:before{content:"\e126"}.glyphicon-hand-right:before{content:"\e127"}.glyphicon-hand-left:before{content:"\e128"}.glyphicon-hand-up:before{content:"\e129"}.glyphicon-hand-down:before{content:"\e130"}.glyphicon-circle-arrow-right:before{content:"\e131"}.glyphicon-circle-arrow-left:before{content:"\e132"}.glyphicon-circle-arrow-up:before{content:"\e133"}.glyphicon-circle-arrow-down:before{content:"\e134"}.glyphicon-globe:before{content:"\e135"}.glyphicon-wrench:before{content:"\e136"}.glyphicon-tasks:before{content:"\e137"}.glyphicon-filter:before{content:"\e138"}.glyphicon-briefcase:before{content:"\e139"}.glyphicon-fullscreen:before{content:"\e140"}.glyphicon-dashboard:before{content:"\e141"}.glyphicon-paperclip:before{content:"\e142"}.glyphicon-heart-empty:before{content:"\e143"}.glyphicon-link:before{content:"\e144"}.glyphicon-phone:before{content:"\e145"}.glyphicon-pushpin:before{content:"\e146"}.glyphicon-usd:before{content:"\e148"}.glyphicon-gbp:before{content:"\e149"}.glyphicon-sort:before{content:"\e150"}.glyphicon-sort-by-alphabet:before{content:"\e151"}.glyphicon-sort-by-alphabet-alt:before{content:"\e152"}.glyphicon-sort-by-order:before{content:"\e153"}.glyphicon-sort-by-order-alt:before{content:"\e154"}.glyphicon-sort-by-attributes:before{content:"\e155"}.glyphicon-sort-by-attributes-alt:before{content:"\e156"}.glyphicon-unchecked:before{content:"\e157"}.glyphicon-expand:before{content:"\e158"}.glyphicon-collapse-down:before{content:"\e159"}.glyphicon-collapse-up:before{content:"\e160"}.glyphicon-log-in:before{content:"\e161"}.glyphicon-flash:before{content:"\e162"}.glyphicon-log-out:before{content:"\e163"}.glyphicon-new-window:before{content:"\e164"}.glyphicon-record:before{content:"\e165"}.glyphicon-save:before{content:"\e166"}.glyphicon-open:before{content:"\e167"}.glyphicon-saved:before{content:"\e168"}.glyphicon-import:before{content:"\e169"}.glyphicon-export:before{content:"\e170"}.glyphicon-send:before{content:"\e171"}.glyphicon-floppy-disk:before{content:"\e172"}.glyphicon-floppy-saved:before{content:"\e173"}.glyphicon-floppy-remove:before{content:"\e174"}.glyphicon-floppy-save:before{content:"\e175"}.glyphicon-floppy-open:before{content:"\e176"}.glyphicon-credit-card:before{content:"\e177"}.glyphicon-transfer:before{content:"\e178"}.glyphicon-cutlery:before{content:"\e179"}.glyphicon-header:before{content:"\e180"}.glyphicon-compressed:before{content:"\e181"}.glyphicon-earphone:before{content:"\e182"}.glyphicon-phone-alt:before{content:"\e183"}.glyphicon-tower:before{content:"\e184"}.glyphicon-stats:before{content:"\e185"}.glyphicon-sd-video:before{content:"\e186"}.glyphicon-hd-video:before{content:"\e187"}.glyphicon-subtitles:before{content:"\e188"}.glyphicon-sound-stereo:before{content:"\e189"}.glyphicon-sound-dolby:before{content:"\e190"}.glyphicon-sound-5-1:before{content:"\e191"}.glyphicon-sound-6-1:before{content:"\e192"}.glyphicon-sound-7-1:before{content:"\e193"}.glyphicon-copyright-mark:before{content:"\e194"}.glyphicon-registration-mark:before{content:"\e195"}.glyphicon-cloud-download:before{content:"\e197"}.glyphicon-cloud-upload:before{content:"\e198"}.glyphicon-tree-conifer:before{content:"\e199"}.glyphicon-tree-deciduous:before{content:"\e200"}.glyphicon-cd:before{content:"\e201"}.glyphicon-save-file:before{content:"\e202"}.glyphicon-open-file:before{content:"\e203"}.glyphicon-level-up:before{content:"\e204"}.glyphicon-copy:before{content:"\e205"}.glyphicon-paste:before{content:"\e206"}.glyphicon-alert:before{content:"\e209"}.glyphicon-equalizer:before{content:"\e210"}.glyphicon-king:before{content:"\e211"}.glyphicon-queen:before{content:"\e212"}.glyphicon-pawn:before{content:"\e213"}.glyphicon-bishop:before{content:"\e214"}.glyphicon-knight:before{content:"\e215"}.glyphicon-baby-formula:before{content:"\e216"}.glyphicon-tent:before{content:"\26fa"}.glyphicon-blackboard:before{content:"\e218"}.glyphicon-bed:before{content:"\e219"}.glyphicon-apple:before{content:"\f8ff"}.glyphicon-erase:before{content:"\e221"}.glyphicon-hourglass:before{content:"\231b"}.glyphicon-lamp:before{content:"\e223"}.glyphicon-duplicate:before{content:"\e224"}.glyphicon-piggy-bank:before{content:"\e225"}.glyphicon-scissors:before{content:"\e226"}.glyphicon-bitcoin:before{content:"\e227"}.glyphicon-yen:before{content:"\00a5"}.glyphicon-ruble:before{content:"\20bd"}.glyphicon-scale:before{content:"\e230"}.glyphicon-ice-lolly:before{content:"\e231"}.glyphicon-ice-lolly-tasted:before{content:"\e232"}.glyphicon-education:before{content:"\e233"}.glyphicon-option-horizontal:before{content:"\e234"}.glyphicon-option-vertical:before{content:"\e235"}.glyphicon-menu-hamburger:before{content:"\e236"}.glyphicon-modal-window:before{content:"\e237"}.glyphicon-oil:before{content:"\e238"}.glyphicon-grain:before{content:"\e239"}.glyphicon-sunglasses:before{content:"\e240"}.glyphicon-text-size:before{content:"\e241"}.glyphicon-text-color:before{content:"\e242"}.glyphicon-text-background:before{content:"\e243"}.glyphicon-object-align-top:before{content:"\e244"}.glyphicon-object-align-bottom:before{content:"\e245"}.glyphicon-object-align-horizontal:before{content:"\e246"}.glyphicon-object-align-left:before{content:"\e247"}.glyphicon-object-align-vertical:before{content:"\e248"}.glyphicon-object-align-right:before{content:"\e249"}.glyphicon-triangle-right:before{content:"\e250"}.glyphicon-triangle-left:before{content:"\e251"}.glyphicon-triangle-bottom:before{content:"\e252"}.glyphicon-triangle-top:before{content:"\e253"}.glyphicon-console:before{content:"\e254"}.glyphicon-superscript:before{content:"\e255"}.glyphicon-subscript:before{content:"\e256"}.glyphicon-menu-left:before{content:"\e257"}.glyphicon-menu-right:before{content:"\e258"}.glyphicon-menu-down:before{content:"\e259"}.glyphicon-menu-up:before{content:"\e260"}*{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}:after,:before{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}html{font-size:10px;-webkit-tap-highlight-color:rgba(0,0,0,0)}body{font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:14px;line-height:1.42857143;color:#333;background-color:#fff}button,input,select,textarea{font-family:inherit;font-size:inherit;line-height:inherit}a{color:#337ab7;text-decoration:none}a:focus,a:hover{color:#23527c;text-decoration:underline}a:focus{outline:thin dotted;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}figure{margin:0}img{vertical-align:middle}.carousel-inner>.item>a>img,.carousel-inner>.item>img,.img-responsive,.thumbnail a>img,.thumbnail>img{display:block;max-width:100%;height:auto}.img-rounded{border-radius:6px}.img-thumbnail{display:inline-block;max-width:100%;height:auto;padding:4px;line-height:1.42857143;background-color:#fff;border:1px solid #ddd;border-radius:4px;-webkit-transition:all .2s ease-in-out;-o-transition:all .2s ease-in-out;transition:all .2s ease-in-out}.img-circle{border-radius:50%}hr{margin-top:20px;margin-bottom:20px;border:0;border-top:1px solid #eee}.sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);border:0}.sr-only-focusable:active,.sr-only-focusable:focus{position:static;width:auto;height:auto;margin:0;overflow:visible;clip:auto}.h1,.h2,.h3,.h4,.h5,.h6,h1,h2,h3,h4,h5,h6{font-family:inherit;font-weight:500;line-height:1.1;color:inherit}.h1 .small,.h1 small,.h2 .small,.h2 small,.h3 .small,.h3 small,.h4 .small,.h4 small,.h5 .small,.h5 small,.h6 .small,.h6 small,h1 .small,h1 small,h2 .small,h2 small,h3 .small,h3 small,h4 .small,h4 small,h5 .small,h5 small,h6 .small,h6 small{font-weight:400;line-height:1;color:#777}.h1,.h2,.h3,h1,h2,h3{margin-top:20px;margin-bottom:10px}.h1 .small,.h1 small,.h2 .small,.h2 small,.h3 .small,.h3 small,h1 .small,h1 small,h2 .small,h2 small,h3 .small,h3 small{font-size:65%}.h4,.h5,.h6,h4,h5,h6{margin-top:10px;margin-bottom:10px}.h4 .small,.h4 small,.h5 .small,.h5 small,.h6 .small,.h6 small,h4 .small,h4 small,h5 .small,h5 small,h6 .small,h6 small{font-size:75%}.h1,h1{font-size:36px}.h2,h2{font-size:30px}.h3,h3{font-size:24px}.h4,h4{font-size:18px}.h5,h5{font-size:14px}.h6,h6{font-size:12px}p{margin:0 0 10px}.lead{margin-bottom:20px;font-size:16px;font-weight:300;line-height:1.4}@media (min-width:768px){.lead{font-size:21px}}.small,small{font-size:85%}.mark,mark{padding:.2em;background-color:#fcf8e3}.text-left{text-align:left}.text-right{text-align:right}.text-center{text-align:center}.text-justify{text-align:justify}.text-nowrap{white-space:nowrap}.text-lowercase{text-transform:lowercase}.text-uppercase{text-transform:uppercase}.text-capitalize{text-transform:capitalize}.text-muted{color:#777}.text-primary{color:#337ab7}a.text-primary:hover{color:#286090}.text-success{color:#3c763d}a.text-success:hover{color:#2b542c}.text-info{color:#31708f}a.text-info:hover{color:#245269}.text-warning{color:#8a6d3b}a.text-warning:hover{color:#66512c}.text-danger{color:#a94442}a.text-danger:hover{color:#843534}.bg-primary{color:#fff;background-color:#337ab7}a.bg-primary:hover{background-color:#286090}.bg-success{background-color:#dff0d8}a.bg-success:hover{background-color:#c1e2b3}.bg-info{background-color:#d9edf7}a.bg-info:hover{background-color:#afd9ee}.bg-warning{background-color:#fcf8e3}a.bg-warning:hover{background-color:#f7ecb5}.bg-danger{background-color:#f2dede}a.bg-danger:hover{background-color:#e4b9b9}.page-header{padding-bottom:9px;margin:40px 0 20px;border-bottom:1px solid #eee}ol,ul{margin-top:0;margin-bottom:10px}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}.list-unstyled{padding-left:0;list-style:none}.list-inline{padding-left:0;margin-left:-5px;list-style:none}.list-inline>li{display:inline-block;padding-right:5px;padding-left:5px}dl{margin-top:0;margin-bottom:20px}dd,dt{line-height:1.42857143}dt{font-weight:700}dd{margin-left:0}@media (min-width:768px){.dl-horizontal dt{float:left;width:160px;overflow:hidden;clear:left;text-align:right;text-overflow:ellipsis;white-space:nowrap}.dl-horizontal dd{margin-left:180px}}abbr[data-original-title],abbr[title]{cursor:help;border-bottom:1px dotted #777}.initialism{font-size:90%;text-transform:uppercase}blockquote{padding:10px 20px;margin:0 0 20px;font-size:17.5px;border-left:5px solid #eee}blockquote ol:last-child,blockquote p:last-child,blockquote ul:last-child{margin-bottom:0}blockquote .small,blockquote footer,blockquote small{display:block;font-size:80%;line-height:1.42857143;color:#777}blockquote .small:before,blockquote footer:before,blockquote small:before{content:'\2014 \00A0'}.blockquote-reverse,blockquote.pull-right{padding-right:15px;padding-left:0;text-align:right;border-right:5px solid #eee;border-left:0}.blockquote-reverse .small:before,.blockquote-reverse footer:before,.blockquote-reverse small:before,blockquote.pull-right .small:before,blockquote.pull-right footer:before,blockquote.pull-right small:before{content:''}.blockquote-reverse .small:after,.blockquote-reverse footer:after,.blockquote-reverse small:after,blockquote.pull-right .small:after,blockquote.pull-right footer:after,blockquote.pull-right small:after{content:'\00A0 \2014'}address{margin-bottom:20px;font-style:normal;line-height:1.42857143}code,kbd,pre,samp{font-family:Menlo,Monaco,Consolas,"Courier New",monospace}code{padding:2px 4px;font-size:90%;color:#c7254e;background-color:#f9f2f4;border-radius:4px}kbd{padding:2px 4px;font-size:90%;color:#fff;background-color:#333;border-radius:3px;-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,.25);box-shadow:inset 0 -1px 0 rgba(0,0,0,.25)}kbd kbd{padding:0;font-size:100%;font-weight:700;-webkit-box-shadow:none;box-shadow:none}pre{display:block;padding:9.5px;margin:0 0 10px;font-size:13px;line-height:1.42857143;color:#333;word-break:break-all;word-wrap:break-word;background-color:#f5f5f5;border:1px solid #ccc;border-radius:4px}pre code{padding:0;font-size:inherit;color:inherit;white-space:pre-wrap;background-color:transparent;border-radius:0}.pre-scrollable{max-height:340px;overflow-y:scroll}.container{padding-right:15px;padding-left:15px;margin-right:auto;margin-left:auto}@media (min-width:768px){.container{width:750px}}@media (min-width:992px){.container{width:970px}}@media (min-width:1200px){.container{width:1170px}}.container-fluid{padding-right:15px;padding-left:15px;margin-right:auto;margin-left:auto}.row{margin-right:-15px;margin-left:-15px}.col-lg-1,.col-lg-10,.col-lg-11,.col-lg-12,.col-lg-2,.col-lg-3,.col-lg-4,.col-lg-5,.col-lg-6,.col-lg-7,.col-lg-8,.col-lg-9,.col-md-1,.col-md-10,.col-md-11,.col-md-12,.col-md-2,.col-md-3,.col-md-4,.col-md-5,.col-md-6,.col-md-7,.col-md-8,.col-md-9,.col-sm-1,.col-sm-10,.col-sm-11,.col-sm-12,.col-sm-2,.col-sm-3,.col-sm-4,.col-sm-5,.col-sm-6,.col-sm-7,.col-sm-8,.col-sm-9,.col-xs-1,.col-xs-10,.col-xs-11,.col-xs-12,.col-xs-2,.col-xs-3,.col-xs-4,.col-xs-5,.col-xs-6,.col-xs-7,.col-xs-8,.col-xs-9{position:relative;min-height:1px;padding-right:15px;padding-left:15px}.col-xs-1,.col-xs-10,.col-xs-11,.col-xs-12,.col-xs-2,.col-xs-3,.col-xs-4,.col-xs-5,.col-xs-6,.col-xs-7,.col-xs-8,.col-xs-9{float:left}.col-xs-12{width:100%}.col-xs-11{width:91.66666667%}.col-xs-10{width:83.33333333%}.col-xs-9{width:75%}.col-xs-8{width:66.66666667%}.col-xs-7{width:58.33333333%}.col-xs-6{width:50%}.col-xs-5{width:41.66666667%}.col-xs-4{width:33.33333333%}.col-xs-3{width:25%}.col-xs-2{width:16.66666667%}.col-xs-1{width:8.33333333%}.col-xs-pull-12{right:100%}.col-xs-pull-11{right:91.66666667%}.col-xs-pull-10{right:83.33333333%}.col-xs-pull-9{right:75%}.col-xs-pull-8{right:66.66666667%}.col-xs-pull-7{right:58.33333333%}.col-xs-pull-6{right:50%}.col-xs-pull-5{right:41.66666667%}.col-xs-pull-4{right:33.33333333%}.col-xs-pull-3{right:25%}.col-xs-pull-2{right:16.66666667%}.col-xs-pull-1{right:8.33333333%}.col-xs-pull-0{right:auto}.col-xs-push-12{left:100%}.col-xs-push-11{left:91.66666667%}.col-xs-push-10{left:83.33333333%}.col-xs-push-9{left:75%}.col-xs-push-8{left:66.66666667%}.col-xs-push-7{left:58.33333333%}.col-xs-push-6{left:50%}.col-xs-push-5{left:41.66666667%}.col-xs-push-4{left:33.33333333%}.col-xs-push-3{left:25%}.col-xs-push-2{left:16.66666667%}.col-xs-push-1{left:8.33333333%}.col-xs-push-0{left:auto}.col-xs-offset-12{margin-left:100%}.col-xs-offset-11{margin-left:91.66666667%}.col-xs-offset-10{margin-left:83.33333333%}.col-xs-offset-9{margin-left:75%}.col-xs-offset-8{margin-left:66.66666667%}.col-xs-offset-7{margin-left:58.33333333%}.col-xs-offset-6{margin-left:50%}.col-xs-offset-5{margin-left:41.66666667%}.col-xs-offset-4{margin-left:33.33333333%}.col-xs-offset-3{margin-left:25%}.col-xs-offset-2{margin-left:16.66666667%}.col-xs-offset-1{margin-left:8.33333333%}.col-xs-offset-0{margin-left:0}@media (min-width:768px){.col-sm-1,.col-sm-10,.col-sm-11,.col-sm-12,.col-sm-2,.col-sm-3,.col-sm-4,.col-sm-5,.col-sm-6,.col-sm-7,.col-sm-8,.col-sm-9{float:left}.col-sm-12{width:100%}.col-sm-11{width:91.66666667%}.col-sm-10{width:83.33333333%}.col-sm-9{width:75%}.col-sm-8{width:66.66666667%}.col-sm-7{width:58.33333333%}.col-sm-6{width:50%}.col-sm-5{width:41.66666667%}.col-sm-4{width:33.33333333%}.col-sm-3{width:25%}.col-sm-2{width:16.66666667%}.col-sm-1{width:8.33333333%}.col-sm-pull-12{right:100%}.col-sm-pull-11{right:91.66666667%}.col-sm-pull-10{right:83.33333333%}.col-sm-pull-9{right:75%}.col-sm-pull-8{right:66.66666667%}.col-sm-pull-7{right:58.33333333%}.col-sm-pull-6{right:50%}.col-sm-pull-5{right:41.66666667%}.col-sm-pull-4{right:33.33333333%}.col-sm-pull-3{right:25%}.col-sm-pull-2{right:16.66666667%}.col-sm-pull-1{right:8.33333333%}.col-sm-pull-0{right:auto}.col-sm-push-12{left:100%}.col-sm-push-11{left:91.66666667%}.col-sm-push-10{left:83.33333333%}.col-sm-push-9{left:75%}.col-sm-push-8{left:66.66666667%}.col-sm-push-7{left:58.33333333%}.col-sm-push-6{left:50%}.col-sm-push-5{left:41.66666667%}.col-sm-push-4{left:33.33333333%}.col-sm-push-3{left:25%}.col-sm-push-2{left:16.66666667%}.col-sm-push-1{left:8.33333333%}.col-sm-push-0{left:auto}.col-sm-offset-12{margin-left:100%}.col-sm-offset-11{margin-left:91.66666667%}.col-sm-offset-10{margin-left:83.33333333%}.col-sm-offset-9{margin-left:75%}.col-sm-offset-8{margin-left:66.66666667%}.col-sm-offset-7{margin-left:58.33333333%}.col-sm-offset-6{margin-left:50%}.col-sm-offset-5{margin-left:41.66666667%}.col-sm-offset-4{margin-left:33.33333333%}.col-sm-offset-3{margin-left:25%}.col-sm-offset-2{margin-left:16.66666667%}.col-sm-offset-1{margin-left:8.33333333%}.col-sm-offset-0{margin-left:0}}@media (min-width:992px){.col-md-1,.col-md-10,.col-md-11,.col-md-12,.col-md-2,.col-md-3,.col-md-4,.col-md-5,.col-md-6,.col-md-7,.col-md-8,.col-md-9{float:left}.col-md-12{width:100%}.col-md-11{width:91.66666667%}.col-md-10{width:83.33333333%}.col-md-9{width:75%}.col-md-8{width:66.66666667%}.col-md-7{width:58.33333333%}.col-md-6{width:50%}.col-md-5{width:41.66666667%}.col-md-4{width:33.33333333%}.col-md-3{width:25%}.col-md-2{width:16.66666667%}.col-md-1{width:8.33333333%}.col-md-pull-12{right:100%}.col-md-pull-11{right:91.66666667%}.col-md-pull-10{right:83.33333333%}.col-md-pull-9{right:75%}.col-md-pull-8{right:66.66666667%}.col-md-pull-7{right:58.33333333%}.col-md-pull-6{right:50%}.col-md-pull-5{right:41.66666667%}.col-md-pull-4{right:33.33333333%}.col-md-pull-3{right:25%}.col-md-pull-2{right:16.66666667%}.col-md-pull-1{right:8.33333333%}.col-md-pull-0{right:auto}.col-md-push-12{left:100%}.col-md-push-11{left:91.66666667%}.col-md-push-10{left:83.33333333%}.col-md-push-9{left:75%}.col-md-push-8{left:66.66666667%}.col-md-push-7{left:58.33333333%}.col-md-push-6{left:50%}.col-md-push-5{left:41.66666667%}.col-md-push-4{left:33.33333333%}.col-md-push-3{left:25%}.col-md-push-2{left:16.66666667%}.col-md-push-1{left:8.33333333%}.col-md-push-0{left:auto}.col-md-offset-12{margin-left:100%}.col-md-offset-11{margin-left:91.66666667%}.col-md-offset-10{margin-left:83.33333333%}.col-md-offset-9{margin-left:75%}.col-md-offset-8{margin-left:66.66666667%}.col-md-offset-7{margin-left:58.33333333%}.col-md-offset-6{margin-left:50%}.col-md-offset-5{margin-left:41.66666667%}.col-md-offset-4{margin-left:33.33333333%}.col-md-offset-3{margin-left:25%}.col-md-offset-2{margin-left:16.66666667%}.col-md-offset-1{margin-left:8.33333333%}.col-md-offset-0{margin-left:0}}@media (min-width:1200px){.col-lg-1,.col-lg-10,.col-lg-11,.col-lg-12,.col-lg-2,.col-lg-3,.col-lg-4,.col-lg-5,.col-lg-6,.col-lg-7,.col-lg-8,.col-lg-9{float:left}.col-lg-12{width:100%}.col-lg-11{width:91.66666667%}.col-lg-10{width:83.33333333%}.col-lg-9{width:75%}.col-lg-8{width:66.66666667%}.col-lg-7{width:58.33333333%}.col-lg-6{width:50%}.col-lg-5{width:41.66666667%}.col-lg-4{width:33.33333333%}.col-lg-3{width:25%}.col-lg-2{width:16.66666667%}.col-lg-1{width:8.33333333%}.col-lg-pull-12{right:100%}.col-lg-pull-11{right:91.66666667%}.col-lg-pull-10{right:83.33333333%}.col-lg-pull-9{right:75%}.col-lg-pull-8{right:66.66666667%}.col-lg-pull-7{right:58.33333333%}.col-lg-pull-6{right:50%}.col-lg-pull-5{right:41.66666667%}.col-lg-pull-4{right:33.33333333%}.col-lg-pull-3{right:25%}.col-lg-pull-2{right:16.66666667%}.col-lg-pull-1{right:8.33333333%}.col-lg-pull-0{right:auto}.col-lg-push-12{left:100%}.col-lg-push-11{left:91.66666667%}.col-lg-push-10{left:83.33333333%}.col-lg-push-9{left:75%}.col-lg-push-8{left:66.66666667%}.col-lg-push-7{left:58.33333333%}.col-lg-push-6{left:50%}.col-lg-push-5{left:41.66666667%}.col-lg-push-4{left:33.33333333%}.col-lg-push-3{left:25%}.col-lg-push-2{left:16.66666667%}.col-lg-push-1{left:8.33333333%}.col-lg-push-0{left:auto}.col-lg-offset-12{margin-left:100%}.col-lg-offset-11{margin-left:91.66666667%}.col-lg-offset-10{margin-left:83.33333333%}.col-lg-offset-9{margin-left:75%}.col-lg-offset-8{margin-left:66.66666667%}.col-lg-offset-7{margin-left:58.33333333%}.col-lg-offset-6{margin-left:50%}.col-lg-offset-5{margin-left:41.66666667%}.col-lg-offset-4{margin-left:33.33333333%}.col-lg-offset-3{margin-left:25%}.col-lg-offset-2{margin-left:16.66666667%}.col-lg-offset-1{margin-left:8.33333333%}.col-lg-offset-0{margin-left:0}}table{background-color:transparent}caption{padding-top:8px;padding-bottom:8px;color:#777;text-align:left}th{text-align:left}.table{width:100%;max-width:100%;margin-bottom:20px}.table>tbody>tr>td,.table>tbody>tr>th,.table>tfoot>tr>td,.table>tfoot>tr>th,.table>thead>tr>td,.table>thead>tr>th{padding:8px;line-height:1.42857143;vertical-align:top;border-top:1px solid #ddd}.table>thead>tr>th{vertical-align:bottom;border-bottom:2px solid #ddd}.table>caption+thead>tr:first-child>td,.table>caption+thead>tr:first-child>th,.table>colgroup+thead>tr:first-child>td,.table>colgroup+thead>tr:first-child>th,.table>thead:first-child>tr:first-child>td,.table>thead:first-child>tr:first-child>th{border-top:0}.table>tbody+tbody{border-top:2px solid #ddd}.table .table{background-color:#fff}.table-condensed>tbody>tr>td,.table-condensed>tbody>tr>th,.table-condensed>tfoot>tr>td,.table-condensed>tfoot>tr>th,.table-condensed>thead>tr>td,.table-condensed>thead>tr>th{padding:5px}.table-bordered{border:1px solid #ddd}.table-bordered>tbody>tr>td,.table-bordered>tbody>tr>th,.table-bordered>tfoot>tr>td,.table-bordered>tfoot>tr>th,.table-bordered>thead>tr>td,.table-bordered>thead>tr>th{border:1px solid #ddd}.table-bordered>thead>tr>td,.table-bordered>thead>tr>th{border-bottom-width:2px}.table-striped>tbody>tr:nth-of-type(odd){background-color:#f9f9f9}.table-hover>tbody>tr:hover{background-color:#f5f5f5}table col[class*=col-]{position:static;display:table-column;float:none}table td[class*=col-],table th[class*=col-]{position:static;display:table-cell;float:none}.table>tbody>tr.active>td,.table>tbody>tr.active>th,.table>tbody>tr>td.active,.table>tbody>tr>th.active,.table>tfoot>tr.active>td,.table>tfoot>tr.active>th,.table>tfoot>tr>td.active,.table>tfoot>tr>th.active,.table>thead>tr.active>td,.table>thead>tr.active>th,.table>thead>tr>td.active,.table>thead>tr>th.active{background-color:#f5f5f5}.table-hover>tbody>tr.active:hover>td,.table-hover>tbody>tr.active:hover>th,.table-hover>tbody>tr:hover>.active,.table-hover>tbody>tr>td.active:hover,.table-hover>tbody>tr>th.active:hover{background-color:#e8e8e8}.table>tbody>tr.success>td,.table>tbody>tr.success>th,.table>tbody>tr>td.success,.table>tbody>tr>th.success,.table>tfoot>tr.success>td,.table>tfoot>tr.success>th,.table>tfoot>tr>td.success,.table>tfoot>tr>th.success,.table>thead>tr.success>td,.table>thead>tr.success>th,.table>thead>tr>td.success,.table>thead>tr>th.success{background-color:#dff0d8}.table-hover>tbody>tr.success:hover>td,.table-hover>tbody>tr.success:hover>th,.table-hover>tbody>tr:hover>.success,.table-hover>tbody>tr>td.success:hover,.table-hover>tbody>tr>th.success:hover{background-color:#d0e9c6}.table>tbody>tr.info>td,.table>tbody>tr.info>th,.table>tbody>tr>td.info,.table>tbody>tr>th.info,.table>tfoot>tr.info>td,.table>tfoot>tr.info>th,.table>tfoot>tr>td.info,.table>tfoot>tr>th.info,.table>thead>tr.info>td,.table>thead>tr.info>th,.table>thead>tr>td.info,.table>thead>tr>th.info{background-color:#d9edf7}.table-hover>tbody>tr.info:hover>td,.table-hover>tbody>tr.info:hover>th,.table-hover>tbody>tr:hover>.info,.table-hover>tbody>tr>td.info:hover,.table-hover>tbody>tr>th.info:hover{background-color:#c4e3f3}.table>tbody>tr.warning>td,.table>tbody>tr.warning>th,.table>tbody>tr>td.warning,.table>tbody>tr>th.warning,.table>tfoot>tr.warning>td,.table>tfoot>tr.warning>th,.table>tfoot>tr>td.warning,.table>tfoot>tr>th.warning,.table>thead>tr.warning>td,.table>thead>tr.warning>th,.table>thead>tr>td.warning,.table>thead>tr>th.warning{background-color:#fcf8e3}.table-hover>tbody>tr.warning:hover>td,.table-hover>tbody>tr.warning:hover>th,.table-hover>tbody>tr:hover>.warning,.table-hover>tbody>tr>td.warning:hover,.table-hover>tbody>tr>th.warning:hover{background-color:#faf2cc}.table>tbody>tr.danger>td,.table>tbody>tr.danger>th,.table>tbody>tr>td.danger,.table>tbody>tr>th.danger,.table>tfoot>tr.danger>td,.table>tfoot>tr.danger>th,.table>tfoot>tr>td.danger,.table>tfoot>tr>th.danger,.table>thead>tr.danger>td,.table>thead>tr.danger>th,.table>thead>tr>td.danger,.table>thead>tr>th.danger{background-color:#f2dede}.table-hover>tbody>tr.danger:hover>td,.table-hover>tbody>tr.danger:hover>th,.table-hover>tbody>tr:hover>.danger,.table-hover>tbody>tr>td.danger:hover,.table-hover>tbody>tr>th.danger:hover{background-color:#ebcccc}.table-responsive{min-height:.01%;overflow-x:auto}@media screen and (max-width:767px){.table-responsive{width:100%;margin-bottom:15px;overflow-y:hidden;-ms-overflow-style:-ms-autohiding-scrollbar;border:1px solid #ddd}.table-responsive>.table{margin-bottom:0}.table-responsive>.table>tbody>tr>td,.table-responsive>.table>tbody>tr>th,.table-responsive>.table>tfoot>tr>td,.table-responsive>.table>tfoot>tr>th,.table-responsive>.table>thead>tr>td,.table-responsive>.table>thead>tr>th{white-space:nowrap}.table-responsive>.table-bordered{border:0}.table-responsive>.table-bordered>tbody>tr>td:first-child,.table-responsive>.table-bordered>tbody>tr>th:first-child,.table-responsive>.table-bordered>tfoot>tr>td:first-child,.table-responsive>.table-bordered>tfoot>tr>th:first-child,.table-responsive>.table-bordered>thead>tr>td:first-child,.table-responsive>.table-bordered>thead>tr>th:first-child{border-left:0}.table-responsive>.table-bordered>tbody>tr>td:last-child,.table-responsive>.table-bordered>tbody>tr>th:last-child,.table-responsive>.table-bordered>tfoot>tr>td:last-child,.table-responsive>.table-bordered>tfoot>tr>th:last-child,.table-responsive>.table-bordered>thead>tr>td:last-child,.table-responsive>.table-bordered>thead>tr>th:last-child{border-right:0}.table-responsive>.table-bordered>tbody>tr:last-child>td,.table-responsive>.table-bordered>tbody>tr:last-child>th,.table-responsive>.table-bordered>tfoot>tr:last-child>td,.table-responsive>.table-bordered>tfoot>tr:last-child>th{border-bottom:0}}fieldset{min-width:0;padding:0;margin:0;border:0}legend{display:block;width:100%;padding:0;margin-bottom:20px;font-size:21px;line-height:inherit;color:#333;border:0;border-bottom:1px solid #e5e5e5}label{display:inline-block;max-width:100%;margin-bottom:5px;font-weight:700}input[type=search]{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}input[type=checkbox],input[type=radio]{margin:4px 0 0;margin-top:1px \9;line-height:normal}input[type=file]{display:block}input[type=range]{display:block;width:100%}select[multiple],select[size]{height:auto}input[type=file]:focus,input[type=checkbox]:focus,input[type=radio]:focus{outline:thin dotted;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}output{display:block;padding-top:7px;font-size:14px;line-height:1.42857143;color:#555}.form-control{display:block;width:100%;height:34px;padding:6px 12px;font-size:14px;line-height:1.42857143;color:#555;background-color:#fff;background-image:none;border:1px solid #ccc;border-radius:4px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075);-webkit-transition:border-color ease-in-out .15s,-webkit-box-shadow ease-in-out .15s;-o-transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s;transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s}.form-control:focus{border-color:#66afe9;outline:0;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 8px rgba(102,175,233,.6);box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 8px rgba(102,175,233,.6)}.form-control::-moz-placeholder{color:#999;opacity:1}.form-control:-ms-input-placeholder{color:#999}.form-control::-webkit-input-placeholder{color:#999}.form-control[disabled],.form-control[readonly],fieldset[disabled] .form-control{cursor:not-allowed;background-color:#eee;opacity:1}textarea.form-control{height:auto}input[type=search]{-webkit-appearance:none}@media screen and (-webkit-min-device-pixel-ratio:0){input[type=date],input[type=time],input[type=datetime-local],input[type=month]{line-height:34px}.input-group-sm input[type=date],.input-group-sm input[type=time],.input-group-sm input[type=datetime-local],.input-group-sm input[type=month],input[type=date].input-sm,input[type=time].input-sm,input[type=datetime-local].input-sm,input[type=month].input-sm{line-height:30px}.input-group-lg input[type=date],.input-group-lg input[type=time],.input-group-lg input[type=datetime-local],.input-group-lg input[type=month],input[type=date].input-lg,input[type=time].input-lg,input[type=datetime-local].input-lg,input[type=month].input-lg{line-height:46px}}.form-group{margin-bottom:15px}.checkbox,.radio{position:relative;display:block;margin-top:10px;margin-bottom:10px}.checkbox label,.radio label{min-height:20px;padding-left:20px;margin-bottom:0;font-weight:400;cursor:pointer}.checkbox input[type=checkbox],.checkbox-inline input[type=checkbox],.radio input[type=radio],.radio-inline input[type=radio]{position:absolute;margin-top:4px \9;margin-left:-20px}.checkbox+.checkbox,.radio+.radio{margin-top:-5px}.checkbox-inline,.radio-inline{display:inline-block;padding-left:20px;margin-bottom:0;font-weight:400;vertical-align:middle;cursor:pointer}.checkbox-inline+.checkbox-inline,.radio-inline+.radio-inline{margin-top:0;margin-left:10px}fieldset[disabled] input[type=checkbox],fieldset[disabled] input[type=radio],input[type=checkbox].disabled,input[type=checkbox][disabled],input[type=radio].disabled,input[type=radio][disabled]{cursor:not-allowed}.checkbox-inline.disabled,.radio-inline.disabled,fieldset[disabled] .checkbox-inline,fieldset[disabled] .radio-inline{cursor:not-allowed}.checkbox.disabled label,.radio.disabled label,fieldset[disabled] .checkbox label,fieldset[disabled] .radio label{cursor:not-allowed}.form-control-static{padding-top:7px;padding-bottom:7px;margin-bottom:0}.form-control-static.input-lg,.form-control-static.input-sm{padding-right:0;padding-left:0}.input-sm{height:30px;padding:5px 10px;font-size:12px;line-height:1.5;border-radius:3px}select.input-sm{height:30px;line-height:30px}select[multiple].input-sm,textarea.input-sm{height:auto}.form-group-sm .form-control{height:30px;padding:5px 10px;font-size:12px;line-height:1.5;border-radius:3px}select.form-group-sm .form-control{height:30px;line-height:30px}select[multiple].form-group-sm .form-control,textarea.form-group-sm .form-control{height:auto}.form-group-sm .form-control-static{height:30px;padding:5px 10px;font-size:12px;line-height:1.5}.input-lg{height:46px;padding:10px 16px;font-size:18px;line-height:1.3333333;border-radius:6px}select.input-lg{height:46px;line-height:46px}select[multiple].input-lg,textarea.input-lg{height:auto}.form-group-lg .form-control{height:46px;padding:10px 16px;font-size:18px;line-height:1.3333333;border-radius:6px}select.form-group-lg .form-control{height:46px;line-height:46px}select[multiple].form-group-lg .form-control,textarea.form-group-lg .form-control{height:auto}.form-group-lg .form-control-static{height:46px;padding:10px 16px;font-size:18px;line-height:1.3333333}.has-feedback{position:relative}.has-feedback .form-control{padding-right:42.5px}.form-control-feedback{position:absolute;top:0;right:0;z-index:2;display:block;width:34px;height:34px;line-height:34px;text-align:center;pointer-events:none}.input-lg+.form-control-feedback{width:46px;height:46px;line-height:46px}.input-sm+.form-control-feedback{width:30px;height:30px;line-height:30px}.has-success .checkbox,.has-success .checkbox-inline,.has-success .control-label,.has-success .help-block,.has-success .radio,.has-success .radio-inline,.has-success.checkbox label,.has-success.checkbox-inline label,.has-success.radio label,.has-success.radio-inline label{color:#3c763d}.has-success .form-control{border-color:#3c763d;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075)}.has-success .form-control:focus{border-color:#2b542c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #67b168;box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #67b168}.has-success .input-group-addon{color:#3c763d;background-color:#dff0d8;border-color:#3c763d}.has-success .form-control-feedback{color:#3c763d}.has-warning .checkbox,.has-warning .checkbox-inline,.has-warning .control-label,.has-warning .help-block,.has-warning .radio,.has-warning .radio-inline,.has-warning.checkbox label,.has-warning.checkbox-inline label,.has-warning.radio label,.has-warning.radio-inline label{color:#8a6d3b}.has-warning .form-control{border-color:#8a6d3b;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075)}.has-warning .form-control:focus{border-color:#66512c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #c0a16b;box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #c0a16b}.has-warning .input-group-addon{color:#8a6d3b;background-color:#fcf8e3;border-color:#8a6d3b}.has-warning .form-control-feedback{color:#8a6d3b}.has-error .checkbox,.has-error .checkbox-inline,.has-error .control-label,.has-error .help-block,.has-error .radio,.has-error .radio-inline,.has-error.checkbox label,.has-error.checkbox-inline label,.has-error.radio label,.has-error.radio-inline label{color:#a94442}.has-error .form-control{border-color:#a94442;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075)}.has-error .form-control:focus{border-color:#843534;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #ce8483;box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #ce8483}.has-error .input-group-addon{color:#a94442;background-color:#f2dede;border-color:#a94442}.has-error .form-control-feedback{color:#a94442}.has-feedback label~.form-control-feedback{top:25px}.has-feedback label.sr-only~.form-control-feedback{top:0}.help-block{display:block;margin-top:5px;margin-bottom:10px;color:#737373}@media (min-width:768px){.form-inline .form-group{display:inline-block;margin-bottom:0;vertical-align:middle}.form-inline .form-control{display:inline-block;width:auto;vertical-align:middle}.form-inline .form-control-static{display:inline-block}.form-inline .input-group{display:inline-table;vertical-align:middle}.form-inline .input-group .form-control,.form-inline .input-group .input-group-addon,.form-inline .input-group .input-group-btn{width:auto}.form-inline .input-group>.form-control{width:100%}.form-inline .control-label{margin-bottom:0;vertical-align:middle}.form-inline .checkbox,.form-inline .radio{display:inline-block;margin-top:0;margin-bottom:0;vertical-align:middle}.form-inline .checkbox label,.form-inline .radio label{padding-left:0}.form-inline .checkbox input[type=checkbox],.form-inline .radio input[type=radio]{position:relative;margin-left:0}.form-inline .has-feedback .form-control-feedback{top:0}}.form-horizontal .checkbox,.form-horizontal .checkbox-inline,.form-horizontal .radio,.form-horizontal .radio-inline{padding-top:7px;margin-top:0;margin-bottom:0}.form-horizontal .checkbox,.form-horizontal .radio{min-height:27px}.form-horizontal .form-group{margin-right:-15px;margin-left:-15px}@media (min-width:768px){.form-horizontal .control-label{padding-top:7px;margin-bottom:0;text-align:right}}.form-horizontal .has-feedback .form-control-feedback{right:15px}@media (min-width:768px){.form-horizontal .form-group-lg .control-label{padding-top:14.33px}}@media (min-width:768px){.form-horizontal .form-group-sm .control-label{padding-top:6px}}.btn{display:inline-block;padding:6px 12px;margin-bottom:0;font-size:14px;font-weight:400;line-height:1.42857143;text-align:center;white-space:nowrap;vertical-align:middle;-ms-touch-action:manipulation;touch-action:manipulation;cursor:pointer;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;background-image:none;border:1px solid transparent;border-radius:4px}.btn.active.focus,.btn.active:focus,.btn.focus,.btn:active.focus,.btn:active:focus,.btn:focus{outline:thin dotted;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}.btn.focus,.btn:focus,.btn:hover{color:#333;text-decoration:none}.btn.active,.btn:active{background-image:none;outline:0;-webkit-box-shadow:inset 0 3px 5px rgba(0,0,0,.125);box-shadow:inset 0 3px 5px rgba(0,0,0,.125)}.btn.disabled,.btn[disabled],fieldset[disabled] .btn{pointer-events:none;cursor:not-allowed;filter:alpha(opacity=65);-webkit-box-shadow:none;box-shadow:none;opacity:.65}.btn-default{color:#333;background-color:#fff;border-color:#ccc}.btn-default.active,.btn-default.focus,.btn-default:active,.btn-default:focus,.btn-default:hover,.open>.dropdown-toggle.btn-default{color:#333;background-color:#e6e6e6;border-color:#adadad}.btn-default.active,.btn-default:active,.open>.dropdown-toggle.btn-default{background-image:none}.btn-default.disabled,.btn-default.disabled.active,.btn-default.disabled.focus,.btn-default.disabled:active,.btn-default.disabled:focus,.btn-default.disabled:hover,.btn-default[disabled],.btn-default[disabled].active,.btn-default[disabled].focus,.btn-default[disabled]:active,.btn-default[disabled]:focus,.btn-default[disabled]:hover,fieldset[disabled] .btn-default,fieldset[disabled] .btn-default.active,fieldset[disabled] .btn-default.focus,fieldset[disabled] .btn-default:active,fieldset[disabled] .btn-default:focus,fieldset[disabled] .btn-default:hover{background-color:#fff;border-color:#ccc}.btn-default .badge{color:#fff;background-color:#333}.btn-primary{color:#fff;background-color:#337ab7;border-color:#2e6da4}.btn-primary.active,.btn-primary.focus,.btn-primary:active,.btn-primary:focus,.btn-primary:hover,.open>.dropdown-toggle.btn-primary{color:#fff;background-color:#286090;border-color:#204d74}.btn-primary.active,.btn-primary:active,.open>.dropdown-toggle.btn-primary{background-image:none}.btn-primary.disabled,.btn-primary.disabled.active,.btn-primary.disabled.focus,.btn-primary.disabled:active,.btn-primary.disabled:focus,.btn-primary.disabled:hover,.btn-primary[disabled],.btn-primary[disabled].active,.btn-primary[disabled].focus,.btn-primary[disabled]:active,.btn-primary[disabled]:focus,.btn-primary[disabled]:hover,fieldset[disabled] .btn-primary,fieldset[disabled] .btn-primary.active,fieldset[disabled] .btn-primary.focus,fieldset[disabled] .btn-primary:active,fieldset[disabled] .btn-primary:focus,fieldset[disabled] .btn-primary:hover{background-color:#337ab7;border-color:#2e6da4}.btn-primary .badge{color:#337ab7;background-color:#fff}.btn-success{color:#fff;background-color:#5cb85c;border-color:#4cae4c}.btn-success.active,.btn-success.focus,.btn-success:active,.btn-success:focus,.btn-success:hover,.open>.dropdown-toggle.btn-success{color:#fff;background-color:#449d44;border-color:#398439}.btn-success.active,.btn-success:active,.open>.dropdown-toggle.btn-success{background-image:none}.btn-success.disabled,.btn-success.disabled.active,.btn-success.disabled.focus,.btn-success.disabled:active,.btn-success.disabled:focus,.btn-success.disabled:hover,.btn-success[disabled],.btn-success[disabled].active,.btn-success[disabled].focus,.btn-success[disabled]:active,.btn-success[disabled]:focus,.btn-success[disabled]:hover,fieldset[disabled] .btn-success,fieldset[disabled] .btn-success.active,fieldset[disabled] .btn-success.focus,fieldset[disabled] .btn-success:active,fieldset[disabled] .btn-success:focus,fieldset[disabled] .btn-success:hover{background-color:#5cb85c;border-color:#4cae4c}.btn-success .badge{color:#5cb85c;background-color:#fff}.btn-info{color:#fff;background-color:#5bc0de;border-color:#46b8da}.btn-info.active,.btn-info.focus,.btn-info:active,.btn-info:focus,.btn-info:hover,.open>.dropdown-toggle.btn-info{color:#fff;background-color:#31b0d5;border-color:#269abc}.btn-info.active,.btn-info:active,.open>.dropdown-toggle.btn-info{background-image:none}.btn-info.disabled,.btn-info.disabled.active,.btn-info.disabled.focus,.btn-info.disabled:active,.btn-info.disabled:focus,.btn-info.disabled:hover,.btn-info[disabled],.btn-info[disabled].active,.btn-info[disabled].focus,.btn-info[disabled]:active,.btn-info[disabled]:focus,.btn-info[disabled]:hover,fieldset[disabled] .btn-info,fieldset[disabled] .btn-info.active,fieldset[disabled] .btn-info.focus,fieldset[disabled] .btn-info:active,fieldset[disabled] .btn-info:focus,fieldset[disabled] .btn-info:hover{background-color:#5bc0de;border-color:#46b8da}.btn-info .badge{color:#5bc0de;background-color:#fff}.btn-warning{color:#fff;background-color:#f0ad4e;border-color:#eea236}.btn-warning.active,.btn-warning.focus,.btn-warning:active,.btn-warning:focus,.btn-warning:hover,.open>.dropdown-toggle.btn-warning{color:#fff;background-color:#ec971f;border-color:#d58512}.btn-warning.active,.btn-warning:active,.open>.dropdown-toggle.btn-warning{background-image:none}.btn-warning.disabled,.btn-warning.disabled.active,.btn-warning.disabled.focus,.btn-warning.disabled:active,.btn-warning.disabled:focus,.btn-warning.disabled:hover,.btn-warning[disabled],.btn-warning[disabled].active,.btn-warning[disabled].focus,.btn-warning[disabled]:active,.btn-warning[disabled]:focus,.btn-warning[disabled]:hover,fieldset[disabled] .btn-warning,fieldset[disabled] .btn-warning.active,fieldset[disabled] .btn-warning.focus,fieldset[disabled] .btn-warning:active,fieldset[disabled] .btn-warning:focus,fieldset[disabled] .btn-warning:hover{background-color:#f0ad4e;border-color:#eea236}.btn-warning .badge{color:#f0ad4e;background-color:#fff}.btn-danger{color:#fff;background-color:#d9534f;border-color:#d43f3a}.btn-danger.active,.btn-danger.focus,.btn-danger:active,.btn-danger:focus,.btn-danger:hover,.open>.dropdown-toggle.btn-danger{color:#fff;background-color:#c9302c;border-color:#ac2925}.btn-danger.active,.btn-danger:active,.open>.dropdown-toggle.btn-danger{background-image:none}.btn-danger.disabled,.btn-danger.disabled.active,.btn-danger.disabled.focus,.btn-danger.disabled:active,.btn-danger.disabled:focus,.btn-danger.disabled:hover,.btn-danger[disabled],.btn-danger[disabled].active,.btn-danger[disabled].focus,.btn-danger[disabled]:active,.btn-danger[disabled]:focus,.btn-danger[disabled]:hover,fieldset[disabled] .btn-danger,fieldset[disabled] .btn-danger.active,fieldset[disabled] .btn-danger.focus,fieldset[disabled] .btn-danger:active,fieldset[disabled] .btn-danger:focus,fieldset[disabled] .btn-danger:hover{background-color:#d9534f;border-color:#d43f3a}.btn-danger .badge{color:#d9534f;background-color:#fff}.btn-link{font-weight:400;color:#337ab7;border-radius:0}.btn-link,.btn-link.active,.btn-link:active,.btn-link[disabled],fieldset[disabled] .btn-link{background-color:transparent;-webkit-box-shadow:none;box-shadow:none}.btn-link,.btn-link:active,.btn-link:focus,.btn-link:hover{border-color:transparent}.btn-link:focus,.btn-link:hover{color:#23527c;text-decoration:underline;background-color:transparent}.btn-link[disabled]:focus,.btn-link[disabled]:hover,fieldset[disabled] .btn-link:focus,fieldset[disabled] .btn-link:hover{color:#777;text-decoration:none}.btn-group-lg>.btn,.btn-lg{padding:10px 16px;font-size:18px;line-height:1.3333333;border-radius:6px}.btn-group-sm>.btn,.btn-sm{padding:5px 10px;font-size:12px;line-height:1.5;border-radius:3px}.btn-group-xs>.btn,.btn-xs{padding:1px 5px;font-size:12px;line-height:1.5;border-radius:3px}.btn-block{display:block;width:100%}.btn-block+.btn-block{margin-top:5px}input[type=button].btn-block,input[type=reset].btn-block,input[type=submit].btn-block{width:100%}.fade{opacity:0;-webkit-transition:opacity .15s linear;-o-transition:opacity .15s linear;transition:opacity .15s linear}.fade.in{opacity:1}.collapse{display:none;visibility:hidden}.collapse.in{display:block;visibility:visible}tr.collapse.in{display:table-row}tbody.collapse.in{display:table-row-group}.collapsing{position:relative;height:0;overflow:hidden;-webkit-transition-timing-function:ease;-o-transition-timing-function:ease;transition-timing-function:ease;-webkit-transition-duration:.35s;-o-transition-duration:.35s;transition-duration:.35s;-webkit-transition-property:height,visibility;-o-transition-property:height,visibility;transition-property:height,visibility}.caret{display:inline-block;width:0;height:0;margin-left:2px;vertical-align:middle;border-top:4px solid;border-right:4px solid transparent;border-left:4px solid transparent}.dropdown,.dropup{position:relative}.dropdown-toggle:focus{outline:0}.dropdown-menu{position:absolute;top:100%;left:0;z-index:1000;display:none;float:left;min-width:160px;padding:5px 0;margin:2px 0 0;font-size:14px;text-align:left;list-style:none;background-color:#fff;-webkit-background-clip:padding-box;background-clip:padding-box;border:1px solid #ccc;border:1px solid rgba(0,0,0,.15);border-radius:4px;-webkit-box-shadow:0 6px 12px rgba(0,0,0,.175);box-shadow:0 6px 12px rgba(0,0,0,.175)}.dropdown-menu.pull-right{right:0;left:auto}.dropdown-menu .divider{height:1px;margin:9px 0;overflow:hidden;background-color:#e5e5e5}.dropdown-menu>li>a{display:block;padding:3px 20px;clear:both;font-weight:400;line-height:1.42857143;color:#333;white-space:nowrap}.dropdown-menu>li>a:focus,.dropdown-menu>li>a:hover{color:#262626;text-decoration:none;background-color:#f5f5f5}.dropdown-menu>.active>a,.dropdown-menu>.active>a:focus,.dropdown-menu>.active>a:hover{color:#fff;text-decoration:none;background-color:#337ab7;outline:0}.dropdown-menu>.disabled>a,.dropdown-menu>.disabled>a:focus,.dropdown-menu>.disabled>a:hover{color:#777}.dropdown-menu>.disabled>a:focus,.dropdown-menu>.disabled>a:hover{text-decoration:none;cursor:not-allowed;background-color:transparent;background-image:none;filter:progid:DXImageTransform.Microsoft.gradient(enabled=false)}.open>.dropdown-menu{display:block}.open>a{outline:0}.dropdown-menu-right{right:0;left:auto}.dropdown-menu-left{right:auto;left:0}.dropdown-header{display:block;padding:3px 20px;font-size:12px;line-height:1.42857143;color:#777;white-space:nowrap}.dropdown-backdrop{position:fixed;top:0;right:0;bottom:0;left:0;z-index:990}.pull-right>.dropdown-menu{right:0;left:auto}.dropup .caret,.navbar-fixed-bottom .dropdown .caret{content:"";border-top:0;border-bottom:4px solid}.dropup .dropdown-menu,.navbar-fixed-bottom .dropdown .dropdown-menu{top:auto;bottom:100%;margin-bottom:2px}@media (min-width:768px){.navbar-right .dropdown-menu{right:0;left:auto}.navbar-right .dropdown-menu-left{right:auto;left:0}}.btn-group,.btn-group-vertical{position:relative;display:inline-block;vertical-align:middle}.btn-group-vertical>.btn,.btn-group>.btn{position:relative;float:left}.btn-group-vertical>.btn.active,.btn-group-vertical>.btn:active,.btn-group-vertical>.btn:focus,.btn-group-vertical>.btn:hover,.btn-group>.btn.active,.btn-group>.btn:active,.btn-group>.btn:focus,.btn-group>.btn:hover{z-index:2}.btn-group .btn+.btn,.btn-group .btn+.btn-group,.btn-group .btn-group+.btn,.btn-group .btn-group+.btn-group{margin-left:-1px}.btn-toolbar{margin-left:-5px}.btn-toolbar .btn-group,.btn-toolbar .input-group{float:left}.btn-toolbar>.btn,.btn-toolbar>.btn-group,.btn-toolbar>.input-group{margin-left:5px}.btn-group>.btn:not(:first-child):not(:last-child):not(.dropdown-toggle){border-radius:0}.btn-group>.btn:first-child{margin-left:0}.btn-group>.btn:first-child:not(:last-child):not(.dropdown-toggle){border-top-right-radius:0;border-bottom-right-radius:0}.btn-group>.btn:last-child:not(:first-child),.btn-group>.dropdown-toggle:not(:first-child){border-top-left-radius:0;border-bottom-left-radius:0}.btn-group>.btn-group{float:left}.btn-group>.btn-group:not(:first-child):not(:last-child)>.btn{border-radius:0}.btn-group>.btn-group:first-child:not(:last-child)>.btn:last-child,.btn-group>.btn-group:first-child:not(:last-child)>.dropdown-toggle{border-top-right-radius:0;border-bottom-right-radius:0}.btn-group>.btn-group:last-child:not(:first-child)>.btn:first-child{border-top-left-radius:0;border-bottom-left-radius:0}.btn-group .dropdown-toggle:active,.btn-group.open .dropdown-toggle{outline:0}.btn-group>.btn+.dropdown-toggle{padding-right:8px;padding-left:8px}.btn-group>.btn-lg+.dropdown-toggle{padding-right:12px;padding-left:12px}.btn-group.open .dropdown-toggle{-webkit-box-shadow:inset 0 3px 5px rgba(0,0,0,.125);box-shadow:inset 0 3px 5px rgba(0,0,0,.125)}.btn-group.open .dropdown-toggle.btn-link{-webkit-box-shadow:none;box-shadow:none}.btn .caret{margin-left:0}.btn-lg .caret{border-width:5px 5px 0;border-bottom-width:0}.dropup .btn-lg .caret{border-width:0 5px 5px}.btn-group-vertical>.btn,.btn-group-vertical>.btn-group,.btn-group-vertical>.btn-group>.btn{display:block;float:none;width:100%;max-width:100%}.btn-group-vertical>.btn-group>.btn{float:none}.btn-group-vertical>.btn+.btn,.btn-group-vertical>.btn+.btn-group,.btn-group-vertical>.btn-group+.btn,.btn-group-vertical>.btn-group+.btn-group{margin-top:-1px;margin-left:0}.btn-group-vertical>.btn:not(:first-child):not(:last-child){border-radius:0}.btn-group-vertical>.btn:first-child:not(:last-child){border-top-right-radius:4px;border-bottom-right-radius:0;border-bottom-left-radius:0}.btn-group-vertical>.btn:last-child:not(:first-child){border-top-left-radius:0;border-top-right-radius:0;border-bottom-left-radius:4px}.btn-group-vertical>.btn-group:not(:first-child):not(:last-child)>.btn{border-radius:0}.btn-group-vertical>.btn-group:first-child:not(:last-child)>.btn:last-child,.btn-group-vertical>.btn-group:first-child:not(:last-child)>.dropdown-toggle{border-bottom-right-radius:0;border-bottom-left-radius:0}.btn-group-vertical>.btn-group:last-child:not(:first-child)>.btn:first-child{border-top-left-radius:0;border-top-right-radius:0}.btn-group-justified{display:table;width:100%;table-layout:fixed;border-collapse:separate}.btn-group-justified>.btn,.btn-group-justified>.btn-group{display:table-cell;float:none;width:1%}.btn-group-justified>.btn-group .btn{width:100%}.btn-group-justified>.btn-group .dropdown-menu{left:auto}[data-toggle=buttons]>.btn input[type=checkbox],[data-toggle=buttons]>.btn input[type=radio],[data-toggle=buttons]>.btn-group>.btn input[type=checkbox],[data-toggle=buttons]>.btn-group>.btn input[type=radio]{position:absolute;clip:rect(0,0,0,0);pointer-events:none}.input-group{position:relative;display:table;border-collapse:separate}.input-group[class*=col-]{float:none;padding-right:0;padding-left:0}.input-group .form-control{position:relative;z-index:2;float:left;width:100%;margin-bottom:0}.input-group-lg>.form-control,.input-group-lg>.input-group-addon,.input-group-lg>.input-group-btn>.btn{height:46px;padding:10px 16px;font-size:18px;line-height:1.3333333;border-radius:6px}select.input-group-lg>.form-control,select.input-group-lg>.input-group-addon,select.input-group-lg>.input-group-btn>.btn{height:46px;line-height:46px}select[multiple].input-group-lg>.form-control,select[multiple].input-group-lg>.input-group-addon,select[multiple].input-group-lg>.input-group-btn>.btn,textarea.input-group-lg>.form-control,textarea.input-group-lg>.input-group-addon,textarea.input-group-lg>.input-group-btn>.btn{height:auto}.input-group-sm>.form-control,.input-group-sm>.input-group-addon,.input-group-sm>.input-group-btn>.btn{height:30px;padding:5px 10px;font-size:12px;line-height:1.5;border-radius:3px}select.input-group-sm>.form-control,select.input-group-sm>.input-group-addon,select.input-group-sm>.input-group-btn>.btn{height:30px;line-height:30px}select[multiple].input-group-sm>.form-control,select[multiple].input-group-sm>.input-group-addon,select[multiple].input-group-sm>.input-group-btn>.btn,textarea.input-group-sm>.form-control,textarea.input-group-sm>.input-group-addon,textarea.input-group-sm>.input-group-btn>.btn{height:auto}.input-group .form-control,.input-group-addon,.input-group-btn{display:table-cell}.input-group .form-control:not(:first-child):not(:last-child),.input-group-addon:not(:first-child):not(:last-child),.input-group-btn:not(:first-child):not(:last-child){border-radius:0}.input-group-addon,.input-group-btn{width:1%;white-space:nowrap;vertical-align:middle}.input-group-addon{padding:6px 12px;font-size:14px;font-weight:400;line-height:1;color:#555;text-align:center;background-color:#eee;border:1px solid #ccc;border-radius:4px}.input-group-addon.input-sm{padding:5px 10px;font-size:12px;border-radius:3px}.input-group-addon.input-lg{padding:10px 16px;font-size:18px;border-radius:6px}.input-group-addon input[type=checkbox],.input-group-addon input[type=radio]{margin-top:0}.input-group .form-control:first-child,.input-group-addon:first-child,.input-group-btn:first-child>.btn,.input-group-btn:first-child>.btn-group>.btn,.input-group-btn:first-child>.dropdown-toggle,.input-group-btn:last-child>.btn-group:not(:last-child)>.btn,.input-group-btn:last-child>.btn:not(:last-child):not(.dropdown-toggle){border-top-right-radius:0;border-bottom-right-radius:0}.input-group-addon:first-child{border-right:0}.input-group .form-control:last-child,.input-group-addon:last-child,.input-group-btn:first-child>.btn-group:not(:first-child)>.btn,.input-group-btn:first-child>.btn:not(:first-child),.input-group-btn:last-child>.btn,.input-group-btn:last-child>.btn-group>.btn,.input-group-btn:last-child>.dropdown-toggle{border-top-left-radius:0;border-bottom-left-radius:0}.input-group-addon:last-child{border-left:0}.input-group-btn{position:relative;font-size:0;white-space:nowrap}.input-group-btn>.btn{position:relative}.input-group-btn>.btn+.btn{margin-left:-1px}.input-group-btn>.btn:active,.input-group-btn>.btn:focus,.input-group-btn>.btn:hover{z-index:2}.input-group-btn:first-child>.btn,.input-group-btn:first-child>.btn-group{margin-right:-1px}.input-group-btn:last-child>.btn,.input-group-btn:last-child>.btn-group{margin-left:-1px}.nav{padding-left:0;margin-bottom:0;list-style:none}.nav>li{position:relative;display:block}.nav>li>a{position:relative;display:block;padding:10px 15px}.nav>li>a:focus,.nav>li>a:hover{text-decoration:none;background-color:#eee}.nav>li.disabled>a{color:#777}.nav>li.disabled>a:focus,.nav>li.disabled>a:hover{color:#777;text-decoration:none;cursor:not-allowed;background-color:transparent}.nav .open>a,.nav .open>a:focus,.nav .open>a:hover{background-color:#eee;border-color:#337ab7}.nav .nav-divider{height:1px;margin:9px 0;overflow:hidden;background-color:#e5e5e5}.nav>li>a>img{max-width:none}.nav-tabs{border-bottom:1px solid #ddd}.nav-tabs>li{float:left;margin-bottom:-1px}.nav-tabs>li>a{margin-right:2px;line-height:1.42857143;border:1px solid transparent;border-radius:4px 4px 0 0}.nav-tabs>li>a:hover{border-color:#eee #eee #ddd}.nav-tabs>li.active>a,.nav-tabs>li.active>a:focus,.nav-tabs>li.active>a:hover{color:#555;cursor:default;background-color:#fff;border:1px solid #ddd;border-bottom-color:transparent}.nav-tabs.nav-justified{width:100%;border-bottom:0}.nav-tabs.nav-justified>li{float:none}.nav-tabs.nav-justified>li>a{margin-bottom:5px;text-align:center}.nav-tabs.nav-justified>.dropdown .dropdown-menu{top:auto;left:auto}@media (min-width:768px){.nav-tabs.nav-justified>li{display:table-cell;width:1%}.nav-tabs.nav-justified>li>a{margin-bottom:0}}.nav-tabs.nav-justified>li>a{margin-right:0;border-radius:4px}.nav-tabs.nav-justified>.active>a,.nav-tabs.nav-justified>.active>a:focus,.nav-tabs.nav-justified>.active>a:hover{border:1px solid #ddd}@media (min-width:768px){.nav-tabs.nav-justified>li>a{border-bottom:1px solid #ddd;border-radius:4px 4px 0 0}.nav-tabs.nav-justified>.active>a,.nav-tabs.nav-justified>.active>a:focus,.nav-tabs.nav-justified>.active>a:hover{border-bottom-color:#fff}}.nav-pills>li{float:left}.nav-pills>li>a{border-radius:4px}.nav-pills>li+li{margin-left:2px}.nav-pills>li.active>a,.nav-pills>li.active>a:focus,.nav-pills>li.active>a:hover{color:#fff;background-color:#337ab7}.nav-stacked>li{float:none}.nav-stacked>li+li{margin-top:2px;margin-left:0}.nav-justified{width:100%}.nav-justified>li{float:none}.nav-justified>li>a{margin-bottom:5px;text-align:center}.nav-justified>.dropdown .dropdown-menu{top:auto;left:auto}@media (min-width:768px){.nav-justified>li{display:table-cell;width:1%}.nav-justified>li>a{margin-bottom:0}}.nav-tabs-justified{border-bottom:0}.nav-tabs-justified>li>a{margin-right:0;border-radius:4px}.nav-tabs-justified>.active>a,.nav-tabs-justified>.active>a:focus,.nav-tabs-justified>.active>a:hover{border:1px solid #ddd}@media (min-width:768px){.nav-tabs-justified>li>a{border-bottom:1px solid #ddd;border-radius:4px 4px 0 0}.nav-tabs-justified>.active>a,.nav-tabs-justified>.active>a:focus,.nav-tabs-justified>.active>a:hover{border-bottom-color:#fff}}.tab-content>.tab-pane{display:none;visibility:hidden}.tab-content>.active{display:block;visibility:visible}.nav-tabs .dropdown-menu{margin-top:-1px;border-top-left-radius:0;border-top-right-radius:0}.navbar{position:relative;min-height:50px;margin-bottom:20px;border:1px solid transparent}@media (min-width:768px){.navbar{border-radius:4px}}@media (min-width:768px){.navbar-header{float:left}}.navbar-collapse{padding-right:15px;padding-left:15px;overflow-x:visible;-webkit-overflow-scrolling:touch;border-top:1px solid transparent;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,.1);box-shadow:inset 0 1px 0 rgba(255,255,255,.1)}.navbar-collapse.in{overflow-y:auto}@media (min-width:768px){.navbar-collapse{width:auto;border-top:0;-webkit-box-shadow:none;box-shadow:none}.navbar-collapse.collapse{display:block!important;height:auto!important;padding-bottom:0;overflow:visible!important;visibility:visible!important}.navbar-collapse.in{overflow-y:visible}.navbar-fixed-bottom .navbar-collapse,.navbar-fixed-top .navbar-collapse,.navbar-static-top .navbar-collapse{padding-right:0;padding-left:0}}.navbar-fixed-bottom .navbar-collapse,.navbar-fixed-top .navbar-collapse{max-height:340px}@media (max-device-width:480px) and (orientation:landscape){.navbar-fixed-bottom .navbar-collapse,.navbar-fixed-top .navbar-collapse{max-height:200px}}.container-fluid>.navbar-collapse,.container-fluid>.navbar-header,.container>.navbar-collapse,.container>.navbar-header{margin-right:-15px;margin-left:-15px}@media (min-width:768px){.container-fluid>.navbar-collapse,.container-fluid>.navbar-header,.container>.navbar-collapse,.container>.navbar-header{margin-right:0;margin-left:0}}.navbar-static-top{z-index:1000;border-width:0 0 1px}@media (min-width:768px){.navbar-static-top{border-radius:0}}.navbar-fixed-bottom,.navbar-fixed-top{position:fixed;right:0;left:0;z-index:1030}@media (min-width:768px){.navbar-fixed-bottom,.navbar-fixed-top{border-radius:0}}.navbar-fixed-top{top:0;border-width:0 0 1px}.navbar-fixed-bottom{bottom:0;margin-bottom:0;border-width:1px 0 0}.navbar-brand{float:left;height:50px;padding:15px 15px;font-size:18px;line-height:20px}.navbar-brand:focus,.navbar-brand:hover{text-decoration:none}.navbar-brand>img{display:block}@media (min-width:768px){.navbar>.container .navbar-brand,.navbar>.container-fluid .navbar-brand{margin-left:-15px}}.navbar-toggle{position:relative;float:right;padding:9px 10px;margin-top:8px;margin-right:15px;margin-bottom:8px;background-color:transparent;background-image:none;border:1px solid transparent;border-radius:4px}.navbar-toggle:focus{outline:0}.navbar-toggle .icon-bar{display:block;width:22px;height:2px;border-radius:1px}.navbar-toggle .icon-bar+.icon-bar{margin-top:4px}@media (min-width:768px){.navbar-toggle{display:none}}.navbar-nav{margin:7.5px -15px}.navbar-nav>li>a{padding-top:10px;padding-bottom:10px;line-height:20px}@media (max-width:767px){.navbar-nav .open .dropdown-menu{position:static;float:none;width:auto;margin-top:0;background-color:transparent;border:0;-webkit-box-shadow:none;box-shadow:none}.navbar-nav .open .dropdown-menu .dropdown-header,.navbar-nav .open .dropdown-menu>li>a{padding:5px 15px 5px 25px}.navbar-nav .open .dropdown-menu>li>a{line-height:20px}.navbar-nav .open .dropdown-menu>li>a:focus,.navbar-nav .open .dropdown-menu>li>a:hover{background-image:none}}@media (min-width:768px){.navbar-nav{float:left;margin:0}.navbar-nav>li{float:left}.navbar-nav>li>a{padding-top:15px;padding-bottom:15px}}.navbar-form{padding:10px 15px;margin-top:8px;margin-right:-15px;margin-bottom:8px;margin-left:-15px;border-top:1px solid transparent;border-bottom:1px solid transparent;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,.1),0 1px 0 rgba(255,255,255,.1);box-shadow:inset 0 1px 0 rgba(255,255,255,.1),0 1px 0 rgba(255,255,255,.1)}@media (min-width:768px){.navbar-form .form-group{display:inline-block;margin-bottom:0;vertical-align:middle}.navbar-form .form-control{display:inline-block;width:auto;vertical-align:middle}.navbar-form .form-control-static{display:inline-block}.navbar-form .input-group{display:inline-table;vertical-align:middle}.navbar-form .input-group .form-control,.navbar-form .input-group .input-group-addon,.navbar-form .input-group .input-group-btn{width:auto}.navbar-form .input-group>.form-control{width:100%}.navbar-form .control-label{margin-bottom:0;vertical-align:middle}.navbar-form .checkbox,.navbar-form .radio{display:inline-block;margin-top:0;margin-bottom:0;vertical-align:middle}.navbar-form .checkbox label,.navbar-form .radio label{padding-left:0}.navbar-form .checkbox input[type=checkbox],.navbar-form .radio input[type=radio]{position:relative;margin-left:0}.navbar-form .has-feedback .form-control-feedback{top:0}}@media (max-width:767px){.navbar-form .form-group{margin-bottom:5px}.navbar-form .form-group:last-child{margin-bottom:0}}@media (min-width:768px){.navbar-form{width:auto;padding-top:0;padding-bottom:0;margin-right:0;margin-left:0;border:0;-webkit-box-shadow:none;box-shadow:none}}.navbar-nav>li>.dropdown-menu{margin-top:0;border-top-left-radius:0;border-top-right-radius:0}.navbar-fixed-bottom .navbar-nav>li>.dropdown-menu{margin-bottom:0;border-top-left-radius:4px;border-top-right-radius:4px;border-bottom-right-radius:0;border-bottom-left-radius:0}.navbar-btn{margin-top:8px;margin-bottom:8px}.navbar-btn.btn-sm{margin-top:10px;margin-bottom:10px}.navbar-btn.btn-xs{margin-top:14px;margin-bottom:14px}.navbar-text{margin-top:15px;margin-bottom:15px}@media (min-width:768px){.navbar-text{float:left;margin-right:15px;margin-left:15px}}@media (min-width:768px){.navbar-left{float:left!important}.navbar-right{float:right!important;margin-right:-15px}.navbar-right~.navbar-right{margin-right:0}}.navbar-default{background-color:#f8f8f8;border-color:#e7e7e7}.navbar-default .navbar-brand{color:#777}.navbar-default .navbar-brand:focus,.navbar-default .navbar-brand:hover{color:#5e5e5e;background-color:transparent}.navbar-default .navbar-text{color:#777}.navbar-default .navbar-nav>li>a{color:#777}.navbar-default .navbar-nav>li>a:focus,.navbar-default .navbar-nav>li>a:hover{color:#333;background-color:transparent}.navbar-default .navbar-nav>.active>a,.navbar-default .navbar-nav>.active>a:focus,.navbar-default .navbar-nav>.active>a:hover{color:#555;background-color:#e7e7e7}.navbar-default .navbar-nav>.disabled>a,.navbar-default .navbar-nav>.disabled>a:focus,.navbar-default .navbar-nav>.disabled>a:hover{color:#ccc;background-color:transparent}.navbar-default .navbar-toggle{border-color:#ddd}.navbar-default .navbar-toggle:focus,.navbar-default .navbar-toggle:hover{background-color:#ddd}.navbar-default .navbar-toggle .icon-bar{background-color:#888}.navbar-default .navbar-collapse,.navbar-default .navbar-form{border-color:#e7e7e7}.navbar-default .navbar-nav>.open>a,.navbar-default .navbar-nav>.open>a:focus,.navbar-default .navbar-nav>.open>a:hover{color:#555;background-color:#e7e7e7}@media (max-width:767px){.navbar-default .navbar-nav .open .dropdown-menu>li>a{color:#777}.navbar-default .navbar-nav .open .dropdown-menu>li>a:focus,.navbar-default .navbar-nav .open .dropdown-menu>li>a:hover{color:#333;background-color:transparent}.navbar-default .navbar-nav .open .dropdown-menu>.active>a,.navbar-default .navbar-nav .open .dropdown-menu>.active>a:focus,.navbar-default .navbar-nav .open .dropdown-menu>.active>a:hover{color:#555;background-color:#e7e7e7}.navbar-default .navbar-nav .open .dropdown-menu>.disabled>a,.navbar-default .navbar-nav .open .dropdown-menu>.disabled>a:focus,.navbar-default .navbar-nav .open .dropdown-menu>.disabled>a:hover{color:#ccc;background-color:transparent}}.navbar-default .navbar-link{color:#777}.navbar-default .navbar-link:hover{color:#333}.navbar-default .btn-link{color:#777}.navbar-default .btn-link:focus,.navbar-default .btn-link:hover{color:#333}.navbar-default .btn-link[disabled]:focus,.navbar-default .btn-link[disabled]:hover,fieldset[disabled] .navbar-default .btn-link:focus,fieldset[disabled] .navbar-default .btn-link:hover{color:#ccc}.navbar-inverse{background-color:#222;border-color:#080808}.navbar-inverse .navbar-brand{color:#9d9d9d}.navbar-inverse .navbar-brand:focus,.navbar-inverse .navbar-brand:hover{color:#fff;background-color:transparent}.navbar-inverse .navbar-text{color:#9d9d9d}.navbar-inverse .navbar-nav>li>a{color:#9d9d9d}.navbar-inverse .navbar-nav>li>a:focus,.navbar-inverse .navbar-nav>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .navbar-nav>.active>a,.navbar-inverse .navbar-nav>.active>a:focus,.navbar-inverse .navbar-nav>.active>a:hover{color:#fff;background-color:#080808}.navbar-inverse .navbar-nav>.disabled>a,.navbar-inverse .navbar-nav>.disabled>a:focus,.navbar-inverse .navbar-nav>.disabled>a:hover{color:#444;background-color:transparent}.navbar-inverse .navbar-toggle{border-color:#333}.navbar-inverse .navbar-toggle:focus,.navbar-inverse .navbar-toggle:hover{background-color:#333}.navbar-inverse .navbar-toggle .icon-bar{background-color:#fff}.navbar-inverse .navbar-collapse,.navbar-inverse .navbar-form{border-color:#101010}.navbar-inverse .navbar-nav>.open>a,.navbar-inverse .navbar-nav>.open>a:focus,.navbar-inverse .navbar-nav>.open>a:hover{color:#fff;background-color:#080808}@media (max-width:767px){.navbar-inverse .navbar-nav .open .dropdown-menu>.dropdown-header{border-color:#080808}.navbar-inverse .navbar-nav .open .dropdown-menu .divider{background-color:#080808}.navbar-inverse .navbar-nav .open .dropdown-menu>li>a{color:#9d9d9d}.navbar-inverse .navbar-nav .open .dropdown-menu>li>a:focus,.navbar-inverse .navbar-nav .open .dropdown-menu>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .navbar-nav .open .dropdown-menu>.active>a,.navbar-inverse .navbar-nav .open .dropdown-menu>.active>a:focus,.navbar-inverse .navbar-nav .open .dropdown-menu>.active>a:hover{color:#fff;background-color:#080808}.navbar-inverse .navbar-nav .open .dropdown-menu>.disabled>a,.navbar-inverse .navbar-nav .open .dropdown-menu>.disabled>a:focus,.navbar-inverse .navbar-nav .open .dropdown-menu>.disabled>a:hover{color:#444;background-color:transparent}}.navbar-inverse .navbar-link{color:#9d9d9d}.navbar-inverse .navbar-link:hover{color:#fff}.navbar-inverse .btn-link{color:#9d9d9d}.navbar-inverse .btn-link:focus,.navbar-inverse .btn-link:hover{color:#fff}.navbar-inverse .btn-link[disabled]:focus,.navbar-inverse .btn-link[disabled]:hover,fieldset[disabled] .navbar-inverse .btn-link:focus,fieldset[disabled] .navbar-inverse .btn-link:hover{color:#444}.breadcrumb{padding:8px 15px;margin-bottom:20px;list-style:none;background-color:#f5f5f5;border-radius:4px}.breadcrumb>li{display:inline-block}.breadcrumb>li+li:before{padding:0 5px;color:#ccc;content:"/\00a0"}.breadcrumb>.active{color:#777}.pagination{display:inline-block;padding-left:0;margin:20px 0;border-radius:4px}.pagination>li{display:inline}.pagination>li>a,.pagination>li>span{position:relative;float:left;padding:6px 12px;margin-left:-1px;line-height:1.42857143;color:#337ab7;text-decoration:none;background-color:#fff;border:1px solid #ddd}.pagination>li:first-child>a,.pagination>li:first-child>span{margin-left:0;border-top-left-radius:4px;border-bottom-left-radius:4px}.pagination>li:last-child>a,.pagination>li:last-child>span{border-top-right-radius:4px;border-bottom-right-radius:4px}.pagination>li>a:focus,.pagination>li>a:hover,.pagination>li>span:focus,.pagination>li>span:hover{color:#23527c;background-color:#eee;border-color:#ddd}.pagination>.active>a,.pagination>.active>a:focus,.pagination>.active>a:hover,.pagination>.active>span,.pagination>.active>span:focus,.pagination>.active>span:hover{z-index:2;color:#fff;cursor:default;background-color:#337ab7;border-color:#337ab7}.pagination>.disabled>a,.pagination>.disabled>a:focus,.pagination>.disabled>a:hover,.pagination>.disabled>span,.pagination>.disabled>span:focus,.pagination>.disabled>span:hover{color:#777;cursor:not-allowed;background-color:#fff;border-color:#ddd}.pagination-lg>li>a,.pagination-lg>li>span{padding:10px 16px;font-size:18px}.pagination-lg>li:first-child>a,.pagination-lg>li:first-child>span{border-top-left-radius:6px;border-bottom-left-radius:6px}.pagination-lg>li:last-child>a,.pagination-lg>li:last-child>span{border-top-right-radius:6px;border-bottom-right-radius:6px}.pagination-sm>li>a,.pagination-sm>li>span{padding:5px 10px;font-size:12px}.pagination-sm>li:first-child>a,.pagination-sm>li:first-child>span{border-top-left-radius:3px;border-bottom-left-radius:3px}.pagination-sm>li:last-child>a,.pagination-sm>li:last-child>span{border-top-right-radius:3px;border-bottom-right-radius:3px}.pager{padding-left:0;margin:20px 0;text-align:center;list-style:none}.pager li{display:inline}.pager li>a,.pager li>span{display:inline-block;padding:5px 14px;background-color:#fff;border:1px solid #ddd;border-radius:15px}.pager li>a:focus,.pager li>a:hover{text-decoration:none;background-color:#eee}.pager .next>a,.pager .next>span{float:right}.pager .previous>a,.pager .previous>span{float:left}.pager .disabled>a,.pager .disabled>a:focus,.pager .disabled>a:hover,.pager .disabled>span{color:#777;cursor:not-allowed;background-color:#fff}.label{display:inline;padding:.2em .6em .3em;font-size:75%;font-weight:700;line-height:1;color:#fff;text-align:center;white-space:nowrap;vertical-align:baseline;border-radius:.25em}a.label:focus,a.label:hover{color:#fff;text-decoration:none;cursor:pointer}.label:empty{display:none}.btn .label{position:relative;top:-1px}.label-default{background-color:#777}.label-default[href]:focus,.label-default[href]:hover{background-color:#5e5e5e}.label-primary{background-color:#337ab7}.label-primary[href]:focus,.label-primary[href]:hover{background-color:#286090}.label-success{background-color:#5cb85c}.label-success[href]:focus,.label-success[href]:hover{background-color:#449d44}.label-info{background-color:#5bc0de}.label-info[href]:focus,.label-info[href]:hover{background-color:#31b0d5}.label-warning{background-color:#f0ad4e}.label-warning[href]:focus,.label-warning[href]:hover{background-color:#ec971f}.label-danger{background-color:#d9534f}.label-danger[href]:focus,.label-danger[href]:hover{background-color:#c9302c}.badge{display:inline-block;min-width:10px;padding:3px 7px;font-size:12px;font-weight:700;line-height:1;color:#fff;text-align:center;white-space:nowrap;vertical-align:baseline;background-color:#777;border-radius:10px}.badge:empty{display:none}.btn .badge{position:relative;top:-1px}.btn-xs .badge{top:0;padding:1px 5px}a.badge:focus,a.badge:hover{color:#fff;text-decoration:none;cursor:pointer}.list-group-item.active>.badge,.nav-pills>.active>a>.badge{color:#337ab7;background-color:#fff}.list-group-item>.badge{float:right}.list-group-item>.badge+.badge{margin-right:5px}.nav-pills>li>a>.badge{margin-left:3px}.jumbotron{padding:30px 15px;margin-bottom:30px;color:inherit;background-color:#eee}.jumbotron .h1,.jumbotron h1{color:inherit}.jumbotron p{margin-bottom:15px;font-size:21px;font-weight:200}.jumbotron>hr{border-top-color:#d5d5d5}.container .jumbotron,.container-fluid .jumbotron{border-radius:6px}.jumbotron .container{max-width:100%}@media screen and (min-width:768px){.jumbotron{padding:48px 0}.container .jumbotron,.container-fluid .jumbotron{padding-right:60px;padding-left:60px}.jumbotron .h1,.jumbotron h1{font-size:63px}}.thumbnail{display:block;padding:4px;margin-bottom:20px;line-height:1.42857143;background-color:#fff;border:1px solid #ddd;border-radius:4px;-webkit-transition:border .2s ease-in-out;-o-transition:border .2s ease-in-out;transition:border .2s ease-in-out}.thumbnail a>img,.thumbnail>img{margin-right:auto;margin-left:auto}a.thumbnail.active,a.thumbnail:focus,a.thumbnail:hover{border-color:#337ab7}.thumbnail .caption{padding:9px;color:#333}.alert{padding:15px;margin-bottom:20px;border:1px solid transparent;border-radius:4px}.alert h4{margin-top:0;color:inherit}.alert .alert-link{font-weight:700}.alert>p,.alert>ul{margin-bottom:0}.alert>p+p{margin-top:5px}.alert-dismissable,.alert-dismissible{padding-right:35px}.alert-dismissable .close,.alert-dismissible .close{position:relative;top:-2px;right:-21px;color:inherit}.alert-success{color:#3c763d;background-color:#dff0d8;border-color:#d6e9c6}.alert-success hr{border-top-color:#c9e2b3}.alert-success .alert-link{color:#2b542c}.alert-info{color:#31708f;background-color:#d9edf7;border-color:#bce8f1}.alert-info hr{border-top-color:#a6e1ec}.alert-info .alert-link{color:#245269}.alert-warning{color:#8a6d3b;background-color:#fcf8e3;border-color:#faebcc}.alert-warning hr{border-top-color:#f7e1b5}.alert-warning .alert-link{color:#66512c}.alert-danger{color:#a94442;background-color:#f2dede;border-color:#ebccd1}.alert-danger hr{border-top-color:#e4b9c0}.alert-danger .alert-link{color:#843534}@-webkit-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@-o-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}.progress{height:20px;margin-bottom:20px;overflow:hidden;background-color:#f5f5f5;border-radius:4px;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,.1);box-shadow:inset 0 1px 2px rgba(0,0,0,.1)}.progress-bar{float:left;width:0;height:100%;font-size:12px;line-height:20px;color:#fff;text-align:center;background-color:#337ab7;-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,.15);box-shadow:inset 0 -1px 0 rgba(0,0,0,.15);-webkit-transition:width .6s ease;-o-transition:width .6s ease;transition:width .6s ease}.progress-bar-striped,.progress-striped .progress-bar{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);-webkit-background-size:40px 40px;background-size:40px 40px}.progress-bar.active,.progress.active .progress-bar{-webkit-animation:progress-bar-stripes 2s linear infinite;-o-animation:progress-bar-stripes 2s linear infinite;animation:progress-bar-stripes 2s linear infinite}.progress-bar-success{background-color:#5cb85c}.progress-striped .progress-bar-success{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.progress-bar-info{background-color:#5bc0de}.progress-striped .progress-bar-info{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.progress-bar-warning{background-color:#f0ad4e}.progress-striped .progress-bar-warning{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.progress-bar-danger{background-color:#d9534f}.progress-striped .progress-bar-danger{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.media{margin-top:15px}.media:first-child{margin-top:0}.media,.media-body{overflow:hidden;zoom:1}.media-body{width:10000px}.media-object{display:block}.media-right,.media>.pull-right{padding-left:10px}.media-left,.media>.pull-left{padding-right:10px}.media-body,.media-left,.media-right{display:table-cell;vertical-align:top}.media-middle{vertical-align:middle}.media-bottom{vertical-align:bottom}.media-heading{margin-top:0;margin-bottom:5px}.media-list{padding-left:0;list-style:none}.list-group{padding-left:0;margin-bottom:20px}.list-group-item{position:relative;display:block;padding:10px 15px;margin-bottom:-1px;background-color:#fff;border:1px solid #ddd}.list-group-item:first-child{border-top-left-radius:4px;border-top-right-radius:4px}.list-group-item:last-child{margin-bottom:0;border-bottom-right-radius:4px;border-bottom-left-radius:4px}a.list-group-item{color:#555}a.list-group-item .list-group-item-heading{color:#333}a.list-group-item:focus,a.list-group-item:hover{color:#555;text-decoration:none;background-color:#f5f5f5}.list-group-item.disabled,.list-group-item.disabled:focus,.list-group-item.disabled:hover{color:#777;cursor:not-allowed;background-color:#eee}.list-group-item.disabled .list-group-item-heading,.list-group-item.disabled:focus .list-group-item-heading,.list-group-item.disabled:hover .list-group-item-heading{color:inherit}.list-group-item.disabled .list-group-item-text,.list-group-item.disabled:focus .list-group-item-text,.list-group-item.disabled:hover .list-group-item-text{color:#777}.list-group-item.active,.list-group-item.active:focus,.list-group-item.active:hover{z-index:2;color:#fff;background-color:#337ab7;border-color:#337ab7}.list-group-item.active .list-group-item-heading,.list-group-item.active .list-group-item-heading>.small,.list-group-item.active .list-group-item-heading>small,.list-group-item.active:focus .list-group-item-heading,.list-group-item.active:focus .list-group-item-heading>.small,.list-group-item.active:focus .list-group-item-heading>small,.list-group-item.active:hover .list-group-item-heading,.list-group-item.active:hover .list-group-item-heading>.small,.list-group-item.active:hover .list-group-item-heading>small{color:inherit}.list-group-item.active .list-group-item-text,.list-group-item.active:focus .list-group-item-text,.list-group-item.active:hover .list-group-item-text{color:#c7ddef}.list-group-item-success{color:#3c763d;background-color:#dff0d8}a.list-group-item-success{color:#3c763d}a.list-group-item-success .list-group-item-heading{color:inherit}a.list-group-item-success:focus,a.list-group-item-success:hover{color:#3c763d;background-color:#d0e9c6}a.list-group-item-success.active,a.list-group-item-success.active:focus,a.list-group-item-success.active:hover{color:#fff;background-color:#3c763d;border-color:#3c763d}.list-group-item-info{color:#31708f;background-color:#d9edf7}a.list-group-item-info{color:#31708f}a.list-group-item-info .list-group-item-heading{color:inherit}a.list-group-item-info:focus,a.list-group-item-info:hover{color:#31708f;background-color:#c4e3f3}a.list-group-item-info.active,a.list-group-item-info.active:focus,a.list-group-item-info.active:hover{color:#fff;background-color:#31708f;border-color:#31708f}.list-group-item-warning{color:#8a6d3b;background-color:#fcf8e3}a.list-group-item-warning{color:#8a6d3b}a.list-group-item-warning .list-group-item-heading{color:inherit}a.list-group-item-warning:focus,a.list-group-item-warning:hover{color:#8a6d3b;background-color:#faf2cc}a.list-group-item-warning.active,a.list-group-item-warning.active:focus,a.list-group-item-warning.active:hover{color:#fff;background-color:#8a6d3b;border-color:#8a6d3b}.list-group-item-danger{color:#a94442;background-color:#f2dede}a.list-group-item-danger{color:#a94442}a.list-group-item-danger .list-group-item-heading{color:inherit}a.list-group-item-danger:focus,a.list-group-item-danger:hover{color:#a94442;background-color:#ebcccc}a.list-group-item-danger.active,a.list-group-item-danger.active:focus,a.list-group-item-danger.active:hover{color:#fff;background-color:#a94442;border-color:#a94442}.list-group-item-heading{margin-top:0;margin-bottom:5px}.list-group-item-text{margin-bottom:0;line-height:1.3}.panel{margin-bottom:20px;background-color:#fff;border:1px solid transparent;border-radius:4px;-webkit-box-shadow:0 1px 1px rgba(0,0,0,.05);box-shadow:0 1px 1px rgba(0,0,0,.05)}.panel-body{padding:15px}.panel-heading{padding:10px 15px;border-bottom:1px solid transparent;border-top-left-radius:3px;border-top-right-radius:3px}.panel-heading>.dropdown .dropdown-toggle{color:inherit}.panel-title{margin-top:0;margin-bottom:0;font-size:16px;color:inherit}.panel-title>.small,.panel-title>.small>a,.panel-title>a,.panel-title>small,.panel-title>small>a{color:inherit}.panel-footer{padding:10px 15px;background-color:#f5f5f5;border-top:1px solid #ddd;border-bottom-right-radius:3px;border-bottom-left-radius:3px}.panel>.list-group,.panel>.panel-collapse>.list-group{margin-bottom:0}.panel>.list-group .list-group-item,.panel>.panel-collapse>.list-group .list-group-item{border-width:1px 0;border-radius:0}.panel>.list-group:first-child .list-group-item:first-child,.panel>.panel-collapse>.list-group:first-child .list-group-item:first-child{border-top:0;border-top-left-radius:3px;border-top-right-radius:3px}.panel>.list-group:last-child .list-group-item:last-child,.panel>.panel-collapse>.list-group:last-child .list-group-item:last-child{border-bottom:0;border-bottom-right-radius:3px;border-bottom-left-radius:3px}.panel-heading+.list-group .list-group-item:first-child{border-top-width:0}.list-group+.panel-footer{border-top-width:0}.panel>.panel-collapse>.table,.panel>.table,.panel>.table-responsive>.table{margin-bottom:0}.panel>.panel-collapse>.table caption,.panel>.table caption,.panel>.table-responsive>.table caption{padding-right:15px;padding-left:15px}.panel>.table-responsive:first-child>.table:first-child,.panel>.table:first-child{border-top-left-radius:3px;border-top-right-radius:3px}.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child,.panel>.table:first-child>tbody:first-child>tr:first-child,.panel>.table:first-child>thead:first-child>tr:first-child{border-top-left-radius:3px;border-top-right-radius:3px}.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child td:first-child,.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child th:first-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child td:first-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child th:first-child,.panel>.table:first-child>tbody:first-child>tr:first-child td:first-child,.panel>.table:first-child>tbody:first-child>tr:first-child th:first-child,.panel>.table:first-child>thead:first-child>tr:first-child td:first-child,.panel>.table:first-child>thead:first-child>tr:first-child th:first-child{border-top-left-radius:3px}.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child td:last-child,.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child th:last-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child td:last-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child th:last-child,.panel>.table:first-child>tbody:first-child>tr:first-child td:last-child,.panel>.table:first-child>tbody:first-child>tr:first-child th:last-child,.panel>.table:first-child>thead:first-child>tr:first-child td:last-child,.panel>.table:first-child>thead:first-child>tr:first-child th:last-child{border-top-right-radius:3px}.panel>.table-responsive:last-child>.table:last-child,.panel>.table:last-child{border-bottom-right-radius:3px;border-bottom-left-radius:3px}.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child,.panel>.table:last-child>tbody:last-child>tr:last-child,.panel>.table:last-child>tfoot:last-child>tr:last-child{border-bottom-right-radius:3px;border-bottom-left-radius:3px}.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child td:first-child,.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child th:first-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child td:first-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child th:first-child,.panel>.table:last-child>tbody:last-child>tr:last-child td:first-child,.panel>.table:last-child>tbody:last-child>tr:last-child th:first-child,.panel>.table:last-child>tfoot:last-child>tr:last-child td:first-child,.panel>.table:last-child>tfoot:last-child>tr:last-child th:first-child{border-bottom-left-radius:3px}.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child td:last-child,.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child th:last-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child td:last-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child th:last-child,.panel>.table:last-child>tbody:last-child>tr:last-child td:last-child,.panel>.table:last-child>tbody:last-child>tr:last-child th:last-child,.panel>.table:last-child>tfoot:last-child>tr:last-child td:last-child,.panel>.table:last-child>tfoot:last-child>tr:last-child th:last-child{border-bottom-right-radius:3px}.panel>.panel-body+.table,.panel>.panel-body+.table-responsive,.panel>.table+.panel-body,.panel>.table-responsive+.panel-body{border-top:1px solid #ddd}.panel>.table>tbody:first-child>tr:first-child td,.panel>.table>tbody:first-child>tr:first-child th{border-top:0}.panel>.table-bordered,.panel>.table-responsive>.table-bordered{border:0}.panel>.table-bordered>tbody>tr>td:first-child,.panel>.table-bordered>tbody>tr>th:first-child,.panel>.table-bordered>tfoot>tr>td:first-child,.panel>.table-bordered>tfoot>tr>th:first-child,.panel>.table-bordered>thead>tr>td:first-child,.panel>.table-bordered>thead>tr>th:first-child,.panel>.table-responsive>.table-bordered>tbody>tr>td:first-child,.panel>.table-responsive>.table-bordered>tbody>tr>th:first-child,.panel>.table-responsive>.table-bordered>tfoot>tr>td:first-child,.panel>.table-responsive>.table-bordered>tfoot>tr>th:first-child,.panel>.table-responsive>.table-bordered>thead>tr>td:first-child,.panel>.table-responsive>.table-bordered>thead>tr>th:first-child{border-left:0}.panel>.table-bordered>tbody>tr>td:last-child,.panel>.table-bordered>tbody>tr>th:last-child,.panel>.table-bordered>tfoot>tr>td:last-child,.panel>.table-bordered>tfoot>tr>th:last-child,.panel>.table-bordered>thead>tr>td:last-child,.panel>.table-bordered>thead>tr>th:last-child,.panel>.table-responsive>.table-bordered>tbody>tr>td:last-child,.panel>.table-responsive>.table-bordered>tbody>tr>th:last-child,.panel>.table-responsive>.table-bordered>tfoot>tr>td:last-child,.panel>.table-responsive>.table-bordered>tfoot>tr>th:last-child,.panel>.table-responsive>.table-bordered>thead>tr>td:last-child,.panel>.table-responsive>.table-bordered>thead>tr>th:last-child{border-right:0}.panel>.table-bordered>tbody>tr:first-child>td,.panel>.table-bordered>tbody>tr:first-child>th,.panel>.table-bordered>thead>tr:first-child>td,.panel>.table-bordered>thead>tr:first-child>th,.panel>.table-responsive>.table-bordered>tbody>tr:first-child>td,.panel>.table-responsive>.table-bordered>tbody>tr:first-child>th,.panel>.table-responsive>.table-bordered>thead>tr:first-child>td,.panel>.table-responsive>.table-bordered>thead>tr:first-child>th{border-bottom:0}.panel>.table-bordered>tbody>tr:last-child>td,.panel>.table-bordered>tbody>tr:last-child>th,.panel>.table-bordered>tfoot>tr:last-child>td,.panel>.table-bordered>tfoot>tr:last-child>th,.panel>.table-responsive>.table-bordered>tbody>tr:last-child>td,.panel>.table-responsive>.table-bordered>tbody>tr:last-child>th,.panel>.table-responsive>.table-bordered>tfoot>tr:last-child>td,.panel>.table-responsive>.table-bordered>tfoot>tr:last-child>th{border-bottom:0}.panel>.table-responsive{margin-bottom:0;border:0}.panel-group{margin-bottom:20px}.panel-group .panel{margin-bottom:0;border-radius:4px}.panel-group .panel+.panel{margin-top:5px}.panel-group .panel-heading{border-bottom:0}.panel-group .panel-heading+.panel-collapse>.list-group,.panel-group .panel-heading+.panel-collapse>.panel-body{border-top:1px solid #ddd}.panel-group .panel-footer{border-top:0}.panel-group .panel-footer+.panel-collapse .panel-body{border-bottom:1px solid #ddd}.panel-default{border-color:#ddd}.panel-default>.panel-heading{color:#333;background-color:#f5f5f5;border-color:#ddd}.panel-default>.panel-heading+.panel-collapse>.panel-body{border-top-color:#ddd}.panel-default>.panel-heading .badge{color:#f5f5f5;background-color:#333}.panel-default>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#ddd}.panel-primary{border-color:#337ab7}.panel-primary>.panel-heading{color:#fff;background-color:#337ab7;border-color:#337ab7}.panel-primary>.panel-heading+.panel-collapse>.panel-body{border-top-color:#337ab7}.panel-primary>.panel-heading .badge{color:#337ab7;background-color:#fff}.panel-primary>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#337ab7}.panel-success{border-color:#d6e9c6}.panel-success>.panel-heading{color:#3c763d;background-color:#dff0d8;border-color:#d6e9c6}.panel-success>.panel-heading+.panel-collapse>.panel-body{border-top-color:#d6e9c6}.panel-success>.panel-heading .badge{color:#dff0d8;background-color:#3c763d}.panel-success>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#d6e9c6}.panel-info{border-color:#bce8f1}.panel-info>.panel-heading{color:#31708f;background-color:#d9edf7;border-color:#bce8f1}.panel-info>.panel-heading+.panel-collapse>.panel-body{border-top-color:#bce8f1}.panel-info>.panel-heading .badge{color:#d9edf7;background-color:#31708f}.panel-info>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#bce8f1}.panel-warning{border-color:#faebcc}.panel-warning>.panel-heading{color:#8a6d3b;background-color:#fcf8e3;border-color:#faebcc}.panel-warning>.panel-heading+.panel-collapse>.panel-body{border-top-color:#faebcc}.panel-warning>.panel-heading .badge{color:#fcf8e3;background-color:#8a6d3b}.panel-warning>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#faebcc}.panel-danger{border-color:#ebccd1}.panel-danger>.panel-heading{color:#a94442;background-color:#f2dede;border-color:#ebccd1}.panel-danger>.panel-heading+.panel-collapse>.panel-body{border-top-color:#ebccd1}.panel-danger>.panel-heading .badge{color:#f2dede;background-color:#a94442}.panel-danger>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#ebccd1}.embed-responsive{position:relative;display:block;height:0;padding:0;overflow:hidden}.embed-responsive .embed-responsive-item,.embed-responsive embed,.embed-responsive iframe,.embed-responsive object,.embed-responsive video{position:absolute;top:0;bottom:0;left:0;width:100%;height:100%;border:0}.embed-responsive.embed-responsive-16by9{padding-bottom:56.25%}.embed-responsive.embed-responsive-4by3{padding-bottom:75%}.well{min-height:20px;padding:19px;margin-bottom:20px;background-color:#f5f5f5;border:1px solid #e3e3e3;border-radius:4px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.05);box-shadow:inset 0 1px 1px rgba(0,0,0,.05)}.well blockquote{border-color:#ddd;border-color:rgba(0,0,0,.15)}.well-lg{padding:24px;border-radius:6px}.well-sm{padding:9px;border-radius:3px}.close{float:right;font-size:21px;font-weight:700;line-height:1;color:#000;text-shadow:0 1px 0 #fff;filter:alpha(opacity=20);opacity:.2}.close:focus,.close:hover{color:#000;text-decoration:none;cursor:pointer;filter:alpha(opacity=50);opacity:.5}button.close{-webkit-appearance:none;padding:0;cursor:pointer;background:0 0;border:0}.modal-open{overflow:hidden}.modal{position:fixed;top:0;right:0;bottom:0;left:0;z-index:1040;display:none;overflow:hidden;-webkit-overflow-scrolling:touch;outline:0}.modal.fade .modal-dialog{-webkit-transition:-webkit-transform .3s ease-out;-o-transition:-o-transform .3s ease-out;transition:transform .3s ease-out;-webkit-transform:translate(0,-25%);-ms-transform:translate(0,-25%);-o-transform:translate(0,-25%);transform:translate(0,-25%)}.modal.in .modal-dialog{-webkit-transform:translate(0,0);-ms-transform:translate(0,0);-o-transform:translate(0,0);transform:translate(0,0)}.modal-open .modal{overflow-x:hidden;overflow-y:auto}.modal-dialog{position:relative;width:auto;margin:10px}.modal-content{position:relative;background-color:#fff;-webkit-background-clip:padding-box;background-clip:padding-box;border:1px solid #999;border:1px solid rgba(0,0,0,.2);border-radius:6px;outline:0;-webkit-box-shadow:0 3px 9px rgba(0,0,0,.5);box-shadow:0 3px 9px rgba(0,0,0,.5)}.modal-backdrop{position:absolute;top:0;right:0;left:0;background-color:#000}.modal-backdrop.fade{filter:alpha(opacity=0);opacity:0}.modal-backdrop.in{filter:alpha(opacity=50);opacity:.5}.modal-header{min-height:16.43px;padding:15px;border-bottom:1px solid #e5e5e5}.modal-header .close{margin-top:-2px}.modal-title{margin:0;line-height:1.42857143}.modal-body{position:relative;padding:15px}.modal-footer{padding:15px;text-align:right;border-top:1px solid #e5e5e5}.modal-footer .btn+.btn{margin-bottom:0;margin-left:5px}.modal-footer .btn-group .btn+.btn{margin-left:-1px}.modal-footer .btn-block+.btn-block{margin-left:0}.modal-scrollbar-measure{position:absolute;top:-9999px;width:50px;height:50px;overflow:scroll}@media (min-width:768px){.modal-dialog{width:600px;margin:30px auto}.modal-content{-webkit-box-shadow:0 5px 15px rgba(0,0,0,.5);box-shadow:0 5px 15px rgba(0,0,0,.5)}.modal-sm{width:300px}}@media (min-width:992px){.modal-lg{width:900px}}.tooltip{position:absolute;z-index:1070;display:block;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:12px;font-weight:400;line-height:1.4;visibility:visible;filter:alpha(opacity=0);opacity:0}.tooltip.in{filter:alpha(opacity=90);opacity:.9}.tooltip.top{padding:5px 0;margin-top:-3px}.tooltip.right{padding:0 5px;margin-left:3px}.tooltip.bottom{padding:5px 0;margin-top:3px}.tooltip.left{padding:0 5px;margin-left:-3px}.tooltip-inner{max-width:200px;padding:3px 8px;color:#fff;text-align:center;text-decoration:none;background-color:#000;border-radius:4px}.tooltip-arrow{position:absolute;width:0;height:0;border-color:transparent;border-style:solid}.tooltip.top .tooltip-arrow{bottom:0;left:50%;margin-left:-5px;border-width:5px 5px 0;border-top-color:#000}.tooltip.top-left .tooltip-arrow{right:5px;bottom:0;margin-bottom:-5px;border-width:5px 5px 0;border-top-color:#000}.tooltip.top-right .tooltip-arrow{bottom:0;left:5px;margin-bottom:-5px;border-width:5px 5px 0;border-top-color:#000}.tooltip.right .tooltip-arrow{top:50%;left:0;margin-top:-5px;border-width:5px 5px 5px 0;border-right-color:#000}.tooltip.left .tooltip-arrow{top:50%;right:0;margin-top:-5px;border-width:5px 0 5px 5px;border-left-color:#000}.tooltip.bottom .tooltip-arrow{top:0;left:50%;margin-left:-5px;border-width:0 5px 5px;border-bottom-color:#000}.tooltip.bottom-left .tooltip-arrow{top:0;right:5px;margin-top:-5px;border-width:0 5px 5px;border-bottom-color:#000}.tooltip.bottom-right .tooltip-arrow{top:0;left:5px;margin-top:-5px;border-width:0 5px 5px;border-bottom-color:#000}.popover{position:absolute;top:0;left:0;z-index:1060;display:none;max-width:276px;padding:1px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:14px;font-weight:400;line-height:1.42857143;text-align:left;white-space:normal;background-color:#fff;-webkit-background-clip:padding-box;background-clip:padding-box;border:1px solid #ccc;border:1px solid rgba(0,0,0,.2);border-radius:6px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,.2);box-shadow:0 5px 10px rgba(0,0,0,.2)}.popover.top{margin-top:-10px}.popover.right{margin-left:10px}.popover.bottom{margin-top:10px}.popover.left{margin-left:-10px}.popover-title{padding:8px 14px;margin:0;font-size:14px;background-color:#f7f7f7;border-bottom:1px solid #ebebeb;border-radius:5px 5px 0 0}.popover-content{padding:9px 14px}.popover>.arrow,.popover>.arrow:after{position:absolute;display:block;width:0;height:0;border-color:transparent;border-style:solid}.popover>.arrow{border-width:11px}.popover>.arrow:after{content:"";border-width:10px}.popover.top>.arrow{bottom:-11px;left:50%;margin-left:-11px;border-top-color:#999;border-top-color:rgba(0,0,0,.25);border-bottom-width:0}.popover.top>.arrow:after{bottom:1px;margin-left:-10px;content:" ";border-top-color:#fff;border-bottom-width:0}.popover.right>.arrow{top:50%;left:-11px;margin-top:-11px;border-right-color:#999;border-right-color:rgba(0,0,0,.25);border-left-width:0}.popover.right>.arrow:after{bottom:-10px;left:1px;content:" ";border-right-color:#fff;border-left-width:0}.popover.bottom>.arrow{top:-11px;left:50%;margin-left:-11px;border-top-width:0;border-bottom-color:#999;border-bottom-color:rgba(0,0,0,.25)}.popover.bottom>.arrow:after{top:1px;margin-left:-10px;content:" ";border-top-width:0;border-bottom-color:#fff}.popover.left>.arrow{top:50%;right:-11px;margin-top:-11px;border-right-width:0;border-left-color:#999;border-left-color:rgba(0,0,0,.25)}.popover.left>.arrow:after{right:1px;bottom:-10px;content:" ";border-right-width:0;border-left-color:#fff}.carousel{position:relative}.carousel-inner{position:relative;width:100%;overflow:hidden}.carousel-inner>.item{position:relative;display:none;-webkit-transition:.6s ease-in-out left;-o-transition:.6s ease-in-out left;transition:.6s ease-in-out left}.carousel-inner>.item>a>img,.carousel-inner>.item>img{line-height:1}@media all and (transform-3d),(-webkit-transform-3d){.carousel-inner>.item{-webkit-transition:-webkit-transform .6s ease-in-out;-o-transition:-o-transform .6s ease-in-out;transition:transform .6s ease-in-out;-webkit-backface-visibility:hidden;backface-visibility:hidden;-webkit-perspective:1000;perspective:1000}.carousel-inner>.item.active.right,.carousel-inner>.item.next{left:0;-webkit-transform:translate3d(100%,0,0);transform:translate3d(100%,0,0)}.carousel-inner>.item.active.left,.carousel-inner>.item.prev{left:0;-webkit-transform:translate3d(-100%,0,0);transform:translate3d(-100%,0,0)}.carousel-inner>.item.active,.carousel-inner>.item.next.left,.carousel-inner>.item.prev.right{left:0;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0)}}.carousel-inner>.active,.carousel-inner>.next,.carousel-inner>.prev{display:block}.carousel-inner>.active{left:0}.carousel-inner>.next,.carousel-inner>.prev{position:absolute;top:0;width:100%}.carousel-inner>.next{left:100%}.carousel-inner>.prev{left:-100%}.carousel-inner>.next.left,.carousel-inner>.prev.right{left:0}.carousel-inner>.active.left{left:-100%}.carousel-inner>.active.right{left:100%}.carousel-control{position:absolute;top:0;bottom:0;left:0;width:15%;font-size:20px;color:#fff;text-align:center;text-shadow:0 1px 2px rgba(0,0,0,.6);filter:alpha(opacity=50);opacity:.5}.carousel-control.left{background-image:-webkit-linear-gradient(left,rgba(0,0,0,.5) 0,rgba(0,0,0,.0001) 100%);background-image:-o-linear-gradient(left,rgba(0,0,0,.5) 0,rgba(0,0,0,.0001) 100%);background-image:-webkit-gradient(linear,left top,right top,from(rgba(0,0,0,.5)),to(rgba(0,0,0,.0001)));background-image:linear-gradient(to right,rgba(0,0,0,.5) 0,rgba(0,0,0,.0001) 100%);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#80000000', endColorstr='#00000000', GradientType=1);background-repeat:repeat-x}.carousel-control.right{right:0;left:auto;background-image:-webkit-linear-gradient(left,rgba(0,0,0,.0001) 0,rgba(0,0,0,.5) 100%);background-image:-o-linear-gradient(left,rgba(0,0,0,.0001) 0,rgba(0,0,0,.5) 100%);background-image:-webkit-gradient(linear,left top,right top,from(rgba(0,0,0,.0001)),to(rgba(0,0,0,.5)));background-image:linear-gradient(to right,rgba(0,0,0,.0001) 0,rgba(0,0,0,.5) 100%);filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#00000000', endColorstr='#80000000', GradientType=1);background-repeat:repeat-x}.carousel-control:focus,.carousel-control:hover{color:#fff;text-decoration:none;filter:alpha(opacity=90);outline:0;opacity:.9}.carousel-control .glyphicon-chevron-left,.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next,.carousel-control .icon-prev{position:absolute;top:50%;z-index:5;display:inline-block}.carousel-control .glyphicon-chevron-left,.carousel-control .icon-prev{left:50%;margin-left:-10px}.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next{right:50%;margin-right:-10px}.carousel-control .icon-next,.carousel-control .icon-prev{width:20px;height:20px;margin-top:-10px;font-family:serif;line-height:1}.carousel-control .icon-prev:before{content:'\2039'}.carousel-control .icon-next:before{content:'\203a'}.carousel-indicators{position:absolute;bottom:10px;left:50%;z-index:15;width:60%;padding-left:0;margin-left:-30%;text-align:center;list-style:none}.carousel-indicators li{display:inline-block;width:10px;height:10px;margin:1px;text-indent:-999px;cursor:pointer;background-color:#000 \9;background-color:rgba(0,0,0,0);border:1px solid #fff;border-radius:10px}.carousel-indicators .active{width:12px;height:12px;margin:0;background-color:#fff}.carousel-caption{position:absolute;right:15%;bottom:20px;left:15%;z-index:10;padding-top:20px;padding-bottom:20px;color:#fff;text-align:center;text-shadow:0 1px 2px rgba(0,0,0,.6)}.carousel-caption .btn{text-shadow:none}@media screen and (min-width:768px){.carousel-control .glyphicon-chevron-left,.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next,.carousel-control .icon-prev{width:30px;height:30px;margin-top:-15px;font-size:30px}.carousel-control .glyphicon-chevron-left,.carousel-control .icon-prev{margin-left:-15px}.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next{margin-right:-15px}.carousel-caption{right:20%;left:20%;padding-bottom:30px}.carousel-indicators{bottom:20px}}.btn-group-vertical>.btn-group:after,.btn-group-vertical>.btn-group:before,.btn-toolbar:after,.btn-toolbar:before,.clearfix:after,.clearfix:before,.container-fluid:after,.container-fluid:before,.container:after,.container:before,.dl-horizontal dd:after,.dl-horizontal dd:before,.form-horizontal .form-group:after,.form-horizontal .form-group:before,.modal-footer:after,.modal-footer:before,.nav:after,.nav:before,.navbar-collapse:after,.navbar-collapse:before,.navbar-header:after,.navbar-header:before,.navbar:after,.navbar:before,.pager:after,.pager:before,.panel-body:after,.panel-body:before,.row:after,.row:before{display:table;content:" "}.btn-group-vertical>.btn-group:after,.btn-toolbar:after,.clearfix:after,.container-fluid:after,.container:after,.dl-horizontal dd:after,.form-horizontal .form-group:after,.modal-footer:after,.nav:after,.navbar-collapse:after,.navbar-header:after,.navbar:after,.pager:after,.panel-body:after,.row:after{clear:both}.center-block{display:block;margin-right:auto;margin-left:auto}.pull-right{float:right!important}.pull-left{float:left!important}.hide{display:none!important}.show{display:block!important}.invisible{visibility:hidden}.text-hide{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.hidden{display:none!important;visibility:hidden!important}.affix{position:fixed}@-ms-viewport{width:device-width}.visible-lg,.visible-md,.visible-sm,.visible-xs{display:none!important}.visible-lg-block,.visible-lg-inline,.visible-lg-inline-block,.visible-md-block,.visible-md-inline,.visible-md-inline-block,.visible-sm-block,.visible-sm-inline,.visible-sm-inline-block,.visible-xs-block,.visible-xs-inline,.visible-xs-inline-block{display:none!important}@media (max-width:767px){.visible-xs{display:block!important}table.visible-xs{display:table}tr.visible-xs{display:table-row!important}td.visible-xs,th.visible-xs{display:table-cell!important}}@media (max-width:767px){.visible-xs-block{display:block!important}}@media (max-width:767px){.visible-xs-inline{display:inline!important}}@media (max-width:767px){.visible-xs-inline-block{display:inline-block!important}}@media (min-width:768px) and (max-width:991px){.visible-sm{display:block!important}table.visible-sm{display:table}tr.visible-sm{display:table-row!important}td.visible-sm,th.visible-sm{display:table-cell!important}}@media (min-width:768px) and (max-width:991px){.visible-sm-block{display:block!important}}@media (min-width:768px) and (max-width:991px){.visible-sm-inline{display:inline!important}}@media (min-width:768px) and (max-width:991px){.visible-sm-inline-block{display:inline-block!important}}@media (min-width:992px) and (max-width:1199px){.visible-md{display:block!important}table.visible-md{display:table}tr.visible-md{display:table-row!important}td.visible-md,th.visible-md{display:table-cell!important}}@media (min-width:992px) and (max-width:1199px){.visible-md-block{display:block!important}}@media (min-width:992px) and (max-width:1199px){.visible-md-inline{display:inline!important}}@media (min-width:992px) and (max-width:1199px){.visible-md-inline-block{display:inline-block!important}}@media (min-width:1200px){.visible-lg{display:block!important}table.visible-lg{display:table}tr.visible-lg{display:table-row!important}td.visible-lg,th.visible-lg{display:table-cell!important}}@media (min-width:1200px){.visible-lg-block{display:block!important}}@media (min-width:1200px){.visible-lg-inline{display:inline!important}}@media (min-width:1200px){.visible-lg-inline-block{display:inline-block!important}}@media (max-width:767px){.hidden-xs{display:none!important}}@media (min-width:768px) and (max-width:991px){.hidden-sm{display:none!important}}@media (min-width:992px) and (max-width:1199px){.hidden-md{display:none!important}}@media (min-width:1200px){.hidden-lg{display:none!important}}.visible-print{display:none!important}@media print{.visible-print{display:block!important}table.visible-print{display:table}tr.visible-print{display:table-row!important}td.visible-print,th.visible-print{display:table-cell!important}}.visible-print-block{display:none!important}@media print{.visible-print-block{display:block!important}}.visible-print-inline{display:none!important}@media print{.visible-print-inline{display:inline!important}}.visible-print-inline-block{display:none!important}@media print{.visible-print-inline-block{display:inline-block!important}}@media print{.hidden-print{display:none!important}}
\ No newline at end of file
diff --git a/twitter/mysos/scheduler/assets/static/bootstrap.min.js b/twitter/mysos/scheduler/assets/static/bootstrap.min.js
new file mode 100644
index 0000000..c6d3692
--- /dev/null
+++ b/twitter/mysos/scheduler/assets/static/bootstrap.min.js
@@ -0,0 +1,7 @@
+/*!
+ * Bootstrap v3.3.2 (http://getbootstrap.com)
+ * Copyright 2011-2015 Twitter, Inc.
+ * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
+ */
+if("undefined"==typeof jQuery)throw new Error("Bootstrap's JavaScript requires jQuery");+function(a){"use strict";var b=a.fn.jquery.split(" ")[0].split(".");if(b[0]<2&&b[1]<9||1==b[0]&&9==b[1]&&b[2]<1)throw new Error("Bootstrap's JavaScript requires jQuery version 1.9.1 or higher")}(jQuery),+function(a){"use strict";function b(){var a=document.createElement("bootstrap"),b={WebkitTransition:"webkitTransitionEnd",MozTransition:"transitionend",OTransition:"oTransitionEnd otransitionend",transition:"transitionend"};for(var c in b)if(void 0!==a.style[c])return{end:b[c]};return!1}a.fn.emulateTransitionEnd=function(b){var c=!1,d=this;a(this).one("bsTransitionEnd",function(){c=!0});var e=function(){c||a(d).trigger(a.support.transition.end)};return setTimeout(e,b),this},a(function(){a.support.transition=b(),a.support.transition&&(a.event.special.bsTransitionEnd={bindType:a.support.transition.end,delegateType:a.support.transition.end,handle:function(b){return a(b.target).is(this)?b.handleObj.handler.apply(this,arguments):void 0}})})}(jQuery),+function(a){"use strict";function b(b){return this.each(function(){var c=a(this),e=c.data("bs.alert");e||c.data("bs.alert",e=new d(this)),"string"==typeof b&&e[b].call(c)})}var c='[data-dismiss="alert"]',d=function(b){a(b).on("click",c,this.close)};d.VERSION="3.3.2",d.TRANSITION_DURATION=150,d.prototype.close=function(b){function c(){g.detach().trigger("closed.bs.alert").remove()}var e=a(this),f=e.attr("data-target");f||(f=e.attr("href"),f=f&&f.replace(/.*(?=#[^\s]*$)/,""));var g=a(f);b&&b.preventDefault(),g.length||(g=e.closest(".alert")),g.trigger(b=a.Event("close.bs.alert")),b.isDefaultPrevented()||(g.removeClass("in"),a.support.transition&&g.hasClass("fade")?g.one("bsTransitionEnd",c).emulateTransitionEnd(d.TRANSITION_DURATION):c())};var e=a.fn.alert;a.fn.alert=b,a.fn.alert.Constructor=d,a.fn.alert.noConflict=function(){return a.fn.alert=e,this},a(document).on("click.bs.alert.data-api",c,d.prototype.close)}(jQuery),+function(a){"use strict";function b(b){return this.each(function(){var d=a(this),e=d.data("bs.button"),f="object"==typeof b&&b;e||d.data("bs.button",e=new c(this,f)),"toggle"==b?e.toggle():b&&e.setState(b)})}var c=function(b,d){this.$element=a(b),this.options=a.extend({},c.DEFAULTS,d),this.isLoading=!1};c.VERSION="3.3.2",c.DEFAULTS={loadingText:"loading..."},c.prototype.setState=function(b){var c="disabled",d=this.$element,e=d.is("input")?"val":"html",f=d.data();b+="Text",null==f.resetText&&d.data("resetText",d[e]()),setTimeout(a.proxy(function(){d[e](null==f[b]?this.options[b]:f[b]),"loadingText"==b?(this.isLoading=!0,d.addClass(c).attr(c,c)):this.isLoading&&(this.isLoading=!1,d.removeClass(c).removeAttr(c))},this),0)},c.prototype.toggle=function(){var a=!0,b=this.$element.closest('[data-toggle="buttons"]');if(b.length){var c=this.$element.find("input");"radio"==c.prop("type")&&(c.prop("checked")&&this.$element.hasClass("active")?a=!1:b.find(".active").removeClass("active")),a&&c.prop("checked",!this.$element.hasClass("active")).trigger("change")}else this.$element.attr("aria-pressed",!this.$element.hasClass("active"));a&&this.$element.toggleClass("active")};var d=a.fn.button;a.fn.button=b,a.fn.button.Constructor=c,a.fn.button.noConflict=function(){return a.fn.button=d,this},a(document).on("click.bs.button.data-api",'[data-toggle^="button"]',function(c){var d=a(c.target);d.hasClass("btn")||(d=d.closest(".btn")),b.call(d,"toggle"),c.preventDefault()}).on("focus.bs.button.data-api blur.bs.button.data-api",'[data-toggle^="button"]',function(b){a(b.target).closest(".btn").toggleClass("focus",/^focus(in)?$/.test(b.type))})}(jQuery),+function(a){"use strict";function b(b){return this.each(function(){var d=a(this),e=d.data("bs.carousel"),f=a.extend({},c.DEFAULTS,d.data(),"object"==typeof b&&b),g="string"==typeof b?b:f.slide;e||d.data("bs.carousel",e=new c(this,f)),"number"==typeof b?e.to(b):g?e[g]():f.interval&&e.pause().cycle()})}var c=function(b,c){this.$element=a(b),this.$indicators=this.$element.find(".carousel-indicators"),this.options=c,this.paused=this.sliding=this.interval=this.$active=this.$items=null,this.options.keyboard&&this.$element.on("keydown.bs.carousel",a.proxy(this.keydown,this)),"hover"==this.options.pause&&!("ontouchstart"in document.documentElement)&&this.$element.on("mouseenter.bs.carousel",a.proxy(this.pause,this)).on("mouseleave.bs.carousel",a.proxy(this.cycle,this))};c.VERSION="3.3.2",c.TRANSITION_DURATION=600,c.DEFAULTS={interval:5e3,pause:"hover",wrap:!0,keyboard:!0},c.prototype.keydown=function(a){if(!/input|textarea/i.test(a.target.tagName)){switch(a.which){case 37:this.prev();break;case 39:this.next();break;default:return}a.preventDefault()}},c.prototype.cycle=function(b){return b||(this.paused=!1),this.interval&&clearInterval(this.interval),this.options.interval&&!this.paused&&(this.interval=setInterval(a.proxy(this.next,this),this.options.interval)),this},c.prototype.getItemIndex=function(a){return this.$items=a.parent().children(".item"),this.$items.index(a||this.$active)},c.prototype.getItemForDirection=function(a,b){var c=this.getItemIndex(b),d="prev"==a&&0===c||"next"==a&&c==this.$items.length-1;if(d&&!this.options.wrap)return b;var e="prev"==a?-1:1,f=(c+e)%this.$items.length;return this.$items.eq(f)},c.prototype.to=function(a){var b=this,c=this.getItemIndex(this.$active=this.$element.find(".item.active"));return a>this.$items.length-1||0>a?void 0:this.sliding?this.$element.one("slid.bs.carousel",function(){b.to(a)}):c==a?this.pause().cycle():this.slide(a>c?"next":"prev",this.$items.eq(a))},c.prototype.pause=function(b){return b||(this.paused=!0),this.$element.find(".next, .prev").length&&a.support.transition&&(this.$element.trigger(a.support.transition.end),this.cycle(!0)),this.interval=clearInterval(this.interval),this},c.prototype.next=function(){return this.sliding?void 0:this.slide("next")},c.prototype.prev=function(){return this.sliding?void 0:this.slide("prev")},c.prototype.slide=function(b,d){var e=this.$element.find(".item.active"),f=d||this.getItemForDirection(b,e),g=this.interval,h="next"==b?"left":"right",i=this;if(f.hasClass("active"))return this.sliding=!1;var j=f[0],k=a.Event("slide.bs.carousel",{relatedTarget:j,direction:h});if(this.$element.trigger(k),!k.isDefaultPrevented()){if(this.sliding=!0,g&&this.pause(),this.$indicators.length){this.$indicators.find(".active").removeClass("active");var l=a(this.$indicators.children()[this.getItemIndex(f)]);l&&l.addClass("active")}var m=a.Event("slid.bs.carousel",{relatedTarget:j,direction:h});return a.support.transition&&this.$element.hasClass("slide")?(f.addClass(b),f[0].offsetWidth,e.addClass(h),f.addClass(h),e.one("bsTransitionEnd",function(){f.removeClass([b,h].join(" ")).addClass("active"),e.removeClass(["active",h].join(" ")),i.sliding=!1,setTimeout(function(){i.$element.trigger(m)},0)}).emulateTransitionEnd(c.TRANSITION_DURATION)):(e.removeClass("active"),f.addClass("active"),this.sliding=!1,this.$element.trigger(m)),g&&this.cycle(),this}};var d=a.fn.carousel;a.fn.carousel=b,a.fn.carousel.Constructor=c,a.fn.carousel.noConflict=function(){return a.fn.carousel=d,this};var e=function(c){var d,e=a(this),f=a(e.attr("data-target")||(d=e.attr("href"))&&d.replace(/.*(?=#[^\s]+$)/,""));if(f.hasClass("carousel")){var g=a.extend({},f.data(),e.data()),h=e.attr("data-slide-to");h&&(g.interval=!1),b.call(f,g),h&&f.data("bs.carousel").to(h),c.preventDefault()}};a(document).on("click.bs.carousel.data-api","[data-slide]",e).on("click.bs.carousel.data-api","[data-slide-to]",e),a(window).on("load",function(){a('[data-ride="carousel"]').each(function(){var c=a(this);b.call(c,c.data())})})}(jQuery),+function(a){"use strict";function b(b){var c,d=b.attr("data-target")||(c=b.attr("href"))&&c.replace(/.*(?=#[^\s]+$)/,"");return a(d)}function c(b){return this.each(function(){var c=a(this),e=c.data("bs.collapse"),f=a.extend({},d.DEFAULTS,c.data(),"object"==typeof b&&b);!e&&f.toggle&&"show"==b&&(f.toggle=!1),e||c.data("bs.collapse",e=new d(this,f)),"string"==typeof b&&e[b]()})}var d=function(b,c){this.$element=a(b),this.options=a.extend({},d.DEFAULTS,c),this.$trigger=a(this.options.trigger).filter('[href="#'+b.id+'"], [data-target="#'+b.id+'"]'),this.transitioning=null,this.options.parent?this.$parent=this.getParent():this.addAriaAndCollapsedClass(this.$element,this.$trigger),this.options.toggle&&this.toggle()};d.VERSION="3.3.2",d.TRANSITION_DURATION=350,d.DEFAULTS={toggle:!0,trigger:'[data-toggle="collapse"]'},d.prototype.dimension=function(){var a=this.$element.hasClass("width");return a?"width":"height"},d.prototype.show=function(){if(!this.transitioning&&!this.$element.hasClass("in")){var b,e=this.$parent&&this.$parent.children(".panel").children(".in, .collapsing");if(!(e&&e.length&&(b=e.data("bs.collapse"),b&&b.transitioning))){var f=a.Event("show.bs.collapse");if(this.$element.trigger(f),!f.isDefaultPrevented()){e&&e.length&&(c.call(e,"hide"),b||e.data("bs.collapse",null));var g=this.dimension();this.$element.removeClass("collapse").addClass("collapsing")[g](0).attr("aria-expanded",!0),this.$trigger.removeClass("collapsed").attr("aria-expanded",!0),this.transitioning=1;var h=function(){this.$element.removeClass("collapsing").addClass("collapse in")[g](""),this.transitioning=0,this.$element.trigger("shown.bs.collapse")};if(!a.support.transition)return h.call(this);var i=a.camelCase(["scroll",g].join("-"));this.$element.one("bsTransitionEnd",a.proxy(h,this)).emulateTransitionEnd(d.TRANSITION_DURATION)[g](this.$element[0][i])}}}},d.prototype.hide=function(){if(!this.transitioning&&this.$element.hasClass("in")){var b=a.Event("hide.bs.collapse");if(this.$element.trigger(b),!b.isDefaultPrevented()){var c=this.dimension();this.$element[c](this.$element[c]())[0].offsetHeight,this.$element.addClass("collapsing").removeClass("collapse in").attr("aria-expanded",!1),this.$trigger.addClass("collapsed").attr("aria-expanded",!1),this.transitioning=1;var e=function(){this.transitioning=0,this.$element.removeClass("collapsing").addClass("collapse").trigger("hidden.bs.collapse")};return a.support.transition?void this.$element[c](0).one("bsTransitionEnd",a.proxy(e,this)).emulateTransitionEnd(d.TRANSITION_DURATION):e.call(this)}}},d.prototype.toggle=function(){this[this.$element.hasClass("in")?"hide":"show"]()},d.prototype.getParent=function(){return a(this.options.parent).find('[data-toggle="collapse"][data-parent="'+this.options.parent+'"]').each(a.proxy(function(c,d){var e=a(d);this.addAriaAndCollapsedClass(b(e),e)},this)).end()},d.prototype.addAriaAndCollapsedClass=function(a,b){var c=a.hasClass("in");a.attr("aria-expanded",c),b.toggleClass("collapsed",!c).attr("aria-expanded",c)};var e=a.fn.collapse;a.fn.collapse=c,a.fn.collapse.Constructor=d,a.fn.collapse.noConflict=function(){return a.fn.collapse=e,this},a(document).on("click.bs.collapse.data-api",'[data-toggle="collapse"]',function(d){var e=a(this);e.attr("data-target")||d.preventDefault();var f=b(e),g=f.data("bs.collapse"),h=g?"toggle":a.extend({},e.data(),{trigger:this});c.call(f,h)})}(jQuery),+function(a){"use strict";function b(b){b&&3===b.which||(a(e).remove(),a(f).each(function(){var d=a(this),e=c(d),f={relatedTarget:this};e.hasClass("open")&&(e.trigger(b=a.Event("hide.bs.dropdown",f)),b.isDefaultPrevented()||(d.attr("aria-expanded","false"),e.removeClass("open").trigger("hidden.bs.dropdown",f)))}))}function c(b){var c=b.attr("data-target");c||(c=b.attr("href"),c=c&&/#[A-Za-z]/.test(c)&&c.replace(/.*(?=#[^\s]*$)/,""));var d=c&&a(c);return d&&d.length?d:b.parent()}function d(b){return this.each(function(){var c=a(this),d=c.data("bs.dropdown");d||c.data("bs.dropdown",d=new g(this)),"string"==typeof b&&d[b].call(c)})}var e=".dropdown-backdrop",f='[data-toggle="dropdown"]',g=function(b){a(b).on("click.bs.dropdown",this.toggle)};g.VERSION="3.3.2",g.prototype.toggle=function(d){var e=a(this);if(!e.is(".disabled, :disabled")){var f=c(e),g=f.hasClass("open");if(b(),!g){"ontouchstart"in document.documentElement&&!f.closest(".navbar-nav").length&&a('<div class="dropdown-backdrop"/>').insertAfter(a(this)).on("click",b);var h={relatedTarget:this};if(f.trigger(d=a.Event("show.bs.dropdown",h)),d.isDefaultPrevented())return;e.trigger("focus").attr("aria-expanded","true"),f.toggleClass("open").trigger("shown.bs.dropdown",h)}return!1}},g.prototype.keydown=function(b){if(/(38|40|27|32)/.test(b.which)&&!/input|textarea/i.test(b.target.tagName)){var d=a(this);if(b.preventDefault(),b.stopPropagation(),!d.is(".disabled, :disabled")){var e=c(d),g=e.hasClass("open");if(!g&&27!=b.which||g&&27==b.which)return 27==b.which&&e.find(f).trigger("focus"),d.trigger("click");var h=" li:not(.divider):visible a",i=e.find('[role="menu"]'+h+', [role="listbox"]'+h);if(i.length){var j=i.index(b.target);38==b.which&&j>0&&j--,40==b.which&&j<i.length-1&&j++,~j||(j=0),i.eq(j).trigger("focus")}}}};var h=a.fn.dropdown;a.fn.dropdown=d,a.fn.dropdown.Constructor=g,a.fn.dropdown.noConflict=function(){return a.fn.dropdown=h,this},a(document).on("click.bs.dropdown.data-api",b).on("click.bs.dropdown.data-api",".dropdown form",function(a){a.stopPropagation()}).on("click.bs.dropdown.data-api",f,g.prototype.toggle).on("keydown.bs.dropdown.data-api",f,g.prototype.keydown).on("keydown.bs.dropdown.data-api",'[role="menu"]',g.prototype.keydown).on("keydown.bs.dropdown.data-api",'[role="listbox"]',g.prototype.keydown)}(jQuery),+function(a){"use strict";function b(b,d){return this.each(function(){var e=a(this),f=e.data("bs.modal"),g=a.extend({},c.DEFAULTS,e.data(),"object"==typeof b&&b);f||e.data("bs.modal",f=new c(this,g)),"string"==typeof b?f[b](d):g.show&&f.show(d)})}var c=function(b,c){this.options=c,this.$body=a(document.body),this.$element=a(b),this.$backdrop=this.isShown=null,this.scrollbarWidth=0,this.options.remote&&this.$element.find(".modal-content").load(this.options.remote,a.proxy(function(){this.$element.trigger("loaded.bs.modal")},this))};c.VERSION="3.3.2",c.TRANSITION_DURATION=300,c.BACKDROP_TRANSITION_DURATION=150,c.DEFAULTS={backdrop:!0,keyboard:!0,show:!0},c.prototype.toggle=function(a){return this.isShown?this.hide():this.show(a)},c.prototype.show=function(b){var d=this,e=a.Event("show.bs.modal",{relatedTarget:b});this.$element.trigger(e),this.isShown||e.isDefaultPrevented()||(this.isShown=!0,this.checkScrollbar(),this.setScrollbar(),this.$body.addClass("modal-open"),this.escape(),this.resize(),this.$element.on("click.dismiss.bs.modal",'[data-dismiss="modal"]',a.proxy(this.hide,this)),this.backdrop(function(){var e=a.support.transition&&d.$element.hasClass("fade");d.$element.parent().length||d.$element.appendTo(d.$body),d.$element.show().scrollTop(0),d.options.backdrop&&d.adjustBackdrop(),d.adjustDialog(),e&&d.$element[0].offsetWidth,d.$element.addClass("in").attr("aria-hidden",!1),d.enforceFocus();var f=a.Event("shown.bs.modal",{relatedTarget:b});e?d.$element.find(".modal-dialog").one("bsTransitionEnd",function(){d.$element.trigger("focus").trigger(f)}).emulateTransitionEnd(c.TRANSITION_DURATION):d.$element.trigger("focus").trigger(f)}))},c.prototype.hide=function(b){b&&b.preventDefault(),b=a.Event("hide.bs.modal"),this.$element.trigger(b),this.isShown&&!b.isDefaultPrevented()&&(this.isShown=!1,this.escape(),this.resize(),a(document).off("focusin.bs.modal"),this.$element.removeClass("in").attr("aria-hidden",!0).off("click.dismiss.bs.modal"),a.support.transition&&this.$element.hasClass("fade")?this.$element.one("bsTransitionEnd",a.proxy(this.hideModal,this)).emulateTransitionEnd(c.TRANSITION_DURATION):this.hideModal())},c.prototype.enforceFocus=function(){a(document).off("focusin.bs.modal").on("focusin.bs.modal",a.proxy(function(a){this.$element[0]===a.target||this.$element.has(a.target).length||this.$element.trigger("focus")},this))},c.prototype.escape=function(){this.isShown&&this.options.keyboard?this.$element.on("keydown.dismiss.bs.modal",a.proxy(function(a){27==a.which&&this.hide()},this)):this.isShown||this.$element.off("keydown.dismiss.bs.modal")},c.prototype.resize=function(){this.isShown?a(window).on("resize.bs.modal",a.proxy(this.handleUpdate,this)):a(window).off("resize.bs.modal")},c.prototype.hideModal=function(){var a=this;this.$element.hide(),this.backdrop(function(){a.$body.removeClass("modal-open"),a.resetAdjustments(),a.resetScrollbar(),a.$element.trigger("hidden.bs.modal")})},c.prototype.removeBackdrop=function(){this.$backdrop&&this.$backdrop.remove(),this.$backdrop=null},c.prototype.backdrop=function(b){var d=this,e=this.$element.hasClass("fade")?"fade":"";if(this.isShown&&this.options.backdrop){var f=a.support.transition&&e;if(this.$backdrop=a('<div class="modal-backdrop '+e+'" />').prependTo(this.$element).on("click.dismiss.bs.modal",a.proxy(function(a){a.target===a.currentTarget&&("static"==this.options.backdrop?this.$element[0].focus.call(this.$element[0]):this.hide.call(this))},this)),f&&this.$backdrop[0].offsetWidth,this.$backdrop.addClass("in"),!b)return;f?this.$backdrop.one("bsTransitionEnd",b).emulateTransitionEnd(c.BACKDROP_TRANSITION_DURATION):b()}else if(!this.isShown&&this.$backdrop){this.$backdrop.removeClass("in");var g=function(){d.removeBackdrop(),b&&b()};a.support.transition&&this.$element.hasClass("fade")?this.$backdrop.one("bsTransitionEnd",g).emulateTransitionEnd(c.BACKDROP_TRANSITION_DURATION):g()}else b&&b()},c.prototype.handleUpdate=function(){this.options.backdrop&&this.adjustBackdrop(),this.adjustDialog()},c.prototype.adjustBackdrop=function(){this.$backdrop.css("height",0).css("height",this.$element[0].scrollHeight)},c.prototype.adjustDialog=function(){var a=this.$element[0].scrollHeight>document.documentElement.clientHeight;this.$element.css({paddingLeft:!this.bodyIsOverflowing&&a?this.scrollbarWidth:"",paddingRight:this.bodyIsOverflowing&&!a?this.scrollbarWidth:""})},c.prototype.resetAdjustments=function(){this.$element.css({paddingLeft:"",paddingRight:""})},c.prototype.checkScrollbar=function(){this.bodyIsOverflowing=document.body.scrollHeight>document.documentElement.clientHeight,this.scrollbarWidth=this.measureScrollbar()},c.prototype.setScrollbar=function(){var a=parseInt(this.$body.css("padding-right")||0,10);this.bodyIsOverflowing&&this.$body.css("padding-right",a+this.scrollbarWidth)},c.prototype.resetScrollbar=function(){this.$body.css("padding-right","")},c.prototype.measureScrollbar=function(){var a=document.createElement("div");a.className="modal-scrollbar-measure",this.$body.append(a);var b=a.offsetWidth-a.clientWidth;return this.$body[0].removeChild(a),b};var d=a.fn.modal;a.fn.modal=b,a.fn.modal.Constructor=c,a.fn.modal.noConflict=function(){return a.fn.modal=d,this},a(document).on("click.bs.modal.data-api",'[data-toggle="modal"]',function(c){var d=a(this),e=d.attr("href"),f=a(d.attr("data-target")||e&&e.replace(/.*(?=#[^\s]+$)/,"")),g=f.data("bs.modal")?"toggle":a.extend({remote:!/#/.test(e)&&e},f.data(),d.data());d.is("a")&&c.preventDefault(),f.one("show.bs.modal",function(a){a.isDefaultPrevented()||f.one("hidden.bs.modal",function(){d.is(":visible")&&d.trigger("focus")})}),b.call(f,g,this)})}(jQuery),+function(a){"use strict";function b(b){return this.each(function(){var d=a(this),e=d.data("bs.tooltip"),f="object"==typeof b&&b;(e||"destroy"!=b)&&(e||d.data("bs.tooltip",e=new c(this,f)),"string"==typeof b&&e[b]())})}var c=function(a,b){this.type=this.options=this.enabled=this.timeout=this.hoverState=this.$element=null,this.init("tooltip",a,b)};c.VERSION="3.3.2",c.TRANSITION_DURATION=150,c.DEFAULTS={animation:!0,placement:"top",selector:!1,template:'<div class="tooltip" role="tooltip"><div class="tooltip-arrow"></div><div class="tooltip-inner"></div></div>',trigger:"hover focus",title:"",delay:0,html:!1,container:!1,viewport:{selector:"body",padding:0}},c.prototype.init=function(b,c,d){this.enabled=!0,this.type=b,this.$element=a(c),this.options=this.getOptions(d),this.$viewport=this.options.viewport&&a(this.options.viewport.selector||this.options.viewport);for(var e=this.options.trigger.split(" "),f=e.length;f--;){var g=e[f];if("click"==g)this.$element.on("click."+this.type,this.options.selector,a.proxy(this.toggle,this));else if("manual"!=g){var h="hover"==g?"mouseenter":"focusin",i="hover"==g?"mouseleave":"focusout";this.$element.on(h+"."+this.type,this.options.selector,a.proxy(this.enter,this)),this.$element.on(i+"."+this.type,this.options.selector,a.proxy(this.leave,this))}}this.options.selector?this._options=a.extend({},this.options,{trigger:"manual",selector:""}):this.fixTitle()},c.prototype.getDefaults=function(){return c.DEFAULTS},c.prototype.getOptions=function(b){return b=a.extend({},this.getDefaults(),this.$element.data(),b),b.delay&&"number"==typeof b.delay&&(b.delay={show:b.delay,hide:b.delay}),b},c.prototype.getDelegateOptions=function(){var b={},c=this.getDefaults();return this._options&&a.each(this._options,function(a,d){c[a]!=d&&(b[a]=d)}),b},c.prototype.enter=function(b){var c=b instanceof this.constructor?b:a(b.currentTarget).data("bs."+this.type);return c&&c.$tip&&c.$tip.is(":visible")?void(c.hoverState="in"):(c||(c=new this.constructor(b.currentTarget,this.getDelegateOptions()),a(b.currentTarget).data("bs."+this.type,c)),clearTimeout(c.timeout),c.hoverState="in",c.options.delay&&c.options.delay.show?void(c.timeout=setTimeout(function(){"in"==c.hoverState&&c.show()},c.options.delay.show)):c.show())},c.prototype.leave=function(b){var c=b instanceof this.constructor?b:a(b.currentTarget).data("bs."+this.type);return c||(c=new this.constructor(b.currentTarget,this.getDelegateOptions()),a(b.currentTarget).data("bs."+this.type,c)),clearTimeout(c.timeout),c.hoverState="out",c.options.delay&&c.options.delay.hide?void(c.timeout=setTimeout(function(){"out"==c.hoverState&&c.hide()},c.options.delay.hide)):c.hide()},c.prototype.show=function(){var b=a.Event("show.bs."+this.type);if(this.hasContent()&&this.enabled){this.$element.trigger(b);var d=a.contains(this.$element[0].ownerDocument.documentElement,this.$element[0]);if(b.isDefaultPrevented()||!d)return;var e=this,f=this.tip(),g=this.getUID(this.type);this.setContent(),f.attr("id",g),this.$element.attr("aria-describedby",g),this.options.animation&&f.addClass("fade");var h="function"==typeof this.options.placement?this.options.placement.call(this,f[0],this.$element[0]):this.options.placement,i=/\s?auto?\s?/i,j=i.test(h);j&&(h=h.replace(i,"")||"top"),f.detach().css({top:0,left:0,display:"block"}).addClass(h).data("bs."+this.type,this),this.options.container?f.appendTo(this.options.container):f.insertAfter(this.$element);var k=this.getPosition(),l=f[0].offsetWidth,m=f[0].offsetHeight;if(j){var n=h,o=this.options.container?a(this.options.container):this.$element.parent(),p=this.getPosition(o);h="bottom"==h&&k.bottom+m>p.bottom?"top":"top"==h&&k.top-m<p.top?"bottom":"right"==h&&k.right+l>p.width?"left":"left"==h&&k.left-l<p.left?"right":h,f.removeClass(n).addClass(h)}var q=this.getCalculatedOffset(h,k,l,m);this.applyPlacement(q,h);var r=function(){var a=e.hoverState;e.$element.trigger("shown.bs."+e.type),e.hoverState=null,"out"==a&&e.leave(e)};a.support.transition&&this.$tip.hasClass("fade")?f.one("bsTransitionEnd",r).emulateTransitionEnd(c.TRANSITION_DURATION):r()}},c.prototype.applyPlacement=function(b,c){var d=this.tip(),e=d[0].offsetWidth,f=d[0].offsetHeight,g=parseInt(d.css("margin-top"),10),h=parseInt(d.css("margin-left"),10);isNaN(g)&&(g=0),isNaN(h)&&(h=0),b.top=b.top+g,b.left=b.left+h,a.offset.setOffset(d[0],a.extend({using:function(a){d.css({top:Math.round(a.top),left:Math.round(a.left)})}},b),0),d.addClass("in");var i=d[0].offsetWidth,j=d[0].offsetHeight;"top"==c&&j!=f&&(b.top=b.top+f-j);var k=this.getViewportAdjustedDelta(c,b,i,j);k.left?b.left+=k.left:b.top+=k.top;var l=/top|bottom/.test(c),m=l?2*k.left-e+i:2*k.top-f+j,n=l?"offsetWidth":"offsetHeight";d.offset(b),this.replaceArrow(m,d[0][n],l)},c.prototype.replaceArrow=function(a,b,c){this.arrow().css(c?"left":"top",50*(1-a/b)+"%").css(c?"top":"left","")},c.prototype.setContent=function(){var a=this.tip(),b=this.getTitle();a.find(".tooltip-inner")[this.options.html?"html":"text"](b),a.removeClass("fade in top bottom left right")},c.prototype.hide=function(b){function d(){"in"!=e.hoverState&&f.detach(),e.$element.removeAttr("aria-describedby").trigger("hidden.bs."+e.type),b&&b()}var e=this,f=this.tip(),g=a.Event("hide.bs."+this.type);return this.$element.trigger(g),g.isDefaultPrevented()?void 0:(f.removeClass("in"),a.support.transition&&this.$tip.hasClass("fade")?f.one("bsTransitionEnd",d).emulateTransitionEnd(c.TRANSITION_DURATION):d(),this.hoverState=null,this)},c.prototype.fixTitle=function(){var a=this.$element;(a.attr("title")||"string"!=typeof a.attr("data-original-title"))&&a.attr("data-original-title",a.attr("title")||"").attr("title","")},c.prototype.hasContent=function(){return this.getTitle()},c.prototype.getPosition=function(b){b=b||this.$element;var c=b[0],d="BODY"==c.tagName,e=c.getBoundingClientRect();null==e.width&&(e=a.extend({},e,{width:e.right-e.left,height:e.bottom-e.top}));var f=d?{top:0,left:0}:b.offset(),g={scroll:d?document.documentElement.scrollTop||document.body.scrollTop:b.scrollTop()},h=d?{width:a(window).width(),height:a(window).height()}:null;return a.extend({},e,g,h,f)},c.prototype.getCalculatedOffset=function(a,b,c,d){return"bottom"==a?{top:b.top+b.height,left:b.left+b.width/2-c/2}:"top"==a?{top:b.top-d,left:b.left+b.width/2-c/2}:"left"==a?{top:b.top+b.height/2-d/2,left:b.left-c}:{top:b.top+b.height/2-d/2,left:b.left+b.width}},c.prototype.getViewportAdjustedDelta=function(a,b,c,d){var e={top:0,left:0};if(!this.$viewport)return e;var f=this.options.viewport&&this.options.viewport.padding||0,g=this.getPosition(this.$viewport);if(/right|left/.test(a)){var h=b.top-f-g.scroll,i=b.top+f-g.scroll+d;h<g.top?e.top=g.top-h:i>g.top+g.height&&(e.top=g.top+g.height-i)}else{var j=b.left-f,k=b.left+f+c;j<g.left?e.left=g.left-j:k>g.width&&(e.left=g.left+g.width-k)}return e},c.prototype.getTitle=function(){var a,b=this.$element,c=this.options;return a=b.attr("data-original-title")||("function"==typeof c.title?c.title.call(b[0]):c.title)},c.prototype.getUID=function(a){do a+=~~(1e6*Math.random());while(document.getElementById(a));return a},c.prototype.tip=function(){return this.$tip=this.$tip||a(this.options.template)},c.prototype.arrow=function(){return this.$arrow=this.$arrow||this.tip().find(".tooltip-arrow")},c.prototype.enable=function(){this.enabled=!0},c.prototype.disable=function(){this.enabled=!1},c.prototype.toggleEnabled=function(){this.enabled=!this.enabled},c.prototype.toggle=function(b){var c=this;b&&(c=a(b.currentTarget).data("bs."+this.type),c||(c=new this.constructor(b.currentTarget,this.getDelegateOptions()),a(b.currentTarget).data("bs."+this.type,c))),c.tip().hasClass("in")?c.leave(c):c.enter(c)},c.prototype.destroy=function(){var a=this;clearTimeout(this.timeout),this.hide(function(){a.$element.off("."+a.type).removeData("bs."+a.type)})};var d=a.fn.tooltip;a.fn.tooltip=b,a.fn.tooltip.Constructor=c,a.fn.tooltip.noConflict=function(){return a.fn.tooltip=d,this}}(jQuery),+function(a){"use strict";function b(b){return this.each(function(){var d=a(this),e=d.data("bs.popover"),f="object"==typeof b&&b;(e||"destroy"!=b)&&(e||d.data("bs.popover",e=new c(this,f)),"string"==typeof b&&e[b]())})}var c=function(a,b){this.init("popover",a,b)};if(!a.fn.tooltip)throw new Error("Popover requires tooltip.js");c.VERSION="3.3.2",c.DEFAULTS=a.extend({},a.fn.tooltip.Constructor.DEFAULTS,{placement:"right",trigger:"click",content:"",template:'<div class="popover" role="tooltip"><div class="arrow"></div><h3 class="popover-title"></h3><div class="popover-content"></div></div>'}),c.prototype=a.extend({},a.fn.tooltip.Constructor.prototype),c.prototype.constructor=c,c.prototype.getDefaults=function(){return c.DEFAULTS},c.prototype.setContent=function(){var a=this.tip(),b=this.getTitle(),c=this.getContent();a.find(".popover-title")[this.options.html?"html":"text"](b),a.find(".popover-content").children().detach().end()[this.options.html?"string"==typeof c?"html":"append":"text"](c),a.removeClass("fade top bottom left right in"),a.find(".popover-title").html()||a.find(".popover-title").hide()},c.prototype.hasContent=function(){return this.getTitle()||this.getContent()},c.prototype.getContent=function(){var a=this.$element,b=this.options;return a.attr("data-content")||("function"==typeof b.content?b.content.call(a[0]):b.content)},c.prototype.arrow=function(){return this.$arrow=this.$arrow||this.tip().find(".arrow")},c.prototype.tip=function(){return this.$tip||(this.$tip=a(this.options.template)),this.$tip};var d=a.fn.popover;a.fn.popover=b,a.fn.popover.Constructor=c,a.fn.popover.noConflict=function(){return a.fn.popover=d,this}}(jQuery),+function(a){"use strict";function b(c,d){var e=a.proxy(this.process,this);this.$body=a("body"),this.$scrollElement=a(a(c).is("body")?window:c),this.options=a.extend({},b.DEFAULTS,d),this.selector=(this.options.target||"")+" .nav li > a",this.offsets=[],this.targets=[],this.activeTarget=null,this.scrollHeight=0,this.$scrollElement.on("scroll.bs.scrollspy",e),this.refresh(),this.process()}function c(c){return this.each(function(){var d=a(this),e=d.data("bs.scrollspy"),f="object"==typeof c&&c;e||d.data("bs.scrollspy",e=new b(this,f)),"string"==typeof c&&e[c]()})}b.VERSION="3.3.2",b.DEFAULTS={offset:10},b.prototype.getScrollHeight=function(){return this.$scrollElement[0].scrollHeight||Math.max(this.$body[0].scrollHeight,document.documentElement.scrollHeight)},b.prototype.refresh=function(){var b="offset",c=0;a.isWindow(this.$scrollElement[0])||(b="position",c=this.$scrollElement.scrollTop()),this.offsets=[],this.targets=[],this.scrollHeight=this.getScrollHeight();var d=this;this.$body.find(this.selector).map(function(){var d=a(this),e=d.data("target")||d.attr("href"),f=/^#./.test(e)&&a(e);return f&&f.length&&f.is(":visible")&&[[f[b]().top+c,e]]||null}).sort(function(a,b){return a[0]-b[0]}).each(function(){d.offsets.push(this[0]),d.targets.push(this[1])})},b.prototype.process=function(){var a,b=this.$scrollElement.scrollTop()+this.options.offset,c=this.getScrollHeight(),d=this.options.offset+c-this.$scrollElement.height(),e=this.offsets,f=this.targets,g=this.activeTarget;if(this.scrollHeight!=c&&this.refresh(),b>=d)return g!=(a=f[f.length-1])&&this.activate(a);if(g&&b<e[0])return this.activeTarget=null,this.clear();for(a=e.length;a--;)g!=f[a]&&b>=e[a]&&(!e[a+1]||b<=e[a+1])&&this.activate(f[a])},b.prototype.activate=function(b){this.activeTarget=b,this.clear();var c=this.selector+'[data-target="'+b+'"],'+this.selector+'[href="'+b+'"]',d=a(c).parents("li").addClass("active");d.parent(".dropdown-menu").length&&(d=d.closest("li.dropdown").addClass("active")),d.trigger("activate.bs.scrollspy")},b.prototype.clear=function(){a(this.selector).parentsUntil(this.options.target,".active").removeClass("active")};var d=a.fn.scrollspy;a.fn.scrollspy=c,a.fn.scrollspy.Constructor=b,a.fn.scrollspy.noConflict=function(){return a.fn.scrollspy=d,this},a(window).on("load.bs.scrollspy.data-api",function(){a('[data-spy="scroll"]').each(function(){var b=a(this);c.call(b,b.data())})})}(jQuery),+function(a){"use strict";function b(b){return this.each(function(){var d=a(this),e=d.data("bs.tab");e||d.data("bs.tab",e=new c(this)),"string"==typeof b&&e[b]()})}var c=function(b){this.element=a(b)};c.VERSION="3.3.2",c.TRANSITION_DURATION=150,c.prototype.show=function(){var b=this.element,c=b.closest("ul:not(.dropdown-menu)"),d=b.data("target");if(d||(d=b.attr("href"),d=d&&d.replace(/.*(?=#[^\s]*$)/,"")),!b.parent("li").hasClass("active")){var e=c.find(".active:last a"),f=a.Event("hide.bs.tab",{relatedTarget:b[0]}),g=a.Event("show.bs.tab",{relatedTarget:e[0]});if(e.trigger(f),b.trigger(g),!g.isDefaultPrevented()&&!f.isDefaultPrevented()){var h=a(d);this.activate(b.closest("li"),c),this.activate(h,h.parent(),function(){e.trigger({type:"hidden.bs.tab",relatedTarget:b[0]}),b.trigger({type:"shown.bs.tab",relatedTarget:e[0]})})}}},c.prototype.activate=function(b,d,e){function f(){g.removeClass("active").find("> .dropdown-menu > .active").removeClass("active").end().find('[data-toggle="tab"]').attr("aria-expanded",!1),b.addClass("active").find('[data-toggle="tab"]').attr("aria-expanded",!0),h?(b[0].offsetWidth,b.addClass("in")):b.removeClass("fade"),b.parent(".dropdown-menu")&&b.closest("li.dropdown").addClass("active").end().find('[data-toggle="tab"]').attr("aria-expanded",!0),e&&e()
+}var g=d.find("> .active"),h=e&&a.support.transition&&(g.length&&g.hasClass("fade")||!!d.find("> .fade").length);g.length&&h?g.one("bsTransitionEnd",f).emulateTransitionEnd(c.TRANSITION_DURATION):f(),g.removeClass("in")};var d=a.fn.tab;a.fn.tab=b,a.fn.tab.Constructor=c,a.fn.tab.noConflict=function(){return a.fn.tab=d,this};var e=function(c){c.preventDefault(),b.call(a(this),"show")};a(document).on("click.bs.tab.data-api",'[data-toggle="tab"]',e).on("click.bs.tab.data-api",'[data-toggle="pill"]',e)}(jQuery),+function(a){"use strict";function b(b){return this.each(function(){var d=a(this),e=d.data("bs.affix"),f="object"==typeof b&&b;e||d.data("bs.affix",e=new c(this,f)),"string"==typeof b&&e[b]()})}var c=function(b,d){this.options=a.extend({},c.DEFAULTS,d),this.$target=a(this.options.target).on("scroll.bs.affix.data-api",a.proxy(this.checkPosition,this)).on("click.bs.affix.data-api",a.proxy(this.checkPositionWithEventLoop,this)),this.$element=a(b),this.affixed=this.unpin=this.pinnedOffset=null,this.checkPosition()};c.VERSION="3.3.2",c.RESET="affix affix-top affix-bottom",c.DEFAULTS={offset:0,target:window},c.prototype.getState=function(a,b,c,d){var e=this.$target.scrollTop(),f=this.$element.offset(),g=this.$target.height();if(null!=c&&"top"==this.affixed)return c>e?"top":!1;if("bottom"==this.affixed)return null!=c?e+this.unpin<=f.top?!1:"bottom":a-d>=e+g?!1:"bottom";var h=null==this.affixed,i=h?e:f.top,j=h?g:b;return null!=c&&c>=e?"top":null!=d&&i+j>=a-d?"bottom":!1},c.prototype.getPinnedOffset=function(){if(this.pinnedOffset)return this.pinnedOffset;this.$element.removeClass(c.RESET).addClass("affix");var a=this.$target.scrollTop(),b=this.$element.offset();return this.pinnedOffset=b.top-a},c.prototype.checkPositionWithEventLoop=function(){setTimeout(a.proxy(this.checkPosition,this),1)},c.prototype.checkPosition=function(){if(this.$element.is(":visible")){var b=this.$element.height(),d=this.options.offset,e=d.top,f=d.bottom,g=a("body").height();"object"!=typeof d&&(f=e=d),"function"==typeof e&&(e=d.top(this.$element)),"function"==typeof f&&(f=d.bottom(this.$element));var h=this.getState(g,b,e,f);if(this.affixed!=h){null!=this.unpin&&this.$element.css("top","");var i="affix"+(h?"-"+h:""),j=a.Event(i+".bs.affix");if(this.$element.trigger(j),j.isDefaultPrevented())return;this.affixed=h,this.unpin="bottom"==h?this.getPinnedOffset():null,this.$element.removeClass(c.RESET).addClass(i).trigger(i.replace("affix","affixed")+".bs.affix")}"bottom"==h&&this.$element.offset({top:g-b-f})}};var d=a.fn.affix;a.fn.affix=b,a.fn.affix.Constructor=c,a.fn.affix.noConflict=function(){return a.fn.affix=d,this},a(window).on("load",function(){a('[data-spy="affix"]').each(function(){var c=a(this),d=c.data();d.offset=d.offset||{},null!=d.offsetBottom&&(d.offset.bottom=d.offsetBottom),null!=d.offsetTop&&(d.offset.top=d.offsetTop),b.call(c,d)})})}(jQuery);
\ No newline at end of file
diff --git a/twitter/mysos/scheduler/assets/static/jquery.min.js b/twitter/mysos/scheduler/assets/static/jquery.min.js
new file mode 100644
index 0000000..e6a051d
--- /dev/null
+++ b/twitter/mysos/scheduler/assets/static/jquery.min.js
@@ -0,0 +1,4 @@
+/*! jQuery v1.11.2 | (c) 2005, 2014 jQuery Foundation, Inc. | jquery.org/license */
+!function(a,b){"object"==typeof module&&"object"==typeof module.exports?module.exports=a.document?b(a,!0):function(a){if(!a.document)throw new Error("jQuery requires a window with a document");return b(a)}:b(a)}("undefined"!=typeof window?window:this,function(a,b){var c=[],d=c.slice,e=c.concat,f=c.push,g=c.indexOf,h={},i=h.toString,j=h.hasOwnProperty,k={},l="1.11.2",m=function(a,b){return new m.fn.init(a,b)},n=/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,o=/^-ms-/,p=/-([\da-z])/gi,q=function(a,b){return b.toUpperCase()};m.fn=m.prototype={jquery:l,constructor:m,selector:"",length:0,toArray:function(){return d.call(this)},get:function(a){return null!=a?0>a?this[a+this.length]:this[a]:d.call(this)},pushStack:function(a){var b=m.merge(this.constructor(),a);return b.prevObject=this,b.context=this.context,b},each:function(a,b){return m.each(this,a,b)},map:function(a){return this.pushStack(m.map(this,function(b,c){return a.call(b,c,b)}))},slice:function(){return this.pushStack(d.apply(this,arguments))},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},eq:function(a){var b=this.length,c=+a+(0>a?b:0);return this.pushStack(c>=0&&b>c?[this[c]]:[])},end:function(){return this.prevObject||this.constructor(null)},push:f,sort:c.sort,splice:c.splice},m.extend=m.fn.extend=function(){var a,b,c,d,e,f,g=arguments[0]||{},h=1,i=arguments.length,j=!1;for("boolean"==typeof g&&(j=g,g=arguments[h]||{},h++),"object"==typeof g||m.isFunction(g)||(g={}),h===i&&(g=this,h--);i>h;h++)if(null!=(e=arguments[h]))for(d in e)a=g[d],c=e[d],g!==c&&(j&&c&&(m.isPlainObject(c)||(b=m.isArray(c)))?(b?(b=!1,f=a&&m.isArray(a)?a:[]):f=a&&m.isPlainObject(a)?a:{},g[d]=m.extend(j,f,c)):void 0!==c&&(g[d]=c));return g},m.extend({expando:"jQuery"+(l+Math.random()).replace(/\D/g,""),isReady:!0,error:function(a){throw new Error(a)},noop:function(){},isFunction:function(a){return"function"===m.type(a)},isArray:Array.isArray||function(a){return"array"===m.type(a)},isWindow:function(a){return null!=a&&a==a.window},isNumeric:function(a){return!m.isArray(a)&&a-parseFloat(a)+1>=0},isEmptyObject:function(a){var b;for(b in a)return!1;return!0},isPlainObject:function(a){var b;if(!a||"object"!==m.type(a)||a.nodeType||m.isWindow(a))return!1;try{if(a.constructor&&!j.call(a,"constructor")&&!j.call(a.constructor.prototype,"isPrototypeOf"))return!1}catch(c){return!1}if(k.ownLast)for(b in a)return j.call(a,b);for(b in a);return void 0===b||j.call(a,b)},type:function(a){return null==a?a+"":"object"==typeof a||"function"==typeof a?h[i.call(a)]||"object":typeof a},globalEval:function(b){b&&m.trim(b)&&(a.execScript||function(b){a.eval.call(a,b)})(b)},camelCase:function(a){return a.replace(o,"ms-").replace(p,q)},nodeName:function(a,b){return a.nodeName&&a.nodeName.toLowerCase()===b.toLowerCase()},each:function(a,b,c){var d,e=0,f=a.length,g=r(a);if(c){if(g){for(;f>e;e++)if(d=b.apply(a[e],c),d===!1)break}else for(e in a)if(d=b.apply(a[e],c),d===!1)break}else if(g){for(;f>e;e++)if(d=b.call(a[e],e,a[e]),d===!1)break}else for(e in a)if(d=b.call(a[e],e,a[e]),d===!1)break;return a},trim:function(a){return null==a?"":(a+"").replace(n,"")},makeArray:function(a,b){var c=b||[];return null!=a&&(r(Object(a))?m.merge(c,"string"==typeof a?[a]:a):f.call(c,a)),c},inArray:function(a,b,c){var d;if(b){if(g)return g.call(b,a,c);for(d=b.length,c=c?0>c?Math.max(0,d+c):c:0;d>c;c++)if(c in b&&b[c]===a)return c}return-1},merge:function(a,b){var c=+b.length,d=0,e=a.length;while(c>d)a[e++]=b[d++];if(c!==c)while(void 0!==b[d])a[e++]=b[d++];return a.length=e,a},grep:function(a,b,c){for(var d,e=[],f=0,g=a.length,h=!c;g>f;f++)d=!b(a[f],f),d!==h&&e.push(a[f]);return e},map:function(a,b,c){var d,f=0,g=a.length,h=r(a),i=[];if(h)for(;g>f;f++)d=b(a[f],f,c),null!=d&&i.push(d);else for(f in a)d=b(a[f],f,c),null!=d&&i.push(d);return e.apply([],i)},guid:1,proxy:function(a,b){var c,e,f;return"string"==typeof b&&(f=a[b],b=a,a=f),m.isFunction(a)?(c=d.call(arguments,2),e=function(){return a.apply(b||this,c.concat(d.call(arguments)))},e.guid=a.guid=a.guid||m.guid++,e):void 0},now:function(){return+new Date},support:k}),m.each("Boolean Number String Function Array Date RegExp Object Error".split(" "),function(a,b){h["[object "+b+"]"]=b.toLowerCase()});function r(a){var b=a.length,c=m.type(a);return"function"===c||m.isWindow(a)?!1:1===a.nodeType&&b?!0:"array"===c||0===b||"number"==typeof b&&b>0&&b-1 in a}var s=function(a){var b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u="sizzle"+1*new Date,v=a.document,w=0,x=0,y=hb(),z=hb(),A=hb(),B=function(a,b){return a===b&&(l=!0),0},C=1<<31,D={}.hasOwnProperty,E=[],F=E.pop,G=E.push,H=E.push,I=E.slice,J=function(a,b){for(var c=0,d=a.length;d>c;c++)if(a[c]===b)return c;return-1},K="checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|ismap|loop|multiple|open|readonly|required|scoped",L="[\\x20\\t\\r\\n\\f]",M="(?:\\\\.|[\\w-]|[^\\x00-\\xa0])+",N=M.replace("w","w#"),O="\\["+L+"*("+M+")(?:"+L+"*([*^$|!~]?=)"+L+"*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|("+N+"))|)"+L+"*\\]",P=":("+M+")(?:\\((('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|((?:\\\\.|[^\\\\()[\\]]|"+O+")*)|.*)\\)|)",Q=new RegExp(L+"+","g"),R=new RegExp("^"+L+"+|((?:^|[^\\\\])(?:\\\\.)*)"+L+"+$","g"),S=new RegExp("^"+L+"*,"+L+"*"),T=new RegExp("^"+L+"*([>+~]|"+L+")"+L+"*"),U=new RegExp("="+L+"*([^\\]'\"]*?)"+L+"*\\]","g"),V=new RegExp(P),W=new RegExp("^"+N+"$"),X={ID:new RegExp("^#("+M+")"),CLASS:new RegExp("^\\.("+M+")"),TAG:new RegExp("^("+M.replace("w","w*")+")"),ATTR:new RegExp("^"+O),PSEUDO:new RegExp("^"+P),CHILD:new RegExp("^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\("+L+"*(even|odd|(([+-]|)(\\d*)n|)"+L+"*(?:([+-]|)"+L+"*(\\d+)|))"+L+"*\\)|)","i"),bool:new RegExp("^(?:"+K+")$","i"),needsContext:new RegExp("^"+L+"*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\("+L+"*((?:-\\d)?\\d*)"+L+"*\\)|)(?=[^-]|$)","i")},Y=/^(?:input|select|textarea|button)$/i,Z=/^h\d$/i,$=/^[^{]+\{\s*\[native \w/,_=/^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/,ab=/[+~]/,bb=/'|\\/g,cb=new RegExp("\\\\([\\da-f]{1,6}"+L+"?|("+L+")|.)","ig"),db=function(a,b,c){var d="0x"+b-65536;return d!==d||c?b:0>d?String.fromCharCode(d+65536):String.fromCharCode(d>>10|55296,1023&d|56320)},eb=function(){m()};try{H.apply(E=I.call(v.childNodes),v.childNodes),E[v.childNodes.length].nodeType}catch(fb){H={apply:E.length?function(a,b){G.apply(a,I.call(b))}:function(a,b){var c=a.length,d=0;while(a[c++]=b[d++]);a.length=c-1}}}function gb(a,b,d,e){var f,h,j,k,l,o,r,s,w,x;if((b?b.ownerDocument||b:v)!==n&&m(b),b=b||n,d=d||[],k=b.nodeType,"string"!=typeof a||!a||1!==k&&9!==k&&11!==k)return d;if(!e&&p){if(11!==k&&(f=_.exec(a)))if(j=f[1]){if(9===k){if(h=b.getElementById(j),!h||!h.parentNode)return d;if(h.id===j)return d.push(h),d}else if(b.ownerDocument&&(h=b.ownerDocument.getElementById(j))&&t(b,h)&&h.id===j)return d.push(h),d}else{if(f[2])return H.apply(d,b.getElementsByTagName(a)),d;if((j=f[3])&&c.getElementsByClassName)return H.apply(d,b.getElementsByClassName(j)),d}if(c.qsa&&(!q||!q.test(a))){if(s=r=u,w=b,x=1!==k&&a,1===k&&"object"!==b.nodeName.toLowerCase()){o=g(a),(r=b.getAttribute("id"))?s=r.replace(bb,"\\$&"):b.setAttribute("id",s),s="[id='"+s+"'] ",l=o.length;while(l--)o[l]=s+rb(o[l]);w=ab.test(a)&&pb(b.parentNode)||b,x=o.join(",")}if(x)try{return H.apply(d,w.querySelectorAll(x)),d}catch(y){}finally{r||b.removeAttribute("id")}}}return i(a.replace(R,"$1"),b,d,e)}function hb(){var a=[];function b(c,e){return a.push(c+" ")>d.cacheLength&&delete b[a.shift()],b[c+" "]=e}return b}function ib(a){return a[u]=!0,a}function jb(a){var b=n.createElement("div");try{return!!a(b)}catch(c){return!1}finally{b.parentNode&&b.parentNode.removeChild(b),b=null}}function kb(a,b){var c=a.split("|"),e=a.length;while(e--)d.attrHandle[c[e]]=b}function lb(a,b){var c=b&&a,d=c&&1===a.nodeType&&1===b.nodeType&&(~b.sourceIndex||C)-(~a.sourceIndex||C);if(d)return d;if(c)while(c=c.nextSibling)if(c===b)return-1;return a?1:-1}function mb(a){return function(b){var c=b.nodeName.toLowerCase();return"input"===c&&b.type===a}}function nb(a){return function(b){var c=b.nodeName.toLowerCase();return("input"===c||"button"===c)&&b.type===a}}function ob(a){return ib(function(b){return b=+b,ib(function(c,d){var e,f=a([],c.length,b),g=f.length;while(g--)c[e=f[g]]&&(c[e]=!(d[e]=c[e]))})})}function pb(a){return a&&"undefined"!=typeof a.getElementsByTagName&&a}c=gb.support={},f=gb.isXML=function(a){var b=a&&(a.ownerDocument||a).documentElement;return b?"HTML"!==b.nodeName:!1},m=gb.setDocument=function(a){var b,e,g=a?a.ownerDocument||a:v;return g!==n&&9===g.nodeType&&g.documentElement?(n=g,o=g.documentElement,e=g.defaultView,e&&e!==e.top&&(e.addEventListener?e.addEventListener("unload",eb,!1):e.attachEvent&&e.attachEvent("onunload",eb)),p=!f(g),c.attributes=jb(function(a){return a.className="i",!a.getAttribute("className")}),c.getElementsByTagName=jb(function(a){return a.appendChild(g.createComment("")),!a.getElementsByTagName("*").length}),c.getElementsByClassName=$.test(g.getElementsByClassName),c.getById=jb(function(a){return o.appendChild(a).id=u,!g.getElementsByName||!g.getElementsByName(u).length}),c.getById?(d.find.ID=function(a,b){if("undefined"!=typeof b.getElementById&&p){var c=b.getElementById(a);return c&&c.parentNode?[c]:[]}},d.filter.ID=function(a){var b=a.replace(cb,db);return function(a){return a.getAttribute("id")===b}}):(delete d.find.ID,d.filter.ID=function(a){var b=a.replace(cb,db);return function(a){var c="undefined"!=typeof a.getAttributeNode&&a.getAttributeNode("id");return c&&c.value===b}}),d.find.TAG=c.getElementsByTagName?function(a,b){return"undefined"!=typeof b.getElementsByTagName?b.getElementsByTagName(a):c.qsa?b.querySelectorAll(a):void 0}:function(a,b){var c,d=[],e=0,f=b.getElementsByTagName(a);if("*"===a){while(c=f[e++])1===c.nodeType&&d.push(c);return d}return f},d.find.CLASS=c.getElementsByClassName&&function(a,b){return p?b.getElementsByClassName(a):void 0},r=[],q=[],(c.qsa=$.test(g.querySelectorAll))&&(jb(function(a){o.appendChild(a).innerHTML="<a id='"+u+"'></a><select id='"+u+"-\f]' msallowcapture=''><option selected=''></option></select>",a.querySelectorAll("[msallowcapture^='']").length&&q.push("[*^$]="+L+"*(?:''|\"\")"),a.querySelectorAll("[selected]").length||q.push("\\["+L+"*(?:value|"+K+")"),a.querySelectorAll("[id~="+u+"-]").length||q.push("~="),a.querySelectorAll(":checked").length||q.push(":checked"),a.querySelectorAll("a#"+u+"+*").length||q.push(".#.+[+~]")}),jb(function(a){var b=g.createElement("input");b.setAttribute("type","hidden"),a.appendChild(b).setAttribute("name","D"),a.querySelectorAll("[name=d]").length&&q.push("name"+L+"*[*^$|!~]?="),a.querySelectorAll(":enabled").length||q.push(":enabled",":disabled"),a.querySelectorAll("*,:x"),q.push(",.*:")})),(c.matchesSelector=$.test(s=o.matches||o.webkitMatchesSelector||o.mozMatchesSelector||o.oMatchesSelector||o.msMatchesSelector))&&jb(function(a){c.disconnectedMatch=s.call(a,"div"),s.call(a,"[s!='']:x"),r.push("!=",P)}),q=q.length&&new RegExp(q.join("|")),r=r.length&&new RegExp(r.join("|")),b=$.test(o.compareDocumentPosition),t=b||$.test(o.contains)?function(a,b){var c=9===a.nodeType?a.documentElement:a,d=b&&b.parentNode;return a===d||!(!d||1!==d.nodeType||!(c.contains?c.contains(d):a.compareDocumentPosition&&16&a.compareDocumentPosition(d)))}:function(a,b){if(b)while(b=b.parentNode)if(b===a)return!0;return!1},B=b?function(a,b){if(a===b)return l=!0,0;var d=!a.compareDocumentPosition-!b.compareDocumentPosition;return d?d:(d=(a.ownerDocument||a)===(b.ownerDocument||b)?a.compareDocumentPosition(b):1,1&d||!c.sortDetached&&b.compareDocumentPosition(a)===d?a===g||a.ownerDocument===v&&t(v,a)?-1:b===g||b.ownerDocument===v&&t(v,b)?1:k?J(k,a)-J(k,b):0:4&d?-1:1)}:function(a,b){if(a===b)return l=!0,0;var c,d=0,e=a.parentNode,f=b.parentNode,h=[a],i=[b];if(!e||!f)return a===g?-1:b===g?1:e?-1:f?1:k?J(k,a)-J(k,b):0;if(e===f)return lb(a,b);c=a;while(c=c.parentNode)h.unshift(c);c=b;while(c=c.parentNode)i.unshift(c);while(h[d]===i[d])d++;return d?lb(h[d],i[d]):h[d]===v?-1:i[d]===v?1:0},g):n},gb.matches=function(a,b){return gb(a,null,null,b)},gb.matchesSelector=function(a,b){if((a.ownerDocument||a)!==n&&m(a),b=b.replace(U,"='$1']"),!(!c.matchesSelector||!p||r&&r.test(b)||q&&q.test(b)))try{var d=s.call(a,b);if(d||c.disconnectedMatch||a.document&&11!==a.document.nodeType)return d}catch(e){}return gb(b,n,null,[a]).length>0},gb.contains=function(a,b){return(a.ownerDocument||a)!==n&&m(a),t(a,b)},gb.attr=function(a,b){(a.ownerDocument||a)!==n&&m(a);var e=d.attrHandle[b.toLowerCase()],f=e&&D.call(d.attrHandle,b.toLowerCase())?e(a,b,!p):void 0;return void 0!==f?f:c.attributes||!p?a.getAttribute(b):(f=a.getAttributeNode(b))&&f.specified?f.value:null},gb.error=function(a){throw new Error("Syntax error, unrecognized expression: "+a)},gb.uniqueSort=function(a){var b,d=[],e=0,f=0;if(l=!c.detectDuplicates,k=!c.sortStable&&a.slice(0),a.sort(B),l){while(b=a[f++])b===a[f]&&(e=d.push(f));while(e--)a.splice(d[e],1)}return k=null,a},e=gb.getText=function(a){var b,c="",d=0,f=a.nodeType;if(f){if(1===f||9===f||11===f){if("string"==typeof a.textContent)return a.textContent;for(a=a.firstChild;a;a=a.nextSibling)c+=e(a)}else if(3===f||4===f)return a.nodeValue}else while(b=a[d++])c+=e(b);return c},d=gb.selectors={cacheLength:50,createPseudo:ib,match:X,attrHandle:{},find:{},relative:{">":{dir:"parentNode",first:!0}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:!0},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(a){return a[1]=a[1].replace(cb,db),a[3]=(a[3]||a[4]||a[5]||"").replace(cb,db),"~="===a[2]&&(a[3]=" "+a[3]+" "),a.slice(0,4)},CHILD:function(a){return a[1]=a[1].toLowerCase(),"nth"===a[1].slice(0,3)?(a[3]||gb.error(a[0]),a[4]=+(a[4]?a[5]+(a[6]||1):2*("even"===a[3]||"odd"===a[3])),a[5]=+(a[7]+a[8]||"odd"===a[3])):a[3]&&gb.error(a[0]),a},PSEUDO:function(a){var b,c=!a[6]&&a[2];return X.CHILD.test(a[0])?null:(a[3]?a[2]=a[4]||a[5]||"":c&&V.test(c)&&(b=g(c,!0))&&(b=c.indexOf(")",c.length-b)-c.length)&&(a[0]=a[0].slice(0,b),a[2]=c.slice(0,b)),a.slice(0,3))}},filter:{TAG:function(a){var b=a.replace(cb,db).toLowerCase();return"*"===a?function(){return!0}:function(a){return a.nodeName&&a.nodeName.toLowerCase()===b}},CLASS:function(a){var b=y[a+" "];return b||(b=new RegExp("(^|"+L+")"+a+"("+L+"|$)"))&&y(a,function(a){return b.test("string"==typeof a.className&&a.className||"undefined"!=typeof a.getAttribute&&a.getAttribute("class")||"")})},ATTR:function(a,b,c){return function(d){var e=gb.attr(d,a);return null==e?"!="===b:b?(e+="","="===b?e===c:"!="===b?e!==c:"^="===b?c&&0===e.indexOf(c):"*="===b?c&&e.indexOf(c)>-1:"$="===b?c&&e.slice(-c.length)===c:"~="===b?(" "+e.replace(Q," ")+" ").indexOf(c)>-1:"|="===b?e===c||e.slice(0,c.length+1)===c+"-":!1):!0}},CHILD:function(a,b,c,d,e){var f="nth"!==a.slice(0,3),g="last"!==a.slice(-4),h="of-type"===b;return 1===d&&0===e?function(a){return!!a.parentNode}:function(b,c,i){var j,k,l,m,n,o,p=f!==g?"nextSibling":"previousSibling",q=b.parentNode,r=h&&b.nodeName.toLowerCase(),s=!i&&!h;if(q){if(f){while(p){l=b;while(l=l[p])if(h?l.nodeName.toLowerCase()===r:1===l.nodeType)return!1;o=p="only"===a&&!o&&"nextSibling"}return!0}if(o=[g?q.firstChild:q.lastChild],g&&s){k=q[u]||(q[u]={}),j=k[a]||[],n=j[0]===w&&j[1],m=j[0]===w&&j[2],l=n&&q.childNodes[n];while(l=++n&&l&&l[p]||(m=n=0)||o.pop())if(1===l.nodeType&&++m&&l===b){k[a]=[w,n,m];break}}else if(s&&(j=(b[u]||(b[u]={}))[a])&&j[0]===w)m=j[1];else while(l=++n&&l&&l[p]||(m=n=0)||o.pop())if((h?l.nodeName.toLowerCase()===r:1===l.nodeType)&&++m&&(s&&((l[u]||(l[u]={}))[a]=[w,m]),l===b))break;return m-=e,m===d||m%d===0&&m/d>=0}}},PSEUDO:function(a,b){var c,e=d.pseudos[a]||d.setFilters[a.toLowerCase()]||gb.error("unsupported pseudo: "+a);return e[u]?e(b):e.length>1?(c=[a,a,"",b],d.setFilters.hasOwnProperty(a.toLowerCase())?ib(function(a,c){var d,f=e(a,b),g=f.length;while(g--)d=J(a,f[g]),a[d]=!(c[d]=f[g])}):function(a){return e(a,0,c)}):e}},pseudos:{not:ib(function(a){var b=[],c=[],d=h(a.replace(R,"$1"));return d[u]?ib(function(a,b,c,e){var f,g=d(a,null,e,[]),h=a.length;while(h--)(f=g[h])&&(a[h]=!(b[h]=f))}):function(a,e,f){return b[0]=a,d(b,null,f,c),b[0]=null,!c.pop()}}),has:ib(function(a){return function(b){return gb(a,b).length>0}}),contains:ib(function(a){return a=a.replace(cb,db),function(b){return(b.textContent||b.innerText||e(b)).indexOf(a)>-1}}),lang:ib(function(a){return W.test(a||"")||gb.error("unsupported lang: "+a),a=a.replace(cb,db).toLowerCase(),function(b){var c;do if(c=p?b.lang:b.getAttribute("xml:lang")||b.getAttribute("lang"))return c=c.toLowerCase(),c===a||0===c.indexOf(a+"-");while((b=b.parentNode)&&1===b.nodeType);return!1}}),target:function(b){var c=a.location&&a.location.hash;return c&&c.slice(1)===b.id},root:function(a){return a===o},focus:function(a){return a===n.activeElement&&(!n.hasFocus||n.hasFocus())&&!!(a.type||a.href||~a.tabIndex)},enabled:function(a){return a.disabled===!1},disabled:function(a){return a.disabled===!0},checked:function(a){var b=a.nodeName.toLowerCase();return"input"===b&&!!a.checked||"option"===b&&!!a.selected},selected:function(a){return a.parentNode&&a.parentNode.selectedIndex,a.selected===!0},empty:function(a){for(a=a.firstChild;a;a=a.nextSibling)if(a.nodeType<6)return!1;return!0},parent:function(a){return!d.pseudos.empty(a)},header:function(a){return Z.test(a.nodeName)},input:function(a){return Y.test(a.nodeName)},button:function(a){var b=a.nodeName.toLowerCase();return"input"===b&&"button"===a.type||"button"===b},text:function(a){var b;return"input"===a.nodeName.toLowerCase()&&"text"===a.type&&(null==(b=a.getAttribute("type"))||"text"===b.toLowerCase())},first:ob(function(){return[0]}),last:ob(function(a,b){return[b-1]}),eq:ob(function(a,b,c){return[0>c?c+b:c]}),even:ob(function(a,b){for(var c=0;b>c;c+=2)a.push(c);return a}),odd:ob(function(a,b){for(var c=1;b>c;c+=2)a.push(c);return a}),lt:ob(function(a,b,c){for(var d=0>c?c+b:c;--d>=0;)a.push(d);return a}),gt:ob(function(a,b,c){for(var d=0>c?c+b:c;++d<b;)a.push(d);return a})}},d.pseudos.nth=d.pseudos.eq;for(b in{radio:!0,checkbox:!0,file:!0,password:!0,image:!0})d.pseudos[b]=mb(b);for(b in{submit:!0,reset:!0})d.pseudos[b]=nb(b);function qb(){}qb.prototype=d.filters=d.pseudos,d.setFilters=new qb,g=gb.tokenize=function(a,b){var c,e,f,g,h,i,j,k=z[a+" "];if(k)return b?0:k.slice(0);h=a,i=[],j=d.preFilter;while(h){(!c||(e=S.exec(h)))&&(e&&(h=h.slice(e[0].length)||h),i.push(f=[])),c=!1,(e=T.exec(h))&&(c=e.shift(),f.push({value:c,type:e[0].replace(R," ")}),h=h.slice(c.length));for(g in d.filter)!(e=X[g].exec(h))||j[g]&&!(e=j[g](e))||(c=e.shift(),f.push({value:c,type:g,matches:e}),h=h.slice(c.length));if(!c)break}return b?h.length:h?gb.error(a):z(a,i).slice(0)};function rb(a){for(var b=0,c=a.length,d="";c>b;b++)d+=a[b].value;return d}function sb(a,b,c){var d=b.dir,e=c&&"parentNode"===d,f=x++;return b.first?function(b,c,f){while(b=b[d])if(1===b.nodeType||e)return a(b,c,f)}:function(b,c,g){var h,i,j=[w,f];if(g){while(b=b[d])if((1===b.nodeType||e)&&a(b,c,g))return!0}else while(b=b[d])if(1===b.nodeType||e){if(i=b[u]||(b[u]={}),(h=i[d])&&h[0]===w&&h[1]===f)return j[2]=h[2];if(i[d]=j,j[2]=a(b,c,g))return!0}}}function tb(a){return a.length>1?function(b,c,d){var e=a.length;while(e--)if(!a[e](b,c,d))return!1;return!0}:a[0]}function ub(a,b,c){for(var d=0,e=b.length;e>d;d++)gb(a,b[d],c);return c}function vb(a,b,c,d,e){for(var f,g=[],h=0,i=a.length,j=null!=b;i>h;h++)(f=a[h])&&(!c||c(f,d,e))&&(g.push(f),j&&b.push(h));return g}function wb(a,b,c,d,e,f){return d&&!d[u]&&(d=wb(d)),e&&!e[u]&&(e=wb(e,f)),ib(function(f,g,h,i){var j,k,l,m=[],n=[],o=g.length,p=f||ub(b||"*",h.nodeType?[h]:h,[]),q=!a||!f&&b?p:vb(p,m,a,h,i),r=c?e||(f?a:o||d)?[]:g:q;if(c&&c(q,r,h,i),d){j=vb(r,n),d(j,[],h,i),k=j.length;while(k--)(l=j[k])&&(r[n[k]]=!(q[n[k]]=l))}if(f){if(e||a){if(e){j=[],k=r.length;while(k--)(l=r[k])&&j.push(q[k]=l);e(null,r=[],j,i)}k=r.length;while(k--)(l=r[k])&&(j=e?J(f,l):m[k])>-1&&(f[j]=!(g[j]=l))}}else r=vb(r===g?r.splice(o,r.length):r),e?e(null,g,r,i):H.apply(g,r)})}function xb(a){for(var b,c,e,f=a.length,g=d.relative[a[0].type],h=g||d.relative[" "],i=g?1:0,k=sb(function(a){return a===b},h,!0),l=sb(function(a){return J(b,a)>-1},h,!0),m=[function(a,c,d){var e=!g&&(d||c!==j)||((b=c).nodeType?k(a,c,d):l(a,c,d));return b=null,e}];f>i;i++)if(c=d.relative[a[i].type])m=[sb(tb(m),c)];else{if(c=d.filter[a[i].type].apply(null,a[i].matches),c[u]){for(e=++i;f>e;e++)if(d.relative[a[e].type])break;return wb(i>1&&tb(m),i>1&&rb(a.slice(0,i-1).concat({value:" "===a[i-2].type?"*":""})).replace(R,"$1"),c,e>i&&xb(a.slice(i,e)),f>e&&xb(a=a.slice(e)),f>e&&rb(a))}m.push(c)}return tb(m)}function yb(a,b){var c=b.length>0,e=a.length>0,f=function(f,g,h,i,k){var l,m,o,p=0,q="0",r=f&&[],s=[],t=j,u=f||e&&d.find.TAG("*",k),v=w+=null==t?1:Math.random()||.1,x=u.length;for(k&&(j=g!==n&&g);q!==x&&null!=(l=u[q]);q++){if(e&&l){m=0;while(o=a[m++])if(o(l,g,h)){i.push(l);break}k&&(w=v)}c&&((l=!o&&l)&&p--,f&&r.push(l))}if(p+=q,c&&q!==p){m=0;while(o=b[m++])o(r,s,g,h);if(f){if(p>0)while(q--)r[q]||s[q]||(s[q]=F.call(i));s=vb(s)}H.apply(i,s),k&&!f&&s.length>0&&p+b.length>1&&gb.uniqueSort(i)}return k&&(w=v,j=t),r};return c?ib(f):f}return h=gb.compile=function(a,b){var c,d=[],e=[],f=A[a+" "];if(!f){b||(b=g(a)),c=b.length;while(c--)f=xb(b[c]),f[u]?d.push(f):e.push(f);f=A(a,yb(e,d)),f.selector=a}return f},i=gb.select=function(a,b,e,f){var i,j,k,l,m,n="function"==typeof a&&a,o=!f&&g(a=n.selector||a);if(e=e||[],1===o.length){if(j=o[0]=o[0].slice(0),j.length>2&&"ID"===(k=j[0]).type&&c.getById&&9===b.nodeType&&p&&d.relative[j[1].type]){if(b=(d.find.ID(k.matches[0].replace(cb,db),b)||[])[0],!b)return e;n&&(b=b.parentNode),a=a.slice(j.shift().value.length)}i=X.needsContext.test(a)?0:j.length;while(i--){if(k=j[i],d.relative[l=k.type])break;if((m=d.find[l])&&(f=m(k.matches[0].replace(cb,db),ab.test(j[0].type)&&pb(b.parentNode)||b))){if(j.splice(i,1),a=f.length&&rb(j),!a)return H.apply(e,f),e;break}}}return(n||h(a,o))(f,b,!p,e,ab.test(a)&&pb(b.parentNode)||b),e},c.sortStable=u.split("").sort(B).join("")===u,c.detectDuplicates=!!l,m(),c.sortDetached=jb(function(a){return 1&a.compareDocumentPosition(n.createElement("div"))}),jb(function(a){return a.innerHTML="<a href='#'></a>","#"===a.firstChild.getAttribute("href")})||kb("type|href|height|width",function(a,b,c){return c?void 0:a.getAttribute(b,"type"===b.toLowerCase()?1:2)}),c.attributes&&jb(function(a){return a.innerHTML="<input/>",a.firstChild.setAttribute("value",""),""===a.firstChild.getAttribute("value")})||kb("value",function(a,b,c){return c||"input"!==a.nodeName.toLowerCase()?void 0:a.defaultValue}),jb(function(a){return null==a.getAttribute("disabled")})||kb(K,function(a,b,c){var d;return c?void 0:a[b]===!0?b.toLowerCase():(d=a.getAttributeNode(b))&&d.specified?d.value:null}),gb}(a);m.find=s,m.expr=s.selectors,m.expr[":"]=m.expr.pseudos,m.unique=s.uniqueSort,m.text=s.getText,m.isXMLDoc=s.isXML,m.contains=s.contains;var t=m.expr.match.needsContext,u=/^<(\w+)\s*\/?>(?:<\/\1>|)$/,v=/^.[^:#\[\.,]*$/;function w(a,b,c){if(m.isFunction(b))return m.grep(a,function(a,d){return!!b.call(a,d,a)!==c});if(b.nodeType)return m.grep(a,function(a){return a===b!==c});if("string"==typeof b){if(v.test(b))return m.filter(b,a,c);b=m.filter(b,a)}return m.grep(a,function(a){return m.inArray(a,b)>=0!==c})}m.filter=function(a,b,c){var d=b[0];return c&&(a=":not("+a+")"),1===b.length&&1===d.nodeType?m.find.matchesSelector(d,a)?[d]:[]:m.find.matches(a,m.grep(b,function(a){return 1===a.nodeType}))},m.fn.extend({find:function(a){var b,c=[],d=this,e=d.length;if("string"!=typeof a)return this.pushStack(m(a).filter(function(){for(b=0;e>b;b++)if(m.contains(d[b],this))return!0}));for(b=0;e>b;b++)m.find(a,d[b],c);return c=this.pushStack(e>1?m.unique(c):c),c.selector=this.selector?this.selector+" "+a:a,c},filter:function(a){return this.pushStack(w(this,a||[],!1))},not:function(a){return this.pushStack(w(this,a||[],!0))},is:function(a){return!!w(this,"string"==typeof a&&t.test(a)?m(a):a||[],!1).length}});var x,y=a.document,z=/^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]*))$/,A=m.fn.init=function(a,b){var c,d;if(!a)return this;if("string"==typeof a){if(c="<"===a.charAt(0)&&">"===a.charAt(a.length-1)&&a.length>=3?[null,a,null]:z.exec(a),!c||!c[1]&&b)return!b||b.jquery?(b||x).find(a):this.constructor(b).find(a);if(c[1]){if(b=b instanceof m?b[0]:b,m.merge(this,m.parseHTML(c[1],b&&b.nodeType?b.ownerDocument||b:y,!0)),u.test(c[1])&&m.isPlainObject(b))for(c in b)m.isFunction(this[c])?this[c](b[c]):this.attr(c,b[c]);return this}if(d=y.getElementById(c[2]),d&&d.parentNode){if(d.id!==c[2])return x.find(a);this.length=1,this[0]=d}return this.context=y,this.selector=a,this}return a.nodeType?(this.context=this[0]=a,this.length=1,this):m.isFunction(a)?"undefined"!=typeof x.ready?x.ready(a):a(m):(void 0!==a.selector&&(this.selector=a.selector,this.context=a.context),m.makeArray(a,this))};A.prototype=m.fn,x=m(y);var B=/^(?:parents|prev(?:Until|All))/,C={children:!0,contents:!0,next:!0,prev:!0};m.extend({dir:function(a,b,c){var d=[],e=a[b];while(e&&9!==e.nodeType&&(void 0===c||1!==e.nodeType||!m(e).is(c)))1===e.nodeType&&d.push(e),e=e[b];return d},sibling:function(a,b){for(var c=[];a;a=a.nextSibling)1===a.nodeType&&a!==b&&c.push(a);return c}}),m.fn.extend({has:function(a){var b,c=m(a,this),d=c.length;return this.filter(function(){for(b=0;d>b;b++)if(m.contains(this,c[b]))return!0})},closest:function(a,b){for(var c,d=0,e=this.length,f=[],g=t.test(a)||"string"!=typeof a?m(a,b||this.context):0;e>d;d++)for(c=this[d];c&&c!==b;c=c.parentNode)if(c.nodeType<11&&(g?g.index(c)>-1:1===c.nodeType&&m.find.matchesSelector(c,a))){f.push(c);break}return this.pushStack(f.length>1?m.unique(f):f)},index:function(a){return a?"string"==typeof a?m.inArray(this[0],m(a)):m.inArray(a.jquery?a[0]:a,this):this[0]&&this[0].parentNode?this.first().prevAll().length:-1},add:function(a,b){return this.pushStack(m.unique(m.merge(this.get(),m(a,b))))},addBack:function(a){return this.add(null==a?this.prevObject:this.prevObject.filter(a))}});function D(a,b){do a=a[b];while(a&&1!==a.nodeType);return a}m.each({parent:function(a){var b=a.parentNode;return b&&11!==b.nodeType?b:null},parents:function(a){return m.dir(a,"parentNode")},parentsUntil:function(a,b,c){return m.dir(a,"parentNode",c)},next:function(a){return D(a,"nextSibling")},prev:function(a){return D(a,"previousSibling")},nextAll:function(a){return m.dir(a,"nextSibling")},prevAll:function(a){return m.dir(a,"previousSibling")},nextUntil:function(a,b,c){return m.dir(a,"nextSibling",c)},prevUntil:function(a,b,c){return m.dir(a,"previousSibling",c)},siblings:function(a){return m.sibling((a.parentNode||{}).firstChild,a)},children:function(a){return m.sibling(a.firstChild)},contents:function(a){return m.nodeName(a,"iframe")?a.contentDocument||a.contentWindow.document:m.merge([],a.childNodes)}},function(a,b){m.fn[a]=function(c,d){var e=m.map(this,b,c);return"Until"!==a.slice(-5)&&(d=c),d&&"string"==typeof d&&(e=m.filter(d,e)),this.length>1&&(C[a]||(e=m.unique(e)),B.test(a)&&(e=e.reverse())),this.pushStack(e)}});var E=/\S+/g,F={};function G(a){var b=F[a]={};return m.each(a.match(E)||[],function(a,c){b[c]=!0}),b}m.Callbacks=function(a){a="string"==typeof a?F[a]||G(a):m.extend({},a);var b,c,d,e,f,g,h=[],i=!a.once&&[],j=function(l){for(c=a.memory&&l,d=!0,f=g||0,g=0,e=h.length,b=!0;h&&e>f;f++)if(h[f].apply(l[0],l[1])===!1&&a.stopOnFalse){c=!1;break}b=!1,h&&(i?i.length&&j(i.shift()):c?h=[]:k.disable())},k={add:function(){if(h){var d=h.length;!function f(b){m.each(b,function(b,c){var d=m.type(c);"function"===d?a.unique&&k.has(c)||h.push(c):c&&c.length&&"string"!==d&&f(c)})}(arguments),b?e=h.length:c&&(g=d,j(c))}return this},remove:function(){return h&&m.each(arguments,function(a,c){var d;while((d=m.inArray(c,h,d))>-1)h.splice(d,1),b&&(e>=d&&e--,f>=d&&f--)}),this},has:function(a){return a?m.inArray(a,h)>-1:!(!h||!h.length)},empty:function(){return h=[],e=0,this},disable:function(){return h=i=c=void 0,this},disabled:function(){return!h},lock:function(){return i=void 0,c||k.disable(),this},locked:function(){return!i},fireWith:function(a,c){return!h||d&&!i||(c=c||[],c=[a,c.slice?c.slice():c],b?i.push(c):j(c)),this},fire:function(){return k.fireWith(this,arguments),this},fired:function(){return!!d}};return k},m.extend({Deferred:function(a){var b=[["resolve","done",m.Callbacks("once memory"),"resolved"],["reject","fail",m.Callbacks("once memory"),"rejected"],["notify","progress",m.Callbacks("memory")]],c="pending",d={state:function(){return c},always:function(){return e.done(arguments).fail(arguments),this},then:function(){var a=arguments;return m.Deferred(function(c){m.each(b,function(b,f){var g=m.isFunction(a[b])&&a[b];e[f[1]](function(){var a=g&&g.apply(this,arguments);a&&m.isFunction(a.promise)?a.promise().done(c.resolve).fail(c.reject).progress(c.notify):c[f[0]+"With"](this===d?c.promise():this,g?[a]:arguments)})}),a=null}).promise()},promise:function(a){return null!=a?m.extend(a,d):d}},e={};return d.pipe=d.then,m.each(b,function(a,f){var g=f[2],h=f[3];d[f[1]]=g.add,h&&g.add(function(){c=h},b[1^a][2].disable,b[2][2].lock),e[f[0]]=function(){return e[f[0]+"With"](this===e?d:this,arguments),this},e[f[0]+"With"]=g.fireWith}),d.promise(e),a&&a.call(e,e),e},when:function(a){var b=0,c=d.call(arguments),e=c.length,f=1!==e||a&&m.isFunction(a.promise)?e:0,g=1===f?a:m.Deferred(),h=function(a,b,c){return function(e){b[a]=this,c[a]=arguments.length>1?d.call(arguments):e,c===i?g.notifyWith(b,c):--f||g.resolveWith(b,c)}},i,j,k;if(e>1)for(i=new Array(e),j=new Array(e),k=new Array(e);e>b;b++)c[b]&&m.isFunction(c[b].promise)?c[b].promise().done(h(b,k,c)).fail(g.reject).progress(h(b,j,i)):--f;return f||g.resolveWith(k,c),g.promise()}});var H;m.fn.ready=function(a){return m.ready.promise().done(a),this},m.extend({isReady:!1,readyWait:1,holdReady:function(a){a?m.readyWait++:m.ready(!0)},ready:function(a){if(a===!0?!--m.readyWait:!m.isReady){if(!y.body)return setTimeout(m.ready);m.isReady=!0,a!==!0&&--m.readyWait>0||(H.resolveWith(y,[m]),m.fn.triggerHandler&&(m(y).triggerHandler("ready"),m(y).off("ready")))}}});function I(){y.addEventListener?(y.removeEventListener("DOMContentLoaded",J,!1),a.removeEventListener("load",J,!1)):(y.detachEvent("onreadystatechange",J),a.detachEvent("onload",J))}function J(){(y.addEventListener||"load"===event.type||"complete"===y.readyState)&&(I(),m.ready())}m.ready.promise=function(b){if(!H)if(H=m.Deferred(),"complete"===y.readyState)setTimeout(m.ready);else if(y.addEventListener)y.addEventListener("DOMContentLoaded",J,!1),a.addEventListener("load",J,!1);else{y.attachEvent("onreadystatechange",J),a.attachEvent("onload",J);var c=!1;try{c=null==a.frameElement&&y.documentElement}catch(d){}c&&c.doScroll&&!function e(){if(!m.isReady){try{c.doScroll("left")}catch(a){return setTimeout(e,50)}I(),m.ready()}}()}return H.promise(b)};var K="undefined",L;for(L in m(k))break;k.ownLast="0"!==L,k.inlineBlockNeedsLayout=!1,m(function(){var a,b,c,d;c=y.getElementsByTagName("body")[0],c&&c.style&&(b=y.createElement("div"),d=y.createElement("div"),d.style.cssText="position:absolute;border:0;width:0;height:0;top:0;left:-9999px",c.appendChild(d).appendChild(b),typeof b.style.zoom!==K&&(b.style.cssText="display:inline;margin:0;border:0;padding:1px;width:1px;zoom:1",k.inlineBlockNeedsLayout=a=3===b.offsetWidth,a&&(c.style.zoom=1)),c.removeChild(d))}),function(){var a=y.createElement("div");if(null==k.deleteExpando){k.deleteExpando=!0;try{delete a.test}catch(b){k.deleteExpando=!1}}a=null}(),m.acceptData=function(a){var b=m.noData[(a.nodeName+" ").toLowerCase()],c=+a.nodeType||1;return 1!==c&&9!==c?!1:!b||b!==!0&&a.getAttribute("classid")===b};var M=/^(?:\{[\w\W]*\}|\[[\w\W]*\])$/,N=/([A-Z])/g;function O(a,b,c){if(void 0===c&&1===a.nodeType){var d="data-"+b.replace(N,"-$1").toLowerCase();if(c=a.getAttribute(d),"string"==typeof c){try{c="true"===c?!0:"false"===c?!1:"null"===c?null:+c+""===c?+c:M.test(c)?m.parseJSON(c):c}catch(e){}m.data(a,b,c)}else c=void 0}return c}function P(a){var b;for(b in a)if(("data"!==b||!m.isEmptyObject(a[b]))&&"toJSON"!==b)return!1;
+return!0}function Q(a,b,d,e){if(m.acceptData(a)){var f,g,h=m.expando,i=a.nodeType,j=i?m.cache:a,k=i?a[h]:a[h]&&h;if(k&&j[k]&&(e||j[k].data)||void 0!==d||"string"!=typeof b)return k||(k=i?a[h]=c.pop()||m.guid++:h),j[k]||(j[k]=i?{}:{toJSON:m.noop}),("object"==typeof b||"function"==typeof b)&&(e?j[k]=m.extend(j[k],b):j[k].data=m.extend(j[k].data,b)),g=j[k],e||(g.data||(g.data={}),g=g.data),void 0!==d&&(g[m.camelCase(b)]=d),"string"==typeof b?(f=g[b],null==f&&(f=g[m.camelCase(b)])):f=g,f}}function R(a,b,c){if(m.acceptData(a)){var d,e,f=a.nodeType,g=f?m.cache:a,h=f?a[m.expando]:m.expando;if(g[h]){if(b&&(d=c?g[h]:g[h].data)){m.isArray(b)?b=b.concat(m.map(b,m.camelCase)):b in d?b=[b]:(b=m.camelCase(b),b=b in d?[b]:b.split(" ")),e=b.length;while(e--)delete d[b[e]];if(c?!P(d):!m.isEmptyObject(d))return}(c||(delete g[h].data,P(g[h])))&&(f?m.cleanData([a],!0):k.deleteExpando||g!=g.window?delete g[h]:g[h]=null)}}}m.extend({cache:{},noData:{"applet ":!0,"embed ":!0,"object ":"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000"},hasData:function(a){return a=a.nodeType?m.cache[a[m.expando]]:a[m.expando],!!a&&!P(a)},data:function(a,b,c){return Q(a,b,c)},removeData:function(a,b){return R(a,b)},_data:function(a,b,c){return Q(a,b,c,!0)},_removeData:function(a,b){return R(a,b,!0)}}),m.fn.extend({data:function(a,b){var c,d,e,f=this[0],g=f&&f.attributes;if(void 0===a){if(this.length&&(e=m.data(f),1===f.nodeType&&!m._data(f,"parsedAttrs"))){c=g.length;while(c--)g[c]&&(d=g[c].name,0===d.indexOf("data-")&&(d=m.camelCase(d.slice(5)),O(f,d,e[d])));m._data(f,"parsedAttrs",!0)}return e}return"object"==typeof a?this.each(function(){m.data(this,a)}):arguments.length>1?this.each(function(){m.data(this,a,b)}):f?O(f,a,m.data(f,a)):void 0},removeData:function(a){return this.each(function(){m.removeData(this,a)})}}),m.extend({queue:function(a,b,c){var d;return a?(b=(b||"fx")+"queue",d=m._data(a,b),c&&(!d||m.isArray(c)?d=m._data(a,b,m.makeArray(c)):d.push(c)),d||[]):void 0},dequeue:function(a,b){b=b||"fx";var c=m.queue(a,b),d=c.length,e=c.shift(),f=m._queueHooks(a,b),g=function(){m.dequeue(a,b)};"inprogress"===e&&(e=c.shift(),d--),e&&("fx"===b&&c.unshift("inprogress"),delete f.stop,e.call(a,g,f)),!d&&f&&f.empty.fire()},_queueHooks:function(a,b){var c=b+"queueHooks";return m._data(a,c)||m._data(a,c,{empty:m.Callbacks("once memory").add(function(){m._removeData(a,b+"queue"),m._removeData(a,c)})})}}),m.fn.extend({queue:function(a,b){var c=2;return"string"!=typeof a&&(b=a,a="fx",c--),arguments.length<c?m.queue(this[0],a):void 0===b?this:this.each(function(){var c=m.queue(this,a,b);m._queueHooks(this,a),"fx"===a&&"inprogress"!==c[0]&&m.dequeue(this,a)})},dequeue:function(a){return this.each(function(){m.dequeue(this,a)})},clearQueue:function(a){return this.queue(a||"fx",[])},promise:function(a,b){var c,d=1,e=m.Deferred(),f=this,g=this.length,h=function(){--d||e.resolveWith(f,[f])};"string"!=typeof a&&(b=a,a=void 0),a=a||"fx";while(g--)c=m._data(f[g],a+"queueHooks"),c&&c.empty&&(d++,c.empty.add(h));return h(),e.promise(b)}});var S=/[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/.source,T=["Top","Right","Bottom","Left"],U=function(a,b){return a=b||a,"none"===m.css(a,"display")||!m.contains(a.ownerDocument,a)},V=m.access=function(a,b,c,d,e,f,g){var h=0,i=a.length,j=null==c;if("object"===m.type(c)){e=!0;for(h in c)m.access(a,b,h,c[h],!0,f,g)}else if(void 0!==d&&(e=!0,m.isFunction(d)||(g=!0),j&&(g?(b.call(a,d),b=null):(j=b,b=function(a,b,c){return j.call(m(a),c)})),b))for(;i>h;h++)b(a[h],c,g?d:d.call(a[h],h,b(a[h],c)));return e?a:j?b.call(a):i?b(a[0],c):f},W=/^(?:checkbox|radio)$/i;!function(){var a=y.createElement("input"),b=y.createElement("div"),c=y.createDocumentFragment();if(b.innerHTML=" <link/><table></table><a href='/a'>a</a><input type='checkbox'/>",k.leadingWhitespace=3===b.firstChild.nodeType,k.tbody=!b.getElementsByTagName("tbody").length,k.htmlSerialize=!!b.getElementsByTagName("link").length,k.html5Clone="<:nav></:nav>"!==y.createElement("nav").cloneNode(!0).outerHTML,a.type="checkbox",a.checked=!0,c.appendChild(a),k.appendChecked=a.checked,b.innerHTML="<textarea>x</textarea>",k.noCloneChecked=!!b.cloneNode(!0).lastChild.defaultValue,c.appendChild(b),b.innerHTML="<input type='radio' checked='checked' name='t'/>",k.checkClone=b.cloneNode(!0).cloneNode(!0).lastChild.checked,k.noCloneEvent=!0,b.attachEvent&&(b.attachEvent("onclick",function(){k.noCloneEvent=!1}),b.cloneNode(!0).click()),null==k.deleteExpando){k.deleteExpando=!0;try{delete b.test}catch(d){k.deleteExpando=!1}}}(),function(){var b,c,d=y.createElement("div");for(b in{submit:!0,change:!0,focusin:!0})c="on"+b,(k[b+"Bubbles"]=c in a)||(d.setAttribute(c,"t"),k[b+"Bubbles"]=d.attributes[c].expando===!1);d=null}();var X=/^(?:input|select|textarea)$/i,Y=/^key/,Z=/^(?:mouse|pointer|contextmenu)|click/,$=/^(?:focusinfocus|focusoutblur)$/,_=/^([^.]*)(?:\.(.+)|)$/;function ab(){return!0}function bb(){return!1}function cb(){try{return y.activeElement}catch(a){}}m.event={global:{},add:function(a,b,c,d,e){var f,g,h,i,j,k,l,n,o,p,q,r=m._data(a);if(r){c.handler&&(i=c,c=i.handler,e=i.selector),c.guid||(c.guid=m.guid++),(g=r.events)||(g=r.events={}),(k=r.handle)||(k=r.handle=function(a){return typeof m===K||a&&m.event.triggered===a.type?void 0:m.event.dispatch.apply(k.elem,arguments)},k.elem=a),b=(b||"").match(E)||[""],h=b.length;while(h--)f=_.exec(b[h])||[],o=q=f[1],p=(f[2]||"").split(".").sort(),o&&(j=m.event.special[o]||{},o=(e?j.delegateType:j.bindType)||o,j=m.event.special[o]||{},l=m.extend({type:o,origType:q,data:d,handler:c,guid:c.guid,selector:e,needsContext:e&&m.expr.match.needsContext.test(e),namespace:p.join(".")},i),(n=g[o])||(n=g[o]=[],n.delegateCount=0,j.setup&&j.setup.call(a,d,p,k)!==!1||(a.addEventListener?a.addEventListener(o,k,!1):a.attachEvent&&a.attachEvent("on"+o,k))),j.add&&(j.add.call(a,l),l.handler.guid||(l.handler.guid=c.guid)),e?n.splice(n.delegateCount++,0,l):n.push(l),m.event.global[o]=!0);a=null}},remove:function(a,b,c,d,e){var f,g,h,i,j,k,l,n,o,p,q,r=m.hasData(a)&&m._data(a);if(r&&(k=r.events)){b=(b||"").match(E)||[""],j=b.length;while(j--)if(h=_.exec(b[j])||[],o=q=h[1],p=(h[2]||"").split(".").sort(),o){l=m.event.special[o]||{},o=(d?l.delegateType:l.bindType)||o,n=k[o]||[],h=h[2]&&new RegExp("(^|\\.)"+p.join("\\.(?:.*\\.|)")+"(\\.|$)"),i=f=n.length;while(f--)g=n[f],!e&&q!==g.origType||c&&c.guid!==g.guid||h&&!h.test(g.namespace)||d&&d!==g.selector&&("**"!==d||!g.selector)||(n.splice(f,1),g.selector&&n.delegateCount--,l.remove&&l.remove.call(a,g));i&&!n.length&&(l.teardown&&l.teardown.call(a,p,r.handle)!==!1||m.removeEvent(a,o,r.handle),delete k[o])}else for(o in k)m.event.remove(a,o+b[j],c,d,!0);m.isEmptyObject(k)&&(delete r.handle,m._removeData(a,"events"))}},trigger:function(b,c,d,e){var f,g,h,i,k,l,n,o=[d||y],p=j.call(b,"type")?b.type:b,q=j.call(b,"namespace")?b.namespace.split("."):[];if(h=l=d=d||y,3!==d.nodeType&&8!==d.nodeType&&!$.test(p+m.event.triggered)&&(p.indexOf(".")>=0&&(q=p.split("."),p=q.shift(),q.sort()),g=p.indexOf(":")<0&&"on"+p,b=b[m.expando]?b:new m.Event(p,"object"==typeof b&&b),b.isTrigger=e?2:3,b.namespace=q.join("."),b.namespace_re=b.namespace?new RegExp("(^|\\.)"+q.join("\\.(?:.*\\.|)")+"(\\.|$)"):null,b.result=void 0,b.target||(b.target=d),c=null==c?[b]:m.makeArray(c,[b]),k=m.event.special[p]||{},e||!k.trigger||k.trigger.apply(d,c)!==!1)){if(!e&&!k.noBubble&&!m.isWindow(d)){for(i=k.delegateType||p,$.test(i+p)||(h=h.parentNode);h;h=h.parentNode)o.push(h),l=h;l===(d.ownerDocument||y)&&o.push(l.defaultView||l.parentWindow||a)}n=0;while((h=o[n++])&&!b.isPropagationStopped())b.type=n>1?i:k.bindType||p,f=(m._data(h,"events")||{})[b.type]&&m._data(h,"handle"),f&&f.apply(h,c),f=g&&h[g],f&&f.apply&&m.acceptData(h)&&(b.result=f.apply(h,c),b.result===!1&&b.preventDefault());if(b.type=p,!e&&!b.isDefaultPrevented()&&(!k._default||k._default.apply(o.pop(),c)===!1)&&m.acceptData(d)&&g&&d[p]&&!m.isWindow(d)){l=d[g],l&&(d[g]=null),m.event.triggered=p;try{d[p]()}catch(r){}m.event.triggered=void 0,l&&(d[g]=l)}return b.result}},dispatch:function(a){a=m.event.fix(a);var b,c,e,f,g,h=[],i=d.call(arguments),j=(m._data(this,"events")||{})[a.type]||[],k=m.event.special[a.type]||{};if(i[0]=a,a.delegateTarget=this,!k.preDispatch||k.preDispatch.call(this,a)!==!1){h=m.event.handlers.call(this,a,j),b=0;while((f=h[b++])&&!a.isPropagationStopped()){a.currentTarget=f.elem,g=0;while((e=f.handlers[g++])&&!a.isImmediatePropagationStopped())(!a.namespace_re||a.namespace_re.test(e.namespace))&&(a.handleObj=e,a.data=e.data,c=((m.event.special[e.origType]||{}).handle||e.handler).apply(f.elem,i),void 0!==c&&(a.result=c)===!1&&(a.preventDefault(),a.stopPropagation()))}return k.postDispatch&&k.postDispatch.call(this,a),a.result}},handlers:function(a,b){var c,d,e,f,g=[],h=b.delegateCount,i=a.target;if(h&&i.nodeType&&(!a.button||"click"!==a.type))for(;i!=this;i=i.parentNode||this)if(1===i.nodeType&&(i.disabled!==!0||"click"!==a.type)){for(e=[],f=0;h>f;f++)d=b[f],c=d.selector+" ",void 0===e[c]&&(e[c]=d.needsContext?m(c,this).index(i)>=0:m.find(c,this,null,[i]).length),e[c]&&e.push(d);e.length&&g.push({elem:i,handlers:e})}return h<b.length&&g.push({elem:this,handlers:b.slice(h)}),g},fix:function(a){if(a[m.expando])return a;var b,c,d,e=a.type,f=a,g=this.fixHooks[e];g||(this.fixHooks[e]=g=Z.test(e)?this.mouseHooks:Y.test(e)?this.keyHooks:{}),d=g.props?this.props.concat(g.props):this.props,a=new m.Event(f),b=d.length;while(b--)c=d[b],a[c]=f[c];return a.target||(a.target=f.srcElement||y),3===a.target.nodeType&&(a.target=a.target.parentNode),a.metaKey=!!a.metaKey,g.filter?g.filter(a,f):a},props:"altKey bubbles cancelable ctrlKey currentTarget eventPhase metaKey relatedTarget shiftKey target timeStamp view which".split(" "),fixHooks:{},keyHooks:{props:"char charCode key keyCode".split(" "),filter:function(a,b){return null==a.which&&(a.which=null!=b.charCode?b.charCode:b.keyCode),a}},mouseHooks:{props:"button buttons clientX clientY fromElement offsetX offsetY pageX pageY screenX screenY toElement".split(" "),filter:function(a,b){var c,d,e,f=b.button,g=b.fromElement;return null==a.pageX&&null!=b.clientX&&(d=a.target.ownerDocument||y,e=d.documentElement,c=d.body,a.pageX=b.clientX+(e&&e.scrollLeft||c&&c.scrollLeft||0)-(e&&e.clientLeft||c&&c.clientLeft||0),a.pageY=b.clientY+(e&&e.scrollTop||c&&c.scrollTop||0)-(e&&e.clientTop||c&&c.clientTop||0)),!a.relatedTarget&&g&&(a.relatedTarget=g===a.target?b.toElement:g),a.which||void 0===f||(a.which=1&f?1:2&f?3:4&f?2:0),a}},special:{load:{noBubble:!0},focus:{trigger:function(){if(this!==cb()&&this.focus)try{return this.focus(),!1}catch(a){}},delegateType:"focusin"},blur:{trigger:function(){return this===cb()&&this.blur?(this.blur(),!1):void 0},delegateType:"focusout"},click:{trigger:function(){return m.nodeName(this,"input")&&"checkbox"===this.type&&this.click?(this.click(),!1):void 0},_default:function(a){return m.nodeName(a.target,"a")}},beforeunload:{postDispatch:function(a){void 0!==a.result&&a.originalEvent&&(a.originalEvent.returnValue=a.result)}}},simulate:function(a,b,c,d){var e=m.extend(new m.Event,c,{type:a,isSimulated:!0,originalEvent:{}});d?m.event.trigger(e,null,b):m.event.dispatch.call(b,e),e.isDefaultPrevented()&&c.preventDefault()}},m.removeEvent=y.removeEventListener?function(a,b,c){a.removeEventListener&&a.removeEventListener(b,c,!1)}:function(a,b,c){var d="on"+b;a.detachEvent&&(typeof a[d]===K&&(a[d]=null),a.detachEvent(d,c))},m.Event=function(a,b){return this instanceof m.Event?(a&&a.type?(this.originalEvent=a,this.type=a.type,this.isDefaultPrevented=a.defaultPrevented||void 0===a.defaultPrevented&&a.returnValue===!1?ab:bb):this.type=a,b&&m.extend(this,b),this.timeStamp=a&&a.timeStamp||m.now(),void(this[m.expando]=!0)):new m.Event(a,b)},m.Event.prototype={isDefaultPrevented:bb,isPropagationStopped:bb,isImmediatePropagationStopped:bb,preventDefault:function(){var a=this.originalEvent;this.isDefaultPrevented=ab,a&&(a.preventDefault?a.preventDefault():a.returnValue=!1)},stopPropagation:function(){var a=this.originalEvent;this.isPropagationStopped=ab,a&&(a.stopPropagation&&a.stopPropagation(),a.cancelBubble=!0)},stopImmediatePropagation:function(){var a=this.originalEvent;this.isImmediatePropagationStopped=ab,a&&a.stopImmediatePropagation&&a.stopImmediatePropagation(),this.stopPropagation()}},m.each({mouseenter:"mouseover",mouseleave:"mouseout",pointerenter:"pointerover",pointerleave:"pointerout"},function(a,b){m.event.special[a]={delegateType:b,bindType:b,handle:function(a){var c,d=this,e=a.relatedTarget,f=a.handleObj;return(!e||e!==d&&!m.contains(d,e))&&(a.type=f.origType,c=f.handler.apply(this,arguments),a.type=b),c}}}),k.submitBubbles||(m.event.special.submit={setup:function(){return m.nodeName(this,"form")?!1:void m.event.add(this,"click._submit keypress._submit",function(a){var b=a.target,c=m.nodeName(b,"input")||m.nodeName(b,"button")?b.form:void 0;c&&!m._data(c,"submitBubbles")&&(m.event.add(c,"submit._submit",function(a){a._submit_bubble=!0}),m._data(c,"submitBubbles",!0))})},postDispatch:function(a){a._submit_bubble&&(delete a._submit_bubble,this.parentNode&&!a.isTrigger&&m.event.simulate("submit",this.parentNode,a,!0))},teardown:function(){return m.nodeName(this,"form")?!1:void m.event.remove(this,"._submit")}}),k.changeBubbles||(m.event.special.change={setup:function(){return X.test(this.nodeName)?(("checkbox"===this.type||"radio"===this.type)&&(m.event.add(this,"propertychange._change",function(a){"checked"===a.originalEvent.propertyName&&(this._just_changed=!0)}),m.event.add(this,"click._change",function(a){this._just_changed&&!a.isTrigger&&(this._just_changed=!1),m.event.simulate("change",this,a,!0)})),!1):void m.event.add(this,"beforeactivate._change",function(a){var b=a.target;X.test(b.nodeName)&&!m._data(b,"changeBubbles")&&(m.event.add(b,"change._change",function(a){!this.parentNode||a.isSimulated||a.isTrigger||m.event.simulate("change",this.parentNode,a,!0)}),m._data(b,"changeBubbles",!0))})},handle:function(a){var b=a.target;return this!==b||a.isSimulated||a.isTrigger||"radio"!==b.type&&"checkbox"!==b.type?a.handleObj.handler.apply(this,arguments):void 0},teardown:function(){return m.event.remove(this,"._change"),!X.test(this.nodeName)}}),k.focusinBubbles||m.each({focus:"focusin",blur:"focusout"},function(a,b){var c=function(a){m.event.simulate(b,a.target,m.event.fix(a),!0)};m.event.special[b]={setup:function(){var d=this.ownerDocument||this,e=m._data(d,b);e||d.addEventListener(a,c,!0),m._data(d,b,(e||0)+1)},teardown:function(){var d=this.ownerDocument||this,e=m._data(d,b)-1;e?m._data(d,b,e):(d.removeEventListener(a,c,!0),m._removeData(d,b))}}}),m.fn.extend({on:function(a,b,c,d,e){var f,g;if("object"==typeof a){"string"!=typeof b&&(c=c||b,b=void 0);for(f in a)this.on(f,b,c,a[f],e);return this}if(null==c&&null==d?(d=b,c=b=void 0):null==d&&("string"==typeof b?(d=c,c=void 0):(d=c,c=b,b=void 0)),d===!1)d=bb;else if(!d)return this;return 1===e&&(g=d,d=function(a){return m().off(a),g.apply(this,arguments)},d.guid=g.guid||(g.guid=m.guid++)),this.each(function(){m.event.add(this,a,d,c,b)})},one:function(a,b,c,d){return this.on(a,b,c,d,1)},off:function(a,b,c){var d,e;if(a&&a.preventDefault&&a.handleObj)return d=a.handleObj,m(a.delegateTarget).off(d.namespace?d.origType+"."+d.namespace:d.origType,d.selector,d.handler),this;if("object"==typeof a){for(e in a)this.off(e,b,a[e]);return this}return(b===!1||"function"==typeof b)&&(c=b,b=void 0),c===!1&&(c=bb),this.each(function(){m.event.remove(this,a,c,b)})},trigger:function(a,b){return this.each(function(){m.event.trigger(a,b,this)})},triggerHandler:function(a,b){var c=this[0];return c?m.event.trigger(a,b,c,!0):void 0}});function db(a){var b=eb.split("|"),c=a.createDocumentFragment();if(c.createElement)while(b.length)c.createElement(b.pop());return c}var eb="abbr|article|aside|audio|bdi|canvas|data|datalist|details|figcaption|figure|footer|header|hgroup|mark|meter|nav|output|progress|section|summary|time|video",fb=/ jQuery\d+="(?:null|\d+)"/g,gb=new RegExp("<(?:"+eb+")[\\s/>]","i"),hb=/^\s+/,ib=/<(?!area|br|col|embed|hr|img|input|link|meta|param)(([\w:]+)[^>]*)\/>/gi,jb=/<([\w:]+)/,kb=/<tbody/i,lb=/<|&#?\w+;/,mb=/<(?:script|style|link)/i,nb=/checked\s*(?:[^=]|=\s*.checked.)/i,ob=/^$|\/(?:java|ecma)script/i,pb=/^true\/(.*)/,qb=/^\s*<!(?:\[CDATA\[|--)|(?:\]\]|--)>\s*$/g,rb={option:[1,"<select multiple='multiple'>","</select>"],legend:[1,"<fieldset>","</fieldset>"],area:[1,"<map>","</map>"],param:[1,"<object>","</object>"],thead:[1,"<table>","</table>"],tr:[2,"<table><tbody>","</tbody></table>"],col:[2,"<table><tbody></tbody><colgroup>","</colgroup></table>"],td:[3,"<table><tbody><tr>","</tr></tbody></table>"],_default:k.htmlSerialize?[0,"",""]:[1,"X<div>","</div>"]},sb=db(y),tb=sb.appendChild(y.createElement("div"));rb.optgroup=rb.option,rb.tbody=rb.tfoot=rb.colgroup=rb.caption=rb.thead,rb.th=rb.td;function ub(a,b){var c,d,e=0,f=typeof a.getElementsByTagName!==K?a.getElementsByTagName(b||"*"):typeof a.querySelectorAll!==K?a.querySelectorAll(b||"*"):void 0;if(!f)for(f=[],c=a.childNodes||a;null!=(d=c[e]);e++)!b||m.nodeName(d,b)?f.push(d):m.merge(f,ub(d,b));return void 0===b||b&&m.nodeName(a,b)?m.merge([a],f):f}function vb(a){W.test(a.type)&&(a.defaultChecked=a.checked)}function wb(a,b){return m.nodeName(a,"table")&&m.nodeName(11!==b.nodeType?b:b.firstChild,"tr")?a.getElementsByTagName("tbody")[0]||a.appendChild(a.ownerDocument.createElement("tbody")):a}function xb(a){return a.type=(null!==m.find.attr(a,"type"))+"/"+a.type,a}function yb(a){var b=pb.exec(a.type);return b?a.type=b[1]:a.removeAttribute("type"),a}function zb(a,b){for(var c,d=0;null!=(c=a[d]);d++)m._data(c,"globalEval",!b||m._data(b[d],"globalEval"))}function Ab(a,b){if(1===b.nodeType&&m.hasData(a)){var c,d,e,f=m._data(a),g=m._data(b,f),h=f.events;if(h){delete g.handle,g.events={};for(c in h)for(d=0,e=h[c].length;e>d;d++)m.event.add(b,c,h[c][d])}g.data&&(g.data=m.extend({},g.data))}}function Bb(a,b){var c,d,e;if(1===b.nodeType){if(c=b.nodeName.toLowerCase(),!k.noCloneEvent&&b[m.expando]){e=m._data(b);for(d in e.events)m.removeEvent(b,d,e.handle);b.removeAttribute(m.expando)}"script"===c&&b.text!==a.text?(xb(b).text=a.text,yb(b)):"object"===c?(b.parentNode&&(b.outerHTML=a.outerHTML),k.html5Clone&&a.innerHTML&&!m.trim(b.innerHTML)&&(b.innerHTML=a.innerHTML)):"input"===c&&W.test(a.type)?(b.defaultChecked=b.checked=a.checked,b.value!==a.value&&(b.value=a.value)):"option"===c?b.defaultSelected=b.selected=a.defaultSelected:("input"===c||"textarea"===c)&&(b.defaultValue=a.defaultValue)}}m.extend({clone:function(a,b,c){var d,e,f,g,h,i=m.contains(a.ownerDocument,a);if(k.html5Clone||m.isXMLDoc(a)||!gb.test("<"+a.nodeName+">")?f=a.cloneNode(!0):(tb.innerHTML=a.outerHTML,tb.removeChild(f=tb.firstChild)),!(k.noCloneEvent&&k.noCloneChecked||1!==a.nodeType&&11!==a.nodeType||m.isXMLDoc(a)))for(d=ub(f),h=ub(a),g=0;null!=(e=h[g]);++g)d[g]&&Bb(e,d[g]);if(b)if(c)for(h=h||ub(a),d=d||ub(f),g=0;null!=(e=h[g]);g++)Ab(e,d[g]);else Ab(a,f);return d=ub(f,"script"),d.length>0&&zb(d,!i&&ub(a,"script")),d=h=e=null,f},buildFragment:function(a,b,c,d){for(var e,f,g,h,i,j,l,n=a.length,o=db(b),p=[],q=0;n>q;q++)if(f=a[q],f||0===f)if("object"===m.type(f))m.merge(p,f.nodeType?[f]:f);else if(lb.test(f)){h=h||o.appendChild(b.createElement("div")),i=(jb.exec(f)||["",""])[1].toLowerCase(),l=rb[i]||rb._default,h.innerHTML=l[1]+f.replace(ib,"<$1></$2>")+l[2],e=l[0];while(e--)h=h.lastChild;if(!k.leadingWhitespace&&hb.test(f)&&p.push(b.createTextNode(hb.exec(f)[0])),!k.tbody){f="table"!==i||kb.test(f)?"<table>"!==l[1]||kb.test(f)?0:h:h.firstChild,e=f&&f.childNodes.length;while(e--)m.nodeName(j=f.childNodes[e],"tbody")&&!j.childNodes.length&&f.removeChild(j)}m.merge(p,h.childNodes),h.textContent="";while(h.firstChild)h.removeChild(h.firstChild);h=o.lastChild}else p.push(b.createTextNode(f));h&&o.removeChild(h),k.appendChecked||m.grep(ub(p,"input"),vb),q=0;while(f=p[q++])if((!d||-1===m.inArray(f,d))&&(g=m.contains(f.ownerDocument,f),h=ub(o.appendChild(f),"script"),g&&zb(h),c)){e=0;while(f=h[e++])ob.test(f.type||"")&&c.push(f)}return h=null,o},cleanData:function(a,b){for(var d,e,f,g,h=0,i=m.expando,j=m.cache,l=k.deleteExpando,n=m.event.special;null!=(d=a[h]);h++)if((b||m.acceptData(d))&&(f=d[i],g=f&&j[f])){if(g.events)for(e in g.events)n[e]?m.event.remove(d,e):m.removeEvent(d,e,g.handle);j[f]&&(delete j[f],l?delete d[i]:typeof d.removeAttribute!==K?d.removeAttribute(i):d[i]=null,c.push(f))}}}),m.fn.extend({text:function(a){return V(this,function(a){return void 0===a?m.text(this):this.empty().append((this[0]&&this[0].ownerDocument||y).createTextNode(a))},null,a,arguments.length)},append:function(){return this.domManip(arguments,function(a){if(1===this.nodeType||11===this.nodeType||9===this.nodeType){var b=wb(this,a);b.appendChild(a)}})},prepend:function(){return this.domManip(arguments,function(a){if(1===this.nodeType||11===this.nodeType||9===this.nodeType){var b=wb(this,a);b.insertBefore(a,b.firstChild)}})},before:function(){return this.domManip(arguments,function(a){this.parentNode&&this.parentNode.insertBefore(a,this)})},after:function(){return this.domManip(arguments,function(a){this.parentNode&&this.parentNode.insertBefore(a,this.nextSibling)})},remove:function(a,b){for(var c,d=a?m.filter(a,this):this,e=0;null!=(c=d[e]);e++)b||1!==c.nodeType||m.cleanData(ub(c)),c.parentNode&&(b&&m.contains(c.ownerDocument,c)&&zb(ub(c,"script")),c.parentNode.removeChild(c));return this},empty:function(){for(var a,b=0;null!=(a=this[b]);b++){1===a.nodeType&&m.cleanData(ub(a,!1));while(a.firstChild)a.removeChild(a.firstChild);a.options&&m.nodeName(a,"select")&&(a.options.length=0)}return this},clone:function(a,b){return a=null==a?!1:a,b=null==b?a:b,this.map(function(){return m.clone(this,a,b)})},html:function(a){return V(this,function(a){var b=this[0]||{},c=0,d=this.length;if(void 0===a)return 1===b.nodeType?b.innerHTML.replace(fb,""):void 0;if(!("string"!=typeof a||mb.test(a)||!k.htmlSerialize&&gb.test(a)||!k.leadingWhitespace&&hb.test(a)||rb[(jb.exec(a)||["",""])[1].toLowerCase()])){a=a.replace(ib,"<$1></$2>");try{for(;d>c;c++)b=this[c]||{},1===b.nodeType&&(m.cleanData(ub(b,!1)),b.innerHTML=a);b=0}catch(e){}}b&&this.empty().append(a)},null,a,arguments.length)},replaceWith:function(){var a=arguments[0];return this.domManip(arguments,function(b){a=this.parentNode,m.cleanData(ub(this)),a&&a.replaceChild(b,this)}),a&&(a.length||a.nodeType)?this:this.remove()},detach:function(a){return this.remove(a,!0)},domManip:function(a,b){a=e.apply([],a);var c,d,f,g,h,i,j=0,l=this.length,n=this,o=l-1,p=a[0],q=m.isFunction(p);if(q||l>1&&"string"==typeof p&&!k.checkClone&&nb.test(p))return this.each(function(c){var d=n.eq(c);q&&(a[0]=p.call(this,c,d.html())),d.domManip(a,b)});if(l&&(i=m.buildFragment(a,this[0].ownerDocument,!1,this),c=i.firstChild,1===i.childNodes.length&&(i=c),c)){for(g=m.map(ub(i,"script"),xb),f=g.length;l>j;j++)d=i,j!==o&&(d=m.clone(d,!0,!0),f&&m.merge(g,ub(d,"script"))),b.call(this[j],d,j);if(f)for(h=g[g.length-1].ownerDocument,m.map(g,yb),j=0;f>j;j++)d=g[j],ob.test(d.type||"")&&!m._data(d,"globalEval")&&m.contains(h,d)&&(d.src?m._evalUrl&&m._evalUrl(d.src):m.globalEval((d.text||d.textContent||d.innerHTML||"").replace(qb,"")));i=c=null}return this}}),m.each({appendTo:"append",prependTo:"prepend",insertBefore:"before",insertAfter:"after",replaceAll:"replaceWith"},function(a,b){m.fn[a]=function(a){for(var c,d=0,e=[],g=m(a),h=g.length-1;h>=d;d++)c=d===h?this:this.clone(!0),m(g[d])[b](c),f.apply(e,c.get());return this.pushStack(e)}});var Cb,Db={};function Eb(b,c){var d,e=m(c.createElement(b)).appendTo(c.body),f=a.getDefaultComputedStyle&&(d=a.getDefaultComputedStyle(e[0]))?d.display:m.css(e[0],"display");return e.detach(),f}function Fb(a){var b=y,c=Db[a];return c||(c=Eb(a,b),"none"!==c&&c||(Cb=(Cb||m("<iframe frameborder='0' width='0' height='0'/>")).appendTo(b.documentElement),b=(Cb[0].contentWindow||Cb[0].contentDocument).document,b.write(),b.close(),c=Eb(a,b),Cb.detach()),Db[a]=c),c}!function(){var a;k.shrinkWrapBlocks=function(){if(null!=a)return a;a=!1;var b,c,d;return c=y.getElementsByTagName("body")[0],c&&c.style?(b=y.createElement("div"),d=y.createElement("div"),d.style.cssText="position:absolute;border:0;width:0;height:0;top:0;left:-9999px",c.appendChild(d).appendChild(b),typeof b.style.zoom!==K&&(b.style.cssText="-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;display:block;margin:0;border:0;padding:1px;width:1px;zoom:1",b.appendChild(y.createElement("div")).style.width="5px",a=3!==b.offsetWidth),c.removeChild(d),a):void 0}}();var Gb=/^margin/,Hb=new RegExp("^("+S+")(?!px)[a-z%]+$","i"),Ib,Jb,Kb=/^(top|right|bottom|left)$/;a.getComputedStyle?(Ib=function(b){return b.ownerDocument.defaultView.opener?b.ownerDocument.defaultView.getComputedStyle(b,null):a.getComputedStyle(b,null)},Jb=function(a,b,c){var d,e,f,g,h=a.style;return c=c||Ib(a),g=c?c.getPropertyValue(b)||c[b]:void 0,c&&(""!==g||m.contains(a.ownerDocument,a)||(g=m.style(a,b)),Hb.test(g)&&Gb.test(b)&&(d=h.width,e=h.minWidth,f=h.maxWidth,h.minWidth=h.maxWidth=h.width=g,g=c.width,h.width=d,h.minWidth=e,h.maxWidth=f)),void 0===g?g:g+""}):y.documentElement.currentStyle&&(Ib=function(a){return a.currentStyle},Jb=function(a,b,c){var d,e,f,g,h=a.style;return c=c||Ib(a),g=c?c[b]:void 0,null==g&&h&&h[b]&&(g=h[b]),Hb.test(g)&&!Kb.test(b)&&(d=h.left,e=a.runtimeStyle,f=e&&e.left,f&&(e.left=a.currentStyle.left),h.left="fontSize"===b?"1em":g,g=h.pixelLeft+"px",h.left=d,f&&(e.left=f)),void 0===g?g:g+""||"auto"});function Lb(a,b){return{get:function(){var c=a();if(null!=c)return c?void delete this.get:(this.get=b).apply(this,arguments)}}}!function(){var b,c,d,e,f,g,h;if(b=y.createElement("div"),b.innerHTML=" <link/><table></table><a href='/a'>a</a><input type='checkbox'/>",d=b.getElementsByTagName("a")[0],c=d&&d.style){c.cssText="float:left;opacity:.5",k.opacity="0.5"===c.opacity,k.cssFloat=!!c.cssFloat,b.style.backgroundClip="content-box",b.cloneNode(!0).style.backgroundClip="",k.clearCloneStyle="content-box"===b.style.backgroundClip,k.boxSizing=""===c.boxSizing||""===c.MozBoxSizing||""===c.WebkitBoxSizing,m.extend(k,{reliableHiddenOffsets:function(){return null==g&&i(),g},boxSizingReliable:function(){return null==f&&i(),f},pixelPosition:function(){return null==e&&i(),e},reliableMarginRight:function(){return null==h&&i(),h}});function i(){var b,c,d,i;c=y.getElementsByTagName("body")[0],c&&c.style&&(b=y.createElement("div"),d=y.createElement("div"),d.style.cssText="position:absolute;border:0;width:0;height:0;top:0;left:-9999px",c.appendChild(d).appendChild(b),b.style.cssText="-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box;display:block;margin-top:1%;top:1%;border:1px;padding:1px;width:4px;position:absolute",e=f=!1,h=!0,a.getComputedStyle&&(e="1%"!==(a.getComputedStyle(b,null)||{}).top,f="4px"===(a.getComputedStyle(b,null)||{width:"4px"}).width,i=b.appendChild(y.createElement("div")),i.style.cssText=b.style.cssText="-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;display:block;margin:0;border:0;padding:0",i.style.marginRight=i.style.width="0",b.style.width="1px",h=!parseFloat((a.getComputedStyle(i,null)||{}).marginRight),b.removeChild(i)),b.innerHTML="<table><tr><td></td><td>t</td></tr></table>",i=b.getElementsByTagName("td"),i[0].style.cssText="margin:0;border:0;padding:0;display:none",g=0===i[0].offsetHeight,g&&(i[0].style.display="",i[1].style.display="none",g=0===i[0].offsetHeight),c.removeChild(d))}}}(),m.swap=function(a,b,c,d){var e,f,g={};for(f in b)g[f]=a.style[f],a.style[f]=b[f];e=c.apply(a,d||[]);for(f in b)a.style[f]=g[f];return e};var Mb=/alpha\([^)]*\)/i,Nb=/opacity\s*=\s*([^)]*)/,Ob=/^(none|table(?!-c[ea]).+)/,Pb=new RegExp("^("+S+")(.*)$","i"),Qb=new RegExp("^([+-])=("+S+")","i"),Rb={position:"absolute",visibility:"hidden",display:"block"},Sb={letterSpacing:"0",fontWeight:"400"},Tb=["Webkit","O","Moz","ms"];function Ub(a,b){if(b in a)return b;var c=b.charAt(0).toUpperCase()+b.slice(1),d=b,e=Tb.length;while(e--)if(b=Tb[e]+c,b in a)return b;return d}function Vb(a,b){for(var c,d,e,f=[],g=0,h=a.length;h>g;g++)d=a[g],d.style&&(f[g]=m._data(d,"olddisplay"),c=d.style.display,b?(f[g]||"none"!==c||(d.style.display=""),""===d.style.display&&U(d)&&(f[g]=m._data(d,"olddisplay",Fb(d.nodeName)))):(e=U(d),(c&&"none"!==c||!e)&&m._data(d,"olddisplay",e?c:m.css(d,"display"))));for(g=0;h>g;g++)d=a[g],d.style&&(b&&"none"!==d.style.display&&""!==d.style.display||(d.style.display=b?f[g]||"":"none"));return a}function Wb(a,b,c){var d=Pb.exec(b);return d?Math.max(0,d[1]-(c||0))+(d[2]||"px"):b}function Xb(a,b,c,d,e){for(var f=c===(d?"border":"content")?4:"width"===b?1:0,g=0;4>f;f+=2)"margin"===c&&(g+=m.css(a,c+T[f],!0,e)),d?("content"===c&&(g-=m.css(a,"padding"+T[f],!0,e)),"margin"!==c&&(g-=m.css(a,"border"+T[f]+"Width",!0,e))):(g+=m.css(a,"padding"+T[f],!0,e),"padding"!==c&&(g+=m.css(a,"border"+T[f]+"Width",!0,e)));return g}function Yb(a,b,c){var d=!0,e="width"===b?a.offsetWidth:a.offsetHeight,f=Ib(a),g=k.boxSizing&&"border-box"===m.css(a,"boxSizing",!1,f);if(0>=e||null==e){if(e=Jb(a,b,f),(0>e||null==e)&&(e=a.style[b]),Hb.test(e))return e;d=g&&(k.boxSizingReliable()||e===a.style[b]),e=parseFloat(e)||0}return e+Xb(a,b,c||(g?"border":"content"),d,f)+"px"}m.extend({cssHooks:{opacity:{get:function(a,b){if(b){var c=Jb(a,"opacity");return""===c?"1":c}}}},cssNumber:{columnCount:!0,fillOpacity:!0,flexGrow:!0,flexShrink:!0,fontWeight:!0,lineHeight:!0,opacity:!0,order:!0,orphans:!0,widows:!0,zIndex:!0,zoom:!0},cssProps:{"float":k.cssFloat?"cssFloat":"styleFloat"},style:function(a,b,c,d){if(a&&3!==a.nodeType&&8!==a.nodeType&&a.style){var e,f,g,h=m.camelCase(b),i=a.style;if(b=m.cssProps[h]||(m.cssProps[h]=Ub(i,h)),g=m.cssHooks[b]||m.cssHooks[h],void 0===c)return g&&"get"in g&&void 0!==(e=g.get(a,!1,d))?e:i[b];if(f=typeof c,"string"===f&&(e=Qb.exec(c))&&(c=(e[1]+1)*e[2]+parseFloat(m.css(a,b)),f="number"),null!=c&&c===c&&("number"!==f||m.cssNumber[h]||(c+="px"),k.clearCloneStyle||""!==c||0!==b.indexOf("background")||(i[b]="inherit"),!(g&&"set"in g&&void 0===(c=g.set(a,c,d)))))try{i[b]=c}catch(j){}}},css:function(a,b,c,d){var e,f,g,h=m.camelCase(b);return b=m.cssProps[h]||(m.cssProps[h]=Ub(a.style,h)),g=m.cssHooks[b]||m.cssHooks[h],g&&"get"in g&&(f=g.get(a,!0,c)),void 0===f&&(f=Jb(a,b,d)),"normal"===f&&b in Sb&&(f=Sb[b]),""===c||c?(e=parseFloat(f),c===!0||m.isNumeric(e)?e||0:f):f}}),m.each(["height","width"],function(a,b){m.cssHooks[b]={get:function(a,c,d){return c?Ob.test(m.css(a,"display"))&&0===a.offsetWidth?m.swap(a,Rb,function(){return Yb(a,b,d)}):Yb(a,b,d):void 0},set:function(a,c,d){var e=d&&Ib(a);return Wb(a,c,d?Xb(a,b,d,k.boxSizing&&"border-box"===m.css(a,"boxSizing",!1,e),e):0)}}}),k.opacity||(m.cssHooks.opacity={get:function(a,b){return Nb.test((b&&a.currentStyle?a.currentStyle.filter:a.style.filter)||"")?.01*parseFloat(RegExp.$1)+"":b?"1":""},set:function(a,b){var c=a.style,d=a.currentStyle,e=m.isNumeric(b)?"alpha(opacity="+100*b+")":"",f=d&&d.filter||c.filter||"";c.zoom=1,(b>=1||""===b)&&""===m.trim(f.replace(Mb,""))&&c.removeAttribute&&(c.removeAttribute("filter"),""===b||d&&!d.filter)||(c.filter=Mb.test(f)?f.replace(Mb,e):f+" "+e)}}),m.cssHooks.marginRight=Lb(k.reliableMarginRight,function(a,b){return b?m.swap(a,{display:"inline-block"},Jb,[a,"marginRight"]):void 0}),m.each({margin:"",padding:"",border:"Width"},function(a,b){m.cssHooks[a+b]={expand:function(c){for(var d=0,e={},f="string"==typeof c?c.split(" "):[c];4>d;d++)e[a+T[d]+b]=f[d]||f[d-2]||f[0];return e}},Gb.test(a)||(m.cssHooks[a+b].set=Wb)}),m.fn.extend({css:function(a,b){return V(this,function(a,b,c){var d,e,f={},g=0;if(m.isArray(b)){for(d=Ib(a),e=b.length;e>g;g++)f[b[g]]=m.css(a,b[g],!1,d);return f}return void 0!==c?m.style(a,b,c):m.css(a,b)},a,b,arguments.length>1)},show:function(){return Vb(this,!0)},hide:function(){return Vb(this)},toggle:function(a){return"boolean"==typeof a?a?this.show():this.hide():this.each(function(){U(this)?m(this).show():m(this).hide()})}});function Zb(a,b,c,d,e){return new Zb.prototype.init(a,b,c,d,e)
+}m.Tween=Zb,Zb.prototype={constructor:Zb,init:function(a,b,c,d,e,f){this.elem=a,this.prop=c,this.easing=e||"swing",this.options=b,this.start=this.now=this.cur(),this.end=d,this.unit=f||(m.cssNumber[c]?"":"px")},cur:function(){var a=Zb.propHooks[this.prop];return a&&a.get?a.get(this):Zb.propHooks._default.get(this)},run:function(a){var b,c=Zb.propHooks[this.prop];return this.pos=b=this.options.duration?m.easing[this.easing](a,this.options.duration*a,0,1,this.options.duration):a,this.now=(this.end-this.start)*b+this.start,this.options.step&&this.options.step.call(this.elem,this.now,this),c&&c.set?c.set(this):Zb.propHooks._default.set(this),this}},Zb.prototype.init.prototype=Zb.prototype,Zb.propHooks={_default:{get:function(a){var b;return null==a.elem[a.prop]||a.elem.style&&null!=a.elem.style[a.prop]?(b=m.css(a.elem,a.prop,""),b&&"auto"!==b?b:0):a.elem[a.prop]},set:function(a){m.fx.step[a.prop]?m.fx.step[a.prop](a):a.elem.style&&(null!=a.elem.style[m.cssProps[a.prop]]||m.cssHooks[a.prop])?m.style(a.elem,a.prop,a.now+a.unit):a.elem[a.prop]=a.now}}},Zb.propHooks.scrollTop=Zb.propHooks.scrollLeft={set:function(a){a.elem.nodeType&&a.elem.parentNode&&(a.elem[a.prop]=a.now)}},m.easing={linear:function(a){return a},swing:function(a){return.5-Math.cos(a*Math.PI)/2}},m.fx=Zb.prototype.init,m.fx.step={};var $b,_b,ac=/^(?:toggle|show|hide)$/,bc=new RegExp("^(?:([+-])=|)("+S+")([a-z%]*)$","i"),cc=/queueHooks$/,dc=[ic],ec={"*":[function(a,b){var c=this.createTween(a,b),d=c.cur(),e=bc.exec(b),f=e&&e[3]||(m.cssNumber[a]?"":"px"),g=(m.cssNumber[a]||"px"!==f&&+d)&&bc.exec(m.css(c.elem,a)),h=1,i=20;if(g&&g[3]!==f){f=f||g[3],e=e||[],g=+d||1;do h=h||".5",g/=h,m.style(c.elem,a,g+f);while(h!==(h=c.cur()/d)&&1!==h&&--i)}return e&&(g=c.start=+g||+d||0,c.unit=f,c.end=e[1]?g+(e[1]+1)*e[2]:+e[2]),c}]};function fc(){return setTimeout(function(){$b=void 0}),$b=m.now()}function gc(a,b){var c,d={height:a},e=0;for(b=b?1:0;4>e;e+=2-b)c=T[e],d["margin"+c]=d["padding"+c]=a;return b&&(d.opacity=d.width=a),d}function hc(a,b,c){for(var d,e=(ec[b]||[]).concat(ec["*"]),f=0,g=e.length;g>f;f++)if(d=e[f].call(c,b,a))return d}function ic(a,b,c){var d,e,f,g,h,i,j,l,n=this,o={},p=a.style,q=a.nodeType&&U(a),r=m._data(a,"fxshow");c.queue||(h=m._queueHooks(a,"fx"),null==h.unqueued&&(h.unqueued=0,i=h.empty.fire,h.empty.fire=function(){h.unqueued||i()}),h.unqueued++,n.always(function(){n.always(function(){h.unqueued--,m.queue(a,"fx").length||h.empty.fire()})})),1===a.nodeType&&("height"in b||"width"in b)&&(c.overflow=[p.overflow,p.overflowX,p.overflowY],j=m.css(a,"display"),l="none"===j?m._data(a,"olddisplay")||Fb(a.nodeName):j,"inline"===l&&"none"===m.css(a,"float")&&(k.inlineBlockNeedsLayout&&"inline"!==Fb(a.nodeName)?p.zoom=1:p.display="inline-block")),c.overflow&&(p.overflow="hidden",k.shrinkWrapBlocks()||n.always(function(){p.overflow=c.overflow[0],p.overflowX=c.overflow[1],p.overflowY=c.overflow[2]}));for(d in b)if(e=b[d],ac.exec(e)){if(delete b[d],f=f||"toggle"===e,e===(q?"hide":"show")){if("show"!==e||!r||void 0===r[d])continue;q=!0}o[d]=r&&r[d]||m.style(a,d)}else j=void 0;if(m.isEmptyObject(o))"inline"===("none"===j?Fb(a.nodeName):j)&&(p.display=j);else{r?"hidden"in r&&(q=r.hidden):r=m._data(a,"fxshow",{}),f&&(r.hidden=!q),q?m(a).show():n.done(function(){m(a).hide()}),n.done(function(){var b;m._removeData(a,"fxshow");for(b in o)m.style(a,b,o[b])});for(d in o)g=hc(q?r[d]:0,d,n),d in r||(r[d]=g.start,q&&(g.end=g.start,g.start="width"===d||"height"===d?1:0))}}function jc(a,b){var c,d,e,f,g;for(c in a)if(d=m.camelCase(c),e=b[d],f=a[c],m.isArray(f)&&(e=f[1],f=a[c]=f[0]),c!==d&&(a[d]=f,delete a[c]),g=m.cssHooks[d],g&&"expand"in g){f=g.expand(f),delete a[d];for(c in f)c in a||(a[c]=f[c],b[c]=e)}else b[d]=e}function kc(a,b,c){var d,e,f=0,g=dc.length,h=m.Deferred().always(function(){delete i.elem}),i=function(){if(e)return!1;for(var b=$b||fc(),c=Math.max(0,j.startTime+j.duration-b),d=c/j.duration||0,f=1-d,g=0,i=j.tweens.length;i>g;g++)j.tweens[g].run(f);return h.notifyWith(a,[j,f,c]),1>f&&i?c:(h.resolveWith(a,[j]),!1)},j=h.promise({elem:a,props:m.extend({},b),opts:m.extend(!0,{specialEasing:{}},c),originalProperties:b,originalOptions:c,startTime:$b||fc(),duration:c.duration,tweens:[],createTween:function(b,c){var d=m.Tween(a,j.opts,b,c,j.opts.specialEasing[b]||j.opts.easing);return j.tweens.push(d),d},stop:function(b){var c=0,d=b?j.tweens.length:0;if(e)return this;for(e=!0;d>c;c++)j.tweens[c].run(1);return b?h.resolveWith(a,[j,b]):h.rejectWith(a,[j,b]),this}}),k=j.props;for(jc(k,j.opts.specialEasing);g>f;f++)if(d=dc[f].call(j,a,k,j.opts))return d;return m.map(k,hc,j),m.isFunction(j.opts.start)&&j.opts.start.call(a,j),m.fx.timer(m.extend(i,{elem:a,anim:j,queue:j.opts.queue})),j.progress(j.opts.progress).done(j.opts.done,j.opts.complete).fail(j.opts.fail).always(j.opts.always)}m.Animation=m.extend(kc,{tweener:function(a,b){m.isFunction(a)?(b=a,a=["*"]):a=a.split(" ");for(var c,d=0,e=a.length;e>d;d++)c=a[d],ec[c]=ec[c]||[],ec[c].unshift(b)},prefilter:function(a,b){b?dc.unshift(a):dc.push(a)}}),m.speed=function(a,b,c){var d=a&&"object"==typeof a?m.extend({},a):{complete:c||!c&&b||m.isFunction(a)&&a,duration:a,easing:c&&b||b&&!m.isFunction(b)&&b};return d.duration=m.fx.off?0:"number"==typeof d.duration?d.duration:d.duration in m.fx.speeds?m.fx.speeds[d.duration]:m.fx.speeds._default,(null==d.queue||d.queue===!0)&&(d.queue="fx"),d.old=d.complete,d.complete=function(){m.isFunction(d.old)&&d.old.call(this),d.queue&&m.dequeue(this,d.queue)},d},m.fn.extend({fadeTo:function(a,b,c,d){return this.filter(U).css("opacity",0).show().end().animate({opacity:b},a,c,d)},animate:function(a,b,c,d){var e=m.isEmptyObject(a),f=m.speed(b,c,d),g=function(){var b=kc(this,m.extend({},a),f);(e||m._data(this,"finish"))&&b.stop(!0)};return g.finish=g,e||f.queue===!1?this.each(g):this.queue(f.queue,g)},stop:function(a,b,c){var d=function(a){var b=a.stop;delete a.stop,b(c)};return"string"!=typeof a&&(c=b,b=a,a=void 0),b&&a!==!1&&this.queue(a||"fx",[]),this.each(function(){var b=!0,e=null!=a&&a+"queueHooks",f=m.timers,g=m._data(this);if(e)g[e]&&g[e].stop&&d(g[e]);else for(e in g)g[e]&&g[e].stop&&cc.test(e)&&d(g[e]);for(e=f.length;e--;)f[e].elem!==this||null!=a&&f[e].queue!==a||(f[e].anim.stop(c),b=!1,f.splice(e,1));(b||!c)&&m.dequeue(this,a)})},finish:function(a){return a!==!1&&(a=a||"fx"),this.each(function(){var b,c=m._data(this),d=c[a+"queue"],e=c[a+"queueHooks"],f=m.timers,g=d?d.length:0;for(c.finish=!0,m.queue(this,a,[]),e&&e.stop&&e.stop.call(this,!0),b=f.length;b--;)f[b].elem===this&&f[b].queue===a&&(f[b].anim.stop(!0),f.splice(b,1));for(b=0;g>b;b++)d[b]&&d[b].finish&&d[b].finish.call(this);delete c.finish})}}),m.each(["toggle","show","hide"],function(a,b){var c=m.fn[b];m.fn[b]=function(a,d,e){return null==a||"boolean"==typeof a?c.apply(this,arguments):this.animate(gc(b,!0),a,d,e)}}),m.each({slideDown:gc("show"),slideUp:gc("hide"),slideToggle:gc("toggle"),fadeIn:{opacity:"show"},fadeOut:{opacity:"hide"},fadeToggle:{opacity:"toggle"}},function(a,b){m.fn[a]=function(a,c,d){return this.animate(b,a,c,d)}}),m.timers=[],m.fx.tick=function(){var a,b=m.timers,c=0;for($b=m.now();c<b.length;c++)a=b[c],a()||b[c]!==a||b.splice(c--,1);b.length||m.fx.stop(),$b=void 0},m.fx.timer=function(a){m.timers.push(a),a()?m.fx.start():m.timers.pop()},m.fx.interval=13,m.fx.start=function(){_b||(_b=setInterval(m.fx.tick,m.fx.interval))},m.fx.stop=function(){clearInterval(_b),_b=null},m.fx.speeds={slow:600,fast:200,_default:400},m.fn.delay=function(a,b){return a=m.fx?m.fx.speeds[a]||a:a,b=b||"fx",this.queue(b,function(b,c){var d=setTimeout(b,a);c.stop=function(){clearTimeout(d)}})},function(){var a,b,c,d,e;b=y.createElement("div"),b.setAttribute("className","t"),b.innerHTML=" <link/><table></table><a href='/a'>a</a><input type='checkbox'/>",d=b.getElementsByTagName("a")[0],c=y.createElement("select"),e=c.appendChild(y.createElement("option")),a=b.getElementsByTagName("input")[0],d.style.cssText="top:1px",k.getSetAttribute="t"!==b.className,k.style=/top/.test(d.getAttribute("style")),k.hrefNormalized="/a"===d.getAttribute("href"),k.checkOn=!!a.value,k.optSelected=e.selected,k.enctype=!!y.createElement("form").enctype,c.disabled=!0,k.optDisabled=!e.disabled,a=y.createElement("input"),a.setAttribute("value",""),k.input=""===a.getAttribute("value"),a.value="t",a.setAttribute("type","radio"),k.radioValue="t"===a.value}();var lc=/\r/g;m.fn.extend({val:function(a){var b,c,d,e=this[0];{if(arguments.length)return d=m.isFunction(a),this.each(function(c){var e;1===this.nodeType&&(e=d?a.call(this,c,m(this).val()):a,null==e?e="":"number"==typeof e?e+="":m.isArray(e)&&(e=m.map(e,function(a){return null==a?"":a+""})),b=m.valHooks[this.type]||m.valHooks[this.nodeName.toLowerCase()],b&&"set"in b&&void 0!==b.set(this,e,"value")||(this.value=e))});if(e)return b=m.valHooks[e.type]||m.valHooks[e.nodeName.toLowerCase()],b&&"get"in b&&void 0!==(c=b.get(e,"value"))?c:(c=e.value,"string"==typeof c?c.replace(lc,""):null==c?"":c)}}}),m.extend({valHooks:{option:{get:function(a){var b=m.find.attr(a,"value");return null!=b?b:m.trim(m.text(a))}},select:{get:function(a){for(var b,c,d=a.options,e=a.selectedIndex,f="select-one"===a.type||0>e,g=f?null:[],h=f?e+1:d.length,i=0>e?h:f?e:0;h>i;i++)if(c=d[i],!(!c.selected&&i!==e||(k.optDisabled?c.disabled:null!==c.getAttribute("disabled"))||c.parentNode.disabled&&m.nodeName(c.parentNode,"optgroup"))){if(b=m(c).val(),f)return b;g.push(b)}return g},set:function(a,b){var c,d,e=a.options,f=m.makeArray(b),g=e.length;while(g--)if(d=e[g],m.inArray(m.valHooks.option.get(d),f)>=0)try{d.selected=c=!0}catch(h){d.scrollHeight}else d.selected=!1;return c||(a.selectedIndex=-1),e}}}}),m.each(["radio","checkbox"],function(){m.valHooks[this]={set:function(a,b){return m.isArray(b)?a.checked=m.inArray(m(a).val(),b)>=0:void 0}},k.checkOn||(m.valHooks[this].get=function(a){return null===a.getAttribute("value")?"on":a.value})});var mc,nc,oc=m.expr.attrHandle,pc=/^(?:checked|selected)$/i,qc=k.getSetAttribute,rc=k.input;m.fn.extend({attr:function(a,b){return V(this,m.attr,a,b,arguments.length>1)},removeAttr:function(a){return this.each(function(){m.removeAttr(this,a)})}}),m.extend({attr:function(a,b,c){var d,e,f=a.nodeType;if(a&&3!==f&&8!==f&&2!==f)return typeof a.getAttribute===K?m.prop(a,b,c):(1===f&&m.isXMLDoc(a)||(b=b.toLowerCase(),d=m.attrHooks[b]||(m.expr.match.bool.test(b)?nc:mc)),void 0===c?d&&"get"in d&&null!==(e=d.get(a,b))?e:(e=m.find.attr(a,b),null==e?void 0:e):null!==c?d&&"set"in d&&void 0!==(e=d.set(a,c,b))?e:(a.setAttribute(b,c+""),c):void m.removeAttr(a,b))},removeAttr:function(a,b){var c,d,e=0,f=b&&b.match(E);if(f&&1===a.nodeType)while(c=f[e++])d=m.propFix[c]||c,m.expr.match.bool.test(c)?rc&&qc||!pc.test(c)?a[d]=!1:a[m.camelCase("default-"+c)]=a[d]=!1:m.attr(a,c,""),a.removeAttribute(qc?c:d)},attrHooks:{type:{set:function(a,b){if(!k.radioValue&&"radio"===b&&m.nodeName(a,"input")){var c=a.value;return a.setAttribute("type",b),c&&(a.value=c),b}}}}}),nc={set:function(a,b,c){return b===!1?m.removeAttr(a,c):rc&&qc||!pc.test(c)?a.setAttribute(!qc&&m.propFix[c]||c,c):a[m.camelCase("default-"+c)]=a[c]=!0,c}},m.each(m.expr.match.bool.source.match(/\w+/g),function(a,b){var c=oc[b]||m.find.attr;oc[b]=rc&&qc||!pc.test(b)?function(a,b,d){var e,f;return d||(f=oc[b],oc[b]=e,e=null!=c(a,b,d)?b.toLowerCase():null,oc[b]=f),e}:function(a,b,c){return c?void 0:a[m.camelCase("default-"+b)]?b.toLowerCase():null}}),rc&&qc||(m.attrHooks.value={set:function(a,b,c){return m.nodeName(a,"input")?void(a.defaultValue=b):mc&&mc.set(a,b,c)}}),qc||(mc={set:function(a,b,c){var d=a.getAttributeNode(c);return d||a.setAttributeNode(d=a.ownerDocument.createAttribute(c)),d.value=b+="","value"===c||b===a.getAttribute(c)?b:void 0}},oc.id=oc.name=oc.coords=function(a,b,c){var d;return c?void 0:(d=a.getAttributeNode(b))&&""!==d.value?d.value:null},m.valHooks.button={get:function(a,b){var c=a.getAttributeNode(b);return c&&c.specified?c.value:void 0},set:mc.set},m.attrHooks.contenteditable={set:function(a,b,c){mc.set(a,""===b?!1:b,c)}},m.each(["width","height"],function(a,b){m.attrHooks[b]={set:function(a,c){return""===c?(a.setAttribute(b,"auto"),c):void 0}}})),k.style||(m.attrHooks.style={get:function(a){return a.style.cssText||void 0},set:function(a,b){return a.style.cssText=b+""}});var sc=/^(?:input|select|textarea|button|object)$/i,tc=/^(?:a|area)$/i;m.fn.extend({prop:function(a,b){return V(this,m.prop,a,b,arguments.length>1)},removeProp:function(a){return a=m.propFix[a]||a,this.each(function(){try{this[a]=void 0,delete this[a]}catch(b){}})}}),m.extend({propFix:{"for":"htmlFor","class":"className"},prop:function(a,b,c){var d,e,f,g=a.nodeType;if(a&&3!==g&&8!==g&&2!==g)return f=1!==g||!m.isXMLDoc(a),f&&(b=m.propFix[b]||b,e=m.propHooks[b]),void 0!==c?e&&"set"in e&&void 0!==(d=e.set(a,c,b))?d:a[b]=c:e&&"get"in e&&null!==(d=e.get(a,b))?d:a[b]},propHooks:{tabIndex:{get:function(a){var b=m.find.attr(a,"tabindex");return b?parseInt(b,10):sc.test(a.nodeName)||tc.test(a.nodeName)&&a.href?0:-1}}}}),k.hrefNormalized||m.each(["href","src"],function(a,b){m.propHooks[b]={get:function(a){return a.getAttribute(b,4)}}}),k.optSelected||(m.propHooks.selected={get:function(a){var b=a.parentNode;return b&&(b.selectedIndex,b.parentNode&&b.parentNode.selectedIndex),null}}),m.each(["tabIndex","readOnly","maxLength","cellSpacing","cellPadding","rowSpan","colSpan","useMap","frameBorder","contentEditable"],function(){m.propFix[this.toLowerCase()]=this}),k.enctype||(m.propFix.enctype="encoding");var uc=/[\t\r\n\f]/g;m.fn.extend({addClass:function(a){var b,c,d,e,f,g,h=0,i=this.length,j="string"==typeof a&&a;if(m.isFunction(a))return this.each(function(b){m(this).addClass(a.call(this,b,this.className))});if(j)for(b=(a||"").match(E)||[];i>h;h++)if(c=this[h],d=1===c.nodeType&&(c.className?(" "+c.className+" ").replace(uc," "):" ")){f=0;while(e=b[f++])d.indexOf(" "+e+" ")<0&&(d+=e+" ");g=m.trim(d),c.className!==g&&(c.className=g)}return this},removeClass:function(a){var b,c,d,e,f,g,h=0,i=this.length,j=0===arguments.length||"string"==typeof a&&a;if(m.isFunction(a))return this.each(function(b){m(this).removeClass(a.call(this,b,this.className))});if(j)for(b=(a||"").match(E)||[];i>h;h++)if(c=this[h],d=1===c.nodeType&&(c.className?(" "+c.className+" ").replace(uc," "):"")){f=0;while(e=b[f++])while(d.indexOf(" "+e+" ")>=0)d=d.replace(" "+e+" "," ");g=a?m.trim(d):"",c.className!==g&&(c.className=g)}return this},toggleClass:function(a,b){var c=typeof a;return"boolean"==typeof b&&"string"===c?b?this.addClass(a):this.removeClass(a):this.each(m.isFunction(a)?function(c){m(this).toggleClass(a.call(this,c,this.className,b),b)}:function(){if("string"===c){var b,d=0,e=m(this),f=a.match(E)||[];while(b=f[d++])e.hasClass(b)?e.removeClass(b):e.addClass(b)}else(c===K||"boolean"===c)&&(this.className&&m._data(this,"__className__",this.className),this.className=this.className||a===!1?"":m._data(this,"__className__")||"")})},hasClass:function(a){for(var b=" "+a+" ",c=0,d=this.length;d>c;c++)if(1===this[c].nodeType&&(" "+this[c].className+" ").replace(uc," ").indexOf(b)>=0)return!0;return!1}}),m.each("blur focus focusin focusout load resize scroll unload click dblclick mousedown mouseup mousemove mouseover mouseout mouseenter mouseleave change select submit keydown keypress keyup error contextmenu".split(" "),function(a,b){m.fn[b]=function(a,c){return arguments.length>0?this.on(b,null,a,c):this.trigger(b)}}),m.fn.extend({hover:function(a,b){return this.mouseenter(a).mouseleave(b||a)},bind:function(a,b,c){return this.on(a,null,b,c)},unbind:function(a,b){return this.off(a,null,b)},delegate:function(a,b,c,d){return this.on(b,a,c,d)},undelegate:function(a,b,c){return 1===arguments.length?this.off(a,"**"):this.off(b,a||"**",c)}});var vc=m.now(),wc=/\?/,xc=/(,)|(\[|{)|(}|])|"(?:[^"\\\r\n]|\\["\\\/bfnrt]|\\u[\da-fA-F]{4})*"\s*:?|true|false|null|-?(?!0\d)\d+(?:\.\d+|)(?:[eE][+-]?\d+|)/g;m.parseJSON=function(b){if(a.JSON&&a.JSON.parse)return a.JSON.parse(b+"");var c,d=null,e=m.trim(b+"");return e&&!m.trim(e.replace(xc,function(a,b,e,f){return c&&b&&(d=0),0===d?a:(c=e||b,d+=!f-!e,"")}))?Function("return "+e)():m.error("Invalid JSON: "+b)},m.parseXML=function(b){var c,d;if(!b||"string"!=typeof b)return null;try{a.DOMParser?(d=new DOMParser,c=d.parseFromString(b,"text/xml")):(c=new ActiveXObject("Microsoft.XMLDOM"),c.async="false",c.loadXML(b))}catch(e){c=void 0}return c&&c.documentElement&&!c.getElementsByTagName("parsererror").length||m.error("Invalid XML: "+b),c};var yc,zc,Ac=/#.*$/,Bc=/([?&])_=[^&]*/,Cc=/^(.*?):[ \t]*([^\r\n]*)\r?$/gm,Dc=/^(?:about|app|app-storage|.+-extension|file|res|widget):$/,Ec=/^(?:GET|HEAD)$/,Fc=/^\/\//,Gc=/^([\w.+-]+:)(?:\/\/(?:[^\/?#]*@|)([^\/?#:]*)(?::(\d+)|)|)/,Hc={},Ic={},Jc="*/".concat("*");try{zc=location.href}catch(Kc){zc=y.createElement("a"),zc.href="",zc=zc.href}yc=Gc.exec(zc.toLowerCase())||[];function Lc(a){return function(b,c){"string"!=typeof b&&(c=b,b="*");var d,e=0,f=b.toLowerCase().match(E)||[];if(m.isFunction(c))while(d=f[e++])"+"===d.charAt(0)?(d=d.slice(1)||"*",(a[d]=a[d]||[]).unshift(c)):(a[d]=a[d]||[]).push(c)}}function Mc(a,b,c,d){var e={},f=a===Ic;function g(h){var i;return e[h]=!0,m.each(a[h]||[],function(a,h){var j=h(b,c,d);return"string"!=typeof j||f||e[j]?f?!(i=j):void 0:(b.dataTypes.unshift(j),g(j),!1)}),i}return g(b.dataTypes[0])||!e["*"]&&g("*")}function Nc(a,b){var c,d,e=m.ajaxSettings.flatOptions||{};for(d in b)void 0!==b[d]&&((e[d]?a:c||(c={}))[d]=b[d]);return c&&m.extend(!0,a,c),a}function Oc(a,b,c){var d,e,f,g,h=a.contents,i=a.dataTypes;while("*"===i[0])i.shift(),void 0===e&&(e=a.mimeType||b.getResponseHeader("Content-Type"));if(e)for(g in h)if(h[g]&&h[g].test(e)){i.unshift(g);break}if(i[0]in c)f=i[0];else{for(g in c){if(!i[0]||a.converters[g+" "+i[0]]){f=g;break}d||(d=g)}f=f||d}return f?(f!==i[0]&&i.unshift(f),c[f]):void 0}function Pc(a,b,c,d){var e,f,g,h,i,j={},k=a.dataTypes.slice();if(k[1])for(g in a.converters)j[g.toLowerCase()]=a.converters[g];f=k.shift();while(f)if(a.responseFields[f]&&(c[a.responseFields[f]]=b),!i&&d&&a.dataFilter&&(b=a.dataFilter(b,a.dataType)),i=f,f=k.shift())if("*"===f)f=i;else if("*"!==i&&i!==f){if(g=j[i+" "+f]||j["* "+f],!g)for(e in j)if(h=e.split(" "),h[1]===f&&(g=j[i+" "+h[0]]||j["* "+h[0]])){g===!0?g=j[e]:j[e]!==!0&&(f=h[0],k.unshift(h[1]));break}if(g!==!0)if(g&&a["throws"])b=g(b);else try{b=g(b)}catch(l){return{state:"parsererror",error:g?l:"No conversion from "+i+" to "+f}}}return{state:"success",data:b}}m.extend({active:0,lastModified:{},etag:{},ajaxSettings:{url:zc,type:"GET",isLocal:Dc.test(yc[1]),global:!0,processData:!0,async:!0,contentType:"application/x-www-form-urlencoded; charset=UTF-8",accepts:{"*":Jc,text:"text/plain",html:"text/html",xml:"application/xml, text/xml",json:"application/json, text/javascript"},contents:{xml:/xml/,html:/html/,json:/json/},responseFields:{xml:"responseXML",text:"responseText",json:"responseJSON"},converters:{"* text":String,"text html":!0,"text json":m.parseJSON,"text xml":m.parseXML},flatOptions:{url:!0,context:!0}},ajaxSetup:function(a,b){return b?Nc(Nc(a,m.ajaxSettings),b):Nc(m.ajaxSettings,a)},ajaxPrefilter:Lc(Hc),ajaxTransport:Lc(Ic),ajax:function(a,b){"object"==typeof a&&(b=a,a=void 0),b=b||{};var c,d,e,f,g,h,i,j,k=m.ajaxSetup({},b),l=k.context||k,n=k.context&&(l.nodeType||l.jquery)?m(l):m.event,o=m.Deferred(),p=m.Callbacks("once memory"),q=k.statusCode||{},r={},s={},t=0,u="canceled",v={readyState:0,getResponseHeader:function(a){var b;if(2===t){if(!j){j={};while(b=Cc.exec(f))j[b[1].toLowerCase()]=b[2]}b=j[a.toLowerCase()]}return null==b?null:b},getAllResponseHeaders:function(){return 2===t?f:null},setRequestHeader:function(a,b){var c=a.toLowerCase();return t||(a=s[c]=s[c]||a,r[a]=b),this},overrideMimeType:function(a){return t||(k.mimeType=a),this},statusCode:function(a){var b;if(a)if(2>t)for(b in a)q[b]=[q[b],a[b]];else v.always(a[v.status]);return this},abort:function(a){var b=a||u;return i&&i.abort(b),x(0,b),this}};if(o.promise(v).complete=p.add,v.success=v.done,v.error=v.fail,k.url=((a||k.url||zc)+"").replace(Ac,"").replace(Fc,yc[1]+"//"),k.type=b.method||b.type||k.method||k.type,k.dataTypes=m.trim(k.dataType||"*").toLowerCase().match(E)||[""],null==k.crossDomain&&(c=Gc.exec(k.url.toLowerCase()),k.crossDomain=!(!c||c[1]===yc[1]&&c[2]===yc[2]&&(c[3]||("http:"===c[1]?"80":"443"))===(yc[3]||("http:"===yc[1]?"80":"443")))),k.data&&k.processData&&"string"!=typeof k.data&&(k.data=m.param(k.data,k.traditional)),Mc(Hc,k,b,v),2===t)return v;h=m.event&&k.global,h&&0===m.active++&&m.event.trigger("ajaxStart"),k.type=k.type.toUpperCase(),k.hasContent=!Ec.test(k.type),e=k.url,k.hasContent||(k.data&&(e=k.url+=(wc.test(e)?"&":"?")+k.data,delete k.data),k.cache===!1&&(k.url=Bc.test(e)?e.replace(Bc,"$1_="+vc++):e+(wc.test(e)?"&":"?")+"_="+vc++)),k.ifModified&&(m.lastModified[e]&&v.setRequestHeader("If-Modified-Since",m.lastModified[e]),m.etag[e]&&v.setRequestHeader("If-None-Match",m.etag[e])),(k.data&&k.hasContent&&k.contentType!==!1||b.contentType)&&v.setRequestHeader("Content-Type",k.contentType),v.setRequestHeader("Accept",k.dataTypes[0]&&k.accepts[k.dataTypes[0]]?k.accepts[k.dataTypes[0]]+("*"!==k.dataTypes[0]?", "+Jc+"; q=0.01":""):k.accepts["*"]);for(d in k.headers)v.setRequestHeader(d,k.headers[d]);if(k.beforeSend&&(k.beforeSend.call(l,v,k)===!1||2===t))return v.abort();u="abort";for(d in{success:1,error:1,complete:1})v[d](k[d]);if(i=Mc(Ic,k,b,v)){v.readyState=1,h&&n.trigger("ajaxSend",[v,k]),k.async&&k.timeout>0&&(g=setTimeout(function(){v.abort("timeout")},k.timeout));try{t=1,i.send(r,x)}catch(w){if(!(2>t))throw w;x(-1,w)}}else x(-1,"No Transport");function x(a,b,c,d){var j,r,s,u,w,x=b;2!==t&&(t=2,g&&clearTimeout(g),i=void 0,f=d||"",v.readyState=a>0?4:0,j=a>=200&&300>a||304===a,c&&(u=Oc(k,v,c)),u=Pc(k,u,v,j),j?(k.ifModified&&(w=v.getResponseHeader("Last-Modified"),w&&(m.lastModified[e]=w),w=v.getResponseHeader("etag"),w&&(m.etag[e]=w)),204===a||"HEAD"===k.type?x="nocontent":304===a?x="notmodified":(x=u.state,r=u.data,s=u.error,j=!s)):(s=x,(a||!x)&&(x="error",0>a&&(a=0))),v.status=a,v.statusText=(b||x)+"",j?o.resolveWith(l,[r,x,v]):o.rejectWith(l,[v,x,s]),v.statusCode(q),q=void 0,h&&n.trigger(j?"ajaxSuccess":"ajaxError",[v,k,j?r:s]),p.fireWith(l,[v,x]),h&&(n.trigger("ajaxComplete",[v,k]),--m.active||m.event.trigger("ajaxStop")))}return v},getJSON:function(a,b,c){return m.get(a,b,c,"json")},getScript:function(a,b){return m.get(a,void 0,b,"script")}}),m.each(["get","post"],function(a,b){m[b]=function(a,c,d,e){return m.isFunction(c)&&(e=e||d,d=c,c=void 0),m.ajax({url:a,type:b,dataType:e,data:c,success:d})}}),m._evalUrl=function(a){return m.ajax({url:a,type:"GET",dataType:"script",async:!1,global:!1,"throws":!0})},m.fn.extend({wrapAll:function(a){if(m.isFunction(a))return this.each(function(b){m(this).wrapAll(a.call(this,b))});if(this[0]){var b=m(a,this[0].ownerDocument).eq(0).clone(!0);this[0].parentNode&&b.insertBefore(this[0]),b.map(function(){var a=this;while(a.firstChild&&1===a.firstChild.nodeType)a=a.firstChild;return a}).append(this)}return this},wrapInner:function(a){return this.each(m.isFunction(a)?function(b){m(this).wrapInner(a.call(this,b))}:function(){var b=m(this),c=b.contents();c.length?c.wrapAll(a):b.append(a)})},wrap:function(a){var b=m.isFunction(a);return this.each(function(c){m(this).wrapAll(b?a.call(this,c):a)})},unwrap:function(){return this.parent().each(function(){m.nodeName(this,"body")||m(this).replaceWith(this.childNodes)}).end()}}),m.expr.filters.hidden=function(a){return a.offsetWidth<=0&&a.offsetHeight<=0||!k.reliableHiddenOffsets()&&"none"===(a.style&&a.style.display||m.css(a,"display"))},m.expr.filters.visible=function(a){return!m.expr.filters.hidden(a)};var Qc=/%20/g,Rc=/\[\]$/,Sc=/\r?\n/g,Tc=/^(?:submit|button|image|reset|file)$/i,Uc=/^(?:input|select|textarea|keygen)/i;function Vc(a,b,c,d){var e;if(m.isArray(b))m.each(b,function(b,e){c||Rc.test(a)?d(a,e):Vc(a+"["+("object"==typeof e?b:"")+"]",e,c,d)});else if(c||"object"!==m.type(b))d(a,b);else for(e in b)Vc(a+"["+e+"]",b[e],c,d)}m.param=function(a,b){var c,d=[],e=function(a,b){b=m.isFunction(b)?b():null==b?"":b,d[d.length]=encodeURIComponent(a)+"="+encodeURIComponent(b)};if(void 0===b&&(b=m.ajaxSettings&&m.ajaxSettings.traditional),m.isArray(a)||a.jquery&&!m.isPlainObject(a))m.each(a,function(){e(this.name,this.value)});else for(c in a)Vc(c,a[c],b,e);return d.join("&").replace(Qc,"+")},m.fn.extend({serialize:function(){return m.param(this.serializeArray())},serializeArray:function(){return this.map(function(){var a=m.prop(this,"elements");return a?m.makeArray(a):this}).filter(function(){var a=this.type;return this.name&&!m(this).is(":disabled")&&Uc.test(this.nodeName)&&!Tc.test(a)&&(this.checked||!W.test(a))}).map(function(a,b){var c=m(this).val();return null==c?null:m.isArray(c)?m.map(c,function(a){return{name:b.name,value:a.replace(Sc,"\r\n")}}):{name:b.name,value:c.replace(Sc,"\r\n")}}).get()}}),m.ajaxSettings.xhr=void 0!==a.ActiveXObject?function(){return!this.isLocal&&/^(get|post|head|put|delete|options)$/i.test(this.type)&&Zc()||$c()}:Zc;var Wc=0,Xc={},Yc=m.ajaxSettings.xhr();a.attachEvent&&a.attachEvent("onunload",function(){for(var a in Xc)Xc[a](void 0,!0)}),k.cors=!!Yc&&"withCredentials"in Yc,Yc=k.ajax=!!Yc,Yc&&m.ajaxTransport(function(a){if(!a.crossDomain||k.cors){var b;return{send:function(c,d){var e,f=a.xhr(),g=++Wc;if(f.open(a.type,a.url,a.async,a.username,a.password),a.xhrFields)for(e in a.xhrFields)f[e]=a.xhrFields[e];a.mimeType&&f.overrideMimeType&&f.overrideMimeType(a.mimeType),a.crossDomain||c["X-Requested-With"]||(c["X-Requested-With"]="XMLHttpRequest");for(e in c)void 0!==c[e]&&f.setRequestHeader(e,c[e]+"");f.send(a.hasContent&&a.data||null),b=function(c,e){var h,i,j;if(b&&(e||4===f.readyState))if(delete Xc[g],b=void 0,f.onreadystatechange=m.noop,e)4!==f.readyState&&f.abort();else{j={},h=f.status,"string"==typeof f.responseText&&(j.text=f.responseText);try{i=f.statusText}catch(k){i=""}h||!a.isLocal||a.crossDomain?1223===h&&(h=204):h=j.text?200:404}j&&d(h,i,j,f.getAllResponseHeaders())},a.async?4===f.readyState?setTimeout(b):f.onreadystatechange=Xc[g]=b:b()},abort:function(){b&&b(void 0,!0)}}}});function Zc(){try{return new a.XMLHttpRequest}catch(b){}}function $c(){try{return new a.ActiveXObject("Microsoft.XMLHTTP")}catch(b){}}m.ajaxSetup({accepts:{script:"text/javascript, application/javascript, application/ecmascript, application/x-ecmascript"},contents:{script:/(?:java|ecma)script/},converters:{"text script":function(a){return m.globalEval(a),a}}}),m.ajaxPrefilter("script",function(a){void 0===a.cache&&(a.cache=!1),a.crossDomain&&(a.type="GET",a.global=!1)}),m.ajaxTransport("script",function(a){if(a.crossDomain){var b,c=y.head||m("head")[0]||y.documentElement;return{send:function(d,e){b=y.createElement("script"),b.async=!0,a.scriptCharset&&(b.charset=a.scriptCharset),b.src=a.url,b.onload=b.onreadystatechange=function(a,c){(c||!b.readyState||/loaded|complete/.test(b.readyState))&&(b.onload=b.onreadystatechange=null,b.parentNode&&b.parentNode.removeChild(b),b=null,c||e(200,"success"))},c.insertBefore(b,c.firstChild)},abort:function(){b&&b.onload(void 0,!0)}}}});var _c=[],ad=/(=)\?(?=&|$)|\?\?/;m.ajaxSetup({jsonp:"callback",jsonpCallback:function(){var a=_c.pop()||m.expando+"_"+vc++;return this[a]=!0,a}}),m.ajaxPrefilter("json jsonp",function(b,c,d){var e,f,g,h=b.jsonp!==!1&&(ad.test(b.url)?"url":"string"==typeof b.data&&!(b.contentType||"").indexOf("application/x-www-form-urlencoded")&&ad.test(b.data)&&"data");return h||"jsonp"===b.dataTypes[0]?(e=b.jsonpCallback=m.isFunction(b.jsonpCallback)?b.jsonpCallback():b.jsonpCallback,h?b[h]=b[h].replace(ad,"$1"+e):b.jsonp!==!1&&(b.url+=(wc.test(b.url)?"&":"?")+b.jsonp+"="+e),b.converters["script json"]=function(){return g||m.error(e+" was not called"),g[0]},b.dataTypes[0]="json",f=a[e],a[e]=function(){g=arguments},d.always(function(){a[e]=f,b[e]&&(b.jsonpCallback=c.jsonpCallback,_c.push(e)),g&&m.isFunction(f)&&f(g[0]),g=f=void 0}),"script"):void 0}),m.parseHTML=function(a,b,c){if(!a||"string"!=typeof a)return null;"boolean"==typeof b&&(c=b,b=!1),b=b||y;var d=u.exec(a),e=!c&&[];return d?[b.createElement(d[1])]:(d=m.buildFragment([a],b,e),e&&e.length&&m(e).remove(),m.merge([],d.childNodes))};var bd=m.fn.load;m.fn.load=function(a,b,c){if("string"!=typeof a&&bd)return bd.apply(this,arguments);var d,e,f,g=this,h=a.indexOf(" ");return h>=0&&(d=m.trim(a.slice(h,a.length)),a=a.slice(0,h)),m.isFunction(b)?(c=b,b=void 0):b&&"object"==typeof b&&(f="POST"),g.length>0&&m.ajax({url:a,type:f,dataType:"html",data:b}).done(function(a){e=arguments,g.html(d?m("<div>").append(m.parseHTML(a)).find(d):a)}).complete(c&&function(a,b){g.each(c,e||[a.responseText,b,a])}),this},m.each(["ajaxStart","ajaxStop","ajaxComplete","ajaxError","ajaxSuccess","ajaxSend"],function(a,b){m.fn[b]=function(a){return this.on(b,a)}}),m.expr.filters.animated=function(a){return m.grep(m.timers,function(b){return a===b.elem}).length};var cd=a.document.documentElement;function dd(a){return m.isWindow(a)?a:9===a.nodeType?a.defaultView||a.parentWindow:!1}m.offset={setOffset:function(a,b,c){var d,e,f,g,h,i,j,k=m.css(a,"position"),l=m(a),n={};"static"===k&&(a.style.position="relative"),h=l.offset(),f=m.css(a,"top"),i=m.css(a,"left"),j=("absolute"===k||"fixed"===k)&&m.inArray("auto",[f,i])>-1,j?(d=l.position(),g=d.top,e=d.left):(g=parseFloat(f)||0,e=parseFloat(i)||0),m.isFunction(b)&&(b=b.call(a,c,h)),null!=b.top&&(n.top=b.top-h.top+g),null!=b.left&&(n.left=b.left-h.left+e),"using"in b?b.using.call(a,n):l.css(n)}},m.fn.extend({offset:function(a){if(arguments.length)return void 0===a?this:this.each(function(b){m.offset.setOffset(this,a,b)});var b,c,d={top:0,left:0},e=this[0],f=e&&e.ownerDocument;if(f)return b=f.documentElement,m.contains(b,e)?(typeof e.getBoundingClientRect!==K&&(d=e.getBoundingClientRect()),c=dd(f),{top:d.top+(c.pageYOffset||b.scrollTop)-(b.clientTop||0),left:d.left+(c.pageXOffset||b.scrollLeft)-(b.clientLeft||0)}):d},position:function(){if(this[0]){var a,b,c={top:0,left:0},d=this[0];return"fixed"===m.css(d,"position")?b=d.getBoundingClientRect():(a=this.offsetParent(),b=this.offset(),m.nodeName(a[0],"html")||(c=a.offset()),c.top+=m.css(a[0],"borderTopWidth",!0),c.left+=m.css(a[0],"borderLeftWidth",!0)),{top:b.top-c.top-m.css(d,"marginTop",!0),left:b.left-c.left-m.css(d,"marginLeft",!0)}}},offsetParent:function(){return this.map(function(){var a=this.offsetParent||cd;while(a&&!m.nodeName(a,"html")&&"static"===m.css(a,"position"))a=a.offsetParent;return a||cd})}}),m.each({scrollLeft:"pageXOffset",scrollTop:"pageYOffset"},function(a,b){var c=/Y/.test(b);m.fn[a]=function(d){return V(this,function(a,d,e){var f=dd(a);return void 0===e?f?b in f?f[b]:f.document.documentElement[d]:a[d]:void(f?f.scrollTo(c?m(f).scrollLeft():e,c?e:m(f).scrollTop()):a[d]=e)},a,d,arguments.length,null)}}),m.each(["top","left"],function(a,b){m.cssHooks[b]=Lb(k.pixelPosition,function(a,c){return c?(c=Jb(a,b),Hb.test(c)?m(a).position()[b]+"px":c):void 0})}),m.each({Height:"height",Width:"width"},function(a,b){m.each({padding:"inner"+a,content:b,"":"outer"+a},function(c,d){m.fn[d]=function(d,e){var f=arguments.length&&(c||"boolean"!=typeof d),g=c||(d===!0||e===!0?"margin":"border");return V(this,function(b,c,d){var e;return m.isWindow(b)?b.document.documentElement["client"+a]:9===b.nodeType?(e=b.documentElement,Math.max(b.body["scroll"+a],e["scroll"+a],b.body["offset"+a],e["offset"+a],e["client"+a])):void 0===d?m.css(b,c,g):m.style(b,c,d,g)},b,f?d:void 0,f,null)}})}),m.fn.size=function(){return this.length},m.fn.andSelf=m.fn.addBack,"function"==typeof define&&define.amd&&define("jquery",[],function(){return m});var ed=a.jQuery,fd=a.$;return m.noConflict=function(b){return a.$===m&&(a.$=fd),b&&a.jQuery===m&&(a.jQuery=ed),m},typeof b===K&&(a.jQuery=a.$=m),m});
diff --git a/twitter/mysos/scheduler/assets/templates/clusters.html b/twitter/mysos/scheduler/assets/templates/clusters.html
new file mode 100644
index 0000000..903893e
--- /dev/null
+++ b/twitter/mysos/scheduler/assets/templates/clusters.html
@@ -0,0 +1,67 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <title>Mysos Clusters</title>
+ <meta http-equiv="refresh" content="300">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="stylesheet" href="/static/bootstrap.min.css">
+ <style>
+ body {
+ padding-top: 50px;
+ }
+
+ tr.cluster {
+ cursor: pointer;
+ }
+ </style>
+</head>
+<body>
+<div class="navbar navbar-default navbar-fixed-top" role="navigation">
+ <div class="container">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse"
+ data-target="#navbar-collapse">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/">Mysos</a>
+ </div>
+
+ <!-- Collect the nav links, forms, and other content for toggling -->
+ <div class="collapse navbar-collapse" id="navbar-collapse">
+ <ul class="nav navbar-nav">
+ <li class="active"><a href="/">Clusters</a></li>
+ </ul>
+ </div>
+ <!-- /.navbar-collapse -->
+ </div>
+ <!-- /.container -->
+</div>
+
+<div class="container">
+ <h1>Clusters</h1>
+ <!-- TODO: ajax -->
+ <table class="table table-hover">
+ <thead>
+ <tr>
+ <th>name</th>
+ <th>user</th>
+ <th>num nodes</th>
+ </tr>
+ </thead>
+ <tbody>
+ %for cluster in clusters:
+ <tr class="cluster">
+ <td>${cluster.name}</td>
+ <td>${cluster.user}</td>
+ <td>${cluster.num_nodes}</td>
+ </tr>
+ %endfor
+ </tbody>
+ </table>
+</div>
+<!-- container -->
+<script src="/static/jquery.min.js"></script>
+<script src="/static/bootstrap.min.js"></script>
+</body>
+</html>
diff --git a/twitter/mysos/scheduler/elector.py b/twitter/mysos/scheduler/elector.py
new file mode 100644
index 0000000..cc80d90
--- /dev/null
+++ b/twitter/mysos/scheduler/elector.py
@@ -0,0 +1,247 @@
+from collections import OrderedDict
+from datetime import datetime, timedelta
+import threading
+
+from twitter.common import log
+from twitter.common.exceptions import ExceptionalThread
+from twitter.common.quantity import Amount, Time
+
+import mesos.interface.mesos_pb2 as mesos_pb2
+
+
+class MySQLMasterElector(ExceptionalThread):
+ """
+ Elect the most current slave in the cluster to be the new master.
+
+ The current election algorithm:
+ The elector queries slaves periodically (to tolerate message loss) until all slaves have
+ responded with their current positions and the slave with the highest (most current)
+ position is elected the new master. If the election timeout has passed and not all slaves have
+ responded, the master is elected from the ones that have.
+
+ Thread-safety:
+ The elector runs as a separate thread and it periodically queries the slaves util the election
+ completes or is aborted. Its public methods are thread-safe.
+
+ Usage:
+ After constructing the elector, add all currently known candidates of the election through
+ 'elector.add_slave()' before calling 'elector.start()' to make sure it waits for all of them.
+
+ NOTE: An elector is not reusable. To restart the election, create another MySQLMasterElector
+ instance.
+ """
+
+ def __init__(
+ self,
+ driver,
+ cluster_name,
+ epoch,
+ master_callback,
+ election_timeout,
+ query_interval=Amount(1, Time.SECONDS)):
+ """
+ :param driver: The SchedulerDriver for querying the slaves.
+ :param cluster_name: The name of the MySQL cluster.
+ :param epoch: The master epoch that identifies this election.
+ :param master_callback: The callback function with one argument: the 'task_id' of the elected
+ master which could be None if no one is electable.
+ :param election_timeout: The amount of time the elector waits for all slaves to respond. If
+ not all slaves have responded within the timeout, then the master is
+ elected from the ones who have.
+ :param query_interval: The timeout before the elector re-sends queries for positions.
+
+ :type epoch: int
+ :type query_interval: Amount
+ :type election_timeout: Amount
+ :type master_callback: function
+ """
+ super(MySQLMasterElector, self).__init__()
+
+ if not isinstance(epoch, int):
+ raise TypeError("'epoch' should be an int")
+ if not isinstance(query_interval, Amount) or not isinstance(query_interval.unit(), Time):
+ raise ValueError("'query_interval' must be an Amount of Time")
+ if not isinstance(election_timeout, Amount) or not isinstance(election_timeout.unit(), Time):
+ raise ValueError("'election_timeout' must be an Amount of Time")
+ if not hasattr(master_callback, '__call__'):
+ raise TypeError("master_callback must be a function")
+
+ self._query_interval = query_interval.as_(Time.SECONDS)
+
+ self._election_deadline = (
+ datetime.utcnow() + timedelta(seconds=election_timeout.as_(Time.SECONDS)))
+
+ self._driver = driver
+ self._cluster_name = cluster_name # For logging.
+ self._epoch = epoch
+ self._master_callback = master_callback
+
+ self._positions = OrderedDict() # Slave {Task ID: Position} mappings. Use OrderedDict so we can
+ # easily locate the first added slave.
+ self._mesos_slaves = {} # Slave {Task ID, Mesos slave ID)} mappings.
+ self._master = None # Elected master (its TaskID); initially None and can still be None after
+ # the election has timed out and there are no slaves to elect from.
+
+ self._lock = threading.Lock()
+ self._aborted = threading.Event() # Elector thread aborted (don't invoke callback).
+ self._completed = threading.Event() # Election process completed (invoke callback).
+
+ def add_slave(self, task_id, mesos_slave_id):
+ """
+ Add a new MySQL slave (Task ID, Mesos Slave ID) which could affect the ongoing election.
+ """
+ with self._lock:
+ if self._completed.is_set():
+ log.debug("Ignoring addition of slave %s because the election has completed" % task_id)
+ return
+
+ log.info('Adding slave %s to the election for cluster %s' % (
+ str((task_id, mesos_slave_id)),
+ self._cluster_name))
+
+ self._positions[task_id] = None
+ self._mesos_slaves[task_id] = mesos_slave_id
+
+ def remove_slave(self, task_id):
+ """
+ Remove a slave from the election process.
+ """
+ with self._lock:
+ if self._completed.is_set():
+ log.debug("Ignoring removal of slave %s because the election has completed" % task_id)
+ return
+
+ log.info('Removing slave %s from election for cluster %s' % (
+ str((task_id, self._mesos_slaves[task_id])), self._cluster_name))
+
+ assert task_id in self._positions
+ assert task_id in self._mesos_slaves
+ del self._positions[task_id]
+ del self._mesos_slaves[task_id]
+
+ def update_position(self, epoch, task_id, position):
+ """
+ Called by the launcher upon receiving the executor's response to position query.
+
+ :type epoch: int
+ :param epoch: The master epoch this position is for.
+ :param position: The position of the slave's log (not necessarily numeric). The elector
+ doesn't care about its exact format as long as it is comparable.
+ """
+ if not isinstance(epoch, int):
+ raise TypeError("'epoch' should be an int")
+
+ with self._lock:
+ if self._completed.is_set():
+ log.debug("Ignoring position %s from slave %s because the election has completed" % (
+ position, task_id))
+ return
+
+ if epoch != self._epoch:
+ log.info(
+ "Ignoring position %s from slave %s due to epoch mismatch: (expected: %s, actual: %s)" %
+ (position, task_id, self._epoch, epoch))
+ return
+
+ if task_id not in self._mesos_slaves:
+ log.warn("Ignoring unsolicited position response %s from MySQL slave %s" % (
+ str(position), task_id))
+ return
+
+ log.info('Updating position to %s (epoch=%s) for slave %s of cluster %s' % (
+ str(position),
+ epoch,
+ str((task_id, self._mesos_slaves[task_id])),
+ self._cluster_name))
+
+ self._positions[task_id] = position
+
+ def run(self):
+ # Re-run the election in a loop periodically until a master can be elected or the elector is
+ # aborted.
+ while not self._aborted.is_set() and not self._completed.wait(self._query_interval):
+ if datetime.utcnow() < self._election_deadline:
+ self._elect(timedout=False)
+ else:
+ log.info("Timed out waiting for all slaves to respond. Now elect from existing responses")
+ self._elect(timedout=True)
+ if not self._completed.is_set():
+ log.warn("No slave is electable after timeout")
+
+ if self._aborted.is_set(): # If asked to stop, directly return without triggering the callback.
+ log.info("Asked to stop the elector thread for cluster %s. Stopping..." % self._cluster_name)
+ return
+
+ self._master_callback(self._master) # Invoke the callback from the elector thread.
+ log.info("Stopping the elector thread for cluster %s because the election has completed" %
+ self._cluster_name)
+
+ def _elect(self, timedout=False):
+ """
+ Try to elect the master from MySQL slaves.
+
+ Elect the slave with the highest position if all slaves have responded, otherwise re-query the
+ remaining slaves for log positions (unless 'timedout' is True).
+
+ :param timedout: If True, just elect the slave with the highest position from the ones who
+ have responded.
+
+ NOTE: If 'timedout' is True and no slaves have responded (but there are running slaves in
+ the cluster), theoretically we can still randomly elect one from them but currently we don't
+ as this would suggest some major issues with the MySQL or Mesos cluster and it's dangerous to
+ do so.
+ """
+ with self._lock:
+ # Special-casing the first epoch because this is when the cluster first starts and every
+ # slave should be the same: just pick the first slave that comes up.
+ if self._epoch == 0 and len(self._positions) > 0:
+ self._master = next(iter(self._positions))
+ self._completed.set()
+ return
+
+ if timedout or (self._positions and _all(self._positions.values())):
+ # Pick the slave with the highest position (value of the key-value pair). If all items have
+ # the same value, an arbitrary (but deterministic) one is chosen.
+ if _any(self._positions.values()): # Need at least one position.
+ master_task, _ = max(self._positions.items(), key=lambda kv: kv[1])
+ log.info('Elected master %s for cluster %s' % (master_task, self._cluster_name))
+ self._master = master_task
+ self._completed.set()
+ else:
+ # (Re)query the remaining slaves for log positions.
+ for task_id in self._positions:
+ if not self._positions[task_id]:
+ self._query_slave(task_id)
+
+ def _query_slave(self, task_id):
+ assert task_id in self._mesos_slaves
+
+ log.info('Querying MySQL slave %s for its log position (epoch=%s)' % (
+ str((task_id, self._mesos_slaves[task_id])), self._epoch))
+
+ # Because the elector re-sends messages, it's necessary to use the epoch to differentiate
+ # responses for each election.
+ self._driver.sendFrameworkMessage(
+ mesos_pb2.ExecutorID(value=task_id),
+ mesos_pb2.SlaveID(value=self._mesos_slaves[task_id]),
+ str(self._epoch)) # Send the slave the epoch so it can be included in the response.
+
+ def abort(self):
+ """Stop the elector thread."""
+ self._aborted.set()
+
+
+def _all(iterable):
+ """Same as built-in version of all() except it explicitly checks if an element "is None"."""
+ for element in iterable:
+ if element is None:
+ return False
+ return True
+
+
+def _any(iterable):
+ """Same as built-in version of any() except it explicitly checks if an element "is not None"."""
+ for element in iterable:
+ if element is not None:
+ return True
+ return False
diff --git a/twitter/mysos/scheduler/http.py b/twitter/mysos/scheduler/http.py
new file mode 100644
index 0000000..e2a0636
--- /dev/null
+++ b/twitter/mysos/scheduler/http.py
@@ -0,0 +1,74 @@
+import json
+import os
+
+from twitter.common.http import HttpServer, route, static_file
+
+from .launcher import MySQLClusterLauncher
+from .scheduler import MysosScheduler
+
+import bottle
+from mako.template import Template
+
+
+class MysosServer(HttpServer):
+ def __init__(self, scheduler, asset_dir):
+ super(MysosServer, self).__init__()
+ self._scheduler = scheduler
+ self._asset_dir = asset_dir
+
+ self._static_dir = os.path.join(self._asset_dir, 'static')
+ self._template_dir = os.path.join(self._asset_dir, 'templates')
+
+ self._clusters_template = Template(filename=os.path.join(self._template_dir, 'clusters.html'))
+
+ @route('/clusters/<clustername>', method=['POST'])
+ def create(self, clustername):
+ """Create a db cluster."""
+ cluster_name = clustername # For naming consistency.
+ num_nodes = bottle.request.forms.get('num_nodes', default=3)
+ cluster_user = bottle.request.forms.get('cluster_user', default=None)
+ backup_id = bottle.request.forms.get('backup_id', default=None)
+
+ try:
+ cluster_zk_url, cluster_password = self._scheduler.create_cluster(
+ cluster_name,
+ cluster_user,
+ num_nodes,
+ backup_id=backup_id)
+ return json.dumps(dict(cluster_url=cluster_zk_url, cluster_password=cluster_password))
+ except MysosScheduler.ClusterExists as e:
+ raise bottle.HTTPResponse(e.message, status=409)
+ except MysosScheduler.InvalidUser as e:
+ raise bottle.HTTPResponse(e.message, status=400)
+ except MysosScheduler.ServiceUnavailable as e:
+ raise bottle.HTTPResponse(e.message, status=503)
+ except ValueError as e:
+ raise bottle.HTTPResponse(e.message, status=400)
+
+
+ @route('/clusters/<clustername>', method=['DELETE'])
+ def remove(self, clustername):
+ """Remove a db cluster."""
+ cluster_name = clustername # For naming consistency.
+
+ password = bottle.request.forms.get('password', default=None)
+
+ try:
+ cluster_zk_url = self._scheduler.delete_cluster(cluster_name, password)
+ return json.dumps(dict(cluster_url=cluster_zk_url))
+ except MysosScheduler.ClusterNotFound as e:
+ raise bottle.HTTPResponse(e.message, status=404)
+ except MySQLClusterLauncher.PermissionError as e:
+ raise bottle.HTTPResponse(e.message, status=403)
+
+ @route('/', method=['GET'])
+ def clusters(self):
+ """Landing page, showing the list of managed clusters."""
+ if not self._scheduler.connected.is_set():
+ return "<h1>Mysos scheduler is still connecting...</h1>"
+
+ return self._clusters_template.render(clusters=self._scheduler.clusters)
+
+ @route('/static/<filepath:path>', method=['GET'])
+ def serve_static(self, filepath):
+ return static_file(filepath, root=self._static_dir)
diff --git a/twitter/mysos/scheduler/launcher.py b/twitter/mysos/scheduler/launcher.py
new file mode 100644
index 0000000..c405158
--- /dev/null
+++ b/twitter/mysos/scheduler/launcher.py
@@ -0,0 +1,539 @@
+from collections import namedtuple
+import json
+import random
+import threading
+
+from twitter.common import log
+from twitter.common.quantity import Amount, Time
+from twitter.common.zookeeper.serverset.endpoint import Endpoint, ServiceInstance
+from twitter.mysos.common.cluster import ClusterManager, get_cluster_path
+from twitter.mysos.common import zookeeper
+
+from .elector import MySQLMasterElector
+from .state import MySQLCluster, MySQLTask, StateProvider
+
+import mesos.interface.mesos_pb2 as mesos_pb2
+
+
+# TODO(jyx): Replace this when we start taking tasks from an HTTP API.
+TASK_CPUS = 1
+TASK_MEM = 512
+
+
+class MySQLClusterLauncher(object):
+ """
+ Responsible for launching and maintaining a MySQL cluster.
+
+ Thread-safety:
+ The launcher is thread-safe. It uses a separate thread to wait for the election result and
+ can launch a new election within that thread. All other public methods are called from the
+ scheduler driver thread and the web UI threads.
+ """
+
+ class Error(Exception): pass
+ class IncompatibleRoleError(Error): pass
+ class PermissionError(Error): pass
+
+ def __init__(
+ self,
+ driver,
+ cluster,
+ state_provider,
+ zk_url,
+ kazoo,
+ framework_user,
+ executor_uri,
+ executor_cmd,
+ election_timeout,
+ admin_keypath,
+ installer_args=None,
+ backup_store_args=None,
+ executor_environ=None,
+ framework_role='*',
+ query_interval=Amount(1, Time.SECONDS)):
+ """
+ :param driver: Mesos scheduler driver.
+ :param cluster: The MySQLCluster state object.
+ :param state_provider: For restoring and persisting the cluster state.
+ :param zk_url: The ZooKeeper URL for cluster member discovery and master election.
+ :param kazoo: The Kazoo client to access ZooKeeper with.
+ :param executor_uri: See flags.
+ :param executor_cmd: See flags.
+ :param election_timeout: See flags.
+ :param admin_keypath: See flags.
+ :param installer_args: See flags.
+ :param backup_store_args: See flags.
+ :param executor_environ: See flags.
+ :param framework_role: See flags.
+ :param query_interval: See MySQLMasterElector. Use the default value for production and allow
+ tests to use a different value.
+ """
+ self._driver = driver
+
+ if not isinstance(cluster, MySQLCluster):
+ raise TypeError("'cluster' should be an instance of MySQLCluster")
+ self._cluster = cluster
+
+ if not isinstance(state_provider, StateProvider):
+ raise TypeError("'state_provider' should be an instance of StateProvider")
+ self._state_provider = state_provider
+
+ self._framework_role = framework_role
+
+ # Passed along to executors.
+ self._zk_url = zk_url
+ self._framework_user = framework_user
+ self._executor_uri = executor_uri
+ self._executor_cmd = executor_cmd
+ self._election_timeout = election_timeout
+ self._admin_keypath = admin_keypath
+ self._installer_args = installer_args
+ self._backup_store_args = backup_store_args
+ self._executor_environ = executor_environ
+
+ # Used by the elector.
+ self._query_interval = query_interval
+
+ zk_root = zookeeper.parse(zk_url)[2]
+ self._cluster_manager = ClusterManager(kazoo, get_cluster_path(zk_root, cluster.name))
+
+ self._lock = threading.Lock()
+
+ if self._cluster.master_id:
+ log.info("Republish master %s for cluster %s in case it's not published" % (
+ self._cluster.master_id, self.cluster_name))
+ self._cluster_manager.promote_member(self._cluster.master_id)
+
+ if len(self._cluster.tasks) > 0:
+ log.info("Recovered %s tasks for cluster '%s'" % (
+ len(self._cluster.tasks), self.cluster_name))
+
+ # A recovered launcher should continue the election if the previous one was incomplete when the
+ # scheduler failed over. Mesos will deliver all missed events that affect the election to the
+ # scheduler.
+ if len(self._cluster.running_tasks) > 0 and not self._cluster.master_id:
+ log.info("Restarting election for the recovered launcher")
+ self._elector = self._new_elector()
+ # Add current slaves.
+ for t in self._cluster.running_tasks:
+ self._elector.add_slave(t.task_id, t.mesos_slave_id)
+ self._elector.start()
+ else:
+ # New launcher, the elector is set when the election starts and reset to None when it ends.
+ self._elector = None
+
+ self._terminating = False
+
+ @property
+ def cluster_name(self):
+ return self._cluster.name
+
+ @property
+ def cluster_info(self):
+ with self._lock:
+ ClusterInfo = namedtuple('ClusterInfo', ('name, user, num_nodes'))
+ return ClusterInfo(
+ name=self._cluster.name, user=self._cluster.user, num_nodes=self._cluster.num_nodes)
+
+ def launch(self, offer):
+ """
+ Try to launch a MySQL task with the given offer.
+
+ :returns:
+ Task ID: Either the task ID of the task just launched or None if this offer is not used.
+ Remaining resources: Resources from this offer that are unused by the task. If no task is
+ launched, all the resources from the offer are returned.
+
+ :raises IncompatibleRoleError: Raised when the offer has some resource with incompatible role.
+ """
+ with self._lock:
+ if len(self._cluster.active_tasks) == self._cluster.num_nodes:
+ # All nodes of this cluster have been launched and none have died.
+ return None, offer.resources
+
+ if self._terminating:
+ return None, offer.resources
+
+ cpus, mem, ports = self._get_resources(offer.resources)
+
+ # TODO(jyx): Replace the static resource requirements with what the user requests.
+ task_cpus = TASK_CPUS
+ task_mem = TASK_MEM
+
+ if cpus < task_cpus or mem < task_mem or len(ports) == 0:
+ # Offer doesn't fit.
+ return None, offer.resources
+
+ log.info("Launcher %s accepted offer %s on Mesos slave %s (%s)" % (
+ self.cluster_name, offer.id.value, offer.slave_id.value, offer.hostname))
+
+ task_port = random.choice(list(ports)) # Randomly pick a port in the offer.
+
+ task_info = self._new_task(offer, task_cpus, task_mem, task_port)
+ self._cluster.tasks[task_info.task_id.value] = MySQLTask(
+ self._cluster.name,
+ task_info.task_id.value,
+ task_info.slave_id.value,
+ offer.hostname,
+ task_port)
+ self._cluster.next_id += 1
+
+ # Checkpoint task data. The task can fail to launch. The problem is solved by the TODO below.
+ self._state_provider.dump_cluster_state(self._cluster)
+
+ log.info('Launching task %s on Mesos slave %s (%s)' % (
+ task_info.task_id.value, offer.slave_id.value, offer.hostname))
+
+ # Mysos launches at most a single task for each offer. Note that the SchedulerDriver API
+ # expects a list of tasks.
+ # TODO(jyx): Reconcile after failover because the scheduler can crash before successfully
+ # launching the task. Also run implicit reconciliation periodically.
+ self._driver.launchTasks(offer.id, [task_info])
+
+ # Update the offer's resources and return them for other clusters to use.
+ remaining = create_resources(
+ cpus - task_cpus, mem - task_mem, ports - set([task_port]), role=self._framework_role)
+ return task_info.task_id.value, remaining
+
+ def kill(self, password):
+ """
+ Kill the cluster.
+
+ NOTE: Cluster killing is asynchronous. Use 'terminated' property to check if all tasks in the
+ cluster are killed.
+ """
+ with self._lock:
+ if self._cluster.password != password:
+ raise self.PermissionError("No permission to kill cluster %s" % self.cluster_name)
+
+ self._terminating = True
+
+ # TODO(jyx): Task killing is unreliable. Reconciliation should retry killing.
+ for task_id in self._cluster.tasks:
+ log.info("Killing task %s of cluster %s" % (task_id, self.cluster_name))
+ self._driver.killTask(mesos_pb2.TaskID(value=task_id))
+
+ @property
+ def terminated(self):
+ """True if all tasks in the cluster are killed."""
+ return self._terminating and len(self._cluster.active_tasks) == 0
+
+ def _get_resources(self, resources):
+ """Return a tuple of the resources: cpus, mem, set of ports."""
+ cpus, mem, ports = 0.0, 0, set()
+ for resource in resources:
+ # We do the following check:
+ # 1. We only care about the role of the resources we are going to use.
+ # 2. For this resource if it is not of the role we want we throw an exception. This implies
+ # that when a slave offers resources that include both the '*' role and the Mysos framework
+ # role we'll decline the entire offer. We expect Mesos slave hosts that run Mysos executors
+ # to dedicate *all* its resources to it as we are not currently optimizing for the use
+ # cases where Mysos tasks run side-by-side with tasks from other frameworks. This also
+ # simplifies the launcher's role filtering logic.
+ # TODO(jyx): Revisit this when the above assumption changes.
+ if resource.name in ('cpus', 'mem', 'ports') and resource.role != self._framework_role:
+ raise self.IncompatibleRoleError("Offered resource %s has role %s, expecting %s" % (
+ resource.name, resource.role, self._framework_role))
+
+ if resource.name == 'cpus':
+ cpus = resource.scalar.value
+ elif resource.name == 'mem':
+ mem = resource.scalar.value
+ elif resource.name == 'ports' and resource.ranges.range:
+ for r in resource.ranges.range:
+ ports |= set(range(r.begin, r.end + 1))
+
+ return cpus, mem, ports
+
+ def _new_task(self, offer, task_cpus, task_mem, task_port):
+ """Return a new task with the requested resources."""
+ server_id = self._cluster.next_id
+ task_id = "mysos-" + self.cluster_name + "-" + str(server_id)
+
+ task = mesos_pb2.TaskInfo()
+ task.task_id.value = task_id
+ task.slave_id.value = offer.slave_id.value
+ task.name = task_id
+
+ task.executor.executor_id.value = task_id # Use task_id as executor_id.
+ task.executor.command.value = self._executor_cmd
+
+ if self._executor_environ: # Could be 'None' since it's an optional argument.
+ executor_environ_ = json.loads(self._executor_environ)
+ if executor_environ_:
+ for var_ in executor_environ_:
+ log.info("Executor will use environment variable: %s" % var_)
+ var = task.executor.command.environment.variables.add()
+ var.name = var_['name']
+ var.value = var_['value']
+
+ uri = task.executor.command.uris.add()
+ uri.value = self._executor_uri
+ uri.executable = True
+ uri.extract = False # Don't need to decompress pex.
+
+ task.data = json.dumps({
+ 'framework_user': self._framework_user,
+ 'host': offer.hostname,
+ 'port': task_port,
+ 'cluster': self._cluster.name,
+ 'cluster_user': self._cluster.user,
+ 'cluster_password': self._cluster.password,
+ 'server_id': server_id, # Use the integer Task ID as the server ID.
+ 'zk_url': self._zk_url,
+ 'admin_keypath': self._admin_keypath,
+ 'installer_args': self._installer_args,
+ 'backup_store_args': self._backup_store_args,
+ 'backup_id': self._cluster.backup_id,
+ })
+
+ resources = create_resources(
+ task_cpus, task_mem, set([task_port]), role=self._framework_role)
+ task.resources.extend(resources)
+
+ return task
+
+ def _new_elector(self):
+ """Create a new instance of MySQLMasterElector."""
+ elector = MySQLMasterElector(
+ self._driver,
+ self.cluster_name,
+ self._cluster.next_epoch,
+ self._master_elected,
+ self._election_timeout,
+ query_interval=self._query_interval)
+
+ log.info("Created elector for epoch %s for cluster %s" % (
+ self._cluster.next_epoch, self.cluster_name))
+ self._cluster.next_epoch += 1
+
+ # Save the epoch so the new elector will use a new epoch after scheduler failover.
+ self._state_provider.dump_cluster_state(self._cluster)
+
+ return elector
+
+ def status_update(self, status):
+ """
+ Handle the status update for a task of this cluster.
+
+ NOTE:
+ Duplicate status updates may be handled by either the same scheduler instance or a new
+ instance with the restored state.
+ """
+ with self._lock:
+ task_id = status.task_id.value
+
+ if task_id not in self._cluster.tasks:
+ log.warn("Ignoring status update for unknown task %s" % task_id)
+ return
+
+ task = self._cluster.tasks[task_id]
+ previous_state = task.state
+
+ # We don't want to ignore a duplicate update if the previous one was not successfully handled.
+ # Therefore, we should not checkpoint the status change until we have finished all operations.
+ if previous_state == status.state:
+ log.info('Ignoring duplicate status update %s for task %s' % (
+ mesos_pb2.TaskState.Name(status.state),
+ task_id))
+ return
+
+ if is_terminal(previous_state):
+ log.info('Ignoring status update %s for task %s as it is in terminal state %s' % (
+ mesos_pb2.TaskState.Name(status.state),
+ task_id,
+ mesos_pb2.TaskState.Name(previous_state)))
+ return
+
+ log.info('Updating state of task %s of cluster %s from %s to %s' % (
+ status.task_id.value,
+ self.cluster_name,
+ mesos_pb2.TaskState.Name(previous_state),
+ mesos_pb2.TaskState.Name(status.state)))
+ task.state = status.state
+
+ if status.state == mesos_pb2.TASK_RUNNING:
+ # Register this cluster member.
+ endpoint = Endpoint(
+ self._cluster.tasks[task_id].hostname, self._cluster.tasks[task_id].port)
+
+ # If the scheduler fails over after ZK is updated but before the state change is
+ # checkpointed, it will receive the same status update again and try to publish a duplicate
+ # member to ZK. ClusterManager.add_member() is idempotent and doesn't update ZK in this
+ # case.
+ member_id = self._cluster_manager.add_member(ServiceInstance(endpoint))
+ log.info('Added %s (member id=%s) to cluster %s' % (endpoint, member_id, self.cluster_name))
+ self._cluster.members[task_id] = member_id
+
+ # Checkpoint the status update here. It's OK if the elector fails to launch later because
+ # the new scheduler instance will retry based on the fact that there are running instances
+ # of the cluster but no master.
+ self._state_provider.dump_cluster_state(self._cluster)
+
+ # If MySQL master is already elected for this cluster don't bother adding it to the elector.
+ if self._cluster.master_id:
+ log.info(
+ "MySQL slave task %s on %s started after a master is already elected for cluster %s" %
+ (task_id, endpoint.host, self.cluster_name))
+ return
+
+ if not self._elector:
+ self._elector = self._new_elector()
+ # Add current slaves.
+ for t in self._cluster.running_tasks:
+ self._elector.add_slave(t.task_id, t.mesos_slave_id)
+ self._elector.start()
+ else:
+ self._elector.add_slave(task_id, status.slave_id.value)
+ elif status.state == mesos_pb2.TASK_FINISHED:
+ raise self.Error("Task %s is in unexpected state %s with message '%s'" % (
+ status.task_id.value,
+ mesos_pb2.TaskState.Name(status.state),
+ status.message))
+ elif is_terminal(status.state):
+ if status.state == mesos_pb2.TASK_KILLED:
+ log.info("Task %s was successfully killed" % status.task_id.value)
+ else:
+ log.error("Task %s is now in terminal state %s with message '%s'" % (
+ status.task_id.value,
+ mesos_pb2.TaskState.Name(status.state),
+ status.message))
+ del self._cluster.tasks[task_id]
+
+ if task_id in self._cluster.members:
+ member_id = self._cluster.members[task_id]
+ del self._cluster.members[task_id]
+
+ # If the scheduler fails over after ZK is updated but before its result is persisted, it
+ # will receive the same status update and try to remove the non-existent member.
+ # Removing a non-existent member is a no-op for ClusterManager.remove_member().
+ # Note that if the order is reversed, the scheduler will fail to clean up the orphan ZK
+ # entry.
+ self._cluster_manager.remove_member(member_id)
+
+ if member_id == self._cluster.master_id:
+ self._cluster.master_id = None
+ log.info("Master of cluster %s has terminated. Restarting election" % self.cluster_name)
+
+ assert not self._elector, "Election must not be running since there is a current master"
+ self._elector = self._new_elector()
+
+ # Add current slaves after removing the terminated task.
+ for t in self._cluster.running_tasks:
+ self._elector.add_slave(t.task_id, t.mesos_slave_id)
+ self._elector.start()
+ else:
+ # It will be rescheduled next time the launcher is given an offer.
+ log.info("Slave %s of cluster %s has terminated" % (task_id, self.cluster_name))
+ else:
+ assert previous_state != mesos_pb2.TASK_RUNNING, (
+ "Task must exist in ClusterManager if it was running")
+ log.warn("Slave %s of cluster %s failed to start running" % (task_id, self.cluster_name))
+
+ if self.terminated:
+ log.info("Shutting down launcher for cluster %s" % self.cluster_name)
+ self._shutdown()
+ return
+
+ # Finally, checkpoint the status update.
+ self._state_provider.dump_cluster_state(self._cluster)
+ log.info("Checkpointed the status update for task %s of cluster %s" % (
+ task_id, self.cluster_name))
+
+ def _shutdown(self):
+ self._cluster_manager.delete_cluster()
+ log.info("Deleted cluster %s from ZooKeeper" % self.cluster_name)
+ self._state_provider.remove_cluster_state(self.cluster_name)
+ log.info("Removed the state of cluster %s" % self.cluster_name)
+
+ if self._elector:
+ self._elector.abort()
+ self._elector = None
+
+ def _master_elected(self, master_task):
+ """
+ Invoked by the elector when a master is elected for this cluster.
+
+ :param master_task: The task ID for the elected master.
+
+ NOTE: A new election can be started if the currently elected master has already terminated
+ before the election result arrives at the launcher.
+ """
+ if not master_task:
+ log.error("No master can be elected for cluster %s" % self.cluster_name)
+ return
+
+ with self._lock:
+ self._elector = None # Elector will terminate soon.
+
+ if master_task not in self._cluster.tasks:
+ log.info("Slave %s of cluster %s was elected but has died. Restarting election" % (
+ master_task, self.cluster_name))
+ self._elector = self._new_elector()
+ # Add current slaves.
+ for t in self._cluster.running_tasks:
+ self._elector.add_slave(t.task_id, t.mesos_slave_id)
+ self._elector.start()
+ return
+
+ assert master_task in self._cluster.members, (
+ "Elected master must have been added to 'members'")
+ master_id = self._cluster.members[master_task]
+ slave_host = self._cluster.tasks[master_task].hostname
+
+ log.info('Promoting MySQL task %s on host %s (member ID: %s) as the master for cluster %s' % (
+ master_task, slave_host, master_id, self.cluster_name))
+
+ # Persist the elected master before publishing to ZK. If the scheduler fails over before the
+ # result is persisted, it re-elects one.
+ self._cluster.master_id = master_id
+ self._state_provider.dump_cluster_state(self._cluster)
+
+ # Publish the elected master. If the scheduler fails over before the master is published, it
+ # republishes it.
+ self._cluster_manager.promote_member(master_id)
+
+ def framework_message(self, task_id, slave_id, message):
+ with self._lock:
+ if self._elector:
+ data = json.loads(message)
+ self._elector.update_position(int(data["epoch"]), task_id, data["position"])
+ else:
+ log.info("Received framework message '%s' from task %s (%s) when there is no pending "
+ "election" % (message, task_id, slave_id))
+
+
+# --- Utility methods. ---
+def create_resources(cpus, mem, ports, role='*'):
+ """Return a list of 'Resource' protobuf for the provided resources."""
+ cpus_resources = mesos_pb2.Resource()
+ cpus_resources.name = 'cpus'
+ cpus_resources.type = mesos_pb2.Value.SCALAR
+ cpus_resources.role = role
+ cpus_resources.scalar.value = cpus
+
+ mem_resources = mesos_pb2.Resource()
+ mem_resources.name = 'mem'
+ mem_resources.type = mesos_pb2.Value.SCALAR
+ mem_resources.role = role
+ mem_resources.scalar.value = mem
+
+ ports_resources = mesos_pb2.Resource()
+ ports_resources.name = 'ports'
+ ports_resources.type = mesos_pb2.Value.RANGES
+ ports_resources.role = role
+ for port in ports:
+ port_range = ports_resources.ranges.range.add()
+ port_range.begin = port
+ port_range.end = port
+
+ return [cpus_resources, mem_resources, ports_resources]
+
+
+def is_terminal(state):
+ """Return true if the task reached terminal state."""
+ return state in [
+ mesos_pb2.TASK_FINISHED,
+ mesos_pb2.TASK_KILLED,
+ mesos_pb2.TASK_FAILED,
+ mesos_pb2.TASK_LOST]
diff --git a/twitter/mysos/scheduler/mysos_scheduler.py b/twitter/mysos/scheduler/mysos_scheduler.py
new file mode 100644
index 0000000..fe9273e
--- /dev/null
+++ b/twitter/mysos/scheduler/mysos_scheduler.py
@@ -0,0 +1,295 @@
+import os
+import tempfile
+
+from twitter.common import app, log
+from twitter.common.exceptions import ExceptionalThread
+from twitter.common.http import HttpServer
+from twitter.common.log.options import LogOptions
+from twitter.common.quantity import Time
+from twitter.common.quantity.parse_simple import InvalidTime, parse_time
+from twitter.mysos.common import pkgutil, zookeeper
+
+from .http import MysosServer
+from .scheduler import MysosScheduler
+from .state import LocalStateProvider, Scheduler, StateProvider
+from .zk_state import ZooKeeperStateProvider
+
+from kazoo.client import KazooClient
+
+import mesos.interface
+from mesos.interface.mesos_pb2 import Credential, FrameworkInfo
+import mesos.native
+import yaml
+
+
+FRAMEWORK_NAME = 'mysos'
+MYSOS_MODULE = 'twitter.mysos.scheduler'
+ASSET_RELPATH = 'assets'
+
+
+LogOptions.disable_disk_logging()
+LogOptions.set_stderr_log_level('google:INFO')
+
+
+def proxy_main():
+ app.add_option(
+ '--port',
+ dest='api_port',
+ type='int',
+ default=None,
+ help='Port for the HTTP API server')
+
+ app.add_option(
+ '--mesos_master',
+ dest='mesos_master',
+ default=None,
+ help='Mesos master address. It can be a ZooKeeper URL through which the master can be '
+ 'detected')
+
+ app.add_option(
+ '--framework_user',
+ dest='framework_user',
+ help='The Unix user that Mysos executor runs as')
+
+ app.add_option(
+ '--framework_role',
+ dest='framework_role',
+ default='*',
+ help="The role that Mysos framework runs as. If set, Mysos only uses Mesos pool resources "
+ "with that role. The default value '*' is what Mesos considers as the default role.\n"
+ "NOTE: Mesos master needs to be configured to allow the specified role. See its --roles "
+ "flag")
+
+ app.add_option(
+ '--executor_uri',
+ dest='executor_uri',
+ default=None,
+ help='URI for the Mysos executor package')
+
+ app.add_option(
+ '--executor_cmd',
+ dest='executor_cmd',
+ default=None,
+ help='Command to execute the executor package')
+
+ app.add_option(
+ '--executor_environ',
+ dest='executor_environ',
+ default=None,
+ help="Environment variables for the executors (and the tasks) as a list of dicts keyed by "
+ "{name, value} in JSON. Note that these variables don't affect Mesos slave components "
+ "such as the fetcher")
+
+ app.add_option(
+ '--zk_url',
+ dest='zk_url',
+ default=None,
+ help='ZooKeeper URL for various Mysos operations, in the form of '
+ '"zk://username:password@servers/path". The sub-directory <zk_url>/discover is used for '
+ 'communicating MySQL cluster information between Mysos scheduler and executors')
+
+ # TODO(jyx): This could also be made a per-cluster configuration.
+ app.add_option(
+ '--election_timeout',
+ dest='election_timeout',
+ default='60s',
+ help='The amount of time the scheduler waits for all slaves to respond during a MySQL master '
+ 'election, e.g., 60s. After the timeout the master is elected from only the slaves that '
+ 'have responded')
+
+ app.add_option(
+ '--admin_keypath',
+ dest='admin_keypath',
+ default=None,
+ help='The path to the key file with MySQL admin credentials on Mesos slaves')
+
+ app.add_option(
+ '--work_dir',
+ dest='work_dir',
+ default=os.path.join(tempfile.gettempdir(), 'mysos'),
+ help="Directory path to place Mysos work directories, e.g., web assets, state files if "
+ "--state_storage=local. Default to a system temp directory.")
+
+ app.add_option(
+ '--state_storage',
+ dest='state_storage',
+ default='zk',
+ help="Mechanism to persist scheduler state. Available options are 'zk' and 'local'. If 'zk' "
+ "is chosen, the scheduler state is stored under <zk_url>/state; see --zk_url. Otherwise "
+ "'local' is chosen and the state is persisted under <work_dir>/state; see --work_dir")
+
+ app.add_option(
+ '--framework_failover_timeout',
+ dest='framework_failover_timeout',
+ default='14d',
+ help='Time after which Mysos framework is considered deleted. This implies losing all tasks. '
+ 'SHOULD BE VERY HIGH')
+
+ # TODO(jyx): Flags like this are generally optional but specific executor implementations may
+ # require them. Consider adding validators that can be plugged in so configuration errors can be
+ # caught in the scheduler.
+ app.add_option(
+ '--installer_args',
+ dest='installer_args',
+ default=None,
+ help='Arguments for MySQL installer directly passed along to and parsed by the installer. '
+ 'e.g., a serialized JSON string'
+ )
+
+ app.add_option(
+ '--backup_store_args',
+ dest='backup_store_args',
+ default=None,
+ help="Arguments for the store for MySQL backups. Its use and format are defined by the "
+ "backup store implementation. e.g., It can be a serialized JSON string"
+ )
+
+ app.add_option(
+ '--framework_authentication_file',
+ dest='framework_authentication_file',
+ default=None,
+ help="Path to the key file for authenticating the framework against Mesos master. Framework "
+ "will fail to register with Mesos if authentication is required by Mesos and this "
+ "option is not provided"
+ )
+
+ def main(args, options):
+ log.info("Options in use: %s", options)
+
+ if not options.api_port:
+ app.error('Must specify --port')
+
+ if not options.mesos_master:
+ app.error('Must specify --mesos_master')
+
+ if not options.framework_user:
+ app.error('Must specify --framework_user')
+
+ if not options.executor_uri:
+ app.error('Must specify --executor_uri')
+
+ if not options.executor_cmd:
+ app.error('Must specify --executor_cmd')
+
+ if not options.zk_url:
+ app.error('Must specify --zk_url')
+
+ if not options.admin_keypath:
+ app.error('Must specify --admin_keypath')
+
+ try:
+ election_timeout = parse_time(options.election_timeout)
+ framework_failover_timeout = parse_time(options.framework_failover_timeout)
+ except InvalidTime as e:
+ app.error(e.message)
+
+ try:
+ _, zk_servers, zk_root = zookeeper.parse(options.zk_url)
+ except Exception as e:
+ app.error("Invalid --zk_url: %s" % e.message)
+
+ web_assets_dir = os.path.join(options.work_dir, "web")
+ pkgutil.unpack_assets(web_assets_dir, MYSOS_MODULE, ASSET_RELPATH)
+ log.info("Extracted web assets into %s" % options.work_dir)
+
+ fw_principal = None
+ fw_secret = None
+ if options.framework_authentication_file:
+ try:
+ with open(options.framework_authentication_file, "r") as f:
+ cred = yaml.load(f)
+ fw_principal = cred["principal"]
+ fw_secret = cred["secret"]
+ log.info("Loaded credential (principal=%s) for framework authentication" % fw_principal)
+ except IOError as e:
+ app.error("Unable to read the framework authentication key file: %s" % e)
+ except (KeyError, yaml.YAMLError) as e:
+ app.error("Invalid framework authentication key file format %s" % e)
+
+ log.info("Starting Mysos scheduler")
+
+ kazoo = KazooClient(zk_servers)
+ kazoo.start()
+
+ if options.state_storage == 'zk':
+ log.info("Using ZooKeeper (path: %s) for state storage" % zk_root)
+ state_provider = ZooKeeperStateProvider(kazoo, zk_root)
+ else:
+ log.info("Using local disk for state storage")
+ state_provider = LocalStateProvider(options.work_dir)
+
+ try:
+ state = state_provider.load_scheduler_state()
+ except StateProvider.Error as e:
+ app.error(e.message)
+
+ if state:
+ log.info("Successfully restored scheduler state")
+ framework_info = state.framework_info
+ if framework_info.HasField('id'):
+ log.info("Recovered scheduler's FrameworkID is %s" % framework_info.id.value)
+ else:
+ log.info("No scheduler state to restore")
+ framework_info = FrameworkInfo(
+ user=options.framework_user,
+ name=FRAMEWORK_NAME,
+ checkpoint=True,
+ failover_timeout=framework_failover_timeout.as_(Time.SECONDS),
+ role=options.framework_role)
+ if fw_principal:
+ framework_info.principal = fw_principal
+ state = Scheduler(framework_info)
+ state_provider.dump_scheduler_state(state)
+
+ scheduler = MysosScheduler(
+ state,
+ state_provider,
+ options.framework_user,
+ options.executor_uri,
+ options.executor_cmd,
+ kazoo,
+ options.zk_url,
+ election_timeout,
+ options.admin_keypath,
+ installer_args=options.installer_args,
+ backup_store_args=options.backup_store_args,
+ executor_environ=options.executor_environ,
+ framework_role=options.framework_role)
+
+ if fw_principal and fw_secret:
+ cred = Credential(principal=fw_principal, secret=fw_secret)
+ scheduler_driver = mesos.native.MesosSchedulerDriver(
+ scheduler,
+ framework_info,
+ options.mesos_master,
+ cred)
+ else:
+ scheduler_driver = mesos.native.MesosSchedulerDriver(
+ scheduler,
+ framework_info,
+ options.mesos_master)
+
+ scheduler_driver.start()
+
+ server = HttpServer()
+ server.mount_routes(MysosServer(scheduler, web_assets_dir))
+
+ et = ExceptionalThread(
+ target=server.run, args=('0.0.0.0', options.api_port, 'cherrypy'))
+ et.daemon = True
+ et.start()
+
+ try:
+ # Wait for the scheduler to stop.
+ # The use of 'stopped' event instead of scheduler_driver.join() is necessary to stop the process
+ # with SIGINT.
+ while not scheduler.stopped.wait(timeout=0.5):
+ pass
+ except KeyboardInterrupt:
+ log.info('Interrupted, exiting.')
+ else:
+ log.info('Scheduler exited.')
+
+ app.shutdown(1) # Mysos scheduler is supposed to be long-running thus the use of exit status 1.
+
+ app.main()
diff --git a/twitter/mysos/scheduler/scheduler.py b/twitter/mysos/scheduler/scheduler.py
new file mode 100644
index 0000000..03734ae
--- /dev/null
+++ b/twitter/mysos/scheduler/scheduler.py
@@ -0,0 +1,381 @@
+from collections import OrderedDict
+import posixpath
+import random
+import threading
+import string
+import sys
+
+from twitter.common import log
+from twitter.common.collections.orderedset import OrderedSet
+from twitter.common.quantity import Amount, Time
+from twitter.mysos.common.cluster import get_cluster_path
+from twitter.mysos.common.decorators import logged
+
+from .launcher import MySQLClusterLauncher
+from .state import MySQLCluster, Scheduler, StateProvider
+
+import mesos.interface
+import mesos.interface.mesos_pb2 as mesos_pb2
+
+
+# Refuse the offer for "eternity".
+# NOTE: Using sys.maxint / 2 because sys.maxint causes rounding and precision loss when converted to
+# double 'refuse_seconds' in the ProtoBuf and results in a negative duration on Mesos Master.
+INCOMPATIBLE_ROLE_OFFER_REFUSE_DURATION = Amount(sys.maxint / 2, Time.NANOSECONDS)
+
+
+class MysosScheduler(mesos.interface.Scheduler):
+
+ class Error(Exception): pass
+
+ class ClusterExists(Error): pass
+ class ClusterNotFound(Error): pass
+ class InvalidUser(Error): pass
+ class ServiceUnavailable(Error): pass
+
+ def __init__(
+ self,
+ state,
+ state_provider,
+ framework_user,
+ executor_uri,
+ executor_cmd,
+ kazoo,
+ zk_url,
+ election_timeout,
+ admin_keypath,
+ installer_args=None,
+ backup_store_args=None,
+ executor_environ=None,
+ framework_role='*'):
+ """
+ :param state: The Scheduler object.
+ :param state_provider: The StateProvider instance that the scheduler should use to
+ restore/persist states.
+ :param framework_user: See flags.
+ :param executor_uri: See flags.
+ :param executor_cmd: See flags.
+ :param framework_role: See flags.
+ :param election_timeout: See flags.
+ :param admin_keypath: See flags.
+ :param installer_args: See flags.
+ :param backup_store_args: See flags.
+ :param executor_environ: See flags.
+ :param kazoo: The Kazoo client for communicating MySQL cluster information between the
+ scheduler and the executors.
+ :param zk_url: ZooKeeper URL for used by the scheduler and the executors to access ZooKeeper.
+ """
+ self._lock = threading.Lock()
+
+ if not isinstance(state, Scheduler):
+ raise TypeError("'state' should be an instance of Scheduler")
+ self._state = state
+
+ if not isinstance(state_provider, StateProvider):
+ raise TypeError("'state_provider' should be an instance of StateProvider")
+ self._state_provider = state_provider
+
+ self._framework_user = framework_user
+ self._executor_uri = executor_uri
+ self._executor_cmd = executor_cmd
+ self._framework_role = framework_role
+ self._election_timeout = election_timeout
+ self._admin_keypath = admin_keypath
+ self._installer_args = installer_args
+ self._backup_store_args = backup_store_args
+ self._executor_environ = executor_environ
+
+ self._driver = None # Will be set by registered().
+
+ # Use a subdir to avoid name collision with the state storage.
+ self._discover_zk_url = posixpath.join(zk_url, "discover")
+ self._kazoo = kazoo
+
+ self._tasks = {} # {Task ID: cluster name} mappings.
+ self._launchers = OrderedDict() # Order-preserving {cluster name : MySQLClusterLauncher}
+ # mappings so cluster requests are fulfilled on a first come,
+ # first serve (FCFS) basis.
+
+ self.stopped = threading.Event() # An event set when the scheduler is stopped.
+ self.connected = threading.Event() # An event set when the scheduler is first connected to
+ # Mesos. The scheduler tolerates later disconnections.
+
+ # --- Public interface. ---
+ def create_cluster(self, cluster_name, cluster_user, num_nodes, backup_id=None):
+ """
+ :param cluster_name: Name of the cluster.
+ :param cluster_user: The user account on MySQL server.
+ :param num_nodes: Number of nodes in the cluster.
+ :param backup_id: The 'backup_id' of the backup to restore from. If None then Mysos starts an
+ empty instance.
+
+ :return: a tuple of the following:
+ - ZooKeeper URL for this Mysos cluster that can be used to resolve MySQL cluster info.
+ - The password for the specified user of the specified cluster.
+
+ NOTE:
+ If the scheduler fails over after the cluster state is checkpointed, the new scheduler
+ instance will restore the state and continue to launch the cluster. TODO(jyx): Need to allow
+ users to retrieve the cluster info (e.g. the passwords) afterwards in case the user request
+ has failed but the cluster is successfully created eventually.
+ """
+ with self._lock:
+ if not self._driver:
+ # 'self._driver' is set in (re)registered() after which the scheduler becomes "available".
+ # We need to get hold of the driver to recover the scheduler.
+ raise self.ServiceUnavailable("Service unavailable. Try again later")
+
+ if cluster_name in self._state.clusters:
+ raise self.ClusterExists("Cluster '%s' already exists" % cluster_name)
+
+ if not cluster_user:
+ raise self.InvalidUser('Invalid user name: %s' % cluster_user)
+
+ if int(num_nodes) <= 0:
+ raise ValueError("Invalid number of cluster nodes: %s" % num_nodes)
+
+ self._state.clusters.add(cluster_name)
+ self._state_provider.dump_scheduler_state(self._state)
+
+ cluster = MySQLCluster(
+ cluster_name,
+ cluster_user,
+ gen_password(),
+ int(num_nodes),
+ backup_id=backup_id)
+ self._state_provider.dump_cluster_state(cluster)
+
+ log.info("Creating launcher for cluster %s" % cluster_name)
+ self._launchers[cluster_name] = MySQLClusterLauncher(
+ self._driver,
+ cluster,
+ self._state_provider,
+ self._discover_zk_url,
+ self._kazoo,
+ self._framework_user,
+ self._executor_uri,
+ self._executor_cmd,
+ self._election_timeout,
+ self._admin_keypath,
+ installer_args=self._installer_args,
+ backup_store_args=self._backup_store_args,
+ executor_environ=self._executor_environ,
+ framework_role=self._framework_role)
+
+ return get_cluster_path(self._discover_zk_url, cluster_name), cluster.password
+
+ def delete_cluster(self, cluster_name, password):
+ """
+ :return: ZooKeeper URL for this Mysos cluster that can be used to wait for the termination of
+ the cluster.
+ """
+ with self._lock:
+ if not self._driver:
+ raise self.ServiceUnavailable("Service unavailable. Try again later")
+
+ if cluster_name not in self._state.clusters:
+ raise self.ClusterNotFound("Cluster '%s' not found" % cluster_name)
+
+ launcher = self._launchers[cluster_name]
+ launcher.kill(password)
+ log.info("Attempted to kill cluster %s" % cluster_name)
+
+ return get_cluster_path(self._discover_zk_url, cluster_name)
+
+ @property
+ def clusters(self):
+ """
+ A generator for information about clusters.
+ """
+ with self._lock:
+ for launcher in self._launchers.values():
+ yield launcher.cluster_info
+
+ def _stop(self):
+ """Stop the scheduler."""
+
+ # 'self._driver' is set in registered() so it could be None if the scheduler is asked to stop
+ # before it is registered (e.g. an error is received).
+ if self._driver:
+ self._driver.stop(True) # Set failover to True.
+ self.stopped.set()
+
+ # --- Mesos methods. ---
+ @logged
+ def registered(self, driver, frameworkId, masterInfo):
+ self._driver = driver
+ self._state.framework_info.id.value = frameworkId.value
+ self._state_provider.dump_scheduler_state(self._state)
+
+ # Recover only after the scheduler is connected because it needs '_driver' to be assigned. This
+ # is blocking the scheduler driver thread but we do want further messages to be blocked until
+ # the scheduler state is fully recovered.
+ # TODO(jyx): If performance becomes an issue, we can also restore all the state data while the
+ # driver is connecting and proceed to recover all the internal state objects after the driver is
+ # connected.
+ try:
+ self._recover()
+ except self.Error as e:
+ log.error("Stopping scheduler because: %s" % e)
+ self._stop()
+ return
+
+ self.connected.set()
+
+ def _recover(self):
+ """
+ Recover launchers for existing clusters. A newly created scheduler has no launcher to recover.
+ TODO(jyx): The recovery of clusters can potentially be parallelized.
+ """
+ for cluster_name in OrderedSet(self._state.clusters): # Make a copy so we can remove dead
+ # entries while iterating the copy.
+ log.info("Recovering launcher for cluster %s" % cluster_name)
+ try:
+ cluster = self._state_provider.load_cluster_state(cluster_name)
+ if not cluster:
+ # The scheduler could have failed over before creating the launcher. The user request
+ # should have failed and there is no cluster state to restore.
+ log.info("Skipping cluster %s because its state cannot be found" % cluster_name)
+ self._state.clusters.remove(cluster_name)
+ self._state_provider.dump_scheduler_state(self._state)
+ continue
+ for task_id in cluster.tasks:
+ self._tasks[task_id] = cluster.name # Reconstruct the 'tasks' map.
+ # Order of launchers is preserved thanks to the OrderedSet.
+ # For recovered launchers we use the currently specified --framework_role and
+ # --executor_environ, etc., instead of saving it in cluster state so the change in flags can
+ # be picked up by existing clusters.
+ self._launchers[cluster.name] = MySQLClusterLauncher(
+ self._driver,
+ cluster,
+ self._state_provider,
+ self._discover_zk_url,
+ self._kazoo,
+ self._framework_user,
+ self._executor_uri,
+ self._executor_cmd,
+ self._election_timeout,
+ self._admin_keypath,
+ self._installer_args,
+ self._backup_store_args,
+ self._executor_environ,
+ self._framework_role)
+ except StateProvider.Error as e:
+ raise self.Error("Failed to recover cluster: %s" % e.message)
+
+ log.info("Recovered %s clusters" % len(self._launchers))
+
+ @logged
+ def reregistered(self, driver, masterInfo):
+ self._driver = driver
+ self.connected.set()
+ # TODO(jyx): Reconcile tasks.
+
+ @logged
+ def disconnected(self, driver):
+ pass
+
+ @logged
+ def resourceOffers(self, driver, offers):
+ log.info('Got %d resource offers' % len(offers))
+
+ with self._lock:
+ # Current scheduling algorithm: randomly pick an offer and loop through the list of launchers
+ # until one decides to use this offer to launch a task.
+ # It's possible to launch multiple tasks on the same Mesos slave (in different batches of
+ # offers).
+ for offer in shuffled(offers):
+ task_id = None
+ # 'filters' is set when we need to create a special filter for incompatible roles.
+ filters = None
+ # For each offer, launchers are asked to launch a task in the order they are added (FCFS).
+ for name in self._launchers:
+ launcher = self._launchers[name]
+ try:
+ task_id, _ = launcher.launch(offer)
+ except MySQLClusterLauncher.IncompatibleRoleError as e:
+ # This "error" is not severe and we expect this to occur frequently only when Mysos
+ # first joins the cluster. For a running Mesos cluster this should be somewhat rare
+ # because we refuse the offer "forever".
+ filters = mesos_pb2.Filters()
+ filters.refuse_seconds = INCOMPATIBLE_ROLE_OFFER_REFUSE_DURATION.as_(Time.SECONDS)
+ break # No need to check with other launchers.
+ if task_id:
+ self._tasks[task_id] = launcher.cluster_name
+ # No need to check with other launchers. 'filters' remains unset.
+ break
+ if task_id:
+ break # Some launcher has used this offer. Move on to the next one.
+
+ if filters:
+ log.info("Declining offer %s for %s because '%s'" % (
+ offer.id.value, INCOMPATIBLE_ROLE_OFFER_REFUSE_DURATION, e))
+ else:
+ log.info("Declining offer %s because no launcher can use this offer" % offer.id.value)
+ # Mesos scheduler Python binding doesn't deal with filters='None' properly.
+ # See https://issues.apache.org/jira/browse/MESOS-2567.
+ filters = mesos_pb2.Filters()
+ self._driver.declineOffer(offer.id, filters)
+
+ @logged
+ def statusUpdate(self, driver, status):
+ with self._lock:
+ # Forward the status update to the corresponding launcher.
+ task_id = status.task_id.value
+ launcher = self._get_launcher_by_task_id(task_id)
+ if not launcher:
+ log.info("Cluster for task %s doesn't exist. It could have been removed" % task_id)
+ return
+
+ try:
+ launcher.status_update(status)
+ except MySQLClusterLauncher.Error as e:
+ log.error("Status update failed due to launcher error: %s" % e.message)
+ self._stop()
+
+ if launcher.terminated:
+ log.info("Deleting the launcher for cluster %s because the cluster has terminated" %
+ launcher.cluster_name)
+ self._state.clusters.discard(launcher.cluster_name)
+ self._state_provider.dump_scheduler_state(self._state)
+ del self._launchers[launcher.cluster_name]
+
+ @logged
+ def frameworkMessage(self, driver, executorId, slaveId, message):
+ log.info('Received framework message %s' % message)
+ task_id = executorId.value # task_id == executor_id in Mysos.
+
+ launcher = self._get_launcher_by_task_id(task_id)
+ launcher.framework_message(task_id, slaveId.value, message)
+
+ @logged
+ def slaveLost(self, driver, slaveId):
+ # We receive TASK_LOSTs when a slave is lost so we we don't need to handle it separately here.
+ pass
+
+ @logged
+ def error(self, driver, message):
+ log.error('Received error from mesos: %s' % message)
+ self._stop() # SchedulerDriver aborts when an error message is received.
+
+ def _get_launcher_by_task_id(self, task_id):
+ # TODO(jyx): Currently we don't delete entries from 'self._tasks' so a mapping can always
+ # be found but we should clean it up when tasks die.
+ assert task_id in self._tasks
+ cluster_name = self._tasks[task_id]
+ return self._launchers.get(cluster_name) # Cluster could have been removed.
+
+
+def shuffled(li):
+ """Return a shuffled version of the list."""
+ copy = li[:]
+ random.shuffle(copy)
+ return copy
+
+
+def gen_password():
+ """Return a randomly-generated password of 21 characters."""
+ return ''.join(random.choice(
+ string.ascii_uppercase +
+ string.ascii_lowercase +
+ string.digits) for _ in range(21))
diff --git a/twitter/mysos/scheduler/state.py b/twitter/mysos/scheduler/state.py
new file mode 100644
index 0000000..5e8efd6
--- /dev/null
+++ b/twitter/mysos/scheduler/state.py
@@ -0,0 +1,204 @@
+from abc import abstractmethod
+import cPickle
+from cPickle import PickleError
+import os
+
+from twitter.common import log
+from twitter.common.collections.orderedset import OrderedSet
+from twitter.common.dirutil import safe_mkdir
+
+from mesos.interface import mesos_pb2
+
+
+class StateProvider(object):
+ """
+ StateProvider is responsible for checkpointing and restoring the state of the Mysos scheduler.
+
+ It maintains the following key hierarchy:
+ /state/scheduler # Scheduler-level state.
+ /state/clusters/ # Folder for all cluster-level states.
+ cluster1 # State for 'cluster1'.
+ cluster2 # State for 'cluster2'.
+ ...
+ """
+
+ class Error(Exception): pass
+
+ @abstractmethod
+ def dump_scheduler_state(self, state):
+ """Persist scheduler-level state."""
+ pass
+
+ @abstractmethod
+ def load_scheduler_state(self):
+ """
+ Restore scheduler-level state.
+ :return: The Scheduler object. None if no state is available.
+ """
+ pass
+
+ @abstractmethod
+ def dump_cluster_state(self, state):
+ """Persist cluster-level state."""
+ pass
+
+ @abstractmethod
+ def load_cluster_state(self, cluster_name):
+ """
+ Restore cluster-level state.
+ :return: The MySQLCluster object. None if no state is available.
+ """
+ pass
+
+ @abstractmethod
+ def remove_cluster_state(self, cluster_name):
+ """Remove cluster-level state."""
+ pass
+
+ # --- Helper methods. ---
+ @classmethod
+ def _get_scheduler_state_key(cls):
+ return ['state', 'scheduler']
+
+ @classmethod
+ def _get_cluster_state_key(cls, cluster_name):
+ return ['state', 'clusters', cluster_name]
+
+
+class Scheduler(object):
+ """
+ Scheduler-level state.
+
+ NOTE: It references cluster-level states indirectly through cluster names.
+ """
+
+ def __init__(self, framework_info):
+ self.framework_info = framework_info
+ self.clusters = OrderedSet() # Names of clusters this scheduler manages. cluster creation
+ # order is preserved with the OrderedSet.
+
+
+class MySQLCluster(object):
+ """
+ The state of a MySQL cluster.
+
+ It includes tasks (MySQLTask) for members of the cluster.
+ """
+
+ def __init__(self, name, user, password, num_nodes, backup_id=None):
+ if not isinstance(num_nodes, int):
+ raise TypeError("'num_nodes' should be an int")
+
+ self.name = name
+ self.user = user
+ self.password = password
+ self.num_nodes = num_nodes
+ self.backup_id = backup_id
+
+ self.members = {} # {TaskID : MemberID} mappings. MemberIDs are assigned by ZooKeeper. A task
+ # must be running and published to ZK before it becomes a member.
+ self.master_id = None # MemberID of the MySQL master.
+ self.tasks = {} # {TaskID : MySQLTask} mappings
+ self.next_epoch = 0 # Monotonically increasing number after each master change.
+ self.next_id = 0 # Monotonically increasing number for unique task IDs.
+
+ @property
+ def active_tasks(self):
+ """Tasks that have been launched and have not terminated."""
+ return [t for t in self.tasks.values() if t.state in (
+ mesos_pb2.TASK_STAGING, mesos_pb2.TASK_STARTING, mesos_pb2.TASK_RUNNING)]
+
+ @property
+ def running_tasks(self):
+ return [t for t in self.tasks.values() if t.state == mesos_pb2.TASK_RUNNING]
+
+
+class MySQLTask(object):
+ """The state of a MySQL task."""
+
+ def __init__(self, cluster_name, task_id, mesos_slave_id, hostname, port):
+ if not isinstance(port, int):
+ raise TypeError("'port' should be an int")
+
+ self.cluster_name = cluster_name # So we can refer back to the cluster it belongs to.
+ self.task_id = task_id
+ self.mesos_slave_id = mesos_slave_id
+ self.hostname = hostname
+ self.port = port
+ self.state = mesos_pb2.TASK_STAGING # Initial state. Will be updated by statusUpdate().
+
+
+class LocalStateProvider(StateProvider):
+ """StateProvider implementation that uses local disk to store the state."""
+
+ def __init__(self, work_dir):
+ """
+ :param work_dir: The root directory under which the scheduler state is stored. e.g. The path
+ for 'cluster1' is <work_dir>/state/clusters/cluster1.
+ """
+ self._work_dir = work_dir
+
+ def dump_scheduler_state(self, state):
+ if not isinstance(state, Scheduler):
+ raise TypeError("'state' should be an instance of Scheduler")
+ path = self._get_scheduler_state_path()
+ safe_mkdir(os.path.dirname(path))
+
+ try:
+ with open(path, 'wb') as f:
+ cPickle.dump(state, f)
+ except PickleError as e:
+ raise self.Error('Failed to persist Scheduler: %s' % e)
+
+ def load_scheduler_state(self):
+ path = self._get_scheduler_state_path()
+ if not os.path.isfile(path):
+ log.info("No scheduler state found on path %s" % path)
+ return None
+
+ try:
+ with open(path, 'rb') as f:
+ return cPickle.load(f)
+ except PickleError as e:
+ raise self.Error('Failed to recover Scheduler: %s' % e)
+
+ def dump_cluster_state(self, state):
+ if not isinstance(state, MySQLCluster):
+ raise TypeError("'state' should be an instance of MySQLCluster")
+
+ path = self._get_cluster_state_path(state.name)
+ safe_mkdir(os.path.dirname(path))
+
+ try:
+ with open(path, 'wb') as f:
+ return cPickle.dump(state, f)
+ except PickleError as e:
+ raise self.Error('Failed to persist state for cluster %s: %s' % (state.name, e))
+
+ def load_cluster_state(self, cluster_name):
+ path = self._get_cluster_state_path(cluster_name)
+ if not os.path.isfile(path):
+ log.info("No cluster state found on path %s" % path)
+ return None
+
+ try:
+ with open(path, 'rb') as f:
+ return cPickle.load(f)
+ except PickleError as e:
+ raise self.Error('Failed to recover MySQLCluster: %s' % e)
+
+ @abstractmethod
+ def remove_cluster_state(self, cluster_name):
+ path = self._get_cluster_state_path(cluster_name)
+ if not os.path.isfile(path):
+ log.info("No cluster state found on path %s" % path)
+ return
+
+ os.remove(path)
+
+ # --- Helper methods. ---
+ def _get_scheduler_state_path(self):
+ return os.path.join(self._work_dir, os.path.join(*self._get_scheduler_state_key()))
+
+ def _get_cluster_state_path(self, cluster_name):
+ return os.path.join(self._work_dir, os.path.join(*self._get_cluster_state_key(cluster_name)))
diff --git a/twitter/mysos/scheduler/zk_state.py b/twitter/mysos/scheduler/zk_state.py
new file mode 100644
index 0000000..ca31980
--- /dev/null
+++ b/twitter/mysos/scheduler/zk_state.py
@@ -0,0 +1,100 @@
+import cPickle
+from cPickle import PickleError
+import posixpath
+
+from twitter.common import log
+
+from .state import MySQLCluster, Scheduler, StateProvider
+
+from kazoo.exceptions import KazooException, NoNodeError
+
+
+class ZooKeeperStateProvider(StateProvider):
+ """
+ StateProvider implementation backed by ZooKeeper.
+
+ This class is thread-safe.
+ """
+
+ def __init__(self, client, base_path):
+ """
+ :param client: Kazoo client.
+ :param base_path: The base path for the scheduler state on ZooKeeper.
+ """
+ self._client = client
+ self._base_path = base_path
+
+ def dump_scheduler_state(self, state):
+ if not isinstance(state, Scheduler):
+ raise TypeError("'state' should be an instance of Scheduler")
+
+ path = self._get_scheduler_state_path()
+ self._client.retry(self._client.ensure_path, posixpath.dirname(path))
+
+ content = cPickle.dumps(state)
+ try:
+ self._client.retry(self._create_or_set, path, content)
+ except KazooException as e:
+ raise self.Error('Failed to persist Scheduler: %s' % e)
+
+ def load_scheduler_state(self):
+ path = self._get_scheduler_state_path()
+
+ try:
+ content = self._client.get(path)[0]
+ state = cPickle.loads(content)
+ if not isinstance(state, Scheduler):
+ raise self.Error("Invalid state object. Expect Scheduler, got %s" % type(state))
+ return state
+ except NoNodeError:
+ log.info('No scheduler state found on path %s' % path)
+ return None
+ except (KazooException, PickleError, ValueError) as e:
+ raise self.Error('Failed to recover Scheduler: %s' % e)
+
+ def dump_cluster_state(self, state):
+ if not isinstance(state, MySQLCluster):
+ raise TypeError("'state' should be an instance of MySQLCluster")
+
+ path = self._get_cluster_state_path(state.name)
+ self._client.retry(self._client.ensure_path, posixpath.dirname(path))
+
+ content = cPickle.dumps(state)
+ self._client.retry(self._create_or_set, path, content)
+
+ def load_cluster_state(self, cluster_name):
+ path = self._get_cluster_state_path(cluster_name)
+
+ try:
+ content = self._client.get(path)[0]
+ state = cPickle.loads(content)
+ if not isinstance(state, MySQLCluster):
+ raise self.Error("Invalid state object. Expect MySQLCluster, got %s" % type(state))
+ return state
+ except NoNodeError:
+ log.info('No cluster state found on path %s' % path)
+ return None
+ except (KazooException, PickleError, ValueError) as e:
+ raise self.Error('Failed to recover MySQLCluster: %s' % e)
+
+ def remove_cluster_state(self, cluster_name):
+ path = self._get_cluster_state_path(cluster_name)
+ try:
+ self._client.retry(self._client.delete, path, recursive=True)
+ except KazooException as e:
+ raise self.Error("Failed to remove MySQLCluster: %s" % e)
+
+ # --- Helper methods. ---
+ def _get_scheduler_state_path(self):
+ return posixpath.join(self._base_path, posixpath.join(*self._get_scheduler_state_key()))
+
+ def _get_cluster_state_path(self, cluster_name):
+ return posixpath.join(
+ self._base_path, posixpath.join(*self._get_cluster_state_key(cluster_name)))
+
+ def _create_or_set(self, path, content):
+ """Set the ZNode if the path exists, otherwise create it."""
+ if self._client.exists(path):
+ self._client.set(path, content)
+ else:
+ self._client.create(path, content)
diff --git a/twitter/mysos/testing/__init__.py b/twitter/mysos/testing/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/twitter/mysos/testing/__init__.py
diff --git a/twitter/mysos/testing/mysos_test_client.py b/twitter/mysos/testing/mysos_test_client.py
new file mode 100755
index 0000000..59c1c9e
--- /dev/null
+++ b/twitter/mysos/testing/mysos_test_client.py
@@ -0,0 +1,178 @@
+from __future__ import print_function
+
+import BaseHTTPServer
+import json
+import os
+import tempfile
+import time
+import urllib
+import urllib2
+
+from twitter.common import app, log
+from twitter.common.dirutil import safe_mkdir
+from twitter.common.log.options import LogOptions
+from twitter.mysos.common.cluster import wait_for_master, wait_for_termination
+
+from sqlalchemy import create_engine
+from sqlalchemy.exc import OperationalError
+
+
+LogOptions.disable_disk_logging()
+LogOptions.set_stderr_log_level('google:INFO')
+
+
+app.add_option(
+ '--api_host',
+ dest='api_host',
+ help='Host for the HTTP API server')
+
+
+app.add_option(
+ '--api_port',
+ dest='api_port',
+ type='int',
+ help='Port for the HTTP API server')
+
+
+app.add_option(
+ '--cluster',
+ dest='cluster_name',
+ help='Name of the MySQL cluster to create')
+
+
+app.add_option(
+ '--password_file',
+ dest='password_file',
+ default=os.path.join(tempfile.gettempdir(), 'mysos', 'mysos_test_client', 'password_file'),
+ help="Path to the file for persisting the cluster password for testing purposes")
+
+
+def proxy_main():
+ @app.command
+ @app.command_option(
+ '--num_nodes',
+ dest='num_nodes',
+ type='int',
+ help='Number of nodes this cluster should have')
+ @app.command_option(
+ '--backup_id',
+ dest='backup_id',
+ default=None,
+ help="The 'backup_id' to restore from")
+ @app.command_option(
+ '--cluster_user',
+ dest='cluster_user',
+ help='MySQL user name the of cluster')
+ def create(args, options):
+ validate_common_options(options)
+
+ if not options.num_nodes:
+ app.error("--num_nodes is required")
+
+ if not options.cluster_user:
+ app.error("--cluster_user is required")
+
+ url = 'http://%s:%s/clusters/%s' % (options.api_host, options.api_port, options.cluster_name)
+ values = dict(
+ num_nodes=int(options.num_nodes),
+ cluster_user=options.cluster_user,
+ backup_id=options.backup_id if options.backup_id else '')
+
+ req = urllib2.Request(url, urllib.urlencode(values))
+ try:
+ response = urllib2.urlopen(req).read()
+ except urllib2.HTTPError as e:
+ log.error("POST request failed: %s, %s, %s" % (
+ e.code, BaseHTTPServer.BaseHTTPRequestHandler.responses[e.code], e.read()))
+ app.quit(1)
+
+ try:
+ result = json.loads(response)
+ if not isinstance(result, dict):
+ raise ValueError()
+ except ValueError:
+ log.error("Invalid response: %s" % response)
+ app.quit(1)
+
+ log.info("Cluster created. Cluster info: %s" % str(result))
+ with open(options.password_file, 'w') as f:
+ f.write(result["cluster_password"])
+
+ log.info("Waiting for the master for this cluster to be elected...")
+ master_endpoint = wait_for_master(result['cluster_url']).service_endpoint
+
+ connection_str = "mysql://%s:%s@%s:%d/" % (
+ options.cluster_user, result["cluster_password"], master_endpoint.host, master_endpoint.port)
+ log.info("Connecting to the MySQL cluster master: %s" % connection_str)
+ engine = create_engine(connection_str)
+
+ for i in range(5): # Loop for 5 times/seconds to wait for the master to be promoted.
+ try:
+ # TODO(jyx): Test writing to the master and reading from the slave.
+ result = engine.execute("SELECT 1;").scalar()
+ assert 1 == int(result), "Expecting result to be 1 but got %s" % result
+ break
+ except OperationalError:
+ if i == 4:
+ raise
+ log.debug("MySQL master not ready yet. Sleep for 1 second...")
+ time.sleep(1)
+
+ log.info("Cluster successfully started")
+
+ @app.command
+ def delete(args, options):
+ validate_common_options(options)
+
+ with open(options.password_file, 'r') as f:
+ password = f.read().strip()
+ if not password:
+ app.error("Empty password file")
+
+ url = 'http://%s:%s/clusters/%s' % (options.api_host, options.api_port, options.cluster_name)
+ values = dict(password=password)
+
+ req = urllib2.Request(url, urllib.urlencode(values))
+ req.get_method = lambda: 'DELETE'
+
+ try:
+ response = urllib2.urlopen(req).read()
+ except urllib2.HTTPError as e:
+ log.error("DELETE request failed: %s, %s, %s" % (
+ e.code, BaseHTTPServer.BaseHTTPRequestHandler.responses[e.code], e.read()))
+ app.quit(1)
+
+ try:
+ result = json.loads(response)
+ if not isinstance(result, dict):
+ raise ValueError()
+ except ValueError:
+ log.error("Invalid response: %s" % response)
+ app.quit(1)
+
+ log.info("Cluster deletion result: %s" % result)
+
+ log.info("Waiting for the cluster to terminate...")
+ wait_for_termination(result['cluster_url'])
+
+ log.info("Cluster terminated/deleted")
+
+ def validate_common_options(options):
+ if not options.api_host:
+ app.error("--api_host is required")
+
+ if not options.api_port:
+ app.error("--api_port is required")
+
+ if not options.cluster_name:
+ app.error("--cluster is required")
+
+ if not options.password_file:
+ app.error("--password_file is required")
+ log.info("Using --password_file=%s" % options.password_file)
+ safe_mkdir(os.path.dirname(options.password_file))
+
+ def main(args, options):
+ app.help()
+
+ app.main()
diff --git a/vagrant/bin/mysos_executor.sh b/vagrant/bin/mysos_executor.sh
new file mode 100755
index 0000000..d9738f1
--- /dev/null
+++ b/vagrant/bin/mysos_executor.sh
@@ -0,0 +1,12 @@
+#!/bin/sh
+
+set -uex
+
+virtualenv venv # Create vent in the sandbox.
+
+# Using python to run pip and vagrant_mysos_executor because the shebang in venv/bin/pip can
+# exceed system limit and cannot be executed directly.
+venv/bin/python venv/bin/pip install --find-links /home/vagrant/mysos/deps mesos.native
+venv/bin/python venv/bin/pip install --pre --find-links . mysos
+
+venv/bin/python venv/bin/vagrant_mysos_executor
diff --git a/vagrant/bin/mysos_scheduler.sh b/vagrant/bin/mysos_scheduler.sh
new file mode 100755
index 0000000..e8148af
--- /dev/null
+++ b/vagrant/bin/mysos_scheduler.sh
@@ -0,0 +1,27 @@
+#!/bin/sh
+
+set -uex
+
+TMPDIR=$(mktemp -d)
+
+virtualenv $TMPDIR # Create venv under /tmp.
+$TMPDIR/bin/pip install --find-links /home/vagrant/mysos/deps mesos.native
+$TMPDIR/bin/pip install --pre --find-links /home/vagrant/mysos/dist mysos
+
+ZK_HOST=192.168.33.17
+API_PORT=55001
+
+# NOTE: In --executor_environ we are pointing MYSOS_DEFAULTS_FILE to an empty MySQL defaults file.
+# The file 'my5.6.cnf' is pre-installed by the 'mysql-server-5.6' package on the VM.
+$TMPDIR/bin/mysos_scheduler \
+ --port=$API_PORT \
+ --framework_user=vagrant \
+ --mesos_master=zk://$ZK_HOST:2181/mesos/master \
+ --executor_uri=/home/vagrant/mysos/dist/mysos-0.1.0-dev0.zip \
+ --executor_cmd=/home/vagrant/mysos/vagrant/bin/mysos_executor.sh \
+ --zk_url=zk://$ZK_HOST:2181/mysos \
+ --admin_keypath=/home/vagrant/mysos/vagrant/etc/admin_keyfile.yml \
+ --framework_failover_timeout=1m \
+ --framework_role=mysos \
+ --framework_authentication_file=/home/vagrant/mysos/vagrant/etc/fw_auth_keyfile.yml \
+ --executor_environ='[{"name": "MYSOS_DEFAULTS_FILE", "value": "/etc/mysql/conf.d/my5.6.cnf"}]'
diff --git a/vagrant/etc/admin_keyfile.yml b/vagrant/etc/admin_keyfile.yml
new file mode 100644
index 0000000..013be28
--- /dev/null
+++ b/vagrant/etc/admin_keyfile.yml
@@ -0,0 +1,2 @@
+username: sys.mysos.0
+password: testpassword
\ No newline at end of file
diff --git a/vagrant/etc/framework_keys.txt b/vagrant/etc/framework_keys.txt
new file mode 100644
index 0000000..66ec943
--- /dev/null
+++ b/vagrant/etc/framework_keys.txt
@@ -0,0 +1 @@
+mysos testpassword
\ No newline at end of file
diff --git a/vagrant/etc/fw_auth_keyfile.yml b/vagrant/etc/fw_auth_keyfile.yml
new file mode 100644
index 0000000..321d396
--- /dev/null
+++ b/vagrant/etc/fw_auth_keyfile.yml
@@ -0,0 +1,2 @@
+principal: mysos
+secret: testpassword
\ No newline at end of file
diff --git a/vagrant/provision-dev-cluster.sh b/vagrant/provision-dev-cluster.sh
new file mode 100755
index 0000000..d8390cf
--- /dev/null
+++ b/vagrant/provision-dev-cluster.sh
@@ -0,0 +1,71 @@
+#!/bin/bash -x
+
+# This script requires mysos binaries to be built. Run `tox` first.
+
+# Install dependencies.
+export DEBIAN_FRONTEND=noninteractive
+aptitude update -q
+aptitude install -q -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" \
+ libcurl3-dev \
+ libsasl2-dev \
+ python-dev \
+ zookeeper \
+ mysql-server-5.6 \
+ libmysqlclient-dev \
+ libunwind8 \
+ python-virtualenv \
+ bison flex # For libnl.
+
+# Fix up a dependency issue of Mesos egg: _mesos.so links to libunwind.so.7 but Trusty only has
+# libunwind.so.8.
+ln -sf /usr/lib/x86_64-linux-gnu/libunwind.so.8 /usr/lib/x86_64-linux-gnu/libunwind.so.7
+
+# Fix up Ubuntu mysql-server-5.6 issue: mysql_install_db looks for this file even if we don't need
+# it.
+ln -sf /usr/share/doc/mysql-server-5.6/examples/my-default.cnf /usr/share/mysql/my-default.cnf
+
+# Set the hostname to the IP address. This simplifies things for components that want to advertise
+# the hostname to the user, or other components.
+hostname 192.168.33.17
+
+# Install an update-mysos tool to sync mysos binaries and configs into the VM.
+cat > /usr/local/bin/update-mysos <<EOF
+#!/bin/bash
+mkdir -p /home/vagrant/mysos
+rsync -urzvh /vagrant/vagrant/ /home/vagrant/mysos/vagrant/ --delete
+rsync -urzvh /vagrant/.tox/dist/ /home/vagrant/mysos/dist/ --delete
+rsync -urzvh /vagrant/3rdparty/ /home/vagrant/mysos/3rdparty/ --delete
+
+# Install the upstart configurations.
+sudo cp /home/vagrant/mysos/vagrant/upstart/*.conf /etc/init
+chown -R vagrant:vagrant /home/vagrant/mysos
+EOF
+chmod +x /usr/local/bin/update-mysos
+sudo -u vagrant update-mysos
+
+# Install Mesos.
+MESOS_VERSION=0.20.1
+UBUNTU_YEAR=14
+UBUNTU_MONTH=04
+
+mkdir -p /home/vagrant/mysos/deps
+pushd /home/vagrant/mysos/deps
+if [ ! -f mesos_${MESOS_VERSION}-1.0.ubuntu${UBUNTU_YEAR}${UBUNTU_MONTH}_amd64.deb ]; then
+ curl --silent --output mesos_${MESOS_VERSION}-1.0.ubuntu${UBUNTU_YEAR}${UBUNTU_MONTH}_amd64.deb \
+ http://downloads.mesosphere.io/master/ubuntu/${UBUNTU_YEAR}.${UBUNTU_MONTH}/mesos_${MESOS_VERSION}-1.0.ubuntu${UBUNTU_YEAR}${UBUNTU_MONTH}_amd64.deb
+ dpkg --install --force-confdef --force-confold \
+ /home/vagrant/mysos/deps/mesos_${MESOS_VERSION}-1.0.ubuntu${UBUNTU_YEAR}${UBUNTU_MONTH}_amd64.deb
+fi
+
+if [ ! -f mesos.native-${MESOS_VERSION}-cp27-none-linux_x86_64.whl ]; then
+ # NOTE: We rename the output file here so the egg is properly named.
+ curl --silent --output mesos.native-${MESOS_VERSION}-py2.7-linux-x86_64.egg http://downloads.mesosphere.io/master/ubuntu/${UBUNTU_YEAR}.${UBUNTU_MONTH}/mesos-${MESOS_VERSION}-py2.7-linux-x86_64.egg
+ python -m wheel convert mesos.native-${MESOS_VERSION}-py2.7-linux-x86_64.egg
+fi
+popd
+
+# (Re)start services with new conf.
+stop zookeeper ; start zookeeper
+stop mesos-master ; start mesos-master
+stop mesos-slave ; start mesos-slave
+stop mysos ; start mysos
diff --git a/vagrant/test.sh b/vagrant/test.sh
new file mode 100755
index 0000000..a0b69c9
--- /dev/null
+++ b/vagrant/test.sh
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+# This is an 'end-to-end' test that requires Mysos to be up and running in the Vagrant VM.
+
+set -ue
+
+host="192.168.33.17"
+port=55001
+cluster_name="test_cluster$((RANDOM % 1000))"
+num_nodes=1
+cluster_user="mysos"
+
+HERE="$(cd "$(dirname "$0")" && pwd)"
+executable=$HERE/../.tox/py27/bin/mysos_test_client
+
+if [ ! -f ${executable} ]; then
+ echo "${executable} doesn't exist. Build it first."
+ exit 1
+fi
+
+${executable} create \
+ --api_host=${host} \
+ --api_port=${port} \
+ --cluster_user=${cluster_user} \
+ --cluster=${cluster_name} \
+ --num_nodes=${num_nodes}
+
+echo "Finished creating the cluster, now deleting it"
+
+${executable} delete \
+ --api_host=${host} \
+ --api_port=${port} \
+ --cluster=${cluster_name}
+
+echo "Finished deleting the cluster"
diff --git a/vagrant/upstart/mesos-master.conf b/vagrant/upstart/mesos-master.conf
new file mode 100644
index 0000000..4dee7e7
--- /dev/null
+++ b/vagrant/upstart/mesos-master.conf
@@ -0,0 +1,17 @@
+description "mesos master"
+start on stopped rc RUNLEVEL=[2345]
+respawn
+post-stop exec sleep 5
+
+env MY_HOST=192.168.33.17
+env ZK_HOST=192.168.33.17
+
+exec /usr/local/sbin/mesos-master \
+ --zk=zk://$ZK_HOST:2181/mesos/master \
+ --ip=$MY_HOST \
+ --work_dir=/var/local/mesos/master/db \
+ --quorum=1 \
+ --roles=mysos \
+ --credentials=/home/vagrant/mysos/vagrant/etc/framework_keys.txt
+
+console log
diff --git a/vagrant/upstart/mesos-slave.conf b/vagrant/upstart/mesos-slave.conf
new file mode 100644
index 0000000..1f7b6d9
--- /dev/null
+++ b/vagrant/upstart/mesos-slave.conf
@@ -0,0 +1,17 @@
+description "mesos slave"
+start on stopped rc RUNLEVEL=[2345]
+respawn
+post-stop exec sleep 5
+
+env MY_HOST=192.168.33.17
+env ZK_HOST=192.168.33.17
+
+exec /usr/local/sbin/mesos-slave \
+ --master=zk://$ZK_HOST:2181/mesos/master \
+ --ip=$MY_HOST \
+ --hostname=$MY_HOST \
+ --resources="cpus(mysos):4;mem(mysos):1024;disk(mysos):20000;ports(mysos):[31000-32000]" \
+ --isolation="cgroups/cpu,cgroups/mem" \
+ --cgroups_enable_cfs
+
+console log
diff --git a/vagrant/upstart/mysos.conf b/vagrant/upstart/mysos.conf
new file mode 100644
index 0000000..6205e10
--- /dev/null
+++ b/vagrant/upstart/mysos.conf
@@ -0,0 +1,8 @@
+description "mysos"
+start on stopped rc RUNLEVEL=[2345]
+respawn
+post-stop exec sleep 5
+
+exec /home/vagrant/mysos/vagrant/bin/mysos_scheduler.sh
+
+console log
diff --git a/vagrant/upstart/zookeeper.conf b/vagrant/upstart/zookeeper.conf
new file mode 100644
index 0000000..9e666bc
--- /dev/null
+++ b/vagrant/upstart/zookeeper.conf
@@ -0,0 +1,10 @@
+description "zookeeper"
+start on stopped rc RUNLEVEL=[2345]
+respawn
+post-stop exec sleep 5
+
+env JVMFLAGS='-Djava.net.preferIPv4Stack=true'
+
+exec /usr/share/zookeeper/bin/zkServer.sh start-foreground
+
+console log
