solr/solr-ref-guide/src/package-manager.adoc - lucene-solr - Git at Google

 = Package Management
 :page-children: package-manager-internals
 // Licensed to the Apache Software Foundation (ASF) under one
 // or more contributor license agreements.  See the NOTICE file
 // distributed with this work for additional information
 // regarding copyright ownership.  The ASF licenses this file
 // to you under the Apache License, Version 2.0 (the
 // "License"); you may not use this file except in compliance
 // with the License.  You may obtain a copy of the License at
 //
 //   http://www.apache.org/licenses/LICENSE-2.0
 //
 // Unless required by applicable law or agreed to in writing,
 // software distributed under the License is distributed on an
 // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 // KIND, either express or implied.  See the License for the
 // specific language governing permissions and limitations
 // under the License.

 The package manager in Solr allows installation and updating of Solr-specific packages in Solr's cluster environment.

 In this system, a _package_ is a set of Java jar files (usually one) containing one or more <<solr-plugins.adoc#,Solr plugins>>. Each jar file is also accompanied by a signature string (which can be verified against a supplied public key).

 A key design aspect of this system is the ability to install or update packages in a cluster environment securely without the need to restart every node.

 Other elements of the design include the ability to install from a remote repository; package standardization; a command line interface (CLI); and a package store.

 This section will focus on how to use the package manager to install and update packages.
 For technical details, see the section <<package-manager-internals.adoc#,Package Manager internals>>.

 == Interacting with the Package Manager

 The package manager (CLI) allows you to:

 * Add trusted repositories
 * List packages at a repository
 * Install desired packages
 * Deploy and undeploy packages to/from collections or cluster
 * Update packages when updates are available

 === Enable the Package Manager

 The package manager is disabled by default. To enable it, start all Solr nodes with the `-Denable.packages=true` parameter.

 [source,bash]
 ----
 $ bin/solr -c -Denable.packages=true
 ----

 WARNING: There are security consequences to enabling the package manager.
 If an unauthorized user gained access to the system, they would have write access to ZooKeeper and could install packages from untrusted sources. Always ensure you have secured Solr with firewalls and <<authentication-and-authorization-plugins.adoc#,authentication>> before enabling the package manager.

 === Add Trusted Repositories

 A _repository_ is a location hosting one or many packages. Often, this is a web service that serves meta-information about packages, the package artifacts for downloading, and a public key to validate the jar file signatures while installing.

 In order to install packages into Solr, one has to add a repository hosting the packages.

 [source,bash]
 ----
 $ bin/solr package add-repo <name-of-repo> <repo-url>
 ----

 NOTE: Do not add repositories that you don't trust or control. Only add repositories that are based on HTTPS and avoid repositories based on HTTP to safeguard against MITM attacks.

 === Listing and Installing Packages

 To list installed packages:

 [source,bash]
 ----
 $ bin/solr package list-installed
 ----

 To list packages available for installation from added repositories:

 [source,bash]
 ----
 $ bin/solr package list-available
 ----

 To install a package:

 [source,bash]
 ----
 $ bin/solr package install <package-name>[:<version>]
 ----

 === Deploy a Package

 Once a package has been installed, the plugins contained in it can be used in a collection or at the cluster level.

 There are two ways to do this: either use the CLI's `deploy` command or manually.

 ==== deploy Command

 If the package author states support for it, the package can be deployed with the CLI's `deploy` command:

 [source,bash]
 ----
 $ bin/solr package deploy <package-name>:[version] -collections <collection1>[,<collection2>,...]
 ----

 or

 [source,bash]
 ----
 $ bin/solr package deploy <package-name>:[version] -cluster
 ----

 The author may want you to confirm deployment of a package via a prompt.
 If you pass `-y` to the command, confirmation can be skipped.

 ==== Manual Deploy

 It is also possible to deploy a package's collection level plugins manually by editing a configset (e.g., `solrconfig.xml`, `managed-schema`/`schema.xml`, etc.) and reloading the collection.

 For example, if a package named `mypackage` contains a request handler, we would add it to a configset's `solrconfig.xml` like this:

 [source, xml]
 ----
 <requestHandler name="/myhandler" class="mypackage:full.path.to.MyClass"></requestHandler>
 ----

 Then use either the Collections API <<collection-management.adoc#reload,RELOAD command>> or the <<collections-core-admin.adoc#,Admin UI>> to reload the collection.

 Next, set the package version that this collection is using. If the collection is named `collection1`, the package name is `mypackage`, and the installed version is `1.0.0`, the command would look like this:

 [source,bash]
 ----
 curl  "http://localhost:8983/api/collections/collection1/config/params" \
    -H 'Content-type:application/json' -d "{set: {PKG_VERSIONS: {mypackage: '1.0.0'}}}"
 ----

 For installing cluster level plugins manually, see https://issues.apache.org/jira/browse/SOLR-14404[cluster level request handlers].

 ==== Verify the Deployment
 After deploying, verify that the collection is using the package:

 [source,bash]
 ----
 $ bin/solr package list-deployed -c <collection>
 ----

 === Updating Packages

 In order to update a package, first step is make sure the updated version is available in the added repositories by running `list-available` command shown above in <<Listing and Installing Packages>>.

 Next, install the new version of the package from the repositories.

 [source,bash]
 ----
 $ bin/solr package install <package-name>:<version>
 ----

 Once you have installed the new version, you can selectively update each of your collections or the cluster level plugins. Assuming the old version is `1.0.0` of the package `mypackage`, and the new version is `2.0.0`, the command would be as follows:

 [source,bash]
 ----
 $ bin/solr package deploy mypackage:2.0.0 --update -collections mycollection
 ----
 or
 [source,bash]
 ----
 $ bin/solr package deploy mypackage:2.0.0 --update -cluster
 ----

 You can run the `list-deployed` command to verify that this collection is using the newly added version.

 === Undeploy a Package

 If a package supports undeploying the plugins it contains (check package author's documentation), then a previously deployed package can be undeployed as follows:

 [source,bash]
 ----
 $ bin/solr package undeploy <package-name> -collections <collection1>[,<collection2>,...]
 ----

 === Uninstall a Package

 If a package has been undeployed or was never deployed, then it can be uninstalled as follows:

 [source,bash]
 ----
 $ bin/solr package uninstall <package-name>:<package-version>
 ----

 or

 [source,bash]
 ----
 $ bin/solr package deploy <package-name> -cluster
 ----

 == Security

 The `add-repo` step should only be executed using HTTPS enabled repository urls only so as to prevent against MITM attacks when Solr is fetching the public key for the repository. This `add-repo` step registers the public key of the trusted repository, and hence can only be executed using the package manager (CLI) having direct write access to the trusted store of the package store (a special location in the package store that cannot be written to using the package store APIs). Also, it is critical to protect ZooKeeper from unauthorized write access.

 Also, keep in mind, that it is possible to install *any* package from a repository once it has been added. If you want to use some packages in production, a best practice is to setup your own repository and add that to Solr instead of adding a generic third-party repository that is beyond your administrative control. You might want to re-sign packages from a third-party repository using your own private keys and host them at your own repository.
	= Package Management
	:page-children: package-manager-internals
	// Licensed to the Apache Software Foundation (ASF) under one
	// or more contributor license agreements. See the NOTICE file
	// distributed with this work for additional information
	// regarding copyright ownership. The ASF licenses this file
	// to you under the Apache License, Version 2.0 (the
	// "License"); you may not use this file except in compliance
	// with the License. You may obtain a copy of the License at
	//
	// http://www.apache.org/licenses/LICENSE-2.0
	//
	// Unless required by applicable law or agreed to in writing,
	// software distributed under the License is distributed on an
	// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	// KIND, either express or implied. See the License for the
	// specific language governing permissions and limitations
	// under the License.

	The package manager in Solr allows installation and updating of Solr-specific packages in Solr's cluster environment.

	In this system, a _package_ is a set of Java jar files (usually one) containing one or more <<solr-plugins.adoc#,Solr plugins>>. Each jar file is also accompanied by a signature string (which can be verified against a supplied public key).

	A key design aspect of this system is the ability to install or update packages in a cluster environment securely without the need to restart every node.

	Other elements of the design include the ability to install from a remote repository; package standardization; a command line interface (CLI); and a package store.

	This section will focus on how to use the package manager to install and update packages.
	For technical details, see the section <<package-manager-internals.adoc#,Package Manager internals>>.

	== Interacting with the Package Manager

	The package manager (CLI) allows you to:

	* Add trusted repositories
	* List packages at a repository
	* Install desired packages
	* Deploy and undeploy packages to/from collections or cluster
	* Update packages when updates are available

	=== Enable the Package Manager

	The package manager is disabled by default. To enable it, start all Solr nodes with the `-Denable.packages=true` parameter.

	[source,bash]
	----
	$ bin/solr -c -Denable.packages=true
	----

	WARNING: There are security consequences to enabling the package manager.
	If an unauthorized user gained access to the system, they would have write access to ZooKeeper and could install packages from untrusted sources. Always ensure you have secured Solr with firewalls and <<authentication-and-authorization-plugins.adoc#,authentication>> before enabling the package manager.

	=== Add Trusted Repositories

	A _repository_ is a location hosting one or many packages. Often, this is a web service that serves meta-information about packages, the package artifacts for downloading, and a public key to validate the jar file signatures while installing.

	In order to install packages into Solr, one has to add a repository hosting the packages.

	[source,bash]
	----
	$ bin/solr package add-repo <name-of-repo> <repo-url>
	----

	NOTE: Do not add repositories that you don't trust or control. Only add repositories that are based on HTTPS and avoid repositories based on HTTP to safeguard against MITM attacks.

	=== Listing and Installing Packages

	To list installed packages:

	[source,bash]
	----
	$ bin/solr package list-installed
	----

	To list packages available for installation from added repositories:

	[source,bash]
	----
	$ bin/solr package list-available
	----

	To install a package:

	[source,bash]
	----
	$ bin/solr package install <package-name>[:<version>]
	----

	=== Deploy a Package

	Once a package has been installed, the plugins contained in it can be used in a collection or at the cluster level.

	There are two ways to do this: either use the CLI's `deploy` command or manually.

	==== deploy Command

	If the package author states support for it, the package can be deployed with the CLI's `deploy` command:

	[source,bash]
	----
	$ bin/solr package deploy <package-name>:[version] -collections <collection1>[,<collection2>,...]
	----

	or

	[source,bash]
	----
	$ bin/solr package deploy <package-name>:[version] -cluster
	----

	The author may want you to confirm deployment of a package via a prompt.
	If you pass `-y` to the command, confirmation can be skipped.

	==== Manual Deploy

	It is also possible to deploy a package's collection level plugins manually by editing a configset (e.g., `solrconfig.xml`, `managed-schema`/`schema.xml`, etc.) and reloading the collection.

	For example, if a package named `mypackage` contains a request handler, we would add it to a configset's `solrconfig.xml` like this:

	[source, xml]
	----
	<requestHandler name="/myhandler" class="mypackage:full.path.to.MyClass"></requestHandler>
	----

	Then use either the Collections API <<collection-management.adoc#reload,RELOAD command>> or the <<collections-core-admin.adoc#,Admin UI>> to reload the collection.

	Next, set the package version that this collection is using. If the collection is named `collection1`, the package name is `mypackage`, and the installed version is `1.0.0`, the command would look like this:

	[source,bash]
	----
	curl "http://localhost:8983/api/collections/collection1/config/params" \
	-H 'Content-type:application/json' -d "{set: {PKG_VERSIONS: {mypackage: '1.0.0'}}}"
	----

	For installing cluster level plugins manually, see https://issues.apache.org/jira/browse/SOLR-14404[cluster level request handlers].

	==== Verify the Deployment
	After deploying, verify that the collection is using the package:

	[source,bash]
	----
	$ bin/solr package list-deployed -c <collection>
	----

	=== Updating Packages

	In order to update a package, first step is make sure the updated version is available in the added repositories by running `list-available` command shown above in <<Listing and Installing Packages>>.

	Next, install the new version of the package from the repositories.

	[source,bash]
	----
	$ bin/solr package install <package-name>:<version>
	----

	Once you have installed the new version, you can selectively update each of your collections or the cluster level plugins. Assuming the old version is `1.0.0` of the package `mypackage`, and the new version is `2.0.0`, the command would be as follows:

	[source,bash]
	----
	$ bin/solr package deploy mypackage:2.0.0 --update -collections mycollection
	----
	or
	[source,bash]
	----
	$ bin/solr package deploy mypackage:2.0.0 --update -cluster
	----

	You can run the `list-deployed` command to verify that this collection is using the newly added version.

	=== Undeploy a Package

	If a package supports undeploying the plugins it contains (check package author's documentation), then a previously deployed package can be undeployed as follows:

	[source,bash]
	----
	$ bin/solr package undeploy <package-name> -collections <collection1>[,<collection2>,...]
	----

	=== Uninstall a Package

	If a package has been undeployed or was never deployed, then it can be uninstalled as follows:

	[source,bash]
	----
	$ bin/solr package uninstall <package-name>:<package-version>
	----

	or

	[source,bash]
	----
	$ bin/solr package deploy <package-name> -cluster
	----

	== Security

	The `add-repo` step should only be executed using HTTPS enabled repository urls only so as to prevent against MITM attacks when Solr is fetching the public key for the repository. This `add-repo` step registers the public key of the trusted repository, and hence can only be executed using the package manager (CLI) having direct write access to the trusted store of the package store (a special location in the package store that cannot be written to using the package store APIs). Also, it is critical to protect ZooKeeper from unauthorized write access.

	Also, keep in mind, that it is possible to install any package from a repository once it has been added. If you want to use some packages in production, a best practice is to setup your own repository and add that to Solr instead of adding a generic third-party repository that is beyond your administrative control. You might want to re-sign packages from a third-party repository using your own private keys and host them at your own repository.