maven-core/plugin-manager.txt - maven - Git at Google

 h3. Plugins

 * Execution model
 * Metadata model
 * Metadata tooling
  ** Metadata extractor
  ** Metadata reader
  ** Metadata writer
  ** Metadata adapter (if required for the target system)
 * Maven packaging and lifecycle
 * Maven test harness for plugin execution model
 * Eclipse IDE tooling for plugin execution model and metadata model

 - we also seem to have information like the plugin lifecycle model that's burried inside the maven execution model
 - we also seem to have artifact information tangled inside the plugin model
 - we have to deal with scripting implementations (groovy, beanshell, ruby)
 - we need to deal with a shared context for plugins, like the guarded mojos
 - we need to deal with plugins as core application logic which can also interact with plugins

 * Along with this comes the testing strategies that make this work
 * The repository model where plugins can be stored and cataloged
 * Bridging this into an OSGi system: if this could be done then we can basically take over p2

 TODO

 - dealing with processing of input and adapting GAVs before searching i.e. plugin groups in Maven and default searching
 - how to deal with resolution from different places like local/remote/workspace
 - how to incorporate the metadata processor to start with
 - create a hook to programmatically configure the test
 - create a plugin processor
 - directory
 - jar
 - hooks for loading

 - i could make an annotation that marked another class as its plugin discoverer, i could just add it to the
   the component descriptor and when it's loaded we can add the discoverer, that would be simpler. i could
   also do this with the lister as well
 i'l
 // 1 the metadata -> model plugin/mojo descriptor
 // 2 tools for doing the mapping
 // 3 the component model -> interfaces for the plugin

 // The plugin manager should load up a directory structure of plugins.
 // - a plugin per directory where the plugin is present with its deps
 // - a plugin that has its dependencies packaged up with it
 // - implement filters and create a test where a plugin fails if the right classes are not filtered
 // - plugins with a dependency the same as the core but different versions, make it fail then fix it

   /*
  *
  * h2. Concerns for the plugin manager
  *
  * h3. resolving the dependencies of a plugin - these could be resolved remotely at runtime or, -
  * they could be resolved from a local repository - i think we need a simple dependency model here
  * that is more mercury related and not Maven related i.e no POMs - workspace resolver - we need
  * tools to pre-populate this repository
  *
  * h3. create an isolated classloader
  *
  * h3. lookup the plugin with a configuration
  *
  * h3. execute the plugin
  *
  * h3. plugins may have to deal with particular actions when a plugin is - installed - loaded -
  * unloaded - update - uninstalled
  *
  * h3. plugins should be able to have specific metadata for a plugin model and that be translated -
  * dependencies - resources - configuration - extension points of plugins
  *
  * For a particular application plugin there will be a declarative descriptor for that plugin type.
  *
  * - nexus - the plugin class - UI to contribute - what JS to hook into the UI - what resources to
  * load into the UI - having packed or unpacked plugins, and positioning resources if necessary -
  * maven can work out of the classloader, nexus plugins probably couldn't given the js and image
  * resources
  *
  * We need to look at Maven, and Nexus as use cases and figure out what each of them needs to be
  * able to do
  *
  * - now what is really the difference between this and loading a component in plexus - custom
  * classloading capability - remote resolution of dependencies - do we want a model for sharing
  * information among plugins, is this more like an extension point - do we need a sort of bus for
  * application data - do we need a dictionary for our applications like Apple does. We could easily
  * hook into this and this is the model we need to follow - how many of our REST services do not map
  * directory to a method in the application interface?
  *
  * - research extension points versus plugins
  *
  * from igor: two plugins A and B, both depend on the same library but use different versions, say
  * lib 1.0 and lib 2.0 when debugger hits a breakpoint inside a class from the library, IDE needs to
  * know which version of library the class comes from
  */

 This document outlines the concerns of a general plugin manager that would be used in conjunction with any Plexus-based application. The following outlined concerns are an attempt to describe what a plugin manager would need to do for Maven and for Nexus.

 h2. Concerns for the plugin manager

 h3. Resolving the dependencies of a plugin

 A plugin manager would need to know how to find the dependencies of a plugin whether that be from a workspace of an IDE, a local Maven repository, or a remote Maven repository. During development in an IDE we would need layered resolution approach that would allow resolution from the workspace, then the local Maven repository and then from any number of remote Maven repositories. During development from the command line we would need a layered resolution approach that would allow resolution from the local Maven repository and then from any number of remote Maven repositories. In production the resolution for the dependencies could also be layered but would depend on the preferred model for the given application. In the case of Maven no plugin dependencies are shipped with a Maven plugin, but in the case of Nexus we would probably want to include the dependencies for self-containment.

 h4. Working with POMs during development

 During the development of a plugin we would want to make it easy for a developer to create a plugin which means we would allow the direct use of a POM to state the dependencies of a given plugin and any tooling we created would be responsible for turning the dependency information in the POM into plugin metadata which described the dependencies of the plugin.

 We are looking at using Mercury for the resolution and retrieval of the plugin dependencies so during development we would translate the POMs into dependency information that Mercury can understand, and in production we would have the pre-digested format that Mercury could utilize to resolve and retrieve dependencies. We need to be careful that we are using Mercury during development and production so that we don’t get in the situation where Maven 2.x is resolving different then Mercury is. We can still leverage POMs during development but we need to make sure Mercury is doing the work.

 With respect to development inside an IDE the workspace from which we would like to resolve and retrieve dependencies is a read-only local repository to Mercury which is put at the front-end of the search order of any repository that Mercury will consult for artifacts.

 h4. Generation of plugin metadata and packaging

 To create the packaging for a plugin a special Maven packaging would be created with an accompanying life-cycle in order to create the plugin metadata required and create an archive of the plugin that can be consumed at runtime.

 h3. Creation of an isolated classloader

 The plugin artifact and its dependency artifacts will be loaded in an isolated ClassRealm which is simply an extension of the standard URLClassLoader. Mercury would be responsible for resolving and retrieving the dependencies which would then be used to populate a ClassRealm where there would be a connection to the host applications ClassRealm as the parent but the search order would be child first, and the parent ClassRealm can optionally choose to limit the searching to particular classes. The application ClassRealm, or parent ClassRealm, should only expose the API in the form of and application interface and model classes that may be required by plugins.

 For debugging purposes we also need to keep track of what versions of artifacts are being used so that when we are debugging in the IDE we can find the specific sources for a given version of a library so that the developer can debug the correct version of a library being used for a plugin.

 h3. Discovery all Plexus components

 Once the ClassRealm is populated with the plugin artifact and its dependency artifacts a discovery operation must be performed on the plugin ClassRealm in order to discover any Plexus components that may be present in any of the artifacts required for the plugins execution.

 h3. Looking up the plugin with a configuration

 Once the ClassRealm has been populated and all Plexus components have been discovered a lookup can be performed to retrieve the primary plugin component. Currently the component would need to be looked up and then the BasicComponentConfigurator would need to configure the component with a given Plexus configuration. We will discuss later how we would retrieve the Plexus configuration required for a given plugin.

 When a plugin has a complete and valid configuration, the plugin would be looked up in the Plexus container and be ready to perform any operations requested of it.

 h3. Execution of the plugin

 I still need to do some work here to figure out how a dispatching to a particular plugin would occur because everything needs to be mediated through the host application. For a given action that is possible for a user to perform in the UI we need to know the URI that is to be used with its given parameters. That URI in turn must map internally to a method in a given component which belongs to a plugin. This mapping should actually be more generalized and it shouldn’t matter internally whether this maps to the core application or extension of the application in the form of plugins.

 In the short term we are working with the model where the URI maps to a resource, and the resource itself is a Plexus component which can itself have a reference to the host application in order to perform its work. This may not occur in the first versions of the plugin API but a resource needs to simply become the mapping mechanism by which parameters are taken from the REST side of the application and mapped into a method call within a given Plexus component. Whether that be the core Nexus application or a component provided by a plugin.

 At the very least in the short term the resource needs to know how to lookup the component that is required to perform the work. This needs to be made simple, for the time being we can write and test plugins working in the same realm until we get complete isolation working.

 h3. Plugins need a specific metadata model

 Each application will have its own model for plugins. Much of this model will be common across applications but there will always be issues specific to a particular applications’s plugins. Some of the issues that need to be taken into consideration:

 * Plugin dependencies
 * Plugin resources like JavaScript files or images
 * Configuration metadata and default configuration values
 * Extension points that plugins might expose

 h3. The plugin manager may have to deal with particular actions when a plugin is

 * installation
 * activation
 * update
 * deactivation
 * un-installation

 When a plugin is installed we may want to check the license and make users agree. We may also want to unpack the given plugin in a particular location.

 We might want to separate between installation and activation, it might be nice to allow a user to activate/deactivate a plugin instead of having to uninstall and reinstall a plugin in particular cases. This would prevent having to reconfigure the plugin again. For example it might be nice to turn off LDAP authentication without having to uninstall the plugin.

 These particular phases in a plugin’s lifecycle can probably be generalized to all plugins so could be something implemented in all plugins. Here we could have the descriptor either point to methods for each of these phases to execute or we could create a lifecycle in Plexus for these phases.

 For a particular application plugin there will be a declarative descriptor for that plugin type.

 h3. Some general notes on Nexus

 The plugin metadata:
 * the plugin class
 * UI to contribute
 * what JS to hook into the UI
 * what resources to load into the UI
 * having packed or unpacked plugins, and positioning resources if necessary
 * maven can work out of the classloader, nexus plugins probably couldn't given the js and image resources

 h3. Other issues

 We need to look at Maven, and Nexus as use cases and figure out what each of them needs to be able to do

 * now what is really the difference between this and loading a component in plexus
 * do we want a model for sharing information among plugins, is this more like an extension point
 * do we need a sort of bus for application data
 * do we need a dictionary for our applications like Apple does. We could easily hook into this and this is the model we need to follow. Following the model of apple applications the dictionary is what the REST URI should attach to. Not creating custom logic in the resource code.
 * how many of our REST services do not map directory to a method in the application interface?

 - an example of adding js
 - an example of adding a URI
	h3. Plugins

	* Execution model
	* Metadata model
	* Metadata tooling
	** Metadata extractor
	** Metadata reader
	** Metadata writer
	** Metadata adapter (if required for the target system)
	* Maven packaging and lifecycle
	* Maven test harness for plugin execution model
	* Eclipse IDE tooling for plugin execution model and metadata model

	- we also seem to have information like the plugin lifecycle model that's burried inside the maven execution model
	- we also seem to have artifact information tangled inside the plugin model
	- we have to deal with scripting implementations (groovy, beanshell, ruby)
	- we need to deal with a shared context for plugins, like the guarded mojos
	- we need to deal with plugins as core application logic which can also interact with plugins

	* Along with this comes the testing strategies that make this work
	* The repository model where plugins can be stored and cataloged
	* Bridging this into an OSGi system: if this could be done then we can basically take over p2

	TODO

	- dealing with processing of input and adapting GAVs before searching i.e. plugin groups in Maven and default searching
	- how to deal with resolution from different places like local/remote/workspace
	- how to incorporate the metadata processor to start with
	- create a hook to programmatically configure the test
	- create a plugin processor
	- directory
	- jar
	- hooks for loading

	- i could make an annotation that marked another class as its plugin discoverer, i could just add it to the
	the component descriptor and when it's loaded we can add the discoverer, that would be simpler. i could
	also do this with the lister as well
	i'l
	// 1 the metadata -> model plugin/mojo descriptor
	// 2 tools for doing the mapping
	// 3 the component model -> interfaces for the plugin

	// The plugin manager should load up a directory structure of plugins.
	// - a plugin per directory where the plugin is present with its deps
	// - a plugin that has its dependencies packaged up with it
	// - implement filters and create a test where a plugin fails if the right classes are not filtered
	// - plugins with a dependency the same as the core but different versions, make it fail then fix it

	/*
	*
	* h2. Concerns for the plugin manager
	*
	* h3. resolving the dependencies of a plugin - these could be resolved remotely at runtime or, -
	* they could be resolved from a local repository - i think we need a simple dependency model here
	* that is more mercury related and not Maven related i.e no POMs - workspace resolver - we need
	* tools to pre-populate this repository
	*
	* h3. create an isolated classloader
	*
	* h3. lookup the plugin with a configuration
	*
	* h3. execute the plugin
	*
	* h3. plugins may have to deal with particular actions when a plugin is - installed - loaded -
	* unloaded - update - uninstalled
	*
	* h3. plugins should be able to have specific metadata for a plugin model and that be translated -
	* dependencies - resources - configuration - extension points of plugins
	*
	* For a particular application plugin there will be a declarative descriptor for that plugin type.
	*
	* - nexus - the plugin class - UI to contribute - what JS to hook into the UI - what resources to
	* load into the UI - having packed or unpacked plugins, and positioning resources if necessary -
	* maven can work out of the classloader, nexus plugins probably couldn't given the js and image
	* resources
	*
	* We need to look at Maven, and Nexus as use cases and figure out what each of them needs to be
	* able to do
	*
	* - now what is really the difference between this and loading a component in plexus - custom
	* classloading capability - remote resolution of dependencies - do we want a model for sharing
	* information among plugins, is this more like an extension point - do we need a sort of bus for
	* application data - do we need a dictionary for our applications like Apple does. We could easily
	* hook into this and this is the model we need to follow - how many of our REST services do not map
	* directory to a method in the application interface?
	*
	* - research extension points versus plugins
	*
	* from igor: two plugins A and B, both depend on the same library but use different versions, say
	* lib 1.0 and lib 2.0 when debugger hits a breakpoint inside a class from the library, IDE needs to
	* know which version of library the class comes from
	*/

	This document outlines the concerns of a general plugin manager that would be used in conjunction with any Plexus-based application. The following outlined concerns are an attempt to describe what a plugin manager would need to do for Maven and for Nexus.

	h2. Concerns for the plugin manager

	h3. Resolving the dependencies of a plugin

	A plugin manager would need to know how to find the dependencies of a plugin whether that be from a workspace of an IDE, a local Maven repository, or a remote Maven repository. During development in an IDE we would need layered resolution approach that would allow resolution from the workspace, then the local Maven repository and then from any number of remote Maven repositories. During development from the command line we would need a layered resolution approach that would allow resolution from the local Maven repository and then from any number of remote Maven repositories. In production the resolution for the dependencies could also be layered but would depend on the preferred model for the given application. In the case of Maven no plugin dependencies are shipped with a Maven plugin, but in the case of Nexus we would probably want to include the dependencies for self-containment.

	h4. Working with POMs during development

	During the development of a plugin we would want to make it easy for a developer to create a plugin which means we would allow the direct use of a POM to state the dependencies of a given plugin and any tooling we created would be responsible for turning the dependency information in the POM into plugin metadata which described the dependencies of the plugin.

	We are looking at using Mercury for the resolution and retrieval of the plugin dependencies so during development we would translate the POMs into dependency information that Mercury can understand, and in production we would have the pre-digested format that Mercury could utilize to resolve and retrieve dependencies. We need to be careful that we are using Mercury during development and production so that we don’t get in the situation where Maven 2.x is resolving different then Mercury is. We can still leverage POMs during development but we need to make sure Mercury is doing the work.

	With respect to development inside an IDE the workspace from which we would like to resolve and retrieve dependencies is a read-only local repository to Mercury which is put at the front-end of the search order of any repository that Mercury will consult for artifacts.

	h4. Generation of plugin metadata and packaging

	To create the packaging for a plugin a special Maven packaging would be created with an accompanying life-cycle in order to create the plugin metadata required and create an archive of the plugin that can be consumed at runtime.

	h3. Creation of an isolated classloader

	The plugin artifact and its dependency artifacts will be loaded in an isolated ClassRealm which is simply an extension of the standard URLClassLoader. Mercury would be responsible for resolving and retrieving the dependencies which would then be used to populate a ClassRealm where there would be a connection to the host applications ClassRealm as the parent but the search order would be child first, and the parent ClassRealm can optionally choose to limit the searching to particular classes. The application ClassRealm, or parent ClassRealm, should only expose the API in the form of and application interface and model classes that may be required by plugins.

	For debugging purposes we also need to keep track of what versions of artifacts are being used so that when we are debugging in the IDE we can find the specific sources for a given version of a library so that the developer can debug the correct version of a library being used for a plugin.

	h3. Discovery all Plexus components

	Once the ClassRealm is populated with the plugin artifact and its dependency artifacts a discovery operation must be performed on the plugin ClassRealm in order to discover any Plexus components that may be present in any of the artifacts required for the plugins execution.

	h3. Looking up the plugin with a configuration

	Once the ClassRealm has been populated and all Plexus components have been discovered a lookup can be performed to retrieve the primary plugin component. Currently the component would need to be looked up and then the BasicComponentConfigurator would need to configure the component with a given Plexus configuration. We will discuss later how we would retrieve the Plexus configuration required for a given plugin.

	When a plugin has a complete and valid configuration, the plugin would be looked up in the Plexus container and be ready to perform any operations requested of it.

	h3. Execution of the plugin

	I still need to do some work here to figure out how a dispatching to a particular plugin would occur because everything needs to be mediated through the host application. For a given action that is possible for a user to perform in the UI we need to know the URI that is to be used with its given parameters. That URI in turn must map internally to a method in a given component which belongs to a plugin. This mapping should actually be more generalized and it shouldn’t matter internally whether this maps to the core application or extension of the application in the form of plugins.

	In the short term we are working with the model where the URI maps to a resource, and the resource itself is a Plexus component which can itself have a reference to the host application in order to perform its work. This may not occur in the first versions of the plugin API but a resource needs to simply become the mapping mechanism by which parameters are taken from the REST side of the application and mapped into a method call within a given Plexus component. Whether that be the core Nexus application or a component provided by a plugin.

	At the very least in the short term the resource needs to know how to lookup the component that is required to perform the work. This needs to be made simple, for the time being we can write and test plugins working in the same realm until we get complete isolation working.

	h3. Plugins need a specific metadata model

	Each application will have its own model for plugins. Much of this model will be common across applications but there will always be issues specific to a particular applications’s plugins. Some of the issues that need to be taken into consideration:

	* Plugin dependencies
	* Plugin resources like JavaScript files or images
	* Configuration metadata and default configuration values
	* Extension points that plugins might expose

	h3. The plugin manager may have to deal with particular actions when a plugin is

	* installation
	* activation
	* update
	* deactivation
	* un-installation

	When a plugin is installed we may want to check the license and make users agree. We may also want to unpack the given plugin in a particular location.

	We might want to separate between installation and activation, it might be nice to allow a user to activate/deactivate a plugin instead of having to uninstall and reinstall a plugin in particular cases. This would prevent having to reconfigure the plugin again. For example it might be nice to turn off LDAP authentication without having to uninstall the plugin.

	These particular phases in a plugin’s lifecycle can probably be generalized to all plugins so could be something implemented in all plugins. Here we could have the descriptor either point to methods for each of these phases to execute or we could create a lifecycle in Plexus for these phases.

	For a particular application plugin there will be a declarative descriptor for that plugin type.

	h3. Some general notes on Nexus

	The plugin metadata:
	* the plugin class
	* UI to contribute
	* what JS to hook into the UI
	* what resources to load into the UI
	* having packed or unpacked plugins, and positioning resources if necessary
	* maven can work out of the classloader, nexus plugins probably couldn't given the js and image resources

	h3. Other issues

	We need to look at Maven, and Nexus as use cases and figure out what each of them needs to be able to do

	* now what is really the difference between this and loading a component in plexus
	* do we want a model for sharing information among plugins, is this more like an extension point
	* do we need a sort of bus for application data
	* do we need a dictionary for our applications like Apple does. We could easily hook into this and this is the model we need to follow. Following the model of apple applications the dictionary is what the REST URI should attach to. Not creating custom logic in the resource code.
	* how many of our REST services do not map directory to a method in the application interface?

	- an example of adding js
	- an example of adding a URI