title: “Mastering Component Compatible Dependency” categories:
This article mainly includes three parts.at first,I will introduce compatibility principle(more details see here) briefly.followed by a detailed elaborating about Java component compatible dependency,including the interface-oriented programming,single component signature protection,single component compatibility protection and multi-component compatibility compile time checking.Finally is the review and prospect,especially about Dependency Mediator project.
{% include toc %}
what is compatibility？it is also the key to understanding the compatible dependency.Compatibility, often catch our eyes through two forms,binary compatibility,that means they can run the same executable code, typically machine code for a general purpose computer CPU.within the framework of Release-to-Release Binary Compatibility in SOM (Forman, Conner, Danforth, and Raper, Proceedings of OOPSLA '95), Java programming language binaries are binary compatible under all relevant transformations that the authors identify (with some caveats with respect to the addition of instance variables). here is a list of some important binary compatible changes that the Java programming language supports:
Generally,java language is upwards binary-compatible with previous version,such as Java SE 6 is upwards binary-compatible with J2SE 5.0 except for some minor incompatibilities.You can get details from here.
Another compatibility form,we call it source compatibility,meaning that recompilation is necessary.Java language does not support downward source compatibility. Such as if source files use new language features or Java SE platform APIs, they will not be usable with an earlier version of the Java platform. Usually it will throw similar error like this:
java.lang.UnsupportedClassVersionError: com.alibaba.mq.core.MessageConsumer : Unsupported major.minor version 51.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(Unknown Source)
Java language source compatibility policy is as follows,except for any incompatibilities listed further below:
Let's sum up before going on:
This article don‘t talk about dependency analysis and dependency decoupling,if you care these topics,please read Dependency Analysis and the Modularisation of Java Programs.highly recommending you try to dive into open source project Architecture Explorer,through bcel technology,analyzing and measuring component’s dependency relationship.but in my opinion,Jdepend was surely the most distinguished tool in dependency analysis and dependency decoupling area.it traverses Java class file directories and generates design quality metrics for each Java package,also allows you to automatically measure the quality of a design in terms of its extensibility, reusability, and maintainability to manage package dependencies effectively.
Let's return,ask yourself，have you ever been disturbed by the following problems?
It is the time to introduce the key idea - Mediating Components Compatible Dependency.Specifically,it includs the following basic principles:
Interface-oriented programming,It's easier said than done.I will cite SLF4J design philosophy explain this principle.
Differing from commons logging,SLF4J use static binder technology so as to avoid notorious classloader problem,every concrete log implementation must have a class named StaticLoggerBinder and implementing SLF4J spi LoggerFactoryBinder.thus you just depend SLF4J api,not concerned about underlying implementation.some people may asking,why choose SLF4J not commons logging，I can also depend commons logging api but not concret implementation？it‘s not performance but classloader problem.JCL discovery process relies on classloader hacks to find the logging framework at runtime but this mechanism leads to numerous problems including unexpected behavior,hard to debug classloading problems resulting in increased complexity.This is nicely captured by Ceki (the author of Log4J, SLF4J and Logback) in Think again before adopting the commons-logging API (which also mentions memory leaks problems observed with JCL).and this is why SLF4J, which uses static bindings, has been created. Let’s see.how to find the concrete realizations in SLF4J:
// We need to use the name of the StaticLoggerBinder class, but we can't reference // the class itself. private static String STATIC_LOGGER_BINDER_PATH = "org/slf4j/impl/StaticLoggerBinder.class"; /** * It is LoggerFactory's responsibility to track version changes and manage * the compatibility list. * It is assumed that all versions in the 1.6 are mutually compatible. */ static private final String[] API_COMPATIBILITY_LIST = new String[]{"1.6", "1.7"}; // private constructor prevents instantiation private LoggerFactory() {} private final static void versionSanityCheck() { try { String requested = StaticLoggerBinder.REQUESTED_API_VERSION; boolean match = false; for (int i = 0; i < API_COMPATIBILITY_LIST.length; i++) { if (requested.startsWith(API_COMPATIBILITY_LIST[i])) { match = true; } } if (!match) { Util.report("The requested version " + requested + " by your slf4j binding is not compatible with " + Arrays.asList(API_COMPATIBILITY_LIST).toString()); Util.report("See " + VERSION_MISMATCH + " for further details."); } } catch (java.lang.NoSuchFieldError nsfe) { // given our large user base and SLF4J's commitment to backward // compatibility, we cannot cry here. Only for implementations // which willingly declare a REQUESTED_API_VERSION field do we // emit compatibility warnings. } catch (Throwable e) { // we should never reach here Util.report("Unexpected problem occured during version sanity check", e); } }
you can use these skills at your library.just Let others depend on your interface package,detecting certain implementation statically.
If you have developed an old version library,naming it as 2.x.x.next version,you hope to refactor some implementations,how to process multi-version compatibility?here is a skill from maven dependency plugin:
String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2" : "maven3"; /** * Check the current Maven version to see if it's Maven 2.x. */ protected static boolean isMaven2x() { return !canFindCoreClass( "org.apache.maven.project.DependencyResolutionRequest" ); // Maven 3 specific } /** * Check the current Maven version to see if it's Maven 3.1. */ protected static boolean isMaven31() { return canFindCoreClass( "org.eclipse.aether.artifact.Artifact" ); // Maven 3.1 specific }
Secondly,Single component signature protection,you can using Animal Sniffer project as well-known Guava library,Animal Sniffer provides tools to assist verifying that classes compiled with a newer JDK/API are compatible with an older JDK/API.
<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>animal-sniffer-maven-plugin</artifactId> <configuration> <signature> <groupId>org.codehaus.mojo.signature</groupId> <artifactId>java16-sun</artifactId> <version>1.0</version> </signature> </configuration> <executions> <execution> <id>check-java16-sun</id> <phase>test</phase> <goals> <goal>check</goal> </goals> </execution> </executions> </plugin>
So from now on, when you performing a mvn test or mvn install operation. If there is an incompatibility in one of the projects, you would find some output like this:
[INFO] [animal-sniffer:check {execution: check-java-api}] [INFO] Checking unresolved references to org.codehaus.mojo.signature:java16-sun:1.0 [ERROR] Undefined reference: java/lang/String.contains(Ljava/lang/CharSequence;)Z in /home/admin/workspace/kafka/target/classes/org/kafka/tools...Producer.class
More details please see here.
If your a SOA develper,you almost concerned with single component compatibility protection.[Clirr project] (http://clirr.sourceforge.net/) may be your best choice.Clirr is a tool that checks Java libraries for binary and source compatibility with older releases. Basically you give it two sets of jar files and Clirr dumps out a list of changes in the public api. The Clirr Ant task can be configured to break the build if it detects incompatible api changes. In a continuous integration process Clirr can automatically prevent accidental introduction of binary or source compatibility problems.
How to use it？just add some config snippets in your pom :
<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>clirr-maven-plugin</artifactId> <version>2.6.1</version> </plugin>
then,run mvn clirr:check,output may be like this:
[INFO] --- clirr-maven-plugin:2.6.1:check (default-cli) @ cmq.common --- [INFO] artifact com.alibaba.intl.base.cmq:cmq.common: checking for updates from b2bmirror-all [INFO] Comparing to version: 2.0.1 [ERROR] 7005: com.alibaba.mq.client.CompletionListener: Parameter 1 of 'public void onCompletion(com.alibaba.mq.model.CommonMessage)' has changed its type to com.alibaba.mq.common.CommonMessage [ERROR] 7005: com.alibaba.mq.client.CompletionListener: Parameter 1 of 'public void onException(com.alibaba.mq.model.CommonMessage, java.lang.Exception)' has changed its type to com.alibaba.mq.common.CommonMessage [ERROR] 7005: com.alibaba.mq.client.MessageProducer: Parameter 1 of 'public com.alibaba.mq.client.CommonResult send(com.alibaba.mq.model.CommonMessage)' has changed its type to com.alibaba.mq.common.CommonMessage [ERROR] 7005: com.alibaba.mq.client.MessageProducer: Parameter 1 of 'public com.alibaba.mq.client.CommonResult send(com.alibaba.mq.model.CommonMessage, com.alibaba.mq.client.CompletionListener)' has changed its type to com.alibaba.mq.common.CommonMessage
Some inspiration in Dependency Mediator project just come from here.more topics about compatible,you can reference from series article Evolving Java-based APIs
Last but not least,Multi-component compatibility compile time checking.it's the time to formally recommend project Dependency Mediator.
unlike karaf and other lightness modular technology(like osgi,class names do not need to be unique,but the combination of class names and their defining ClassLoader must to be unique),Dependency Mediator try to remedy component dependency conflicting problem before the runtime rather than using customized classLoader to agree with the coexistence of different version components,thus avoid some well-konwn errors,such as NoSuchMethodError,NoSuchFieldError and NoClassDefFoundError etc.
What is the principle of Dependency Mediator？Dependency Mediator try to mediate various component conflicting problems.It uses jar or class as the smallest component unit,that is ComponentEntry:
public class ComponentEntry implements Comparable<ComponentEntry> { /** * Similar file name */ private String pathName; /** * Component name ,such as fully-qualified class name or jar name */ private String name; /** * Name of the jar which contains this entry,this field may be null */ private String jarName; /** * Jar entry meta info,if it is a jar file,this field may be null */ private JarEntry entry; ....
It could scan directory(also including classpath,if you set system property scanClasspath) and POM.if it happen to jar,but you setting checkJars to false,it could detect inner MANIFEST file,using Build-Jdk and Built-By properties to decide whether duplicated jars.thus,output only report whether including duplicated jar.but if you setting checkJars to true or nothing to do with this property,it will further analyzes whether existing incompatible cases.
It also provides a maven plugin,if your project is a web project,you must run mvn insall before next step. Maven plugin also comply with the latest annotation mojo api rather than version 2.x.x doclet api.
Dependency Mediator take advantage of nearest definition dependency mediation strategy. this determines what version of a dependency will be used when multiple versions of an artifact are encountered. nearest definition,means that it will use the version of the closest dependency to your project in the tree of dependencies. You can always guarantee a version by declaring it explicitly in your project‘s POM. Note that if two dependency versions are at the same depth in the dependency tree, until Maven 2.0.8 it was not defined which one would win, but since Maven 2.0.9 it’s the order in the declaration that counts: the first declaration wins. nearest definition means that the version used will be the closest one to your project in the tree of dependencies, eg. if dependencies for A, B, and C are defined as A -> B -> C -> D 2.0 and A -> E -> D 1.0, then D 1.0 will be used when building A because the path from A to D through E is shorter. You could explicitly add a dependency to D 2.0 in A to force the use of D 2.0.
If you consider the dependency hierarchy as a binary tree structure,Undoubtedly,nearest definition means the minimum depth. so if you put some higher version dependency library after lower version one(may be transitive dependency),you will face incompatible conflicting problems.Such as :
[WARNING] Founded conflicting dependency component:org.apache.thrift:libthrift:jar Resolved version is org.apache.thrift:libthrift:jar:0.8.0:compile But found conflicting artifact org.apache.thrift:libthrift:0.9.1
In this case,you could fix this problem through place libthrift 0.9.1 before the library imported through transitive dependency.
of course,all the establishment must meet a premise,that is:
In Dependency Mediator 2.0 version,you could customize version differentiation strategy,not limited the aformentioned.by the way, also in 2.0,I will improvement compatible dependency mediator, please wait and see.
Through this article, you almost calculated the evolution process of the Dependency Mediator project.in my opinion,integration with the maven enforcer plugin may be a better choice.After all,I hope Dependency Mediator can keep sustainable development.I would try and donate this project to codehaus in the near future.
If you have better idea or improving suggestion, please contact dev@rocketmq.apache.org.