TIKA-4955: make OSGi tika-bundle-standard match the classpath (#3301)
diff --git a/CHANGES.txt b/CHANGES.txt
index 4d40095..8864254 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,17 @@
Release 4.2.0 - unreleased
+ * tika-bundle-standard no longer embeds dependencies that are OSGi bundles
+ themselves (commons-*, pdfbox, fontbox, bouncycastle, jsoup, asm, xz, xmpcore,
+ dd-plist); deploy them alongside it (TIKA-4934). Its imports of those packages,
+ plus log4j-api and jackson, are now mandatory, so a missing bundle fails
+ resolution instead of silently dropping parsers; tika-ml-core and jwarc are now
+ embedded, along with tika-ml-junkdetect. The bundle now registers its
+ EncodingDetector, LanguageDetector and MetadataFilter providers, so tika-core's
+ defaults match the classpath. Restores OOXMLParser, HttpParser, GeoGebraParser and
+ charset detection in OSGi, which had fallen back to BOM and declared charsets
+ only. The bundle now declares its real Java requirement (JavaSE 17, not 1.8)
+ and drops 67 imports of packages it never referenced (TIKA-4955).
+
* PipesParser.start() and PipesForkParser.start() bring the forks up before the
first parse and fail fast if one can't start (TIKA-4953).
diff --git a/docs/modules/ROOT/examples/OsgiParseExample.java b/docs/modules/ROOT/examples/OsgiParseExample.java
new file mode 120000
index 0000000..b14f064
--- /dev/null
+++ b/docs/modules/ROOT/examples/OsgiParseExample.java
@@ -0,0 +1 @@
+../../../../tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/example/OsgiParseExample.java
\ No newline at end of file
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index 10d3a30..973fda5 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -25,6 +25,7 @@
*** xref:using-tika/server/monitoring.adoc[Monitoring with Prometheus]
** xref:using-tika/grpc/index.adoc[gRPC]
** xref:using-tika/docker.adoc[Running Tika in Docker]
+** xref:using-tika/osgi.adoc[Using Tika in OSGi]
* xref:pipes/index.adoc[Pipes]
** xref:pipes/getting-started.adoc[Getting Started]
** xref:pipes/configuration.adoc[Pipeline Configuration]
diff --git a/docs/modules/ROOT/pages/using-tika/index.adoc b/docs/modules/ROOT/pages/using-tika/index.adoc
index bb68ec4..80e5149 100644
--- a/docs/modules/ROOT/pages/using-tika/index.adoc
+++ b/docs/modules/ROOT/pages/using-tika/index.adoc
@@ -52,6 +52,9 @@
To run the server or gRPC service from a published image rather than a distribution zip, see
xref:using-tika/docker.adoc[Running Tika in Docker].
+To run Tika inside an OSGi container such as Apache Felix or Eclipse Equinox, see
+xref:using-tika/osgi.adoc[Using Tika in OSGi].
+
== Resources
* https://tika.apache.org/download.html[Downloads] — jars, the `tika-app` and `tika-server` zips
diff --git a/docs/modules/ROOT/pages/using-tika/osgi.adoc b/docs/modules/ROOT/pages/using-tika/osgi.adoc
new file mode 100644
index 0000000..bf8d35b
--- /dev/null
+++ b/docs/modules/ROOT/pages/using-tika/osgi.adoc
@@ -0,0 +1,97 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+= Using Tika in OSGi
+:toc:
+
+Tika ships two OSGi bundles:
+
+* `tika-core`: the API, the MIME type registry and the default detectors.
+* `tika-bundle-standard`: the standard parsers, with the Tika modules and the libraries
+ that are not OSGi bundles themselves embedded inside it.
+
+Libraries that already ship as OSGi bundles are not embedded. Deploy them alongside the two
+Tika bundles so they can be shared with the rest of your container and updated on their own.
+
+== Bundles to deploy
+
+[cols="1,2"]
+|===
+|Group |Artifacts
+
+|Apache Tika
+|`org.apache.tika:tika-core`, `org.apache.tika:tika-bundle-standard`
+
+|Apache Commons
+|`commons-io:commons-io`, `commons-codec:commons-codec`, `commons-logging:commons-logging`,
+`org.apache.commons:commons-collections4`, `commons-compress`, `commons-csv`, `commons-exec`,
+`commons-lang3`, `commons-math3`, `commons-secure-xml`
+
+|Apache PDFBox
+|`org.apache.pdfbox:pdfbox`, `pdfbox-io`, `fontbox`
+
+|Bouncy Castle
+|`org.bouncycastle:bcprov-jdk18on`, `bcpkix-jdk18on`, `bcutil-jdk18on`, `bcjmail-jdk18on`
+
+|Jackson
+|`com.fasterxml.jackson.core:jackson-databind`, `jackson-core`, `jackson-annotations`
+
+|CommonMark
+|`org.commonmark:commonmark`, `commonmark-ext-gfm-tables`, `commonmark-ext-gfm-strikethrough`
+
+|Other
+|`org.apache.logging.log4j:log4j-api`, `org.jsoup:jsoup`, `org.ow2.asm:asm`,
+`org.tukaani:xz`, `com.adobe.xmp:xmpcore`, `com.googlecode.plist:dd-plist`
+|===
+
+Tika also logs through SLF4J 2.x, so the container needs `org.slf4j` packages and a logging
+backend.
+
+The authoritative list is the one Tika's own OSGi integration tests install:
+link:https://github.com/apache/tika/blob/main/tika-bundles/tika-bundle-standard/test-bundles.xml[`test-bundles.xml`].
+Use the versions that Tika release depends on (`mvn dependency:list` on
+`tika-bundle-standard` prints them); later minor versions of the same major also resolve.
+
+If one of these bundles is missing, `tika-bundle-standard` does not resolve, and the
+container's error names the missing package.
+
+== Services
+
+When it starts, `tika-bundle-standard` registers:
+
+* a `Parser` service (a `DefaultParser` over all the standard parsers)
+* a `Detector` service (a `DefaultDetector` over all the standard detectors)
+* its `EncodingDetector`, `LanguageDetector` and `MetadataFilter` providers, which `tika-core`
+ merges into its own defaults, so charset detection matches a classpath deployment
+
+No Service Loader Mediator (such as Apache Aries SPI Fly) is needed.
+
+To parse a document, including the documents embedded in it, build an `AutoDetectParser` from
+the two services:
+
+[source,java]
+----
+include::example$OsgiParseExample.java[tag=parse,indent=0]
+----
+
+== Tested containers
+
+Every build runs Tika's OSGi integration tests on Apache Felix and Eclipse Equinox. They
+check that both bundles resolve, that `tika-core` resolves on its own, and that detection and
+parsing through the registered services match a classpath deployment. If you run Tika in
+another container, such as Apache Karaf or an Eclipse p2 installation, reports on the
+https://issues.apache.org/jira/projects/TIKA[Tika issue tracker] are very welcome.
diff --git a/tika-bundles/tika-bundle-standard/pom.xml b/tika-bundles/tika-bundle-standard/pom.xml
index 64aad04..4af63ff 100644
--- a/tika-bundles/tika-bundle-standard/pom.xml
+++ b/tika-bundles/tika-bundle-standard/pom.xml
@@ -61,7 +61,7 @@
<version>${project.version}</version>
</dependency>
- <!-- Test: programmatic Felix container + JUnit 5 -->
+ <!-- Test: programmatic Felix and Equinox containers + JUnit 5 -->
<dependency>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.framework</artifactId>
@@ -69,6 +69,12 @@
<scope>test</scope>
</dependency>
<dependency>
+ <groupId>org.eclipse.platform</groupId>
+ <artifactId>org.eclipse.osgi</artifactId>
+ <version>3.24.300</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
<groupId>org.osgi</groupId>
<artifactId>org.osgi.core</artifactId>
<scope>provided</scope>
@@ -89,9 +95,6 @@
<extensions>true</extensions>
<configuration>
<instructions>
- <_runsystempackages>com.sun.xml.bind.marshaller, com.sun.xml.internal.bind.marshaller</_runsystempackages>
- <!-- The file below and the _include entry may be removed once Tika targets OpenJDK 9.0 or above -->
- <_include>src/main/resources/META-INF/MANIFEST.MF</_include>
<Bundle-Activator>
org.apache.tika.bundle.internal.BundleActivator
</Bundle-Activator>
@@ -100,6 +103,7 @@
tika-handler-boilerpipe|
tika-encoding-detector-*|
tika-langdetect-*|
+ tika-ml-*|
jakarta.activation|
junrar|
pdfbox-tools|
@@ -116,6 +120,7 @@
vorbis-java-tika|
metadata-extractor|
json-simple|
+ jwarc|
boilerpipe|
rome|
rome-utils|
@@ -159,137 +164,32 @@
org.slf4j,
org.slf4j.event,
org.slf4j.helpers,
- android.util;resolution:=optional,
- colorspace;resolution:=optional,
- com.apple.eawt;resolution:=optional,
- com.beust.jcommander;resolution:=optional,
- com.github.luben.zstd;resolution:=optional,
- com.github.javaparser;resolution:=optional,
- com.github.javaparser.ast;resolution:=optional,
- com.github.javaparser.ast.body;resolution:=optional,
- com.github.javaparser.ast.expr;resolution:=optional,
- com.github.javaparser.ast.nodeTypes;resolution:=optional,
- com.github.javaparser.ast.type;resolution:=optional,
- com.github.javaparser.resolution;resolution:=optional,
- com.github.javaparser.resolution.declarations;resolution:=optional,
- com.github.javaparser.resolution.types;resolution:=optional,
- com.github.javaparser.symbolsolver;resolution:=optional,
- com.github.javaparser.symbolsolver.resolution.typesolvers;resolution:=optional,
- com.github.javaparser.utils;resolution:=optional,
- com.google.common.base;resolution:=optional,
- com.google.common.math;resolution:=optional,
- com.google.common.util.concurrent.internal;resolution:=optional,
- com.google.errorprone.annotations;resolution:=optional,
- com.google.errorprone.annotations.concurrent;resolution:=optional,
- com.google.protobuf;resolution:=optional,
- com.ibm.icu.text;resolution:=optional,
- com.microsoft.schemas.office.powerpoint;resolution:=optional,
- com.microsoft.schemas.office.word;resolution:=optional,
- com.sleepycat.je;resolution:=optional,
- com.sun.javadoc;resolution:=optional,
- com.sun.jna;resolution:=optional,
- com.sun.jna.ptr;resolution:=optional,
- com.sun.org.apache.xalan.internal;resolution:=optional,
- com.sun.org.apache.xml.internal.resolver;resolution:=optional,
- com.sun.org.apache.xml.internal.resolver.tools;resolution:=optional,
- com.sun.org.apache.xpath.internal.jaxp;resolution:=optional,
- com.sun.xml.bind.marshaller;resolution:=optional,
- com.sun.xml.internal.bind.marshaller;resolution:=optional,
- com.sun.msv.datatype;resolution:=optional,
- com.sun.msv.datatype.xsd;resolution:=optional,
- com.sun.tools.javadoc;resolution:=optional,
- de.rototor.pdfbox.graphics2d;resolution:=optional,
- icc;resolution:=optional,
- jakarta.annotation;resolution:=optional,
- javax.mail;resolution:=optional,
- javax.mail.internet;resolution:=optional,
- javax.net.ssl;resolution:=optional,
- javax.servlet;resolution:=optional,
- javax.servlet.annotation;resolution:=optional,
- javax.servlet.http;resolution:=optional,
- javax.measure.converter;resolution:=optional,
- jakarta.ws.rs.core;resolution:=optional,
- javax.xml.bind;resolution:=optional,
- javax.xml.bind.annotation;resolution:=optional,
- javax.xml.bind.annotation.adapters;resolution:=optional,
- javax.xml.bind.attachment;resolution:=optional,
- javax.xml.bind.helpers;resolution:=optional,
- javax.xml.bind.util;resolution:=optional,
- javax.xml.xpath;resolution:=optional,
- jj2000.j2k.codestream;resolution:=optional,
- jj2000.j2k.codestream.reader;resolution:=optional,
- jj2000.j2k.decoder;resolution:=optional,
- jj2000.j2k.entropy.decoder;resolution:=optional,
- jj2000.j2k.fileformat.reader;resolution:=optional,
- jj2000.j2k.image;resolution:=optional,
- jj2000.j2k.image.invcomptransf;resolution:=optional,
- jj2000.j2k.image.output;resolution:=optional,
- jj2000.j2k.io;resolution:=optional,
- jj2000.j2k.quantization.dequantizer;resolution:=optional,
- jj2000.j2k.roi;resolution:=optional,
- jj2000.j2k.util;resolution:=optional,
- jj2000.j2k.wavelet.synthesis;resolution:=optional,
- net.sf.saxon;resolution:=optional,
- net.sf.saxon.dom;resolution:=optional,
- net.sf.saxon.lib;resolution:=optional,
- net.sf.saxon.ma.map;resolution:=optional,
- net.sf.saxon.om;resolution:=optional,
- net.sf.saxon.query;resolution:=optional,
- net.sf.saxon.sxpath;resolution:=optional,
- net.sf.saxon.trans;resolution:=optional,
- net.sf.saxon.tree.wrapper;resolution:=optional,
- net.sf.saxon.type;resolution:=optional,
- net.sf.saxon.value;resolution:=optional,
- org.apache.batik.anim.dom;resolution:=optional,
- org.apache.batik.bridge;resolution:=optional,
- org.apache.batik.dom;resolution:=optional,
- org.apache.batik.ext.awt;resolution:=optional,
- org.apache.batik.ext.awt.image.renderable;resolution:=optional,
- org.apache.batik.gvt;resolution:=optional,
- org.apache.batik.parser;resolution:=optional,
- org.apache.batik.svggen;resolution:=optional,
- org.apache.batik.util;resolution:=optional,
- org.apache.jcp.xml.dsig.internal.dom;resolution:=optional,
- org.apache.maven.model;resolution:=optional,
- org.apache.maven.plugin;resolution:=optional,
- org.apache.maven.plugin.logging;resolution:=optional,
- org.apache.maven.project;resolution:=optional,
- org.apache.tools.ant;resolution:=optional,
- org.apache.tools.ant.taskdefs;resolution:=optional,
- org.apache.tools.ant.types;resolution:=optional,
- org.apache.xml.resolver;resolution:=optional,
- org.apache.xml.resolver.readers;resolution:=optional,
- org.apache.xml.resolver.tools;resolution:=optional,
- org.apache.xml.security;resolution:=optional,
- org.apache.xml.security.c14n;resolution:=optional,
- org.apache.xml.security.signature;resolution:=optional,
- org.apache.xml.security.utils;resolution:=optional,
- org.apache.xmlbeans.impl.xpath.saxon;resolution:=optional,
- org.apache.xmlbeans.impl.xquery.saxon;resolution:=optional,
- org.brotli.dec;resolution:=optional,
- org.cyberneko.html.xercesbridge;resolution:=optional,
- org.etsi.uri.x01903.v14;resolution:=optional,
- org.itadaki.bzip2;resolution:=optional,
- org.jaxen;resolution:=optional,
- org.jaxen.dom4j;resolution:=optional,
- org.jaxen.pattern;resolution:=optional,
- org.jaxen.saxpath;resolution:=optional,
- org.jaxen.util;resolution:=optional,
- org.openxmlformats.schemas.officeDocument.x2006.math;resolution:=optional,
- org.openxmlformats.schemas.schemaLibrary.x2006.main;resolution:=optional,
- org.w3c.dom;resolution:=optional,
- org.w3c.dom.ranges;resolution:=optional,
- org.w3c.dom.svg;resolution:=optional,
- org.w3c.dom.traversal;resolution:=optional,
- org.xml.sax;resolution:=optional,
- org.xml.sax.ext;resolution:=optional,
- org.xml.sax.helpers;resolution:=optional,
- sun.io;resolution:=optional,
- sun.java2d.cmm.kcms;resolution:=optional,
- sun.misc;resolution:=optional,
- sun.nio.ch;resolution:=optional,
- sun.reflect.generics.reflectiveObjects;resolution:=optional,
- thredds.featurecollection;resolution:=optional,
+ <!-- not exported by any deployed bundle; carved out of the mandatory patterns below -->
+ com.adobe.internal.xmp.impl;resolution:=optional,
+ org.apache.pdfbox.debugger;resolution:=optional,
+ <!-- Deployed as separate bundles (see test-bundles.xml): mandatory, so a missing
+ bundle fails resolution rather than a parse -->
+ com.adobe.internal.xmp;com.adobe.internal.xmp.*,
+ com.dd.plist,
+ com.fasterxml.jackson.core;com.fasterxml.jackson.core.*,
+ com.fasterxml.jackson.databind,
+ org.apache.commons.codec;org.apache.commons.codec.*,
+ org.apache.commons.collections4;org.apache.commons.collections4.*,
+ org.apache.commons.compress;org.apache.commons.compress.*,
+ org.apache.commons.csv,
+ org.apache.commons.exec,
+ <!-- commons-io 2.x also exports a 1.4.9999 compatibility version, which bnd would pick -->
+ org.apache.commons.io;org.apache.commons.io.*;version="${range;[==,+);${commons.io.version}}",
+ org.apache.commons.lang3;org.apache.commons.lang3.*,
+ org.apache.commons.logging,
+ org.apache.commons.math3;org.apache.commons.math3.*,
+ org.apache.fontbox;org.apache.fontbox.*,
+ org.apache.logging.log4j;org.apache.logging.log4j.message;org.apache.logging.log4j.spi;org.apache.logging.log4j.util,
+ org.apache.pdfbox;org.apache.pdfbox.*,
+ org.bouncycastle;org.bouncycastle.*,
+ org.jsoup;org.jsoup.*,
+ org.objectweb.asm,
+ org.tukaani.xz,
*;resolution:=optional
</Import-Package>
</instructions>
@@ -325,12 +225,40 @@
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven.failsafe.version}</version>
+ <!-- Equinox and Felix both ship org.apache.felix.resolver, so one framework per run -->
<executions>
<execution>
+ <id>felix</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
+ <configuration>
+ <includes>
+ <include>**/FelixBundleIT.java</include>
+ </includes>
+ <classpathDependencyExcludes>
+ <classpathDependencyExclude>org.eclipse.platform:org.eclipse.osgi</classpathDependencyExclude>
+ </classpathDependencyExcludes>
+ <reportNameSuffix>felix</reportNameSuffix>
+ </configuration>
+ </execution>
+ <execution>
+ <id>equinox</id>
+ <goals>
+ <goal>integration-test</goal>
+ <goal>verify</goal>
+ </goals>
+ <configuration>
+ <includes>
+ <include>**/EquinoxBundleIT.java</include>
+ </includes>
+ <classpathDependencyExcludes>
+ <classpathDependencyExclude>org.apache.felix:org.apache.felix.framework</classpathDependencyExclude>
+ </classpathDependencyExcludes>
+ <reportNameSuffix>equinox</reportNameSuffix>
+ <summaryFile>${project.build.directory}/failsafe-reports/failsafe-summary-equinox.xml</summaryFile>
+ </configuration>
</execution>
</executions>
</plugin>
diff --git a/tika-bundles/tika-bundle-standard/src/main/java/org/apache/tika/bundle/internal/BundleActivator.java b/tika-bundles/tika-bundle-standard/src/main/java/org/apache/tika/bundle/internal/BundleActivator.java
index c1e029d..258bcc6 100644
--- a/tika-bundles/tika-bundle-standard/src/main/java/org/apache/tika/bundle/internal/BundleActivator.java
+++ b/tika-bundles/tika-bundle-standard/src/main/java/org/apache/tika/bundle/internal/BundleActivator.java
@@ -16,27 +16,40 @@
*/
package org.apache.tika.bundle.internal;
+import java.util.ArrayList;
import java.util.Hashtable;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
import org.osgi.framework.BundleContext;
+import org.osgi.framework.Constants;
import org.osgi.framework.ServiceRegistration;
import org.apache.tika.config.ServiceLoader;
import org.apache.tika.detect.DefaultDetector;
import org.apache.tika.detect.Detector;
+import org.apache.tika.detect.EncodingDetector;
+import org.apache.tika.language.detect.LanguageDetector;
+import org.apache.tika.metadata.filter.MetadataFilter;
import org.apache.tika.mime.MediaTypeRegistry;
import org.apache.tika.mime.MimeTypes;
import org.apache.tika.parser.DefaultParser;
import org.apache.tika.parser.Parser;
/**
- * Registers Tika Parser and Detector services when the bundle starts
- * in an OSGi container.
+ * Registers Tika Parser and Detector services, and the bundle's EncodingDetector,
+ * LanguageDetector and MetadataFilter providers, when the bundle starts in an OSGi container.
*/
public class BundleActivator implements org.osgi.framework.BundleActivator {
+ // the provider types tika-core's TikaActivator merges into its dynamic ServiceLoader
+ private static final Class<?>[] PROVIDER_TYPES = {
+ EncodingDetector.class, LanguageDetector.class, MetadataFilter.class};
+
private ServiceRegistration detectorService;
private ServiceRegistration parserService;
+ private final List<ServiceRegistration> providerServices = new ArrayList<>();
@Override
public void start(BundleContext context) throws Exception {
@@ -47,10 +60,33 @@
Parser parser = new DefaultParser(MediaTypeRegistry.getDefaultRegistry(), loader);
parserService = context.registerService(Parser.class.getName(),
parser, new Hashtable<>());
+
+ // tika-core cannot see providers in this bundle, so register each one, ranked in
+ // classpath order (core's own first), so tika-core's defaults match the classpath
+ ServiceLoader coreLoader = new ServiceLoader(Parser.class.getClassLoader(), false);
+ for (Class<?> type : PROVIDER_TYPES) {
+ Map<String, Object> providers = new LinkedHashMap<>();
+ for (Object p : coreLoader.loadStaticServiceProviders(type)) {
+ providers.putIfAbsent(p.getClass().getName(), p);
+ }
+ for (Object p : loader.loadStaticServiceProviders(type)) {
+ providers.putIfAbsent(p.getClass().getName(), p);
+ }
+ int rank = providers.size();
+ for (Object p : providers.values()) {
+ Hashtable<String, Object> props = new Hashtable<>();
+ props.put(Constants.SERVICE_RANKING, rank--);
+ providerServices.add(context.registerService(type.getName(), p, props));
+ }
+ }
}
@Override
public void stop(BundleContext context) throws Exception {
+ for (ServiceRegistration registration : providerServices) {
+ registration.unregister();
+ }
+ providerServices.clear();
parserService.unregister();
detectorService.unregister();
}
diff --git a/tika-bundles/tika-bundle-standard/src/main/resources/META-INF/MANIFEST.MF b/tika-bundles/tika-bundle-standard/src/main/resources/META-INF/MANIFEST.MF
deleted file mode 100644
index 6314617..0000000
--- a/tika-bundles/tika-bundle-standard/src/main/resources/META-INF/MANIFEST.MF
+++ /dev/null
@@ -1 +0,0 @@
-Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
diff --git a/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/BundleIT.java b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/BundleIT.java
index 18b0687..9d0ad5d 100644
--- a/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/BundleIT.java
+++ b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/BundleIT.java
@@ -17,55 +17,94 @@
package org.apache.tika.bundle;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Enumeration;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
-import java.util.ServiceLoader;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.jar.Attributes;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestInstance;
import org.osgi.framework.Bundle;
import org.osgi.framework.BundleContext;
import org.osgi.framework.Constants;
-import org.osgi.framework.ServiceReference;
import org.osgi.framework.launch.Framework;
import org.osgi.framework.launch.FrameworkFactory;
+import org.osgi.framework.wiring.FrameworkWiring;
import org.xml.sax.ContentHandler;
+import org.apache.tika.detect.DefaultDetector;
+import org.apache.tika.detect.DefaultEncodingDetector;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.HttpHeaders;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.parser.AutoDetectParser;
+import org.apache.tika.parser.DefaultParser;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+import org.apache.tika.sax.BodyContentHandler;
+
/**
- * Integration test that boots an Apache Felix OSGi container, installs the
+ * Integration test that boots an OSGi container, installs the
* tika-core and tika-bundle-standard bundles, and verifies that the bundles
* activate, services register, and parsing works.
* <p>
* The tests run outside the OSGi container (on the JVM classpath), so
* service lookups use string-based names rather than class references.
*/
-public class BundleIT {
+@TestInstance(TestInstance.Lifecycle.PER_CLASS)
+public abstract class BundleIT {
private static final Path TEST_BUNDLES = Paths.get("target", "test-bundles");
- private static Framework framework;
- private static BundleContext ctx;
+ // SPI interfaces whose providers tika-bundle-standard ships
+ private static final String[] SPI_INTERFACES = {
+ "org.apache.tika.parser.Parser",
+ "org.apache.tika.detect.Detector",
+ "org.apache.tika.detect.EncodingDetector",
+ "org.apache.tika.detect.zip.ZipContainerDetector",
+ "org.apache.tika.language.detect.LanguageDetector",
+ "org.apache.tika.metadata.filter.MetadataFilter",
+ "org.apache.tika.renderer.Renderer"};
- @BeforeAll
- static void startFramework() throws Exception {
+ private Framework framework;
+ private BundleContext ctx;
+
+ protected abstract FrameworkFactory frameworkFactory();
+
+ private Framework newFramework(String storage) throws Exception {
Map<String, String> config = new HashMap<>();
config.put(Constants.FRAMEWORK_STORAGE_CLEAN,
Constants.FRAMEWORK_STORAGE_CLEAN_ONFIRSTINIT);
config.put(Constants.FRAMEWORK_STORAGE,
- "target/osgi-cache");
+ "target/osgi-cache/" + getClass().getSimpleName() + "-" + storage);
config.put(Constants.FRAMEWORK_SYSTEMPACKAGES_EXTRA, String.join(",",
"javax.xml.bind",
"org.slf4j;version=2.0.17",
@@ -74,11 +113,14 @@
"org.slf4j.spi;version=2.0.17"
));
// No Service Loader Mediator capabilities: tika-core must resolve without one (TIKA-4945).
+ Framework fw = frameworkFactory().newFramework(config);
+ fw.start();
+ return fw;
+ }
- FrameworkFactory factory = ServiceLoader.load(FrameworkFactory.class)
- .iterator().next();
- framework = factory.newFramework(config);
- framework.start();
+ @BeforeAll
+ void startFramework() throws Exception {
+ framework = newFramework("all");
ctx = framework.getBundleContext();
// Install all bundles first, then start. The test-bundles directory also holds the
@@ -99,28 +141,89 @@
}
@AfterAll
- static void stopFramework() throws Exception {
- if (framework != null) {
- framework.stop();
- framework.waitForStop(10_000);
+ void stopFramework() throws Exception {
+ stop(framework);
+ }
+
+ private static void stop(Framework fw) throws Exception {
+ if (fw != null) {
+ fw.stop();
+ fw.waitForStop(10_000);
}
}
+ // TIKA-4945: a p2 install resolves tika-core on its own, before any provider bundle exists
@Test
- public void testBundleLoaded() {
- boolean hasCore = false, hasBundle = false;
- for (Bundle b : ctx.getBundles()) {
- if ("org.apache.tika.core".equals(b.getSymbolicName())) {
- hasCore = true;
- assertEquals(Bundle.ACTIVE, b.getState(), "Core not activated");
+ public void testCoreResolvesWithoutStandardBundle() throws Exception {
+ assertTrue(resolvesWithout("org.apache.tika.core", "tika-bundle-standard.jar"),
+ "tika-core did not resolve without tika-bundle-standard");
+ }
+
+ // TIKA-4955: a missing dependency bundle must fail resolution, not a parse
+ @Test
+ public void testMissingDependencyBundleFailsResolution() throws Exception {
+ assertFalse(resolvesWithout("org.apache.tika.bundle-standard", "pdfbox-io.jar"),
+ "tika-bundle-standard resolved without pdfbox-io");
+ }
+
+ private boolean resolvesWithout(String symbolicName, String omittedJar) throws Exception {
+ Framework fw = newFramework("without-" + omittedJar);
+ try {
+ Bundle target = null;
+ boolean omitted = false;
+ try (DirectoryStream<Path> jars = Files.newDirectoryStream(TEST_BUNDLES, "*.jar")) {
+ for (Path jar : jars) {
+ if (jar.getFileName().toString().equals(omittedJar)) {
+ omitted = true;
+ continue;
+ }
+ Bundle b = fw.getBundleContext().installBundle(jar.toUri().toString());
+ if (symbolicName.equals(b.getSymbolicName())) {
+ target = b;
+ }
+ }
}
- if ("org.apache.tika.bundle-standard".equals(b.getSymbolicName())) {
- hasBundle = true;
- assertEquals(Bundle.ACTIVE, b.getState(), "Bundle not activated");
+ assertTrue(omitted, omittedJar + " not in " + TEST_BUNDLES);
+ assertNotNull(target, symbolicName + " not installed");
+ return fw.adapt(FrameworkWiring.class).resolveBundles(List.of(target));
+ } finally {
+ stop(fw);
+ }
+ }
+
+ // ServiceLoader logs and skips a provider that fails to load, so a missing embedded
+ // dependency silently drops a parser or detector; instantiate every one here
+ @Test
+ public void testAllServiceProvidersLoad() throws Exception {
+ Bundle tikaBundle = findBundle("org.apache.tika.bundle-standard");
+ Set<String> failures = new TreeSet<>();
+ int count = 0;
+ for (String spi : SPI_INTERFACES) {
+ Enumeration<URL> files =
+ tikaBundle.getResources("META-INF/services/" + spi);
+ while (files != null && files.hasMoreElements()) {
+ for (String line : new String(files.nextElement().openStream().readAllBytes(),
+ StandardCharsets.UTF_8).split("\\R")) {
+ String name = line.replaceAll("#.*", "").trim();
+ if (name.isEmpty()) {
+ continue;
+ }
+ count++;
+ try {
+ tikaBundle.loadClass(name).getConstructor().newInstance();
+ } catch (Throwable t) {
+ Throwable root = t;
+ while (root.getCause() != null) {
+ root = root.getCause();
+ }
+ failures.add(name + " -> " + root);
+ }
+ }
}
}
- assertTrue(hasCore, "Core bundle not found");
- assertTrue(hasBundle, "Standard bundle not found");
+ assertTrue(count > 50, "found only " + count + " service providers");
+ assertTrue(failures.isEmpty(), failures.size() + " of " + count
+ + " service providers failed to load:\n" + String.join("\n", failures));
}
@Test
@@ -130,192 +233,175 @@
}
}
+ // tika-core's defaults inside OSGi must be built from the same providers as on the
+ // classpath; a dependency missing from the bundle drops its providers silently
@Test
- public void testExternalDependenciesWired() throws Exception {
- // All imports of tika-bundle-standard are optional, so check that the
- // packages of dependencies that are not embedded are actually wired.
- Bundle tikaBundle = findBundle("org.apache.tika.bundle-standard");
- assertNotNull(tikaBundle, "tika-bundle-standard not found");
- for (String className : new String[]{
- "com.adobe.internal.xmp.XMPMetaFactory",
- "com.dd.plist.PropertyListParser",
- "org.apache.commons.codec.digest.DigestUtils",
- "org.apache.commons.collections4.MapUtils",
- "org.apache.commons.compress.archivers.ArchiveStreamFactory",
- "org.apache.commons.csv.CSVFormat",
- "org.apache.commons.exec.CommandLine",
- "org.apache.commons.io.IOUtils",
- "org.apache.commons.lang3.StringUtils",
- "org.apache.commons.math3.util.FastMath",
- "org.apache.fontbox.ttf.TrueTypeFont",
- "org.apache.pdfbox.Loader",
- "org.apache.pdfbox.io.RandomAccessRead",
- "org.bouncycastle.cms.CMSSignedData",
- "org.bouncycastle.jce.provider.BouncyCastleProvider",
- "org.jsoup.Jsoup",
- "org.objectweb.asm.ClassReader"}) {
- assertNotNull(tikaBundle.loadClass(className), className);
- }
- Bundle commonsCompress = findBundle("org.apache.commons.commons-compress");
- assertNotNull(commonsCompress, "commons-compress bundle not found");
- assertNotNull(commonsCompress.loadClass("org.tukaani.xz.XZInputStream"));
- Bundle tikaCore = findBundle("org.apache.tika.core");
- assertNotNull(tikaCore, "tika-core bundle not found");
- assertNotNull(tikaCore.loadClass("org.apache.commons.xml.secure.SecureSAXParserFactory"));
- }
-
- @Test
- public void testDetectorServiceRegistered() throws Exception {
- ServiceReference<?>[] refs = ctx.getAllServiceReferences(
- "org.apache.tika.detect.Detector", null);
- assertNotNull(refs, "Detector service not registered");
- assertTrue(refs.length > 0, "Should have at least one Detector service");
- Object detector = ctx.getService(refs[0]);
- assertNotNull(detector);
- assertEquals("org.apache.tika.detect.DefaultDetector",
- detector.getClass().getName());
- }
-
- @Test
- public void testParserServiceRegistered() throws Exception {
- ServiceReference<?>[] refs = ctx.getAllServiceReferences(
- "org.apache.tika.parser.Parser", null);
- assertNotNull(refs, "Parser service not registered");
- assertTrue(refs.length > 0, "Should have at least one Parser service");
- Object parser = ctx.getService(refs[0]);
- assertNotNull(parser);
- assertEquals("org.apache.tika.parser.DefaultParser",
- parser.getClass().getName());
- }
-
- @Test
- public void testDetectorHasMultipleDetectors() throws Exception {
- ServiceReference<?>[] refs = ctx.getAllServiceReferences(
- "org.apache.tika.detect.Detector", null);
- Object detector = ctx.getService(refs[0]);
- Object detectors = detector.getClass()
- .getMethod("getDetectors").invoke(detector);
- int size = ((java.util.List<?>) detectors).size();
- assertTrue(size > 3,
- "Should have several detectors, found " + size);
- }
-
- @Test
- public void testParserHasMultipleParsers() throws Exception {
- ServiceReference<?>[] refs = ctx.getAllServiceReferences(
- "org.apache.tika.parser.Parser", null);
- Object parser = ctx.getService(refs[0]);
- Object parsers = parser.getClass()
- .getMethod("getAllComponentParsers").invoke(parser);
- int size = ((java.util.Collection<?>) parsers).size();
- assertTrue(size > 15,
- "Should have lots of parsers, found " + size);
- }
-
- // parse and detect through the registered services, the way an OSGi consumer does;
- // before the fix both recursed into themselves until the stack overflowed
- @Test
- public void testParseThroughRegisteredServices() throws Exception {
- byte[] html = null;
- try (ZipInputStream zip = new ZipInputStream(
- BundleIT.class.getResourceAsStream("/test-documents.zip"))) {
- for (ZipEntry entry = zip.getNextEntry(); entry != null; entry = zip.getNextEntry()) {
- if ("testHTML.html".equals(entry.getName())) {
- html = zip.readAllBytes();
- }
- }
- }
- assertNotNull(html, "testHTML.html not found");
-
- Bundle tikaCore = findBundle("org.apache.tika.core");
- Class<?> metadataClass = tikaCore.loadClass("org.apache.tika.metadata.Metadata");
- Class<?> tisClass = tikaCore.loadClass("org.apache.tika.io.TikaInputStream");
- Class<?> contextClass = tikaCore.loadClass("org.apache.tika.parser.ParseContext");
- Method tisGet = tisClass.getMethod("get", byte[].class);
- Method metadataSet = metadataClass.getMethod("set", String.class, String.class);
-
+ public void testDefaultsMatchClasspath() throws Exception {
+ Bundle core = findBundle("org.apache.tika.core");
+ Object parser = ctx.getService(ctx.getAllServiceReferences(
+ "org.apache.tika.parser.Parser", null)[0]);
Object detector = ctx.getService(ctx.getAllServiceReferences(
"org.apache.tika.detect.Detector", null)[0]);
- Method detect = tikaCore.loadClass("org.apache.tika.detect.Detector").getMethod("detect",
- tisClass, metadataClass, contextClass);
- Object type;
- try (AutoCloseable tis = (AutoCloseable) tisGet.invoke(null, (Object) html)) {
- type = detect.invoke(detector, tis, metadataClass.getConstructor().newInstance(),
- contextClass.getConstructor().newInstance());
- }
- assertEquals("text/html", type.toString());
+ Object encodingDetector = core.loadClass("org.apache.tika.detect.DefaultEncodingDetector")
+ .getConstructor().newInstance();
- Object parser = ctx.getService(ctx.getAllServiceReferences(
- "org.apache.tika.parser.Parser", null)[0]);
- Method parse = tikaCore.loadClass("org.apache.tika.parser.Parser").getMethod("parse",
- tisClass, ContentHandler.class, metadataClass, contextClass);
- Object metadata = metadataClass.getConstructor().newInstance();
- metadataSet.invoke(metadata, "Content-Type", type.toString());
- ContentHandler handler = (ContentHandler) tikaCore
- .loadClass("org.apache.tika.sax.BodyContentHandler")
- .getConstructor(int.class).newInstance(-1);
- try (AutoCloseable tis = (AutoCloseable) tisGet.invoke(null, (Object) html)) {
- parse.invoke(parser, tis, handler, metadata, contextClass.getConstructor().newInstance());
- }
- assertTrue(handler.toString().contains("Test Indexation Html"), handler.toString());
+ assertEquals(classNames(new DefaultParser().getAllComponentParsers()),
+ classNames((Collection<?>) parser.getClass()
+ .getMethod("getAllComponentParsers").invoke(parser)), "parsers");
+ assertEquals(classNames(new DefaultDetector().getDetectors()),
+ classNames((Collection<?>) detector.getClass()
+ .getMethod("getDetectors").invoke(detector)), "detectors");
+ assertEquals(classNames(new DefaultEncodingDetector().getDetectors()),
+ classNames((Collection<?>) encodingDetector.getClass()
+ .getMethod("getDetectors").invoke(encodingDetector)), "encoding detectors");
}
+ private static List<String> classNames(Collection<?> objects) {
+ List<String> names = new ArrayList<>();
+ for (Object o : objects) {
+ names.add(o.getClass().getName());
+ }
+ return names;
+ }
+
+ // runs the docs page's example as a consumer bundle would
@Test
- public void testPdfParsing() throws Exception {
- byte[] pdf = null;
- try (ZipInputStream zip = new ZipInputStream(
- BundleIT.class.getResourceAsStream("/test-documents.zip"))) {
- for (ZipEntry entry = zip.getNextEntry(); entry != null; entry = zip.getNextEntry()) {
- if ("testPDF.pdf".equals(entry.getName())) {
- pdf = zip.readAllBytes();
+ public void testDocumentedExample() throws Exception {
+ String pkg = "org.apache.tika.bundle.example";
+ Path classes = Paths.get("target", "test-classes", pkg.replace('.', '/'));
+ Path jar = Paths.get("target", "osgi-example.jar");
+ Manifest manifest = new Manifest();
+ Attributes attributes = manifest.getMainAttributes();
+ attributes.put(Attributes.Name.MANIFEST_VERSION, "1.0");
+ attributes.putValue(Constants.BUNDLE_MANIFESTVERSION, "2");
+ attributes.putValue(Constants.BUNDLE_SYMBOLICNAME, pkg);
+ attributes.putValue(Constants.IMPORT_PACKAGE, String.join(",",
+ "org.apache.tika.detect", "org.apache.tika.io", "org.apache.tika.metadata",
+ "org.apache.tika.parser", "org.apache.tika.sax", "org.osgi.framework",
+ "org.xml.sax"));
+ try (JarOutputStream out = new JarOutputStream(Files.newOutputStream(jar), manifest);
+ DirectoryStream<Path> files = Files.newDirectoryStream(classes, "*.class")) {
+ for (Path f : files) {
+ out.putNextEntry(new JarEntry(pkg.replace('.', '/') + "/" + f.getFileName()));
+ out.write(Files.readAllBytes(f));
+ out.closeEntry();
+ }
+ }
+
+ Bundle example = ctx.installBundle(jar.toUri().toString());
+ try {
+ example.start();
+ Path pdf = Files.createTempFile("osgi-example", ".pdf");
+ try {
+ Files.write(pdf, testDocuments().get("testPDF.pdf"));
+ Class<?> exampleClass = example.loadClass(pkg + ".OsgiParseExample");
+ assertNotEquals(BundleIT.class.getClassLoader(), exampleClass.getClassLoader());
+ String text = (String) exampleClass
+ .getMethod("parse", BundleContext.class, Path.class)
+ .invoke(null, example.getBundleContext(), pdf);
+ assertTrue(text.contains("Apache Tika"), text);
+ } finally {
+ Files.delete(pdf);
+ }
+ } finally {
+ example.uninstall();
+ }
+ }
+
+ // Detect and parse every sample, embedded documents included, through the OSGi services
+ // and through the plain classpath; any difference is a class the bundle cannot see
+ @Test
+ public void testParseMatchesClasspath() throws Exception {
+ Bundle core = findBundle("org.apache.tika.core");
+ Class<?> metadataClass = core.loadClass("org.apache.tika.metadata.Metadata");
+ Class<?> propertyClass = core.loadClass("org.apache.tika.metadata.Property");
+ Class<?> tisClass = core.loadClass("org.apache.tika.io.TikaInputStream");
+ Class<?> contextClass = core.loadClass("org.apache.tika.parser.ParseContext");
+ Class<?> parserClass = core.loadClass("org.apache.tika.parser.Parser");
+ Class<?> detectorClass = core.loadClass("org.apache.tika.detect.Detector");
+ Object resourceName = core.loadClass("org.apache.tika.metadata.TikaCoreProperties")
+ .getField("RESOURCE_NAME_KEY").get(null);
+ Method tisGet = tisClass.getMethod("get", byte[].class);
+ Method metadataSet = metadataClass.getMethod("set", propertyClass, String.class);
+ Method metadataGet = metadataClass.getMethod("get", String.class);
+ Method contextSet = contextClass.getMethod("set", Class.class, Object.class);
+ Method parse = parserClass.getMethod("parse", tisClass, ContentHandler.class,
+ metadataClass, contextClass);
+
+ Object parsers = Array.newInstance(parserClass, 1);
+ Array.set(parsers, 0, ctx.getService(ctx.getAllServiceReferences(
+ parserClass.getName(), null)[0]));
+ Object osgiParser = core.loadClass("org.apache.tika.parser.AutoDetectParser")
+ .getConstructor(detectorClass, parsers.getClass())
+ .newInstance(ctx.getService(ctx.getAllServiceReferences(
+ detectorClass.getName(), null)[0]), parsers);
+ AutoDetectParser classpathParser = new AutoDetectParser();
+
+ Map<String, byte[]> docs = testDocuments();
+ assertTrue(docs.size() >= 20, "found only " + docs.size() + " test documents");
+ List<String> failures = new ArrayList<>();
+ for (Map.Entry<String, byte[]> doc : docs.entrySet()) {
+ Metadata expectedMetadata = new Metadata();
+ expectedMetadata.set(TikaCoreProperties.RESOURCE_NAME_KEY, doc.getKey());
+ ParseContext expectedContext = new ParseContext();
+ expectedContext.set(Parser.class, classpathParser);
+ BodyContentHandler expectedText = new BodyContentHandler(-1);
+ try (TikaInputStream tis = TikaInputStream.get(doc.getValue())) {
+ classpathParser.parse(tis, expectedText, expectedMetadata, expectedContext);
+ }
+
+ Object metadata = metadataClass.getConstructor().newInstance();
+ metadataSet.invoke(metadata, resourceName, doc.getKey());
+ Object context = contextClass.getConstructor().newInstance();
+ contextSet.invoke(context, parserClass, osgiParser);
+ ContentHandler text = (ContentHandler) core
+ .loadClass("org.apache.tika.sax.BodyContentHandler")
+ .getConstructor(int.class).newInstance(-1);
+ try (AutoCloseable tis = (AutoCloseable) tisGet.invoke(null, (Object) doc.getValue())) {
+ parse.invoke(osgiParser, tis, text, metadata, context);
+ } catch (InvocationTargetException e) {
+ failures.add(doc.getKey() + ": " + e.getCause());
+ continue;
+ }
+
+ String expectedType = expectedMetadata.get(HttpHeaders.CONTENT_TYPE);
+ Object type = metadataGet.invoke(metadata, HttpHeaders.CONTENT_TYPE.getName());
+ if (!expectedType.equals(type)) {
+ failures.add(doc.getKey() + ": type " + type + ", classpath " + expectedType);
+ } else {
+ String expected = normalize(expectedText.toString());
+ String actual = normalize(text.toString());
+ if (!expected.equals(actual)) {
+ int i = 0;
+ while (i < Math.min(expected.length(), actual.length())
+ && expected.charAt(i) == actual.charAt(i)) {
+ i++;
+ }
+ failures.add(doc.getKey() + ": text differs from classpath at " + i + ": \""
+ + actual.substring(i, Math.min(actual.length(), i + 60)) + "\" vs \""
+ + expected.substring(i, Math.min(expected.length(), i + 60)) + "\"");
}
}
}
- assertNotNull(pdf, "testPDF.pdf not found");
+ assertTrue(failures.isEmpty(), String.join("\n", failures));
+ }
- Bundle tikaCore = findBundle("org.apache.tika.core");
- Class<?> metadataClass = tikaCore.loadClass("org.apache.tika.metadata.Metadata");
- Class<?> tisClass = tikaCore.loadClass("org.apache.tika.io.TikaInputStream");
- Class<?> contextClass = tikaCore.loadClass("org.apache.tika.parser.ParseContext");
- Method parse = tikaCore.loadClass("org.apache.tika.parser.Parser").getMethod("parse",
- tisClass, ContentHandler.class, metadataClass, contextClass);
+ private static String normalize(String text) {
+ return text.replaceAll("\\s+", " ").trim();
+ }
- Object metadata = metadataClass.getConstructor().newInstance();
- metadataClass.getMethod("set", String.class, String.class)
- .invoke(metadata, "Content-Type", "application/pdf");
- ContentHandler handler = (ContentHandler) tikaCore
- .loadClass("org.apache.tika.sax.BodyContentHandler")
- .getConstructor(int.class).newInstance(-1);
-
- Object parser = ctx.getService(ctx.getAllServiceReferences(
- "org.apache.tika.parser.Parser", null)[0]);
- try (AutoCloseable tis = (AutoCloseable) tisClass.getMethod("get", byte[].class)
- .invoke(null, (Object) pdf)) {
- parse.invoke(parser, tis, handler, metadata, contextClass.getConstructor().newInstance());
+ private static Map<String, byte[]> testDocuments() throws Exception {
+ Map<String, byte[]> docs = new TreeMap<>();
+ try (ZipInputStream zip = new ZipInputStream(
+ BundleIT.class.getResourceAsStream("/test-documents.zip"))) {
+ for (ZipEntry entry = zip.getNextEntry(); entry != null; entry = zip.getNextEntry()) {
+ docs.put(entry.getName(), zip.readAllBytes());
+ }
}
-
- Method get = metadataClass.getMethod("get", String.class);
- assertEquals("Apache Tika - Apache Tika", get.invoke(metadata, "dc:title"));
- assertTrue(handler.toString().contains("Apache Tika"), "PDF content not extracted");
+ return docs;
}
- @Test
- public void testTikaClassLoadable() throws Exception {
- // Verify key Tika classes can be loaded from the bundle's classloader
- Bundle tikaCore = findBundle("org.apache.tika.core");
- assertNotNull(tikaCore, "tika-core bundle not found");
- assertNotNull(tikaCore.loadClass("org.apache.tika.Tika"));
- assertNotNull(tikaCore.loadClass("org.apache.tika.parser.AutoDetectParser"));
- assertNotNull(tikaCore.loadClass("org.apache.tika.detect.DefaultDetector"));
-
- Bundle tikaBundle = findBundle("org.apache.tika.bundle-standard");
- assertNotNull(tikaBundle, "tika-bundle-standard not found");
- // Parser implementations should be loadable from the bundle
- assertNotNull(tikaBundle.loadClass("org.apache.tika.parser.pdf.PDFParser"));
- assertNotNull(tikaBundle.loadClass("org.apache.tika.parser.microsoft.ooxml.OOXMLParser"));
- }
-
- private static Bundle findBundle(String symbolicName) {
+ private Bundle findBundle(String symbolicName) {
for (Bundle b : ctx.getBundles()) {
if (symbolicName.equals(b.getSymbolicName())) {
return b;
diff --git a/tika-core/src/main/java/org/apache/tika/config/package-info.java b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/EquinoxBundleIT.java
similarity index 72%
copy from tika-core/src/main/java/org/apache/tika/config/package-info.java
copy to tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/EquinoxBundleIT.java
index 61ee292..aa1ae51 100644
--- a/tika-core/src/main/java/org/apache/tika/config/package-info.java
+++ b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/EquinoxBundleIT.java
@@ -14,9 +14,15 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+package org.apache.tika.bundle;
-/**
- * Tika configuration tools.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.config;
+import org.eclipse.osgi.launch.EquinoxFactory;
+import org.osgi.framework.launch.FrameworkFactory;
+
+public class EquinoxBundleIT extends BundleIT {
+
+ @Override
+ protected FrameworkFactory frameworkFactory() {
+ return new EquinoxFactory();
+ }
+}
diff --git a/tika-core/src/main/java/org/apache/tika/config/package-info.java b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/FelixBundleIT.java
similarity index 74%
rename from tika-core/src/main/java/org/apache/tika/config/package-info.java
rename to tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/FelixBundleIT.java
index 61ee292..8775517 100644
--- a/tika-core/src/main/java/org/apache/tika/config/package-info.java
+++ b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/FelixBundleIT.java
@@ -14,9 +14,14 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+package org.apache.tika.bundle;
-/**
- * Tika configuration tools.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.config;
+import org.osgi.framework.launch.FrameworkFactory;
+
+public class FelixBundleIT extends BundleIT {
+
+ @Override
+ protected FrameworkFactory frameworkFactory() {
+ return new org.apache.felix.framework.FrameworkFactory();
+ }
+}
diff --git a/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/example/OsgiParseExample.java b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/example/OsgiParseExample.java
new file mode 100644
index 0000000..f6921ea
--- /dev/null
+++ b/tika-bundles/tika-bundle-standard/src/test/java/org/apache/tika/bundle/example/OsgiParseExample.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.bundle.example;
+
+import java.nio.file.Path;
+
+import org.osgi.framework.BundleContext;
+
+import org.apache.tika.detect.Detector;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.parser.AutoDetectParser;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+import org.apache.tika.sax.BodyContentHandler;
+
+/**
+ * The example on the "Using Tika in OSGi" docs page; BundleIT installs it as a bundle and runs it.
+ */
+public class OsgiParseExample {
+
+ public static String parse(BundleContext context, Path path) throws Exception {
+ // tag::parse[]
+ Detector detector = context.getService(context.getServiceReference(Detector.class));
+ Parser standard = context.getService(context.getServiceReference(Parser.class));
+ Parser parser = new AutoDetectParser(detector, standard);
+
+ ParseContext parseContext = new ParseContext();
+ parseContext.set(Parser.class, parser); // parse embedded documents too
+ BodyContentHandler handler = new BodyContentHandler(-1);
+ Metadata metadata = new Metadata();
+ try (TikaInputStream tis = TikaInputStream.get(path)) {
+ parser.parse(tis, handler, metadata, parseContext);
+ }
+ // end::parse[]
+ return handler.toString();
+ }
+}
diff --git a/tika-bundles/tika-bundle-standard/src/test/resources/test-documents.zip b/tika-bundles/tika-bundle-standard/src/test/resources/test-documents.zip
index 27d600d..3a1eb3e 100644
--- a/tika-bundles/tika-bundle-standard/src/test/resources/test-documents.zip
+++ b/tika-bundles/tika-bundle-standard/src/test/resources/test-documents.zip
Binary files differ
diff --git a/tika-bundles/tika-bundle-standard/test-bundles.xml b/tika-bundles/tika-bundle-standard/test-bundles.xml
index 1cf9aca..4ddc350 100644
--- a/tika-bundles/tika-bundle-standard/test-bundles.xml
+++ b/tika-bundles/tika-bundle-standard/test-bundles.xml
@@ -31,6 +31,9 @@
<include>org.apache.tika:tika-bundle-standard</include>
<!-- Dependencies that are OSGi bundles themselves (not embedded) -->
<include>com.adobe.xmp:xmpcore</include>
+ <include>com.fasterxml.jackson.core:jackson-annotations</include>
+ <include>com.fasterxml.jackson.core:jackson-core</include>
+ <include>com.fasterxml.jackson.core:jackson-databind</include>
<include>com.googlecode.plist:dd-plist</include>
<include>commons-codec:commons-codec</include>
<include>commons-io:commons-io</include>
@@ -42,6 +45,7 @@
<include>org.apache.commons:commons-lang3</include>
<include>org.apache.commons:commons-math3</include>
<include>org.apache.commons:commons-secure-xml</include>
+ <include>org.apache.logging.log4j:log4j-api</include>
<include>org.apache.pdfbox:fontbox</include>
<include>org.apache.pdfbox:pdfbox</include>
<include>org.apache.pdfbox:pdfbox-io</include>
diff --git a/tika-core/pom.xml b/tika-core/pom.xml
index f418a21..3be97b9 100644
--- a/tika-core/pom.xml
+++ b/tika-core/pom.xml
@@ -63,23 +63,16 @@
<artifactId>commonmark-ext-gfm-strikethrough</artifactId>
</dependency>
- <!-- Optional OSGi dependencies, used only when running within OSGi -->
+ <!-- OSGi API, used only when running within OSGi -->
<dependency>
<groupId>org.osgi</groupId>
<artifactId>org.osgi.core</artifactId>
<scope>provided</scope>
- <optional>true</optional>
</dependency>
<dependency>
<groupId>org.osgi</groupId>
<artifactId>org.osgi.compendium</artifactId>
<scope>provided</scope>
- <optional>true</optional>
- </dependency>
- <dependency>
- <groupId>biz.aQute.bnd</groupId>
- <artifactId>biz.aQute.bndlib</artifactId>
- <scope>provided</scope>
</dependency>
<!-- Annotation processor - contains @TikaComponent and ensures build order.
diff --git a/tika-core/src/main/java/org/apache/tika/detect/package-info.java b/tika-core/src/main/java/org/apache/tika/detect/package-info.java
deleted file mode 100644
index a579b63..0000000
--- a/tika-core/src/main/java/org/apache/tika/detect/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Media type detection.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.detect;
diff --git a/tika-core/src/main/java/org/apache/tika/exception/package-info.java b/tika-core/src/main/java/org/apache/tika/exception/package-info.java
deleted file mode 100644
index 13c7b98..0000000
--- a/tika-core/src/main/java/org/apache/tika/exception/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Tika exception.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.exception;
diff --git a/tika-core/src/main/java/org/apache/tika/extractor/package-info.java b/tika-core/src/main/java/org/apache/tika/extractor/package-info.java
deleted file mode 100644
index ddb1dfd..0000000
--- a/tika-core/src/main/java/org/apache/tika/extractor/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Extraction of component documents.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.extractor;
diff --git a/tika-core/src/main/java/org/apache/tika/io/package-info.java b/tika-core/src/main/java/org/apache/tika/io/package-info.java
deleted file mode 100644
index 3c828b9..0000000
--- a/tika-core/src/main/java/org/apache/tika/io/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * IO utilities.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.io;
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/package-info.java b/tika-core/src/main/java/org/apache/tika/metadata/package-info.java
deleted file mode 100644
index b85b43c..0000000
--- a/tika-core/src/main/java/org/apache/tika/metadata/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Multi-valued metadata container, and set of constant metadata fields.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.metadata;
diff --git a/tika-core/src/main/java/org/apache/tika/mime/package-info.java b/tika-core/src/main/java/org/apache/tika/mime/package-info.java
deleted file mode 100644
index aa45c69..0000000
--- a/tika-core/src/main/java/org/apache/tika/mime/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Media type information.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.mime;
diff --git a/tika-core/src/main/java/org/apache/tika/package-info.java b/tika-core/src/main/java/org/apache/tika/package-info.java
deleted file mode 100644
index 5a18964..0000000
--- a/tika-core/src/main/java/org/apache/tika/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Apache Tika.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika;
diff --git a/tika-core/src/main/java/org/apache/tika/parser/package-info.java b/tika-core/src/main/java/org/apache/tika/parser/package-info.java
deleted file mode 100644
index 8142132..0000000
--- a/tika-core/src/main/java/org/apache/tika/parser/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Tika parsers.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.parser;
diff --git a/tika-core/src/main/java/org/apache/tika/sax/package-info.java b/tika-core/src/main/java/org/apache/tika/sax/package-info.java
deleted file mode 100644
index f2522c4..0000000
--- a/tika-core/src/main/java/org/apache/tika/sax/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * SAX utilities.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.sax;
diff --git a/tika-core/src/main/java/org/apache/tika/sax/xpath/package-info.java b/tika-core/src/main/java/org/apache/tika/sax/xpath/package-info.java
deleted file mode 100644
index 1a12e29..0000000
--- a/tika-core/src/main/java/org/apache/tika/sax/xpath/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * XPath utilities
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.sax.xpath;
diff --git a/tika-core/src/main/java/org/apache/tika/utils/package-info.java b/tika-core/src/main/java/org/apache/tika/utils/package-info.java
deleted file mode 100644
index be106f7..0000000
--- a/tika-core/src/main/java/org/apache/tika/utils/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Utilities.
- */
-@aQute.bnd.annotation.Version("2.0.0")
-package org.apache.tika.utils;
diff --git a/tika-java7/pom.xml b/tika-java7/pom.xml
index dad6757..e4bf1f0 100644
--- a/tika-java7/pom.xml
+++ b/tika-java7/pom.xml
@@ -98,11 +98,6 @@
<version>${project.version}</version>
<type>pom</type>
</dependency>
- <dependency>
- <groupId>biz.aQute.bnd</groupId>
- <artifactId>biz.aQute.bndlib</artifactId>
- <scope>provided</scope>
- </dependency>
</dependencies>
<description>Java-7 reliant components, including FileTypeDetector implementations</description>
diff --git a/tika-java7/src/main/java/org/apache/tika/filetypedetector/package-info.java b/tika-java7/src/main/java/org/apache/tika/filetypedetector/package-info.java
deleted file mode 100644
index a7a82cd..0000000
--- a/tika-java7/src/main/java/org/apache/tika/filetypedetector/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * Tika Java-7 FileTypeDetector implementations.
- */
-@aQute.bnd.annotation.Version("1.0.0")
-package org.apache.tika.filetypedetector;