Tika Text Extract

Extract text from any document by Apache Tika

What?

The Apache Tika™ toolkit detects and extracts metadata and text from over a thousand different file types (such as PPT, XLS, and PDF). All of these file types can be parsed through a single interface, making Tika useful for search engine indexing, content analysis, translation, and much more.

Why?

This was mainly built for convenience usage in AWS Lambda environment.

If you want to use Tika from node.js you are left with these options:

• Spawn a CLI - no, extremely inefficient to pay for Java startup time
• Start HTTP Server
• Use Java ?

Spawning a Tika as CLI is extremely inefficient. Using Java API from node.js is tedious. This module starts a Tika HTTP Server to stream files to and return a string of extracted text.

Requires java to be present on the system.

Install

$ yarn add @shelf/tika-text-extract

Note

By default in tika-text-extract version 3 use tika-server greater than 2.

Usage

import {readFileSync} from 'fs';
import tte from '@shelf/tika-text-extract';

await tte.startServer('/tmp/tika-server-standard-2.2.1.jar');
const testFile = readFileSync('./README.md');

const extractedText = await tte.extract(testFile);

Execute Tika with a custom path to Java binary

const options = {executableJavaPath: '/bin/jre/java'};

await tte.startServer('/tmp/tika-server-standard-2.2.1.jar', options);
// The next command will be executed:
// /bin/jre/java -jar /tmp/tika-server-standard-2.2.1.jar -noFork

Execute Tika with Java version less than 9

By default, the library does not support Java versions less than 9. In order to use it with Java 8, pass an option to startServer function

const options = {alignWithJava8: true};

await tte.startServer('/tmp/tika-server-standard-2.2.1.jar', options);
// The next command will be executed:
// java -jar /tmp/tika-server-standard-2.2.1.jar -noFork

Execute Tika V1

By default in tika-text-extract version 3 use apache-tika greater than 2. To use tika-server less than 2, pass an option to startServer function

const options = {useTikaV1: true};

await tte.startServer('/tmp/tika-server-1.25.jar', options);
// The next command will be executed:
// /bin/jre/java --add-modules=java.xml.bind,java.activation -Duser.home=/tmp -jar /tmp/tika-server-1.25.jar

Note

If you don't use this option with apache-tika less than 2. You will get an error

API

You can see debug messages by setting env var DEBUG=tika-text-extract

tte.startServer(artifactPath)

Params: artifactPath - path to your tika-server.jar file.

Returns: Promise resolved when server is started. Rejects in case of error.

tte.extract(fileInput)

Params: fileInput - Buffer, String, Stream or Promise of file to extract text from.

Returns: Promise resolved with extracted text.

Publish

$ git checkout master
$ yarn version
$ yarn publish
$ git push origin master --tags

How to run tika-text-extract

Download Java, you can accomplish it with these commands:

mkdir java

docker run --rm -v "$PWD"/java:/lambda/opt lambci/yumda:2 yum install -y java-1.8.0-openjdk-headless.x86_64

Move java folder inside tika-text-extract. Download tika-server which you want to use. You can find it in an archive After that you can run this command:

docker run --rm \
-v "$PWD":/var/task \
-v "$PWD/java":/opt/java \
-v "$PWD/tika":/../layer/tika/ \
lambci/lambda:nodejs12.x basic-usage.handler

License

MIT © Shelf