wanted: PDF analyzer (Bugzilla #17751)

[vladak]

status ACCEPTED severity enhancement in component analyzer for ---
Reported in version unspecified on platform ANY/Generic
Assigned to: Lubos Kosco

On 2011-01-20 10:12:05 +0000, Vladimir Kotal wrote:

PDF analyzer would be beneficial to have, e.g. in order to search design documents together with source code (by selecting a project with the source code and a "project" with design documents).

On 2011-02-15 13:54:44 +0000, Lubos Kosco wrote:

we could reuse http://pdfbox.apache.org/

after all old opengrok - arcs - still used for psarcs had it like that ...

forwardport? :-D

On 2011-02-15 13:59:43 +0000, Lubos Kosco wrote:

alternatively is to use pdfbox underneath tika and grant a myriad of supported formats for lucene:

http://tika.apache.org/0.8/formats.html

(pdf, (open)office, mbox, rtf, audio/video metadata alt. java class and jar parser, it also has a compressed files parser, which can be used to satisfy bug 343 )

I have a feeling this might be one of the major features for next version! :)

On 2011-03-15 07:29:16 +0000, Lubos Kosco wrote:

for odf formats we also have:
http://odftoolkit.org/

[nutsh]

It should be a nice feature

[tarzanek]

I have a prototype for this, we can start next release with me pushing this so people can get some deeper testing done
the tika approach can be used for more formats, but binary distro gets increased in size by 25MB

[tarzanek]

(I did a poll, people didn't like the size increase and asked for a "plugin" approach to analyzers, but I simply didn't have time to implement such approach ... :( )

[nutsh]

@tarzanek Thanks,for the KISS project, "plugin" should be a better choice . but your update is welcome....I have also do a similar solution and I'd like to test it.

[nutsh]

@tarzanek Please show how to find the pull ,Sorry I could not find it.Thanks

[tarzanek]

@nutsh don't have it public yet ...
if you have a similar solution and it can be made public, feel free to go ahead and make a pull request
I won't be able to make mine public for next week at least(it needs some clean up and I simply don't have the time to finish it now)

[nutsh]

hi @tarzanek , it seems the opengrok does not have any register new type hook in the AnalyzerGuru.
And also I agree this function is not needed for most of people and the patch with some thirdparty jars is quite lager.We also need to make it as simple as possible.

So I Think the best solution is firstly build and public a friendly plugin interface.
Does the project still has some plan about public some plugin API.

[tarzanek]

well, I agree with plugin API, but so far no one had time to implement it ;)
If you feel like that, please fork and add a sample API
from where I stand the gain is not worth the pain for now ... e.g. jenkins.war is 60MB in size ... opengrok is 7MB ... so if we'd get opengrok to 20MB I am not sure if someone would notice ... also having to add a plugin to get support for something ... umh ... been doing this with netbeans ... and ... I am tired of always downloading a plugin if I need something
it's all about defaults and stuff working out of box ... and this might be worthwhile to be working out of box
so I will just take my prototype and after 0.12 is out I'll put it into the trunk(after review of course) ...
(this will take 1-2 months till we get there)
so if you will have a plugin API by then, I'll gladly adapt