API function to just get all org files without parsed headlines?

[Maltimore]

Hi,
I'm using the telescope-orgmode plugin to find my org files.

Unfortunately, the loading time is starting to get noticeable, now sitting at 5+ seconds. In general, one would expect that the load time is essentially instantanous, as I only have ~20 org files in three locations.

The reason that it's not instantaneous is because telescope-orgmode uses the orgmode.api.load() function to get the files (used here) and this function also parses the headlines, which isn't needed in this case (and I'm assuming there are other use cases that don't need parsed headlines either?).

I'm wondering if an orgmode api function could be exposed which does not parse the headlines (but, I guess, does parse something like the file title and archived status)?

[kristijanhusak]

Yeah we can add that. You basically need only the filenames?

[Maltimore]

I'm just starting out with the telescope-orgmode code to be honest. Digging deeper, I found that telescope-orgmode tries to access the 'TITLE' here via get_directive(). Would providing this 'TITLE' be possible without parsing the entire org file (without major changes to orgmode's code)? If not, I can ask the telescope-orgmode maintainer if they would be fine with using just the filename instead of the 'TITLE' (the filename is used as fallback right now).

[seflue]

@Maltimore Thanks for pointing this out. I am using telescope-orgmode daily and are quite annoyed by the long loading times of orgmode.api.load(). So every API addition which could speed up the loading times I would heavily support.

@kristijanhusak How optimized is orgmode.api.load()? I was already wondering, if a faster Org parser could be implemented in another programming language. Another thing, which would help is some kind of streaming API. Telescope supports streaming and having some smart order would help to improve the interactivity of telescope-orgmode.

In general it would be nice, if an API consumer could hand over some arguments to orgmode.api.load to keep parsing just to the needs. Often you don't need to parse the whole headline tree, but only the first or first two levels. For file indexing (e.g. roam files), as @Maltimore pointed out, only the title is required.

This could be also a speed optimization for the agenda, which is also noticeable slow when invoked the first time.

[kristijanhusak]

@seflue since telescope-orgmode is under the nvim-orgmode org, we could just use the internals directly. I'm fairly satisfied with the structure of the internals, and I don't plan to update it any time soon.
Using the internals should be faster, because we are fetching headlines on demand when calling the get_headlines() method, while API preloads all headlines into the headlines property when the file is loaded.

I would suggest to give it a try and see if it helps.

[seflue]

Thanks @kristijanhusak, I will try this out as soon as I have time again. @Maltimore I am currently a bit busy, so if you're impatient, but willing to create a PR, feel free to do so. I would be happy to review it. 🙂

[Maltimore]

Hey, I made a first PR based on this :) nvim-orgmode/telescope-orgmode.nvim#35
It achieves the goal of having instantaneous loading of the org files, but the TITLE is not parsed anymore, so the telescope entries are just based on the filename. I understand that this can be a serious limitation (although I personally don't use the TITLE field), so maybe look at it as a proof-of-concept PR.