Knowledge - syncing of pdf files results in duplicate folder names being presented in the file tree structure.

[sangee2004]

Steps to reproduce the problem:

1. Create an agent and add knowledge files from cry.org

2. Once sync is completed, notice there is duplicate entries for www.cry.org and there is cry.org with just downloads folder.

Also notice that this folder cry.org ( and all subfolder under it) never shows "Included" state even after it was included ( and files have been ingested successfully).

Expected Behavior:
All pdf files under download folder should be consolidated in this case to be under one www.cry.org folder.

[StrongMonkey]

@sangee2004 these looks like different file sets, how do you tell these are duplicated?

[sangee2004]

@StrongMonkey They are different set of files.
Issue here is why is there 2 download folders under cry.org folder.
It is confusing to see cry.org and www.cry.org as 2 different folder structures in this case ?

[cjellick]

@StrongMonkey why do we have cry.org > cry.org > downloads instead of just cry.org > downloads? Shouldn't it just be cry.org > downloads?

And why do we have cry.org > www.cry.org > downloads and then also have www.cry.org at the top level? shouldnt we just have www.cry.org > downloads at the top-level?

[StrongMonkey]

These are external PDF files. The way we store pdf files is that we store it under the folder of base url(in this case cry.org).
So if there is a pdf file with link like cry.org/download/foo.pdf it will show up as path cry.org/cry.org/download/foo.pdf. The reason for doing that is I want to group all the external pdf files under the base folder to better organize it.

For this case, we can just merge the folder if it is duplicated.

[sangee2004]

Tested with latest build.

We only see 1 download directory under cry.org

@StrongMonkey Can we have the cry.org/downloads directory wrapped into www.cry.org/downloads directory in this case ?

[StrongMonkey]

these are likely two different domains. I think it is fine to separate those.