Compilation database guessing

[Hylen]

No description provided.

[Hylen]

Here is the guessing, as you asked for. I added boost filesystem to be able to produce PoC fast. If you don't want to add it as a dependency, I guess we'll have to re-implement whatever is needed for the guessing.

If it were up to me, I'd gladly use dependencies such as boost filesystem. The benefits are many, we would have less code to maintain, and it would be cross platform by default. I'd like to hear your motivation for not wanting to add such dependencies.

[Sarcasm]

Here is the guessing, as you asked for. I added boost filesystem to be able to produce PoC fast.

I don't think calcScore() is quite correct. You should probably return the number of directory component in the common prefix of the two file compared. There is no reason for /a/foo/bar/baz.cpp to have a better score against /b/foo/bar/qux.cpp than /a/fu.cpp. I'm may have misunderstood the algo.

For the levenstein distance, as much as I would like to have a use for this algorithm, I don't think it is much relevant the way it is used. Maybe it should work only on the filename and not the subpath before the filename.

The guessing algorithms does not look useful to me if headers and source file are in different sub-trees.

For example:

• lib/Support/Mutex.cpp
• include/llvm/Support/Mutex.h

If it were up to me, I'd gladly use dependencies such as boost filesystem. The benefits are many, we would have less code to maintain, and it would be cross platform by default. I'd like to hear your motivation for not wanting to add such dependencies.

I like the simplicity of install of irony-mode. Installing boost can be a pain (e.g: on Windows). For such a little program I don't think it is worth it. Also I think the guessing can be implemented as a simple python script, does not need to be part of irony-server. Python will have everything by default available, JSON, filesystem abstraction. People without python will still be able to use irony-mode. On the other hand, it would be nice to have irony-mode be C++ only I guess.

[Hylen]

I don't think calcScore() is quite correct. You should probably return the number of directory component in the common prefix of the two file compared. There is no reason for /a/foo/bar/baz.cpp to have a better score against /b/foo/bar/qux.cpp than /a/fu.cpp. I'm may have misunderstood the algo.

It is written like that for exactly the reason you mentioned:

The guessing algorithms does not look useful to me if headers and source file are in different sub-trees.

For example:

lib/Support/Mutex.cpp
include/llvm/Support/Mutex.h

If headers and sources are in different sub-trees, this example would still score 1 with the calcScore algorithm. That is why I calculate it backwards. I think it is less common to have headers and sources in the same source tree but not in the same directory. In this example, the guessing would be right from the stem comparison though.

I like the simplicity of install of irony-mode. Installing boost can be a pain (e.g: on Windows). For such a little program I don't think it is worth it. Also I think the guessing can be implemented as a simple python script, does not need to be part of irony-server. Python will have everything by default available, JSON, filesystem abstraction. People without python will still be able to use irony-mode. On the other hand, it would be nice to have irony-mode be C++ only I guess.

I agree that is an advantage. One solution is to make dependencies like this optional, and only use enable features that relies on them if boost (or whatever else is needed) is detected on the system.

[Hylen]

For the levenstein distance, as much as I would like to have a use for this algorithm, I don't think it is much relevant the way it is used. Maybe it should work only on the filename and not the subpath before the filename.

I agree that levenshtein distance maybe isn't perfect here, maybe longest common substring would make more sense. It does work on only the filenames though, and I think it's probably good enough.

[Sarcasm]

lib/Support/Mutex.cpp
include/llvm/Support/Mutex.h

If headers and sources are in different sub-trees, this example would still score 1 with the calcScore algorithm. That is why I calculate it backwards. I think it is less common to have headers and sources in the same source tree but not in the same directory. In this example, the guessing would be right from the stem comparison though.

Interesting. I still think that the prefix matter.
For example:

/home/foo/llvm/include/llvm/Support/Mutex.h // srcPath

/home/foo/llvm/projects/libcxx/src/utility.cpp // score == 3 ?
/home/foo/llvm/include/llvm/lib/Support/Mutex.cpp // score == 1?

I agree that levenshtein distance maybe isn't perfect here, maybe longest common substring would make more sense. It does work on only the filenames though, and I think it's probably good enough.

Indeed, filename() I see filename now.
I guess we could also do the comparison without the file extension.

[Hylen]

Interesting. I still think that the prefix matter.
For example:

/home/foo/llvm/include/llvm/Support/Mutex.h // srcPath

/home/foo/llvm/projects/libcxx/src/utility.cpp // score == 3 ?
/home/foo/llvm/include/llvm/lib/Support/Mutex.cpp // score == 1?

I'm counting backwards in calcDirScore, so it would be

/home/foo/llvm/include/llvm/Support/Mutex.h // srcPath

/home/foo/llvm/projects/libcxx/src/utility.cpp // score == 0
/home/foo/llvm/include/llvm/lib/Support/Mutex.cpp // score == 1

Indeed, filename() I see filename now.
I guess we could also do the comparison without the file extension.

Yes, that makes sense.

[Sarcasm]

Ok, I understand calcScore better now, it's only about the common suffix path indeed.

[Sarcasm]

Something of interest: https://github.com/randomphrase/ede-compdb/tree/db2539738f5f3f279572252a376f4d5d5d6d572f#header-files

[Hylen]

Very interesting indeed. I will look into it when I have more time. Thanks for sharing!