No definitions found when declared in "wrapped declarations"

[ksen-lin]

Hi!
I found that elixir reports "no definitions" for some struct members, though it shows those places as references.

As i see it happens when fields are declared using a macro, e.g. in struct sk_buff https://elixir.bootlin.com/linux/v6.12.6/source/include/linux/skbuff.h#L950:

Looks somewhat similar to recognizing sys_* functions declared with SYSCALL_DEFINE* (thanks for that btw!)

[tleb]

Thanks for the report! Let's look at what ctags is reporting. It is the tool we use under the hood to parse C files.

⟩ curl 'https://elixir.bootlin.com/linux/v6.12.6/source/include/linux/skbuff.h?raw' > /tmp/skbuff.h
⟩ # grep for one specific field in the macro body
⟩ ctags -x --kinds-c=+p+x --extras='-{anonymous}' /tmp/skbuff.h | grep ignore_df

This is not surprising: the struct_group() macro could do anything. To know that it outputs the content of its second argument would be a massive assumption.

---

ctags supports being given macro implementations using -D, which sort of works for our usecase:

⟩ ctags -x --kinds-c=+p+x --extras='-{anonymous}' -D "struct_group(A, B)=B" /tmp/skbuff.h | grep ignore_df
ignore_df        member     1070 /tmp/skbuff.h    ); /* end headers group */
⟩ # yes but line number reported is the end of the macro:
⟩ grep --line-number '\signore_df' /tmp/skbuff.h
956:	__u8			ignore_df:1;

@masatake: Hi! I guess this is an inherent pitfall of C textual macros. Can you confirm or infirm? Any trick to work around the line-number issue? Thanks!

---

We could use ctags -D for ENTRY() and SYSCALL_DEFINE*() (that you noticed) to remove our Perl edge-cases. That should be part of #298.

[masatake]

I'll look into this issue this weekend.

[masatake]

@tleb, Your investigation is correct. You must define struct_group with -D option.

I have been exploring ways to make ctags find definitions from the given source tree so that you don't have to specify the definition manually with the -D option. It will take a long time to get what I want.

On the other hand, the line-number issue may be solved by improving the CPreProcessor macro expander implemented in ctags. I have just started on this topic.

[tleb]

I have been exploring ways to make ctags find definitions from the given source tree so that you don't have to specify the definition manually with the -D option. It will take a long time to get what I want.

We probably wouldn't use that feature as part of Elixir: we parse Git blobs one at a time and don't have a full kernel tree available. Ie we do git cat-blob ... | ctags -x .... We cannot pass headers to ctags for additional context.

---

Do you really want to fix this line-number reporting inside ctags? I'd understand if you say this is out of scope.

---

Workaround idea: we could post-process the output from ctags by ourselves doing something like the pseudocode below.

file = ...
lines = file.split('\n')
ctag_tokens = call_process('ctags ... -D struct_group(A,B)=B ...')
result_tokens = []
for token in ctag_tokens:
	if token.string in lines[token.lineno]:
		# If the token appears on the reported line, all fine
		result_tokens.append(token)
		continue

	# Token does not appear on the reported line, look at previous lines:
	for i in range(token.lineno-1, 0, -1):
		if token.string in lines[token.lineno]:
			# Edit the token reported line number
			token.lineno = i
			result_tokens.append(token)
			break

How safe is this? Are there cases were this wouldn't work? Maybe because identifiers can be defined using some weird syntax where string matching wouldn't work?

[masatake]

On the other hand, the line-number issue may be solved by improving the CPreProcessor macro expander implemented in ctags. I have just started on this topic.

After implementing my idea, I found it was a naive idea; the line numbers could not be adjusted.
However, as a side effect, the implementation fixes universal-ctags/ctags#4018 .

I will look over your workaround in the next time slot.

[masatake]

After writing the last comment, I got one more idea stacked on the original idea.

[tleb]

@masatake: feel free to ping us (@fstachura & I) if you need help testing. Thanks!

[masatake]

In the latest code of Univeral Ctags, I fixed universal-ctags/ctags#4018 . It is the step for fixing
The fix is the step for fixing yet another line number issue reported in #379 (comment) .