ggml : remove src0 and src1 from ggml_tensor and rename opt to src

[ggerganov]

ggml/include/ggml/ggml.h

Lines 415 to 420 in b98cd86

	struct ggml_tensor * grad;
	struct ggml_tensor * src0;
	struct ggml_tensor * src1;
	struct ggml_tensor * opt[GGML_MAX_OPT];

[slaren]

I would suggest considering using an array for all src nodes instead. This will simplify the code when we need to look at the list of parents of a node. This already happens when printing to dot, internally in ggml-cuda, and will also happen when we implement the improved memory management. Having each parent in a different variable forces us to look at each one separately instead of just looping over an array.

[ggerganov]

I updated the issue

[nullhook]

How would the new src indexing work in cases where a 4D tensor is created, but opt[0-2] is being written? It would be out of bounds. for ex: https://github.com/ggerganov/ggml/blob/master/src/ggml.c#L7268

[slaren]

@nullhook I am not sure if I understand what you mean, the total number of source tensors available to ops will be the same, and can be increased as needed.

For instance, see how it is being done in #346:

ggml/src/ggml.c

Lines 7260 to 7266 in b7fd1a4

	result->op = GGML_OP_FLASH_FF;
	result->grad = is_node ? ggml_dup_tensor(ctx, result) : NULL;
	result->src[0] = a;
	result->src[1] = b0;
	result->src[2] = b1;
	result->src[3] = c0;
	result->src[4] = c1;

[nullhook]

I was looking at the PR linked to this issue. So it seems like GGML_MAX_OPT needed to be increased

[ggerganov]

Fixed via ggerganov/llama.cpp#2178