Consider adding operators to FFI `Pointer` objects and deprecate/remove not-very-idiomatic `elementAt`

[mkustermann]

If we have a Pointer<Uint8> and would like to get one that is 1 bytes further we can

Pointer<Uint8> p;

final p1 = p.elementAt(1);
final p1 = Pointer.fromAddress(p.address + 1);

None of which seem very nice.

Also the name elementAt would suggest one actually gets the element at a certain offset, which it doesn't, it gets the address of the element at an offset.

In C one would simply

uint8_t* p = ...;

uint8_t* p10 = p + 1;

We should consider adding extension methods such as

extension on Pointer<Uint8> {
  Pointer<Uint8> operator +(int offset) =>
      Pointer.fromAddress(address + offset);
}

then the example will turn into

Pointer<Uint8> p;
final p10 = p + 1;

Even the following would work out of the box:

Pointer<Uint8> p;
p++;

If we do so, there may no longer be a need to have elementAt and could deprecate & remove it.

There's a question whether we should support operator- and if so, whether it should take two pointers and return the difference as an integer or it should take an int and return a pointer minus the offset.

/cc @dcharkes opinions?

[dcharkes]

I am in favor of renaming elementAt to operator +.

Maybe obvious, but not mentioned above, the address increments would be in terms of sizeOf<element type>(). (To make it consistent with pointer arithmetic in C.)

We should consider adding extension methods such as

For AbiSpecificIntegers and compounds we need some CFE magic, as we currently do for elementAt as well.

Currently, the extension methods for the types with known sizes are in the patch file and invisible for users, we could move these to be in the non-patchfile.

There's a question whether we should support operator- and if so, whether it should take two pointers and return the difference as an integer or it should take an int and return a pointer minus the offset.

In these kind of situations it's really painful that we don't have overloading in Dart, then we could just have both. Maybe Dart should get some kind of overloading support for static methods that cannot be called dynamically. dart-lang/language#1122 dart-lang/language#2456 cc @lrhn

[lrhn]

What we did with DateTime is to let - match + and take a duration, and have a separate difference method to give the difference between two DateTimes.

That is, the same strategy as every other time: different names instead of overloading.

I'd do the same here, so += and ++ are matched by -= and --.
(If you have multiple subtraction operations, use - for the one that returns the same type as the receiver, so the composite operations work.)

[shikharish]

Hello, I would like to work on this issue.

[shikharish]

Hello, can someone help me understand how the extension method should be implemented.

from ffi.dart file

This is how I think it should be implemented(for + for Pointer<Uint8>).

[shikharish]

@dcharkes @mkustermann
Any guidance please?

[dcharkes]

Any guidance please?

One way to find the relevant places to update is to follow the git blame trail on elementAt:

• https://dart-review.googlesource.com/c/sdk/+/276102

Most of these files will have to be edited:

• sdk/lib/ffi/ffi.dart
  • The public API
• runtime/tools/ffi/sdk_lib_ffi_generator.dart
  • The code in the public API is generated by this file.
• sdk/lib/_internal/vm/lib/ffi_patch.dart
  • The implementation for fixed size integers is generated by the same file into this file.
• pkg/vm/lib/transformations/ffi/use_sites.dart
  • Contains the implementation for AbiSpecificInteger elementAt, struct elementAt and union elementAt.
• pkg/vm/lib/transformations/ffi/common.dart
  • Contains some imports for the above file.

Also, there are tests in

• tests/ffi/
  • search for elementAt(. All of these need to be duplicated for +, -, +=, and -=.

You will need to build the SDK locally to test all of this.
Please take a look at https://github.com/dart-lang/sdk/blob/main/CONTRIBUTING.md

[shikharish]

Thank you very much.
I have set up my local development environment and tested for + operator(not rigorously though yet). I will make all the necessary changes in the files.

Regarding - operator,

I'd do the same here, so += and ++ are matched by -= and --. (If you have multiple subtraction operations, use - for the one that returns the same type as the receiver, so the composite operations work.)

It is not clear to me how + would be matched by -. Can you please elaborate or share some readings related to this?

[dcharkes]

It is not clear to me how + would be matched by -. Can you please elaborate or share some readings related to this?

I was referring to:

What we did with DateTime is to let - match + and take a duration, and have a separate difference method to give the difference between two DateTimes.

That is, the same strategy as every other time: different names instead of overloading.

I'd do the same here, so += and ++ are matched by -= and --. (If you have multiple subtraction operations, use - for the one that returns the same type as the receiver, so the composite operations work.)

So pointer += -4 would be the same as pointer -= 4 and pointer + -4 the same as pointer - 4.

[shikharish]

But how would that be implemented? Because for operator -, if we do ptr1 - 4 then it would return a pointer but ptr1 - ptr2 would return an int.