Remove heap allocation from netrecv_from

[rafaelsamenezes]

Feature Request

What feature are you suggesting?

Overview:

• Remove the heap allocation from the netrecv_from method or expose a function that skips the allocation.

Smaller Details:

• Alter the function so it does not need an extra allocation.
• An alternative would be to expose a new function that avoids such allocation

Nature of Request:

• Change

Why would this feature be useful?

Projects that rely on arena allocators can take up the full mem2 space beforehand. This makes network request fail with no memory available even though the provided buffer could be used directly. I did not find any alternatives, please let me known if I am missing something.

For my use case, there is an issue of using retroachievements with retroarch: libretro/RetroArch#16184 (comment)

In my tests, replacing the netrecv_from seems to fix the problem.

[DacoTaco]

euh, i think the correct fix here is that you don't write into the allocated heap of the network stack?
it allocates the stuff using net_alloc ( and free's it after) into the network heap, which is created when init'ing network

how did you end up making that return a fail on allocation without using that specific heap?

EDIT : we are also looking at how its possible you are hitting the limit. are your packets size > 64k ?

[rafaelsamenezes]

euh, i think the correct fix here is that you don't write into the allocated heap of the network stack?
it allocates the stuff using net_alloc ( and free's it after) into the network heap, which is created when init'ing network

how did you end up making that return a fail on allocation without using that specific heap?

Ow, I see. Thanks for the explanation, i misunderstood how the implementation works.

EDIT : we are also looking at how its possible you are hitting the limit. are your packets size > 64k ?

Yeah, I recorded some recv calls with 65544 of length. I also tried to split the request into multiple smaller requests, which helped in some cases. I will try again now that i know about the hard limit.

[DacoTaco]

the network heap is 64k in size, so ye if you request packets to be bigger than that it might bump into the error.

see how far you can get with that information, and let us know. there might be some things we can do to help the issue, but first see how far you can get in splitting it up! :)

[mardy]

Maybe you could add a net_set_heap(void *heap, size_t size) function to be called before initializing the network, that will set the heap to a client-supplied buffer.

[rafaelsamenezes]

Just an update, I tested replacing the direct net_recv calls with:

static inline s32 recv_split(s32 s, void *mem, s32 len, u32 flags) {
  const s32 LIMIT = 48 * 1024;
  s32 max_tries = 1000;
  s32 res = 0;

  while(len) {
    s32 recv_length = len < LIMIT ? len : LIMIT;
    s32 recv_call_res = net_recv(s, mem, recv_length, flags);
 
    if (isagain(recv_call_res) && max_tries--)
      continue;    
    else if(recv_call_res < 0)
      return recv_call_res;
    
    mem = (char *)mem + recv_length;
    len -= recv_length;
    res += recv_call_res;
  }
  
  return res;
}

This seems to solve the allocation issue. Thanks for your support!

[DacoTaco]

@rafaelsamenezes : do you have any code i can test a libogc change with? im currently working on doing the netrecv_from in blocks, so the user code doesn't need to do it in blocks like you did :)

[rafaelsamenezes]

@rafaelsamenezes : do you have any code i can test a libogc change with? im currently working on doing the netrecv_from in blocks, so the user code doesn't need to do it in blocks like you did :)

Ah that's great. The only example that I have is Retroarch, the build process is a bit convoluted though:

1. Clone retraorch (recursively)
2. Build salamander launcher: run make -f Makefile.wii.salamander EXTERNAL_LIBOGC=1 GX_PTHREAD_LEGACY=0 remember to check DEVKITPPC environment var. The external libogc is so makefile use your system version and not the one in the RA repo.
3. Use libretro-super to build some cores: https://docs.libretro.com/development/retroarch/compilation/wii/#core-compilation and copy one of the cores into RA main folder renaming it to libretro_wii.a.
4. Build RA: make -f Makefile.wii -j$(getconf _NPROCESSORS_ONLN) EXTERNAL_LIBOGC=1 GX_PTHREAD_LEGACY=0
5. The file will retroarch_wii.dol be available. This process needs to be done for every core .

Then start retroarch and open up a game that has achievements. For my tests I use Chrono Trigger with Snes9x2005 core.

If you prefer, I can do this test later today (or even generate the build) with your branch.

[DacoTaco]

that kinda sounds like a lot of setup haha.
do you mind building the libogc branch and testing it?
on my end it worked in priiloader

[DacoTaco]

@rafaelsamenezes : did you test it?

[DacoTaco]

closing due to no response, and because branch is merged