SMBJ Performance

[proxymus89]

I noticed that using Windows file explorer I have speeds up to 4 times faster than smbj. why is smbj so slow? I tried to change all the possible parameters of smbj but to no avail. Is there something I'm missing or is it actually a limitation of the library?

[hierynomus]

It highly depends on your method of testing... What types of streams are you using? Are you using the same dialect and encryption standards?

[nashkevichv]

Hi @hierynomus, i can confirm phenomenon @proxymus89 mentioned.
I mounted test share :
net use z: \\HOST\SHARE /user:DOMAIN\USER PASSWORD

and started simple java app
MediaPoolPerformanceTest.java.txt
Below is output.

smbJ

buffer [12288] time [688] ms. speed [53756368] B/s size [36984380]
buffer [12288] time [645] ms. speed [57340124] B/s size [36984380]
buffer [12288] time [654] ms. speed [56551040] B/s size [36984380]
buffer [12288] time [689] ms. speed [53678344] B/s size [36984380]
buffer [12288] time [646] ms. speed [57251364] B/s size [36984380]
buffer [12288] time [624] ms. speed [59269840] B/s size [36984380]
buffer [12288] time [663] ms. speed [55783380] B/s size [36984380]
buffer [12288] time [620] ms. speed [59652228] B/s size [36984380]
buffer [12288] time [635] ms. speed [58243116] B/s size [36984380]
buffer [12288] time [633] ms. speed [58427140] B/s size [36984380]

end of smbJ

local

buffer [16777216] time [349] ms. speed [105972440] B/s size [36984380]
buffer [16777216] time [347] ms. speed [106583224] B/s size [36984380]
buffer [16777216] time [362] ms. speed [102166800] B/s size [36984380]
buffer [16777216] time [349] ms. speed [105972440] B/s size [36984380]
buffer [16777216] time [352] ms. speed [105069256] B/s size [36984380]
buffer [16777216] time [354] ms. speed [104475648] B/s size [36984380]
buffer [16777216] time [352] ms. speed [105069256] B/s size [36984380]
buffer [16777216] time [365] ms. speed [101327072] B/s size [36984380]
buffer [16777216] time [351] ms. speed [105368600] B/s size [36984380]
buffer [16777216] time [354] ms. speed [104475648] B/s size [36984380]

end of local

smbj upload was about 45% slower than upload to mounted share.

What shell we do to improve smbj performance?
Thank you in advance.

[hierynomus]

One thing you should always do is use BufferedInputStream and BufferedOutputStream

Also, have you tried using Smbfiles.copy(sourceFile, share, targetPath, true);, instead of manually opening the file and streaming the bytes?

[nashkevichv]

Thank you for the hint.
Below is output for the SmbFiles.copy(sourceFile, share, targetPath, true) try

smbJ with Smbfiles.copy

time [763] ms. speed [48472320] B/s size [36984380]
time [719] ms. speed [51438636] B/s size [36984380]
time [708] ms. speed [52237824] B/s size [36984380]
time [678] ms. speed [54549236] B/s size [36984380]
time [724] ms. speed [51083400] B/s size [36984380]
time [696] ms. speed [53138476] B/s size [36984380]
time [757] ms. speed [48856512] B/s size [36984380]
time [771] ms. speed [47969364] B/s size [36984380]
time [741] ms. speed [49911444] B/s size [36984380]
time [714] ms. speed [51798852] B/s size [36984380]

smbJ with Smbfiles.copy

overall performance is approximately the same.

[nashkevichv]

I started test with all versions of smbj. Release 0.10.0 was 10% faster than 0.13.0.

On pc with 10Gb network i got the following results :

smbJ

buffer [12288] time [319] ms. speed [115938496] B/s size [36984380]
buffer [12288] time [302] ms. speed [122464832] B/s size [36984380]
buffer [12288] time [300] ms. speed [123281264] B/s size [36984380]
buffer [12288] time [298] ms. speed [124108656] B/s size [36984380]
buffer [12288] time [297] ms. speed [124526528] B/s size [36984380]
buffer [12288] time [290] ms. speed [127532344] B/s size [36984380]
buffer [12288] time [284] ms. speed [130226688] B/s size [36984380]
buffer [12288] time [285] ms. speed [129769760] B/s size [36984380]
buffer [12288] time [283] ms. speed [130686848] B/s size [36984380]
buffer [12288] time [285] ms. speed [129769760] B/s size [36984380]

end of smbJ

smbJ with Smbfiles.copy

time [440] ms. speed [84055408] B/s size [36984380]
time [447] ms. speed [82739104] B/s size [36984380]
time [451] ms. speed [82005272] B/s size [36984380]
time [437] ms. speed [84632448] B/s size [36984380]
time [360] ms. speed [102734392] B/s size [36984380]
time [331] ms. speed [111735288] B/s size [36984380]
time [445] ms. speed [83110968] B/s size [36984380]
time [397] ms. speed [93159648] B/s size [36984380]
time [437] ms. speed [84632448] B/s size [36984380]
time [435] ms. speed [85021560] B/s size [36984380]

smbJ with Smbfiles.copy

local

buffer [16777216] time [77] ms. speed [480316640] B/s size [36984380]
buffer [16777216] time [89] ms. speed [415554848] B/s size [36984380]
buffer [16777216] time [85] ms. speed [435110336] B/s size [36984380]
buffer [16777216] time [93] ms. speed [397681504] B/s size [36984380]
buffer [16777216] time [107] ms. speed [345648416] B/s size [36984380]
buffer [16777216] time [103] ms. speed [359071648] B/s size [36984380]
buffer [16777216] time [83] ms. speed [445594944] B/s size [36984380]
buffer [16777216] time [86] ms. speed [430050944] B/s size [36984380]
buffer [16777216] time [71] ms. speed [520906752] B/s size [36984380]
buffer [16777216] time [92] ms. speed [402004128] B/s size [36984380]

end of local

So copy to mounted share is really 4 times faster than copy with smbj.

[nashkevichv]

Hallo @hierynomus, after some debugging sessions i got a feeling that encryption is a performance killer.

My setup
java : 17
smbj : 0.13.0
file upload, size : 344010288 Bytes

SmbConfig smbConfig = SmbConfig.builder()
.withEncryptData(false)
.withSigningRequired(false)
.withTransportLayerFactory(new AsyncDirectTcpTransportFactory<>())
.withDialects(SMB2Dialect.SMB_3_1_1)
.build()
;
Method AsyncDirectTcpTransport.prepareBufferToSend is called 331 times. Each call lasts about 6 ms. ( 331*6 = 1986 ms just for prepareBufferToSend )
In debugger i can also see that the method AESEngine.encryptBlock is called for each 64k chunk (see stack trace below).

Is it possible to disable encryption somehow?
It seems calling AsyncDirectTcpTransport.prepareBufferToSend asynchronously can also bring some performance boost.

Thank you in advance, Vitali

[hierynomus]

Disabling it is simple :) You can do it in the SmbConfig.... withEncryption(false) is your friend.

[nashkevichv]

I do call SmbConfig..withEncryptData(false) but it changes nothing.
AESEngine.encrypt is called anyway!

[proxymus89]

agree with @nashkevichv

in those last months i've tried every possible combination (different smbj version, different dialects, different envirorments, different smbConfig options, different approach to upload, different file size), I have never achieved anywhere near the performance of Windows file explorer

[nashkevichv]

First of all, @hierynomus, thanks a lot for the SMBJ-library.

As described above we can observe pure SMBJ performance comparing to performance Windows provides by default.

The steps below helped me to improve SMBJ performance.

SmbConfig

final SmbConfig config = SmbConfig.builder()
	.withTransportLayerFactory(new AsyncDirectTcpTransportFactory<>())
     	.build();

By default encryptData = false and signingRequired = false.

1 Gbit network

AsyncDirectTcpTransport solves the problem for 1 G/bit network.
Application test showed average rate 110 - 115 MBytes/s.

10 Gbit network

AsyncDirectTcpTransport did not help. The transfer rate was a little bit higher ( about 140 Mbyres/s ) but far from 10Gbits/s.

After some debugging sessions I found that SMBJ signs data pakages (SMB2WriteRequest).
SmbConfig does not REQUIRE signing but SMBJ simply SIGNS!
Signing takes about 60% of the time. It is 2 times slower than data transfer!

I could not find a way how to disable signing with existing SMBJ api,
but the following trick in the com.hierynomus.smbj.connection.PacketSignatory class effectively disables signing for data packages :

...
private final boolean signDataPackets;

PacketSignatory(SecurityProvider securityProvider, final boolean signDataPackets) {
    this.securityProvider = securityProvider;
    this.signDataPackets = signDataPackets;
  }
  ...
  public SMB2Packet sign(SMB2Packet packet, SecretKey secretKey) {
    if ( secretKey!=null ) 
    {
      if ( packet instanceof SMB2WriteRequest && this.signDataPackets==false )
      {
           return packet;
      }
      return new SignedPacketWrapper(packet, secretKey);
    } 
    logger.debug("Not wrapping {} as signed, as no key is set.", packet.getHeader().getMessage());
    return packet;
  }  

After this change the trasfer rate was 650 - 750 Mbytes/s.
This is a little bit faster than windows does.

How does Windows work?

In my corporate environment after command

net user x: \\server\share 

Windows does neither encryption nor signing.
To activate encryption and signing a /REQUIREPRIVACY switch is required for the net use command.

If '/REQUIREPRIVACY' is set, then Windows performs significantly slower ( ~ 200-250 Mbytes/s ).

[tooptoop4]

amazing! can u raise a PR @nashkevichv ?

[tooptoop4]

@hierynomus would you be keen on this?

[nashkevichv]

Yes, i would. Sent with [Proton Mail](https://proton.me/) secure email.

…

On Thursday, May 2nd, 2024 at 00:34, tooptoop4 ***@***.***> wrote: ***@***.***(https://github.com/hierynomus) would you be keen on this? — Reply to this email directly, [view it on GitHub](#817 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/A77OCHFQULBSD5WRRUE3G23ZAFUY3AVCNFSM6AAAAABD5URJIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBZGI2DCMZUGY). You are receiving this because you were mentioned.Message ID: ***@***.***>