force:source:convert performance degradation in SFDX-cli 7.115

[javilchezffdc]

Summary

We're experiencing a big performance degradation after upgrading from 7.75 to 7.115 when running "force:source:convert" command. Using the same version of our code, the times where from ~1 minute to ~20 minutes

Steps To Reproduce:

Having any SFDX repository already in your local machine

1. Install SFDX-cli 7.75
2. Run force:source:convert --outputdir foo --rootdir bar --packagename baz`
   1. Command time around 1 min
3. Upgrade SFDX-cli to 7.115
4. Run force:source:convert --outputdir foo --rootdir bar --packagename baz`
   1. command time around 20 min

Console output

====================================================================================
   SFDX-cli 7.75 - convert-force-app-core
====================================================================================
➜  ffa git:(develop) ✗ sfdx --version         
sfdx-cli/7.75.0 darwin-x64 node-v12.20.1

➜  ffa git:(develop) ✗ ant convert-force-app-core                     
Buildfile: /Users/javilchez/Dev/repos/ffa/build.xml
     [echo] /Users/javilchez/buildtools/artifacts/ffdctools-3.5.7

convert-force-app-core:
     [exec] Source was successfully converted to Metadata API format and written to the location: /Users/javilchez/Dev/repos/ffa/${packaging.mdapi.outputdir}

BUILD SUCCESSFUL
Total time: 46 seconds


====================================================================================
   SFDX-cli 7.115 - convert-force-app-core
====================================================================================
➜  ffa git:(develop) ✗ sfdx --version                          
sfdx-cli/7.115.1 darwin-x64 node-v12.20.1

➜  ffa git:(develop) ✗ ant convert-force-app-core
Buildfile: /Users/javilchez/Dev/repos/ffa/build.xml
     [echo] /Users/javilchez/buildtools/artifacts/ffdctools-3.5.7

convert-force-app-core:
     [exec] Source was successfully converted to Metadata API format and written to the location: /Users/javilchez/Dev/repos/ffa/${packaging.mdapi.outputdir}

BUILD SUCCESSFUL
Total time: 17 minutes 49 seconds

Expected result

No performance degradation are expected after upgrading to a most recent version.

Actual result

force:source:convert performance has drastically worsened

System Information

sfdx-cli/7.75.0 darwin-x64 node-v12.20.1

@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.0 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.9.0 (core)
@oclif/plugin-update 1.3.10 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.4.3 (core)
alias 1.1.2 (core)
analytics 1.12.1 (core)
config 1.1.8 (core)
generator 1.1.3 (core)
salesforcedx 49.13.0 (core)
├─ templates 49.10.0 (core)
├─ custom-metadata 1.0.10 (core)
├─ salesforce-alm 49.12.1 (core)
├─ apex 0.1.0 (core)
└─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core)

sfdx-cli/7.115.1 darwin-x64 node-v12.20.1

@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.3 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.10.1 (core)
@oclif/plugin-update 1.4.0-3 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.10 (core)
apex 0.2.7 (core)
auth 1.7.1 (core)
config 1.2.24 (core)
custom-metadata 1.0.12 (core)
data 0.6.0 (core)
generator 1.2.0 (core)
limits 1.2.1 (core)
org 1.7.0 (core)
salesforce-alm 52.2.6 (core)
schema 1.0.8 (core)
sfdx-cli 7.115.1 (core)
source 1.0.10 (core)
telemetry 1.2.3 (core)
templates 52.1.0 (core)
user 1.4.0 (core)

Additional information

Target convert-force-app-corre is defined as

<target name="convert-force-app-core">
	<execute-sfdx>
		<arg value="force:source:convert"/>
		<arg value="--outputdir"/>
		<arg value="${packaging.mdapi.outputdir}"/>
		<arg value="--rootdir"/>
		<arg value="${force.app.core.dir}"/>
		<arg value="--packagename"/>
		<arg value="${signup.org.packagename}"/>
	</execute-sfdx>
	<copy todir="${packaging.mdapi.outputdir}/objects">
		<fileset dir="${basedir}/forcevelocity/blankobjects"/>
	</copy>
</target>

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[javilchezffdc]

After some more tests, I can confirm the issue is around labels metadata. The rest of metadata are being converted in the same times as they were doing in CLI-7.75 but when converting the labels/CustomLabels.labels-meta.xml file, it takes up to 15 min (when it should be just a few seconds)

[mshanemc]

@javilchezffdc How many labels are we talking about here? Do you have any metadata that you can share that helps reproduce it, other than "create a project with a lot of labels" ?

[javilchezffdc]

@mshanemc , our CustomLabels.labels-meta.xml file contains 7894 labels at the moment.

What's is weird is how 7.75 only needs few seconds to complete the process and the newer version (7.115) takes more than 15 min. Obviously, I'm using the same file in both tests.

[uip-robot-zz]

This issue has been linked to a new work item: W-9990964

[mshanemc]

Did it get better in the last few versions?

[no-response]

This issue has been automatically closed because there has been no response to our request for more information from the original author. Currently, there is not enough information provided for us to take action. Please reply and reopen this issue if you need additional assistance.

[javilchezffdc]

Sorry @mshanemc, I turned off notifications for that issue.

I can confirm the performance is the expected in the latest versions so we can close the issue as fixed. This fixed the issue forcedotcom/source-deploy-retrieve#470

Thanks