[BUG] Windows Password never available on Jenkins after recent PRs

[peterzhuamazon]

After #289 #291 #292 #293 #294 #295 #296, windows stuck on not able to boot and get password.

EC2 (Amazon_ec2_cloud) - jenkinsAgentNode-Jenkins-Agent-Windows2019-X64-C524xlarge-Single-Host
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.
Waiting for password to be available. Sleeping 10s.

[peterzhuamazon]

Seems like able to connect here:

Connecting to (10.0.101.113) with WinRM as Administrator

Wonder why it is stucking right now.

[peterzhuamazon]

Right after attempt to connect it failed:

<pre id="out"><div>
</div><div>Waiting for WinRM to come up. Sleeping 10s.
</div><div>




<h2>HTTP ERROR 404 Not Found</h2>


URI: | /manage/computer/EC2%20%28Amazon%5Fec2%5Fcloud%29%20%2D%20jenkinsAgentNode%2DJenkins%2DAgent%2DWindows2019%2DX64%2DC524xlarge%2DSingle%2DHost%20%28i%2D019b97c099c560adc%29/logText/progressiveHtml
-- | --
404
Not Found
Stapler

</div></pre>

[peterzhuamazon]

Manually remove queue:

[hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6315, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6330, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6331, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6332, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6333, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6334, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6335, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6337, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6373, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6374, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6397, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6402, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6403, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6404, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6415, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6419, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6430, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6431, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6436, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6441, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6442, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6443, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6446, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6447, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6448, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6449, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6450, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6451, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6452, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6453, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6454, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6455, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6456, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6461, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6475, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6493, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6495, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6499, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6503, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6533, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@<>[gradle-check]:6534, hudson.model.Queue$BlockedItem:org.jenkinsci.plugins.workflow.job.WorkflowJob@a<>[gradle-check]:6535]

Aborting build #16445
Aborting build #16440
Aborting build #16434
Aborting build #16432
Aborting build #16430
Aborting build #16429
Aborting build #16428
Aborting build #16427
Aborting build #16424
Aborting build #16422
Aborting build #16417
Aborting build #16416
Aborting build #16412
Aborting build #16406
Aborting build #16391
Aborting build #16378
Aborting build #16361
Aborting build #16357
Aborting build #16348
Aborting build #16343
All running builds of job 'gradle-check' aborted.

[peterzhuamazon]

Switch all the agents on build.ci.opensearch.org for windows back to 2023/05/19 versions.

[peterzhuamazon]

The above old ami confirms to connect with Jenkins

Waiting for password to be available. Sleeping 10s.
Connecting to (10.0.103.87) with WinRM as Administrator
Waiting for WinRM to come up. Sleeping 10s.
Waiting for WinRM to come up. Sleeping 10s.
Waiting for WinRM to come up. Sleeping 10s.
Connected with WinRM.
Creating tmp directory if it does not exist
Executing init script

C:\Users\Administrator>echo
ECHO is on.
init script ran successfully
remoting.jar sent remotely. Bootstrapping it
Launching via WinRM:java  -jar C:\Windows\Temp\remoting.jar -workDir C:/Users/Administrator/jenkins

Will check 2023/05/25 troubled AMI on personal account.

[peterzhuamazon]

Test running normal windows without any issues on connection, very weird.