UnixPB: Add latest commit SHA to ansible.log file

[Haroon-Khel]

• commit message has one of the standard prefixes
• faq.md updated if appropriate
• other documentation is changed or added (if applicable)
• playbook changes run through VPC or QPC (if you have access)
• VPC/QPC not applicable for this PR
• for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

ref #2745

Adds the latest commit SHA to the logs stored on the remote machine in the jenkins directory. Thankfully, the default working directory of ansible seems to be the directory of the playbook, in this case the main.yml file. Which means git rev-parse HEAD will always be run in the correct directory.

Tested from my workstation and https://awx2.adoptopenjdk.net/#/jobs/playbook/485?job_search=page_size:20;order_by:-finished;not__launch_type:sync (includes debug tasks which I have removed)

[Haroon-Khel]

Log entries now look like this
2022-09-23 04:39:57 d99305e41c4c11c085a8fc4fc195f7d826bc362a

[Haroon-Khel]

VPC https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/1544/console

[sxa]

Code looks ok but we really need to remove the duplication in the implementations.
Can we get this stored in the normal log location on the machine and remove the duplication as mentioned in #2746 (comment)

[Haroon-Khel]

Log path is now /var/log/ansible.log. File is readable by the jenkins user incase test tools need to pick it up

[Haroon-Khel]

#10 816.2 TASK [logs : Get Latest git commit SHA] ****************************************
#10 816.5 fatal: [localhost -> localhost]: FAILED! => {"changed": true, "cmd": "git rev-parse HEAD", "delta": "0:00:00.003055", "end": "2022-09-26 12:53:07.029650", "msg": "non-zero return code", "rc": 128, "start": "2022-09-26 12:53:07.026595", "stderr": "fatal: not a git repository (or any of the parent directories): .git", "stderr_lines": ["fatal: not a git repository (or any of the parent directories): .git"], "stdout": "", "stdout_lines": []}

Failing on the alpine build. A bit strange since when I was testing it on a mac and linux machine, ansible's working directory was always inside the repo. Investigating

[Haroon-Khel]

It might be because the alpine job copies the infra repo into the image and runs the playbook that way

#8 [3/5] COPY . /ansible
#8 DONE 0.7s

Im guessing this copy command doesnt copy the .git folder. The macos build does not have this problem, presumably because it clones infra repo instead of copying it

[Haroon-Khel]

The centos 6 build fails aswell, again because it copies the infra repo into the image instead of git cloning. Need to think of a fix for this use case

[Haroon-Khel]

I think the vpc clones the repo and runs it on the boxes, instead of copying into the boxes and running it locally, so I think the vpc will not hit this error
https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/1546/console
We'll see when it finishes running

[Haroon-Khel]

I've added a debug message to see where ansible's working directory is on the local host. Should provide insight
vpc: https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/1547/console

[Haroon-Khel]

Adding ignore_errors onto the git sha task allows the date and time to still be added to the log without the commit sha in the event of not being able to retrieve the sha
Tested on test-marist-rhel7-s390x-2 (latest entry is without commit sha)

[root@test-marist-rhel7-s390x-2 ~]# cat /var/log/ansible.log 
2022-09-26 07:36:55 d7735cddd36ec8399d18061497a251543b3e47ea
2022-09-26 08:30:49 7e4e33b099b35e388d9a5ee6bc0c679165ae35d8
2022-09-26 08:31:52 7e4e33b099b35e388d9a5ee6bc0c679165ae35d8
2022-09-26 08:32:23 7e4e33b099b35e388d9a5ee6bc0c679165ae35d8
2022-09-26 11:46:15 

[Haroon-Khel]

@sxa ping for review

[sxa]

Some thoughts regarding options here:

• Determine the SHA in the dockerfile and pass that in as a parameter somehow
• Have a line in the dockerfile that writes the line directly to /var/log/ansible inside the generated image (bear in mind it would be highly unusual for the file to ever need to be updated inside a docker image, since we would normally just regenerate the docker image from scratch)
• Just copy from one level up (e.g. copy .. to /infrastructure and run things from /infrasdtructure/ansible

[zdtsw]

Guess it is easier to copy one level up, drawback is it could be slower and need more disk space.

it is better to get the SHA and pass as env to "docker build" then either add as a new instruction in dockerfile to dump it directly to ansible.log or pass in to ansible-playbook -e but in this case, need to handle for both GH action and PVC run

[Haroon-Khel]

Just copy from one level up (e.g. copy .. to /infrastructure and run things from /infrasdtructure/ansible

I was thinking of this, I think it is best for this particular use case

[Haroon-Khel]

I'm comfortable using this solution for the vpc machines, #2751 (comment), since those machines get deleted after the playbook run

[Haroon-Khel]

Still was not able to get the commit SHA despite grabbing the top level directory of the repo,

#10 1991.0 TASK [logs : Get Latest git commit SHA] ****************************************
#10 1991.7 fatal: [localhost -> localhost]: FAILED! => {"changed": true, "cmd": "git rev-parse HEAD", "delta": "0:00:00.532124", "end": "2022-09-26 17:23:20.785741", "msg": "non-zero return code", "rc": 128, "start": "2022-09-26 17:23:20.253617", "stderr": "fatal: Not a git repository (or any of the parent directories): .git", "stderr_lines": ["fatal: Not a git repository (or any of the parent directories): .git"], "stdout": "", "stdout_lines": []}
#10 1991.7 ...ignoring
#10 1991.8 

I've added some debug tasks to get a better idea of the ls output of the /infrastructure and /infrastructure/ansible directories

[Haroon-Khel]

is the "docker build" executed in the "ansble" folder?
then try "docker build -f dockerfile .."

It seems to kick it off from the top level directory of the repo, from the way it calls Dockerfile.Alpine3

/usr/bin/docker buildx build --cache-from type=registry,ref=adoptopenjdk/alpine3_build_image:latest --cache-to type=inline --file ./ansible/docker/Dockerfile.Alpine3

[Haroon-Khel]

Latest commit attempts to pass the SHA into the dockerfile using the github workflows standard variable $GITHUB_SHA, https://docs.github.com/en/github-ae@latest/actions/learn-github-actions/environment-variables

If this doesnt work, I think I have to pass it into the dockerfile from the build.yml, but lets see

[zdtsw]

Latest commit attempts to pass the SHA into the dockerfile using the github workflows standard variable $GITHUB_SHA, https://docs.github.com/en/github-ae@latest/actions/learn-github-actions/environment-variables

If this doesnt work, I think I have to pass it into the dockerfile from the build.yml, but lets see

dont you need pass GITHUB_SHA to the "docker build" first?

[Haroon-Khel]

dont you need pass GITHUB_SHA to the "docker build" first?

Yeah I've done that with the latest commit. Though I did read some docs saying that it could be accessed straight from the dockerfile wihtout needing to be passed but who knows

[Haroon-Khel]

Still getting The workflow is not valid. .github/workflows/build.yml (Line: 46, Col: 9): A sequence was not expected .github/workflows/build.yml (Line: 67, Col: 9): A sequence was not expected

I must not be reading the docs correctly 😅

[Haroon-Khel]

Ayyy the SHA got passed in properly
RUN set -eux; cd /infrastructure; ansible-playbook -i ansible/hosts ansible/playbooks/AdoptOpenJDK_Unix_Playbook/main.yml -e "git_sha=ddf2a3f33fc16345c390815143555b2e1e205c81" --skip-

[Haroon-Khel]

#10 871.0 TASK [logs : Update Log File] **************************************************
#10 871.6 changed: [localhost]

Looks good

[Haroon-Khel]

The linter's acting up again

Error: name Jinja templates should only be at the end of 'name'
Error: name Jinja templates should only be at the end of 'name'
Error: name Jinja templates should only be at the end of 'name'
Error: name Jinja templates should only be at the end of 'name'

[Haroon-Khel]

Logs now look like this

bash-3.2# cat /var/log/ansible.log 
Start 2022-09-27 22:41:52 c0e9e8bc48bd72055123d8bde59a549e080ddf93
End 2022-09-27 22:42:14 c0e9e8bc48bd72055123d8bde59a549e080ddf93

Tested (and works) on linux, solaris and macos

[sxa]

Looks good! Thanks for doing these last changes :-)