2026-04-13 20:57:08.070445 | Job console starting 2026-04-13 20:57:08.106022 | Updating git repos 2026-04-13 20:57:08.288773 | Cloning repos into workspace 2026-04-13 20:57:09.230021 | Restoring repo states 2026-04-13 20:57:09.249822 | Merging changes 2026-04-13 20:57:11.284984 | Checking out repos 2026-04-13 20:57:11.549650 | Preparing playbooks 2026-04-13 20:57:22.068605 | Running Ansible setup 2026-04-13 20:57:26.368334 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-04-13 20:57:27.038744 | 2026-04-13 20:57:27.039220 | PLAY [localhost] 2026-04-13 20:57:27.047397 | 2026-04-13 20:57:27.047483 | TASK [Gathering Facts] 2026-04-13 20:57:28.027960 | localhost | ok 2026-04-13 20:57:28.036615 | 2026-04-13 20:57:28.036795 | TASK [Setup log path fact] 2026-04-13 20:57:28.054460 | localhost | ok 2026-04-13 20:57:28.067037 | 2026-04-13 20:57:28.067186 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-04-13 20:57:28.095205 | localhost | ok 2026-04-13 20:57:28.104301 | 2026-04-13 20:57:28.104457 | TASK [emit-job-header : Print job information] 2026-04-13 20:57:28.143152 | # Job Information 2026-04-13 20:57:28.143347 | Ansible Version: 2.16.16 2026-04-13 20:57:28.143393 | Job: atmosphere-molecule-csi-rbd 2026-04-13 20:57:28.143424 | Pipeline: check 2026-04-13 20:57:28.143454 | Executor: 0a8996d2b663 2026-04-13 20:57:28.143482 | Triggered by: https://github.com/vexxhost/atmosphere/pull/3818 2026-04-13 20:57:28.143514 | Event ID: 339c6a70-377b-11f1-98fb-da2a05131e59 2026-04-13 20:57:28.147680 | 2026-04-13 20:57:28.147776 | LOOP [emit-job-header : Print node information] 2026-04-13 20:57:28.241787 | localhost | ok: 2026-04-13 20:57:28.242230 | localhost | # Node Information 2026-04-13 20:57:28.242263 | localhost | Inventory Hostname: instance 2026-04-13 20:57:28.242287 | localhost | Hostname: np0000165809 2026-04-13 20:57:28.242308 | localhost | Username: zuul 2026-04-13 20:57:28.242331 | localhost | Distro: Ubuntu 22.04 2026-04-13 20:57:28.242352 | localhost | Provider: yul1 2026-04-13 20:57:28.242371 | localhost | Region: ca-ymq-1 2026-04-13 20:57:28.242391 | localhost | Label: ubuntu-jammy 2026-04-13 20:57:28.242411 | localhost | Product Name: OpenStack Nova 2026-04-13 20:57:28.242431 | localhost | Interface IP: 199.19.213.236 2026-04-13 20:57:28.250380 | 2026-04-13 20:57:28.250511 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-04-13 20:57:28.762832 | localhost -> localhost | changed 2026-04-13 20:57:28.770424 | 2026-04-13 20:57:28.770531 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-04-13 20:57:29.959081 | localhost -> localhost | changed 2026-04-13 20:57:30.099558 | 2026-04-13 20:57:30.099758 | PLAY [all] 2026-04-13 20:57:30.112335 | 2026-04-13 20:57:30.112433 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-04-13 20:57:30.552268 | instance -> localhost | ok 2026-04-13 20:57:30.562947 | 2026-04-13 20:57:30.563173 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-04-13 20:57:30.591162 | instance | ok 2026-04-13 20:57:30.610082 | instance | included: /var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-04-13 20:57:30.616475 | 2026-04-13 20:57:30.616582 | TASK [add-build-sshkey : Create Temp SSH key] 2026-04-13 20:57:32.242807 | instance -> localhost | Generating public/private rsa key pair. 2026-04-13 20:57:32.243029 | instance -> localhost | Your identification has been saved in /var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/work/62ecfd380a2641adaac79ab5cc3a0e3a_id_rsa 2026-04-13 20:57:32.243068 | instance -> localhost | Your public key has been saved in /var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/work/62ecfd380a2641adaac79ab5cc3a0e3a_id_rsa.pub 2026-04-13 20:57:32.243093 | instance -> localhost | The key fingerprint is: 2026-04-13 20:57:32.243115 | instance -> localhost | SHA256:kzLGOfn55IQSe0WbjLoC+f3YOz+LWOVRQ/eymeH7eoE zuul-build-sshkey 2026-04-13 20:57:32.243150 | instance -> localhost | The key's randomart image is: 2026-04-13 20:57:32.243172 | instance -> localhost | +---[RSA 3072]----+ 2026-04-13 20:57:32.243197 | instance -> localhost | | . . | 2026-04-13 20:57:32.243219 | instance -> localhost | | . . . | 2026-04-13 20:57:32.243239 | instance -> localhost | | . o o .| 2026-04-13 20:57:32.243259 | instance -> localhost | | . o = + o * | 2026-04-13 20:57:32.243279 | instance -> localhost | | . X S B * | 2026-04-13 20:57:32.243299 | instance -> localhost | | o . O B . E o | 2026-04-13 20:57:32.243318 | instance -> localhost | | o .+ * + . .| 2026-04-13 20:57:32.243338 | instance -> localhost | | o .O.*. ..| 2026-04-13 20:57:32.243360 | instance -> localhost | | .+.=+=o .o.| 2026-04-13 20:57:32.243379 | instance -> localhost | +----[SHA256]-----+ 2026-04-13 20:57:32.243429 | instance -> localhost | ok: Runtime: 0:00:01.119598 2026-04-13 20:57:32.249360 | 2026-04-13 20:57:32.249429 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-04-13 20:57:32.279741 | instance | ok 2026-04-13 20:57:32.295510 | instance | included: /var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-04-13 20:57:32.304435 | 2026-04-13 20:57:32.304520 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-04-13 20:57:32.329195 | instance | skipping: Conditional result was False 2026-04-13 20:57:32.341833 | 2026-04-13 20:57:32.341937 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-04-13 20:57:32.821618 | instance | changed 2026-04-13 20:57:32.829973 | 2026-04-13 20:57:32.830215 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-04-13 20:57:33.006046 | instance | ok 2026-04-13 20:57:33.012320 | 2026-04-13 20:57:33.012425 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-04-13 20:57:33.537547 | instance | changed 2026-04-13 20:57:33.542561 | 2026-04-13 20:57:33.542630 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-04-13 20:57:34.031439 | instance | changed 2026-04-13 20:57:34.125694 | 2026-04-13 20:57:34.125804 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-04-13 20:57:34.149163 | instance | skipping: Conditional result was False 2026-04-13 20:57:34.158055 | 2026-04-13 20:57:34.158159 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-04-13 20:57:34.543902 | instance -> localhost | changed 2026-04-13 20:57:34.559568 | 2026-04-13 20:57:34.559682 | TASK [add-build-sshkey : Add back temp key] 2026-04-13 20:57:34.879145 | instance -> localhost | Identity added: /var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/work/62ecfd380a2641adaac79ab5cc3a0e3a_id_rsa (zuul-build-sshkey) 2026-04-13 20:57:34.879436 | instance -> localhost | ok: Runtime: 0:00:00.012905 2026-04-13 20:57:34.888067 | 2026-04-13 20:57:34.888214 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-04-13 20:57:35.223623 | instance | ok 2026-04-13 20:57:35.231711 | 2026-04-13 20:57:35.231858 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-04-13 20:57:35.256792 | instance | skipping: Conditional result was False 2026-04-13 20:57:35.270635 | 2026-04-13 20:57:35.270761 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-04-13 20:57:35.649454 | instance | ok 2026-04-13 20:57:35.659537 | 2026-04-13 20:57:35.659737 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-04-13 20:57:37.386752 | instance | Output suppressed because no_log was given 2026-04-13 20:57:37.399791 | 2026-04-13 20:57:37.400009 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-04-13 20:57:37.579353 | instance | ok: "logs" 2026-04-13 20:57:37.892324 | instance | ok: All items complete 2026-04-13 20:57:37.892532 | 2026-04-13 20:57:37.904252 | instance | ok: "artifacts" 2026-04-13 20:57:37.913732 | instance | ok: "docs" 2026-04-13 20:57:37.923515 | 2026-04-13 20:57:37.923649 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-04-13 20:57:38.122498 | instance | changed: "logs" 2026-04-13 20:57:38.257123 | instance | changed: "artifacts" 2026-04-13 20:57:38.417903 | instance | changed: "docs" 2026-04-13 20:57:38.433868 | 2026-04-13 20:57:38.433970 | PLAY RECAP 2026-04-13 20:57:38.434019 | instance | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-04-13 20:57:38.434049 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-04-13 20:57:38.434072 | 2026-04-13 20:57:38.560862 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-04-13 20:57:38.647592 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-04-13 20:57:39.339376 | 2026-04-13 20:57:39.339530 | PLAY [all] 2026-04-13 20:57:39.352767 | 2026-04-13 20:57:39.352920 | TASK [setup-uv : Extract archive] 2026-04-13 20:57:41.633569 | instance | changed 2026-04-13 20:57:41.645819 | 2026-04-13 20:57:41.646105 | TASK [setup-uv : Print version] 2026-04-13 20:57:42.015570 | instance | uv 0.8.13 2026-04-13 20:57:42.187945 | instance | ok: Runtime: 0:00:00.010988 2026-04-13 20:57:42.196420 | 2026-04-13 20:57:42.196528 | PLAY RECAP 2026-04-13 20:57:42.196590 | instance | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-04-13 20:57:42.196622 | 2026-04-13 20:57:42.357573 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-04-13 20:57:42.359765 | PRE-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main] 2026-04-13 20:57:43.147615 | 2026-04-13 20:57:43.147764 | PLAY [all] 2026-04-13 20:57:43.161633 | 2026-04-13 20:57:43.161776 | TASK [Install "jq" for log collection] 2026-04-13 20:57:53.872454 | instance | changed 2026-04-13 20:57:53.898337 | 2026-04-13 20:57:53.898550 | TASK [Install pip3 for Python package management] 2026-04-13 20:57:58.854724 | instance | changed 2026-04-13 20:57:58.953882 | 2026-04-13 20:57:58.954020 | TASK [Install Python "kubernetes" library for kubernetes.core modules] 2026-04-13 20:58:01.941654 | instance | changed 2026-04-13 20:58:02.298366 | 2026-04-13 20:58:02.298506 | PLAY [all] 2026-04-13 20:58:02.312354 | 2026-04-13 20:58:02.312463 | TASK [ensure-go : Check installed go version] 2026-04-13 20:58:02.858379 | instance | ok: ERROR (ignored) 2026-04-13 20:58:02.858690 | instance | { 2026-04-13 20:58:02.858726 | instance | "failed_when_result": false, 2026-04-13 20:58:02.858753 | instance | "msg": "[Errno 2] No such file or directory: b'go'", 2026-04-13 20:58:02.858778 | instance | "rc": 2 2026-04-13 20:58:02.858803 | instance | } 2026-04-13 20:58:02.864583 | 2026-04-13 20:58:02.864683 | TASK [ensure-go : Skip if correct version of go is installed] 2026-04-13 20:58:02.918632 | instance | ok 2026-04-13 20:58:02.931309 | instance | included: /var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/untrusted/project_2/opendev.org/zuul/zuul-jobs/roles/ensure-go/tasks/install-go.yaml 2026-04-13 20:58:02.937476 | 2026-04-13 20:58:02.937639 | TASK [ensure-go : Create temp directory] 2026-04-13 20:58:03.339340 | instance | changed 2026-04-13 20:58:05.117493 | 2026-04-13 20:58:05.117602 | TASK [ensure-go : Get archive checksum] 2026-04-13 20:58:05.808583 | instance | ok: OK (64 bytes) 2026-04-13 20:58:05.816859 | 2026-04-13 20:58:05.817078 | TASK [ensure-go : Download go archive] 2026-04-13 20:58:07.488375 | instance | changed: OK (78559214 bytes) 2026-04-13 20:58:07.497597 | 2026-04-13 20:58:07.497929 | TASK [ensure-go : Install go] 2026-04-13 20:58:13.313191 | instance | changed 2026-04-13 20:58:13.338655 | 2026-04-13 20:58:13.338875 | PLAY [all] 2026-04-13 20:58:13.349546 | 2026-04-13 20:58:13.349782 | TASK [Copy inventory file for Zuul] 2026-04-13 20:58:14.155786 | instance | changed 2026-04-13 20:58:14.160616 | 2026-04-13 20:58:14.160678 | TASK [Switch "ansible_host" to private IP] 2026-04-13 20:58:14.589690 | instance | changed: 1 replacements made 2026-04-13 20:58:14.599415 | 2026-04-13 20:58:14.599615 | TASK [Run molecule prepare] 2026-04-13 20:58:14.892443 | instance | Using CPython 3.10.12 interpreter at: /usr/bin/python3 2026-04-13 20:58:14.892524 | instance | Creating virtual environment at: .venv 2026-04-13 20:58:14.918440 | instance | Building atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere 2026-04-13 20:58:14.949937 | instance | Downloading pygments (1.2MiB) 2026-04-13 20:58:14.950132 | instance | Downloading setuptools (1.1MiB) 2026-04-13 20:58:14.950923 | instance | Downloading pydantic-core (2.0MiB) 2026-04-13 20:58:14.951136 | instance | Downloading kubernetes (1.9MiB) 2026-04-13 20:58:14.951344 | instance | Downloading rjsonnet (1.2MiB) 2026-04-13 20:58:14.954491 | instance | Downloading cryptography (4.2MiB) 2026-04-13 20:58:14.954912 | instance | Downloading openstacksdk (1.7MiB) 2026-04-13 20:58:14.955140 | instance | Downloading netaddr (2.2MiB) 2026-04-13 20:58:14.955363 | instance | Downloading ansible-core (2.1MiB) 2026-04-13 20:58:15.294988 | instance | Building pyperclip==1.9.0 2026-04-13 20:58:15.307601 | instance | Downloading rjsonnet 2026-04-13 20:58:15.433525 | instance | Downloading pydantic-core 2026-04-13 20:58:15.478667 | instance | Downloading netaddr 2026-04-13 20:58:15.493108 | instance | Downloading pygments 2026-04-13 20:58:15.510439 | instance | Downloading cryptography 2026-04-13 20:58:15.546350 | instance | Downloading setuptools 2026-04-13 20:58:15.602387 | instance | Downloading kubernetes 2026-04-13 20:58:15.629693 | instance | Downloading ansible-core 2026-04-13 20:58:15.659477 | instance | Downloading openstacksdk 2026-04-13 20:58:16.049032 | instance | Built pyperclip==1.9.0 2026-04-13 20:58:16.189485 | instance | Built atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere 2026-04-13 20:58:16.223980 | instance | Installed 83 packages in 32ms 2026-04-13 20:58:16.859273 | instance | WARNING Molecule scenarios should migrate to 'extensions/molecule' 2026-04-13 20:58:17.474414 | instance | INFO [csi > discovery] scenario test matrix: prepare 2026-04-13 20:58:17.474486 | instance | INFO [csi > prerun] Performing prerun with role_name_check=0... 2026-04-13 20:58:51.775857 | instance | INFO [csi > prepare] Executing 2026-04-13 20:58:52.654720 | instance | 2026-04-13 20:58:52.655212 | instance | PLAY [Prepare] ***************************************************************** 2026-04-13 20:58:52.655469 | instance | 2026-04-13 20:58:52.655750 | instance | TASK [Gathering Facts] ********************************************************* 2026-04-13 20:58:52.656012 | instance | Monday 13 April 2026 20:58:52 +0000 (0:00:00.030) 0:00:00.030 ********** 2026-04-13 20:58:53.788651 | instance | [WARNING]: Platform linux on host instance is using the discovered Python 2026-04-13 20:58:53.788937 | instance | interpreter at /usr/bin/python3.10, but future installation of another Python 2026-04-13 20:58:53.789219 | instance | interpreter could change the meaning of that path. See 2026-04-13 20:58:53.789528 | instance | https://docs.ansible.com/ansible- 2026-04-13 20:58:53.789821 | instance | core/2.17/reference_appendices/interpreter_discovery.html for more information. 2026-04-13 20:58:53.798556 | instance | ok: [instance] 2026-04-13 20:58:53.798816 | instance | 2026-04-13 20:58:53.799093 | instance | TASK [Configure short hostname] ************************************************ 2026-04-13 20:58:53.799370 | instance | Monday 13 April 2026 20:58:53 +0000 (0:00:01.144) 0:00:01.174 ********** 2026-04-13 20:58:54.481220 | instance | changed: [instance] 2026-04-13 20:58:54.481334 | instance | 2026-04-13 20:58:54.481507 | instance | TASK [Ensure hostname inside hosts file] *************************************** 2026-04-13 20:58:54.481631 | instance | Monday 13 April 2026 20:58:54 +0000 (0:00:00.681) 0:00:01.856 ********** 2026-04-13 20:58:54.743896 | instance | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created 2026-04-13 20:58:54.744107 | instance | with a mode of 0700, this may cause issues when running as another user. To 2026-04-13 20:58:54.744380 | instance | avoid this, create the remote_tmp dir with the correct permissions manually 2026-04-13 20:58:54.754860 | instance | changed: [instance] 2026-04-13 20:58:54.755005 | instance | 2026-04-13 20:58:54.755167 | instance | TASK [Purge "snapd" package] *************************************************** 2026-04-13 20:58:54.755326 | instance | Monday 13 April 2026 20:58:54 +0000 (0:00:00.274) 0:00:02.131 ********** 2026-04-13 20:58:55.583128 | instance | ok: [instance] 2026-04-13 20:58:55.583358 | instance | 2026-04-13 20:58:55.583625 | instance | PLAY [Create devices for Ceph] ************************************************* 2026-04-13 20:58:55.583877 | instance | 2026-04-13 20:58:55.584150 | instance | TASK [Gathering Facts] ********************************************************* 2026-04-13 20:58:55.584431 | instance | Monday 13 April 2026 20:58:55 +0000 (0:00:00.827) 0:00:02.959 ********** 2026-04-13 20:58:56.255390 | instance | ok: [instance] 2026-04-13 20:58:56.255673 | instance | 2026-04-13 20:58:56.255999 | instance | TASK [Install depedencies] ***************************************************** 2026-04-13 20:58:56.256297 | instance | Monday 13 April 2026 20:58:56 +0000 (0:00:00.672) 0:00:03.631 ********** 2026-04-13 20:59:16.058029 | instance | changed: [instance] 2026-04-13 20:59:16.058231 | instance | 2026-04-13 20:59:16.058515 | instance | TASK [Start up service] ******************************************************** 2026-04-13 20:59:16.058793 | instance | Monday 13 April 2026 20:59:16 +0000 (0:00:19.802) 0:00:23.433 ********** 2026-04-13 20:59:16.578841 | instance | ok: [instance] 2026-04-13 20:59:16.579097 | instance | 2026-04-13 20:59:16.579379 | instance | TASK [Generate lvm.conf] ******************************************************* 2026-04-13 20:59:16.579654 | instance | Monday 13 April 2026 20:59:16 +0000 (0:00:00.521) 0:00:23.954 ********** 2026-04-13 20:59:16.855524 | instance | ok: [instance] 2026-04-13 20:59:16.855758 | instance | 2026-04-13 20:59:16.856032 | instance | TASK [Write /etc/lvm/lvm.conf] ************************************************* 2026-04-13 20:59:16.856308 | instance | Monday 13 April 2026 20:59:16 +0000 (0:00:00.276) 0:00:24.231 ********** 2026-04-13 20:59:17.470107 | instance | changed: [instance] 2026-04-13 20:59:17.470194 | instance | 2026-04-13 20:59:17.470391 | instance | TASK [Get list of all loopback devices] **************************************** 2026-04-13 20:59:17.470566 | instance | Monday 13 April 2026 20:59:17 +0000 (0:00:00.614) 0:00:24.845 ********** 2026-04-13 20:59:17.667930 | instance | ok: [instance] 2026-04-13 20:59:17.668210 | instance | 2026-04-13 20:59:17.668535 | instance | TASK [Fail if there is any existing loopback devices] ************************** 2026-04-13 20:59:17.668833 | instance | Monday 13 April 2026 20:59:17 +0000 (0:00:00.197) 0:00:25.043 ********** 2026-04-13 20:59:17.688223 | instance | skipping: [instance] 2026-04-13 20:59:17.688493 | instance | 2026-04-13 20:59:17.688786 | instance | TASK [Create devices for Ceph] ************************************************* 2026-04-13 20:59:17.689074 | instance | Monday 13 April 2026 20:59:17 +0000 (0:00:00.020) 0:00:25.064 ********** 2026-04-13 20:59:18.210763 | instance | changed: [instance] => (item=osd0) 2026-04-13 20:59:18.211061 | instance | changed: [instance] => (item=osd1) 2026-04-13 20:59:18.211349 | instance | changed: [instance] => (item=osd2) 2026-04-13 20:59:18.211604 | instance | 2026-04-13 20:59:18.211826 | instance | TASK [Set permissions on loopback devices] ************************************* 2026-04-13 20:59:18.212089 | instance | Monday 13 April 2026 20:59:18 +0000 (0:00:00.521) 0:00:25.586 ********** 2026-04-13 20:59:18.808886 | instance | changed: [instance] => (item=osd0) 2026-04-13 20:59:18.808995 | instance | changed: [instance] => (item=osd1) 2026-04-13 20:59:18.809148 | instance | changed: [instance] => (item=osd2) 2026-04-13 20:59:18.809296 | instance | 2026-04-13 20:59:18.809479 | instance | TASK [Start loop devices] ****************************************************** 2026-04-13 20:59:18.809658 | instance | Monday 13 April 2026 20:59:18 +0000 (0:00:00.598) 0:00:26.185 ********** 2026-04-13 20:59:19.472985 | instance | changed: [instance] => (item=osd0) 2026-04-13 20:59:19.473290 | instance | changed: [instance] => (item=osd1) 2026-04-13 20:59:19.473655 | instance | changed: [instance] => (item=osd2) 2026-04-13 20:59:19.473949 | instance | 2026-04-13 20:59:19.474266 | instance | TASK [Create a volume group for each loop device] ****************************** 2026-04-13 20:59:19.474583 | instance | Monday 13 April 2026 20:59:19 +0000 (0:00:00.663) 0:00:26.848 ********** 2026-04-13 20:59:22.329179 | instance | changed: [instance] => (item=osd0) 2026-04-13 20:59:22.329430 | instance | changed: [instance] => (item=osd1) 2026-04-13 20:59:22.329807 | instance | changed: [instance] => (item=osd2) 2026-04-13 20:59:22.330043 | instance | 2026-04-13 20:59:22.330327 | instance | TASK [Create a logical volume for each loop device] **************************** 2026-04-13 20:59:22.330613 | instance | Monday 13 April 2026 20:59:22 +0000 (0:00:02.856) 0:00:29.705 ********** 2026-04-13 20:59:24.136506 | instance | changed: [instance] => (item=ceph-instance-osd0) 2026-04-13 20:59:24.136772 | instance | changed: [instance] => (item=ceph-instance-osd1) 2026-04-13 20:59:24.137020 | instance | changed: [instance] => (item=ceph-instance-osd2) 2026-04-13 20:59:24.137289 | instance | 2026-04-13 20:59:24.137590 | instance | PLAY RECAP ********************************************************************* 2026-04-13 20:59:24.137887 | instance | instance : ok=15 changed=9 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-04-13 20:59:24.138136 | instance | 2026-04-13 20:59:24.138398 | instance | Monday 13 April 2026 20:59:24 +0000 (0:00:01.807) 0:00:31.512 ********** 2026-04-13 20:59:24.138650 | instance | =============================================================================== 2026-04-13 20:59:24.138913 | instance | Install depedencies ---------------------------------------------------- 19.80s 2026-04-13 20:59:24.139181 | instance | Create a volume group for each loop device ------------------------------ 2.86s 2026-04-13 20:59:24.139447 | instance | Create a logical volume for each loop device ---------------------------- 1.81s 2026-04-13 20:59:24.139706 | instance | Gathering Facts --------------------------------------------------------- 1.14s 2026-04-13 20:59:24.139971 | instance | Purge "snapd" package --------------------------------------------------- 0.83s 2026-04-13 20:59:24.140258 | instance | Configure short hostname ------------------------------------------------ 0.68s 2026-04-13 20:59:24.140513 | instance | Gathering Facts --------------------------------------------------------- 0.67s 2026-04-13 20:59:24.140804 | instance | Start loop devices ------------------------------------------------------ 0.66s 2026-04-13 20:59:24.141054 | instance | Write /etc/lvm/lvm.conf ------------------------------------------------- 0.61s 2026-04-13 20:59:24.141318 | instance | Set permissions on loopback devices ------------------------------------- 0.60s 2026-04-13 20:59:24.141653 | instance | Create devices for Ceph ------------------------------------------------- 0.52s 2026-04-13 20:59:24.141913 | instance | Start up service -------------------------------------------------------- 0.52s 2026-04-13 20:59:24.142177 | instance | Generate lvm.conf ------------------------------------------------------- 0.28s 2026-04-13 20:59:24.142442 | instance | Ensure hostname inside hosts file --------------------------------------- 0.27s 2026-04-13 20:59:24.142708 | instance | Get list of all loopback devices ---------------------------------------- 0.20s 2026-04-13 20:59:24.142974 | instance | Fail if there is any existing loopback devices -------------------------- 0.02s 2026-04-13 20:59:24.194993 | instance | INFO [csi > prepare] Executed: Successful 2026-04-13 20:59:24.195496 | instance | INFO Molecule executed 1 scenario (1 successful) 2026-04-13 20:59:24.750653 | instance | ok: Runtime: 0:01:09.455730 2026-04-13 20:59:24.754390 | 2026-04-13 20:59:24.754477 | PLAY RECAP 2026-04-13 20:59:24.754538 | instance | ok: 12 changed: 9 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-04-13 20:59:24.754570 | 2026-04-13 20:59:24.874363 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main] 2026-04-13 20:59:24.877042 | RUN START: [untrusted : github.com/vexxhost/atmosphere/molecule/csi/converge.yml@main] 2026-04-13 20:59:25.591621 | 2026-04-13 20:59:25.591765 | PLAY [all] 2026-04-13 20:59:25.603047 | 2026-04-13 20:59:25.603141 | TASK [Build atmosphere binary] 2026-04-13 20:59:26.089902 | instance -> localhost | ERROR 2026-04-13 20:59:26.090228 | instance -> localhost | { 2026-04-13 20:59:26.090290 | instance -> localhost | "msg": "[Errno 2] No such file or directory: b'go'", 2026-04-13 20:59:26.090338 | instance -> localhost | "rc": 2 2026-04-13 20:59:26.090412 | instance -> localhost | } failure 2026-04-13 20:59:26.092190 | 2026-04-13 20:59:26.092274 | PLAY RECAP 2026-04-13 20:59:26.092344 | instance | ok: 0 changed: 0 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0 2026-04-13 20:59:26.092374 | 2026-04-13 20:59:26.205140 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/molecule/csi/converge.yml@main] 2026-04-13 20:59:26.207860 | POST-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main] 2026-04-13 20:59:26.876922 | 2026-04-13 20:59:26.877068 | PLAY [all] 2026-04-13 20:59:26.895478 | 2026-04-13 20:59:26.895627 | TASK [gather-host-logs : creating directory for system status] 2026-04-13 20:59:27.327566 | instance | changed 2026-04-13 20:59:27.336010 | 2026-04-13 20:59:27.336236 | TASK [gather-host-logs : Get logs for each host] 2026-04-13 20:59:27.675491 | instance | + systemd-cgls --full --all --no-pager 2026-04-13 20:59:27.686698 | instance | + ip addr 2026-04-13 20:59:27.689092 | instance | + ip route 2026-04-13 20:59:27.692427 | instance | + lsblk 2026-04-13 20:59:27.696833 | instance | + mount 2026-04-13 20:59:27.699143 | instance | + docker images 2026-04-13 20:59:27.699695 | instance | /bin/bash: line 7: docker: command not found 2026-04-13 20:59:27.699947 | instance | + brctl show 2026-04-13 20:59:27.700352 | instance | /bin/bash: line 8: brctl: command not found 2026-04-13 20:59:27.700613 | instance | + ps aux --sort=-%mem 2026-04-13 20:59:27.713168 | instance | + dpkg -l 2026-04-13 20:59:27.723929 | instance | + CONTAINERS=($(docker ps -a --format '{{ .Names }}' --filter label=zuul)) 2026-04-13 20:59:27.724305 | instance | ++ docker ps -a --format '{{ .Names }}' --filter label=zuul 2026-04-13 20:59:27.724364 | instance | /bin/bash: line 11: docker: command not found 2026-04-13 20:59:27.724641 | instance | + '[' '!' -z '' ']' 2026-04-13 20:59:27.873184 | instance | ok: Runtime: 0:00:00.054512 2026-04-13 20:59:27.883116 | 2026-04-13 20:59:27.883263 | TASK [gather-host-logs : Downloads logs to executor] 2026-04-13 20:59:28.517094 | instance | changed: 2026-04-13 20:59:28.517273 | instance | created directory /var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/work/logs/instance 2026-04-13 20:59:28.517315 | instance | cd+++++++++ system/ 2026-04-13 20:59:28.517347 | instance | >f+++++++++ system/brctl-show.txt 2026-04-13 20:59:28.517371 | instance | >f+++++++++ system/docker-images.txt 2026-04-13 20:59:28.517396 | instance | >f+++++++++ system/ip-addr.txt 2026-04-13 20:59:28.517425 | instance | >f+++++++++ system/ip-route.txt 2026-04-13 20:59:28.517474 | instance | >f+++++++++ system/lsblk.txt 2026-04-13 20:59:28.517523 | instance | >f+++++++++ system/mount.txt 2026-04-13 20:59:28.517575 | instance | >f+++++++++ system/packages.txt 2026-04-13 20:59:28.517637 | instance | >f+++++++++ system/ps.txt 2026-04-13 20:59:28.517682 | instance | >f+++++++++ system/systemd-cgls.txt 2026-04-13 20:59:28.536107 | 2026-04-13 20:59:28.536236 | LOOP [helm-release-status : creating directory for helm release status] 2026-04-13 20:59:28.723728 | instance | changed: "values" 2026-04-13 20:59:28.889053 | instance | changed: "releases" 2026-04-13 20:59:28.906252 | 2026-04-13 20:59:28.906547 | TASK [helm-release-status : Gather get release status for helm charts] 2026-04-13 20:59:29.123136 | instance | /bin/bash: line 3: kubectl: command not found 2026-04-13 20:59:29.443496 | instance | ok: Runtime: 0:00:00.006437 2026-04-13 20:59:29.450752 | 2026-04-13 20:59:29.450898 | TASK [helm-release-status : Downloads logs to executor] 2026-04-13 20:59:29.958640 | instance | changed: 2026-04-13 20:59:29.958814 | instance | cd+++++++++ helm/ 2026-04-13 20:59:29.958854 | instance | cd+++++++++ helm/releases/ 2026-04-13 20:59:29.958885 | instance | cd+++++++++ helm/values/ 2026-04-13 20:59:29.972545 | 2026-04-13 20:59:29.972703 | TASK [describe-kubernetes-objects : creating directory for cluster scoped objects] 2026-04-13 20:59:30.169471 | instance | changed 2026-04-13 20:59:30.179611 | 2026-04-13 20:59:30.179698 | TASK [describe-kubernetes-objects : Gathering descriptions for cluster scoped objects] 2026-04-13 20:59:30.386125 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-04-13 20:59:30.386224 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-04-13 20:59:30.388993 | instance | environment: line 1: kubectl: command not found 2026-04-13 20:59:30.390712 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-04-13 20:59:30.393307 | instance | environment: line 1: kubectl: command not found 2026-04-13 20:59:30.394660 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-04-13 20:59:30.395167 | instance | environment: line 1: kubectl: command not found 2026-04-13 20:59:30.396773 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-04-13 20:59:30.397099 | instance | environment: line 1: kubectl: command not found 2026-04-13 20:59:30.398068 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-04-13 20:59:30.399735 | instance | environment: line 1: kubectl: command not found 2026-04-13 20:59:30.400706 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value 2026-04-13 20:59:30.717573 | instance | ok: Runtime: 0:00:00.022819 2026-04-13 20:59:30.725255 | 2026-04-13 20:59:30.725361 | TASK [describe-kubernetes-objects : creating directory for namespace scoped objects] 2026-04-13 20:59:30.920988 | instance | changed 2026-04-13 20:59:30.926387 | 2026-04-13 20:59:30.926462 | TASK [describe-kubernetes-objects : Gathering descriptions for namespace scoped objects] 2026-04-13 20:59:31.153733 | instance | environment: line 5: kubectl: command not found 2026-04-13 20:59:31.154360 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-04-13 20:59:31.155089 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-04-13 20:59:31.155220 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value 2026-04-13 20:59:31.489948 | instance | ok: Runtime: 0:00:00.008731 2026-04-13 20:59:31.497901 | 2026-04-13 20:59:31.498023 | TASK [describe-kubernetes-objects : Downloads logs to executor] 2026-04-13 20:59:32.039106 | instance | changed: 2026-04-13 20:59:32.039420 | instance | cd+++++++++ objects/ 2026-04-13 20:59:32.039478 | instance | cd+++++++++ objects/cluster/ 2026-04-13 20:59:32.039513 | instance | cd+++++++++ objects/namespaced/ 2026-04-13 20:59:32.052697 | 2026-04-13 20:59:32.052813 | TASK [gather-pod-logs : creating directory for pod logs] 2026-04-13 20:59:32.249856 | instance | changed 2026-04-13 20:59:32.257098 | 2026-04-13 20:59:32.257191 | TASK [gather-pod-logs : creating directory for failed pod logs] 2026-04-13 20:59:32.458481 | instance | changed 2026-04-13 20:59:32.464544 | 2026-04-13 20:59:32.464634 | TASK [gather-pod-logs : retrieve all kubernetes logs, current and previous (if they exist)] 2026-04-13 20:59:32.676238 | instance | environment: line 3: kubectl: command not found 2026-04-13 20:59:33.002934 | instance | ok: Runtime: 0:00:00.008317 2026-04-13 20:59:33.008454 | 2026-04-13 20:59:33.008525 | TASK [gather-pod-logs : Downloads pod logs to executor] 2026-04-13 20:59:33.569823 | instance | changed: 2026-04-13 20:59:33.570022 | instance | cd+++++++++ pod-logs/ 2026-04-13 20:59:33.570074 | instance | cd+++++++++ pod-logs/failed-pods/ 2026-04-13 20:59:33.584451 | 2026-04-13 20:59:33.584576 | TASK [gather-prom-metrics : creating directory for helm release descriptions] 2026-04-13 20:59:33.788421 | instance | changed 2026-04-13 20:59:33.795850 | 2026-04-13 20:59:33.795976 | TASK [gather-prom-metrics : Get metrics from exporter services in all namespaces] 2026-04-13 20:59:34.002613 | instance | /bin/bash: line 2: kubectl: command not found 2026-04-13 20:59:34.332604 | instance | ok: Runtime: 0:00:00.036156 2026-04-13 20:59:34.341201 | 2026-04-13 20:59:34.341425 | TASK [gather-prom-metrics : Get ceph metrics from ceph-mgr] 2026-04-13 20:59:34.557749 | instance | /bin/bash: line 2: kubectl: command not found 2026-04-13 20:59:34.588119 | instance | ceph-mgr endpoints: 2026-04-13 20:59:34.880808 | instance | ok: Runtime: 0:00:00.036838 2026-04-13 20:59:34.887175 | 2026-04-13 20:59:34.887248 | TASK [gather-prom-metrics : Get metrics from fluentd pods] 2026-04-13 20:59:35.093448 | instance | /bin/bash: line 4: kubectl: command not found 2026-04-13 20:59:35.425066 | instance | ok: Runtime: 0:00:00.034652 2026-04-13 20:59:35.431233 | 2026-04-13 20:59:35.431335 | TASK [gather-prom-metrics : Downloads logs to executor] 2026-04-13 20:59:35.968402 | instance | changed: cd+++++++++ prometheus/ 2026-04-13 20:59:35.978269 | 2026-04-13 20:59:35.978426 | TASK [gather-selenium-data : creating directory for helm release descriptions] 2026-04-13 20:59:36.188043 | instance | changed 2026-04-13 20:59:36.193729 | 2026-04-13 20:59:36.193864 | TASK [gather-selenium-data : Get selenium data] 2026-04-13 20:59:36.410388 | instance | + cp '/tmp/artifacts/*' /tmp/logs/selenium/. 2026-04-13 20:59:36.411803 | instance | cp: cannot stat '/tmp/artifacts/*': No such file or directory 2026-04-13 20:59:36.737026 | instance | ERROR 2026-04-13 20:59:36.737296 | instance | { 2026-04-13 20:59:36.737349 | instance | "delta": "0:00:00.005980", 2026-04-13 20:59:36.737388 | instance | "end": "2026-04-13 20:59:36.412164", 2026-04-13 20:59:36.737428 | instance | "msg": "non-zero return code", 2026-04-13 20:59:36.737457 | instance | "rc": 1, 2026-04-13 20:59:36.737484 | instance | "start": "2026-04-13 20:59:36.406184" 2026-04-13 20:59:36.737511 | instance | } 2026-04-13 20:59:36.737546 | instance | ERROR: Ignoring Errors 2026-04-13 20:59:36.745955 | 2026-04-13 20:59:36.746060 | TASK [gather-selenium-data : Downloads logs to executor] 2026-04-13 20:59:37.261247 | instance | changed: cd+++++++++ selenium/ 2026-04-13 20:59:37.268360 | 2026-04-13 20:59:37.268440 | PLAY RECAP 2026-04-13 20:59:37.268491 | instance | ok: 23 changed: 23 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 1 2026-04-13 20:59:37.268513 | 2026-04-13 20:59:37.479610 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main] 2026-04-13 20:59:37.484593 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-04-13 20:59:38.146030 | 2026-04-13 20:59:38.146202 | PLAY [all] 2026-04-13 20:59:38.159961 | 2026-04-13 20:59:38.160156 | TASK [fetch-output : Set log path for multiple nodes] 2026-04-13 20:59:38.196162 | instance | skipping: Conditional result was False 2026-04-13 20:59:38.205908 | 2026-04-13 20:59:38.206170 | TASK [fetch-output : Set log path for single node] 2026-04-13 20:59:38.252371 | instance | ok 2026-04-13 20:59:38.261840 | 2026-04-13 20:59:38.262010 | LOOP [fetch-output : Ensure local output dirs] 2026-04-13 20:59:38.667616 | instance -> localhost | ok: "/var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/work/logs" 2026-04-13 20:59:38.891662 | instance -> localhost | changed: "/var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/work/artifacts" 2026-04-13 20:59:39.106670 | instance -> localhost | changed: "/var/lib/zuul/builds/62ecfd380a2641adaac79ab5cc3a0e3a/work/docs" 2026-04-13 20:59:39.123463 | 2026-04-13 20:59:39.123649 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-04-13 20:59:39.776731 | instance | changed: .d..t...... ./ 2026-04-13 20:59:39.776943 | instance | changed: All items complete 2026-04-13 20:59:39.776990 | 2026-04-13 20:59:40.247963 | instance | changed: .d..t...... ./ 2026-04-13 20:59:40.703410 | instance | changed: .d..t...... ./ 2026-04-13 20:59:40.721677 | 2026-04-13 20:59:40.721788 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-04-13 20:59:41.159004 | instance -> localhost | ok: Item: artifacts Runtime: 0:00:00.007175 2026-04-13 20:59:41.386439 | instance -> localhost | ok: Item: docs Runtime: 0:00:00.007585 2026-04-13 20:59:41.408441 | 2026-04-13 20:59:41.408603 | PLAY [all] 2026-04-13 20:59:41.416368 | 2026-04-13 20:59:41.416495 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes] 2026-04-13 20:59:41.813680 | instance | changed 2026-04-13 20:59:41.827226 | 2026-04-13 20:59:41.827516 | PLAY RECAP 2026-04-13 20:59:41.827717 | instance | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-04-13 20:59:41.827837 | 2026-04-13 20:59:42.033387 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-04-13 20:59:42.037283 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main] 2026-04-13 20:59:42.714041 | 2026-04-13 20:59:42.714177 | PLAY [localhost] 2026-04-13 20:59:42.730063 | 2026-04-13 20:59:42.730186 | TASK [Generate Zuul manifest] 2026-04-13 20:59:42.750646 | localhost | ok 2026-04-13 20:59:42.767659 | 2026-04-13 20:59:42.767780 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-04-13 20:59:43.103623 | localhost | changed 2026-04-13 20:59:43.118261 | 2026-04-13 20:59:43.118371 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-04-13 20:59:43.150867 | localhost | ok 2026-04-13 20:59:43.160938 | 2026-04-13 20:59:43.161058 | TASK [Upload logs] 2026-04-13 20:59:43.193304 | localhost | ok 2026-04-13 20:59:43.252849 | 2026-04-13 20:59:43.252984 | TASK [Set zuul-log-path fact] 2026-04-13 20:59:43.275060 | localhost | ok 2026-04-13 20:59:43.289826 | 2026-04-13 20:59:43.289933 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-04-13 20:59:43.333733 | localhost | ok 2026-04-13 20:59:43.343855 | 2026-04-13 20:59:43.343968 | TASK [upload-logs : Create log directories] 2026-04-13 20:59:43.799371 | localhost | changed 2026-04-13 20:59:43.807316 | 2026-04-13 20:59:43.810902 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-04-13 20:59:44.234778 | localhost -> localhost | ok: Runtime: 0:00:00.005540 2026-04-13 20:59:44.244827 | 2026-04-13 20:59:44.245044 | TASK [upload-logs : Upload logs to log server] 2026-04-13 20:59:44.715943 | localhost | Output suppressed because no_log was given 2026-04-13 20:59:44.721293 | 2026-04-13 20:59:44.721392 | LOOP [upload-logs : Compress console log and json output] 2026-04-13 20:59:44.772542 | localhost | skipping: Conditional result was False 2026-04-13 20:59:44.784170 | localhost | skipping: Conditional result was False 2026-04-13 20:59:44.796078 | 2026-04-13 20:59:44.796543 | LOOP [upload-logs : Upload compressed console log and json output] 2026-04-13 20:59:44.843154 | localhost | skipping: Conditional result was False 2026-04-13 20:59:44.843576 | 2026-04-13 20:59:44.849360 | localhost | skipping: Conditional result was False 2026-04-13 20:59:44.861975 | 2026-04-13 20:59:44.862111 | LOOP [upload-logs : Upload console log and json output]