2026-04-02 16:35:34.157503 | Job console starting
2026-04-02 16:35:34.324635 | Updating git repos
2026-04-02 16:35:34.389929 | Cloning repos into workspace
2026-04-02 16:35:35.057977 | Restoring repo states
2026-04-02 16:35:35.080045 | Merging changes
2026-04-02 16:35:37.305673 | Checking out repos
2026-04-02 16:35:37.625769 | Preparing playbooks
2026-04-02 16:35:49.632091 | Running Ansible setup
2026-04-02 16:35:53.135073 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-02 16:35:53.758830 | 
2026-04-02 16:35:53.758943 | PLAY [localhost]
2026-04-02 16:35:53.766568 | 
2026-04-02 16:35:53.766637 | TASK [Gathering Facts]
2026-04-02 16:35:54.766500 | localhost | ok
2026-04-02 16:35:54.779593 | 
2026-04-02 16:35:54.779713 | TASK [Setup log path fact]
2026-04-02 16:35:54.820557 | localhost | ok
2026-04-02 16:35:54.832337 | 
2026-04-02 16:35:54.832446 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-02 16:35:54.872346 | localhost | ok
2026-04-02 16:35:54.881928 | 
2026-04-02 16:35:54.882116 | TASK [emit-job-header : Print job information]
2026-04-02 16:35:54.923326 | # Job Information
2026-04-02 16:35:54.923597 | Ansible Version: 2.16.16
2026-04-02 16:35:54.923688 | Job: atmosphere-molecule-csi-rbd
2026-04-02 16:35:54.923744 | Pipeline: check
2026-04-02 16:35:54.923791 | Executor: 0a8996d2b663
2026-04-02 16:35:54.923835 | Triggered by: https://github.com/vexxhost/atmosphere/pull/3781
2026-04-02 16:35:54.923887 | Event ID: 8a754280-2ea8-11f1-8028-dc95b5af567d
2026-04-02 16:35:54.930167 | 
2026-04-02 16:35:54.930333 | LOOP [emit-job-header : Print node information]
2026-04-02 16:35:55.028084 | localhost | ok:
2026-04-02 16:35:55.116502 | localhost | # Node Information
2026-04-02 16:35:55.116708 | localhost | Inventory Hostname: instance
2026-04-02 16:35:55.116780 | localhost | Hostname: np0000163029
2026-04-02 16:35:55.116842 | localhost | Username: zuul
2026-04-02 16:35:55.116920 | localhost | Distro: Ubuntu 22.04
2026-04-02 16:35:55.117050 | localhost | Provider: yul1
2026-04-02 16:35:55.117113 | localhost | Region: ca-ymq-1
2026-04-02 16:35:55.117167 | localhost | Label: ubuntu-jammy
2026-04-02 16:35:55.117226 | localhost | Product Name: OpenStack Nova
2026-04-02 16:35:55.117281 | localhost | Interface IP: 199.19.213.74
2026-04-02 16:35:55.130429 | 
2026-04-02 16:35:55.130598 | TASK [log-inventory : Ensure Zuul Ansible directory exists]
2026-04-02 16:35:55.605819 | localhost -> localhost | changed
2026-04-02 16:35:55.610737 | 
2026-04-02 16:35:55.610804 | TASK [log-inventory : Copy ansible inventory to logs dir]
2026-04-02 16:35:56.906579 | localhost -> localhost | changed
2026-04-02 16:35:56.915491 | 
2026-04-02 16:35:56.915553 | PLAY [all]
2026-04-02 16:35:56.924173 | 
2026-04-02 16:35:56.924238 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build]
2026-04-02 16:35:57.242831 | instance -> localhost | ok
2026-04-02 16:35:57.254405 | 
2026-04-02 16:35:57.254824 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID]
2026-04-02 16:35:57.287054 | instance | ok
2026-04-02 16:35:57.302742 | instance | included: /var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-04-02 16:35:57.309583 | 
2026-04-02 16:35:57.309663 | TASK [add-build-sshkey : Create Temp SSH key]
2026-04-02 16:35:58.276206 | instance -> localhost | Generating public/private rsa key pair.
2026-04-02 16:35:58.276466 | instance -> localhost | Your identification has been saved in /var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/work/9ff37aa0e09d462cbded0aaa2a6c5801_id_rsa
2026-04-02 16:35:58.276511 | instance -> localhost | Your public key has been saved in /var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/work/9ff37aa0e09d462cbded0aaa2a6c5801_id_rsa.pub
2026-04-02 16:35:58.276546 | instance -> localhost | The key fingerprint is:
2026-04-02 16:35:58.276578 | instance -> localhost | SHA256:Or8NoHRQIRIkx+/rqwa8qpZPt1lhLquFfoJqy//5dCM zuul-build-sshkey
2026-04-02 16:35:58.276632 | instance -> localhost | The key's randomart image is:
2026-04-02 16:35:58.276665 | instance -> localhost | +---[RSA 3072]----+
2026-04-02 16:35:58.276704 | instance -> localhost | |.o=.. o.         |
2026-04-02 16:35:58.276737 | instance -> localhost | | o.. o           |
2026-04-02 16:35:58.276767 | instance -> localhost | |   ..            |
2026-04-02 16:35:58.276798 | instance -> localhost | |    ..           |
2026-04-02 16:35:58.276827 | instance -> localhost | |.  .. ooS        |
2026-04-02 16:35:58.276857 | instance -> localhost | |.. .oooo.        |
2026-04-02 16:35:58.276886 | instance -> localhost | | .+o.=oE.o       |
2026-04-02 16:35:58.276915 | instance -> localhost | |.*+.+.Oooo.      |
2026-04-02 16:35:58.276947 | instance -> localhost | |X+=BBB..o..      |
2026-04-02 16:35:58.277005 | instance -> localhost | +----[SHA256]-----+
2026-04-02 16:35:58.277092 | instance -> localhost | ok: Runtime: 0:00:00.426468
2026-04-02 16:35:58.284017 | 
2026-04-02 16:35:58.284090 | TASK [add-build-sshkey : Remote setup ssh keys (linux)]
2026-04-02 16:35:58.317721 | instance | ok
2026-04-02 16:35:58.326943 | instance | included: /var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-04-02 16:35:58.334397 | 
2026-04-02 16:35:58.334460 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey]
2026-04-02 16:35:58.359865 | instance | skipping: Conditional result was False
2026-04-02 16:35:58.370153 | 
2026-04-02 16:35:58.370281 | TASK [add-build-sshkey : Enable access via build key on all nodes]
2026-04-02 16:35:58.838748 | instance | changed
2026-04-02 16:35:58.845733 | 
2026-04-02 16:35:58.845822 | TASK [add-build-sshkey : Make sure user has a .ssh]
2026-04-02 16:35:59.034208 | instance | ok
2026-04-02 16:35:59.048058 | 
2026-04-02 16:35:59.048202 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes]
2026-04-02 16:35:59.548754 | instance | changed
2026-04-02 16:35:59.558275 | 
2026-04-02 16:35:59.558346 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes]
2026-04-02 16:36:00.013592 | instance | changed
2026-04-02 16:36:00.018490 | 
2026-04-02 16:36:00.018555 | TASK [add-build-sshkey : Remote setup ssh keys (windows)]
2026-04-02 16:36:00.053078 | instance | skipping: Conditional result was False
2026-04-02 16:36:00.064201 | 
2026-04-02 16:36:00.064291 | TASK [remove-zuul-sshkey : Remove master key from local agent]
2026-04-02 16:36:00.467516 | instance -> localhost | changed
2026-04-02 16:36:00.478554 | 
2026-04-02 16:36:00.478642 | TASK [add-build-sshkey : Add back temp key]
2026-04-02 16:36:00.753239 | instance -> localhost | Identity added: /var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/work/9ff37aa0e09d462cbded0aaa2a6c5801_id_rsa (zuul-build-sshkey)
2026-04-02 16:36:00.753447 | instance -> localhost | ok: Runtime: 0:00:00.015745
2026-04-02 16:36:00.759862 | 
2026-04-02 16:36:00.759926 | TASK [add-build-sshkey : Verify we can still SSH to all nodes]
2026-04-02 16:36:01.105185 | instance | ok
2026-04-02 16:36:01.112323 | 
2026-04-02 16:36:01.112431 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)]
2026-04-02 16:36:01.147250 | instance | skipping: Conditional result was False
2026-04-02 16:36:01.157115 | 
2026-04-02 16:36:01.157204 | TASK [prepare-workspace : Start zuul_console daemon.]
2026-04-02 16:36:01.477757 | instance | ok
2026-04-02 16:36:01.484819 | 
2026-04-02 16:36:01.484893 | TASK [prepare-workspace : Synchronize src repos to workspace directory.]
2026-04-02 16:36:03.092267 | instance | Output suppressed because no_log was given
2026-04-02 16:36:03.101052 | 
2026-04-02 16:36:03.101118 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them]
2026-04-02 16:36:03.284619 | instance | ok: "logs"
2026-04-02 16:36:03.284842 | instance | ok: All items complete
2026-04-02 16:36:03.284870 | 
2026-04-02 16:36:03.445634 | instance | ok: "artifacts"
2026-04-02 16:36:03.606255 | instance | ok: "docs"
2026-04-02 16:36:03.687280 | 
2026-04-02 16:36:03.687434 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist]
2026-04-02 16:36:03.888425 | instance | changed: "logs"
2026-04-02 16:36:04.056609 | instance | changed: "artifacts"
2026-04-02 16:36:04.215562 | instance | changed: "docs"
2026-04-02 16:36:04.226320 | 
2026-04-02 16:36:04.226438 | PLAY RECAP
2026-04-02 16:36:04.226504 | instance | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0
2026-04-02 16:36:04.226546 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-02 16:36:04.226581 | 
2026-04-02 16:36:04.370727 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main]
2026-04-02 16:36:04.448407 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-02 16:36:05.059961 | 
2026-04-02 16:36:05.060366 | PLAY [all]
2026-04-02 16:36:05.071936 | 
2026-04-02 16:36:05.072021 | TASK [setup-uv : Extract archive]
2026-04-02 16:36:08.253543 | instance | changed
2026-04-02 16:36:08.260796 | 
2026-04-02 16:36:08.260872 | TASK [setup-uv : Print version]
2026-04-02 16:36:08.614961 | instance | uv 0.8.13
2026-04-02 16:36:08.796032 | instance | ok: Runtime: 0:00:00.014400
2026-04-02 16:36:08.808513 | 
2026-04-02 16:36:08.808636 | PLAY RECAP
2026-04-02 16:36:08.808689 | instance | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-02 16:36:08.808714 | 
2026-04-02 16:36:09.002766 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main]
2026-04-02 16:36:09.057983 | PRE-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-02 16:36:09.618676 | 
2026-04-02 16:36:09.762207 | PLAY [all]
2026-04-02 16:36:09.780406 | 
2026-04-02 16:36:09.780512 | TASK [Install "jq" for log collection]
2026-04-02 16:36:20.362130 | instance | changed
2026-04-02 16:36:20.364674 | 
2026-04-02 16:36:20.364764 | PLAY RECAP
2026-04-02 16:36:20.364839 | instance | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-04-02 16:36:20.364910 | 
2026-04-02 16:36:20.487212 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/pre.yml@main]
2026-04-02 16:36:20.558103 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-02 16:36:21.141116 | 
2026-04-02 16:36:21.188004 | PLAY [all]
2026-04-02 16:36:21.205019 | 
2026-04-02 16:36:21.205143 | TASK [Copy inventory file for Zuul]
2026-04-02 16:36:22.056826 | instance | changed
2026-04-02 16:36:22.061920 | 
2026-04-02 16:36:22.061994 | TASK [Switch "ansible_host" to private IP]
2026-04-02 16:36:22.357864 | instance | changed: 1 replacements made
2026-04-02 16:36:22.364068 | 
2026-04-02 16:36:22.364137 | TASK [Run Molecule scenario]
2026-04-02 16:36:22.755228 | instance | Using CPython 3.10.12 interpreter at: /usr/bin/python3
2026-04-02 16:36:22.755425 | instance | Creating virtual environment at: .venv
2026-04-02 16:36:22.782904 | instance |    Building atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-02 16:36:22.798527 | instance | Downloading pydantic-core (2.0MiB)
2026-04-02 16:36:22.808099 | instance | Downloading openstacksdk (1.7MiB)
2026-04-02 16:36:22.808947 | instance | Downloading ansible-core (2.1MiB)
2026-04-02 16:36:22.809441 | instance | Downloading kubernetes (1.9MiB)
2026-04-02 16:36:22.810233 | instance | Downloading setuptools (1.1MiB)
2026-04-02 16:36:22.810656 | instance | Downloading cryptography (4.2MiB)
2026-04-02 16:36:22.811682 | instance | Downloading rjsonnet (1.2MiB)
2026-04-02 16:36:22.816141 | instance | Downloading pygments (1.2MiB)
2026-04-02 16:36:22.840292 | instance | Downloading netaddr (2.2MiB)
2026-04-02 16:36:23.182022 | instance |    Building pyperclip==1.9.0
2026-04-02 16:36:23.211950 | instance |  Downloading rjsonnet
2026-04-02 16:36:23.374276 | instance |  Downloading pydantic-core
2026-04-02 16:36:23.407496 | instance |  Downloading pygments
2026-04-02 16:36:23.420838 | instance |  Downloading netaddr
2026-04-02 16:36:23.441689 | instance |  Downloading cryptography
2026-04-02 16:36:23.459274 | instance |  Downloading setuptools
2026-04-02 16:36:23.516779 | instance |  Downloading kubernetes
2026-04-02 16:36:23.557521 | instance |  Downloading ansible-core
2026-04-02 16:36:23.587551 | instance |  Downloading openstacksdk
2026-04-02 16:36:24.085307 | instance |       Built pyperclip==1.9.0
2026-04-02 16:36:24.288541 | instance |       Built atmosphere @ file:///home/zuul/src/github.com/vexxhost/atmosphere
2026-04-02 16:36:24.338543 | instance | Installed 83 packages in 48ms
2026-04-02 16:36:25.261647 | instance | WARNING  Molecule scenarios should migrate to 'extensions/molecule'
2026-04-02 16:36:25.855684 | instance | INFO     [csi > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
2026-04-02 16:36:25.855770 | instance | INFO     [csi > prerun] Performing prerun with role_name_check=0...
2026-04-02 16:36:33.102622 | instance | WARNING  Retrying execution failure 1 of: ansible-galaxy collection install -vvv ansible.posix:1.6.0
2026-04-02 16:36:33.102700 | instance | ERROR    Command ansible-galaxy collection install -vvv ansible.posix:1.6.0, returned 1 code:
2026-04-02 16:36:33.102713 | instance | Starting galaxy collection install process
2026-04-02 16:36:33.102723 | instance | Process install dependency map
2026-04-02 16:36:33.102732 | instance | Starting collection install process
2026-04-02 16:36:33.102742 | instance | Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/ansible-posix-1.6.0.tar.gz to /home/zuul/.ansible/tmp/ansible-local-1294l1ykk_5j/tmps0hn0s3t/ansible-posix-1.6.0-vsmcxal8
2026-04-02 16:36:33.102757 | instance |
2026-04-02 16:36:33.102766 | instance | ansible-galaxy
2026-04-02 16:36:33.102775 | instance |   config file = None
2026-04-02 16:36:33.102784 | instance |   configured module search path =
2026-04-02 16:36:33.102793 | instance |   ansible python module location = /home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/ansible
2026-04-02 16:36:33.102804 | instance |   ansible collection location = /home/zuul/.ansible/collections:/usr/share/ansible/collections:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/dist-packages:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3/dist-packages:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/local/lib/python3.10/dist-packages:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages
2026-04-02 16:36:33.102824 | instance |   executable location = /home/zuul/src/github.com/vexxhost/atmosphere/.venv/bin/ansible-galaxy
2026-04-02 16:36:33.102833 | instance |   python version = 3.10.12 (main, Mar  3 2026, 11:56:32)  (/home/zuul/src/github.com/vexxhost/atmosphere/.venv/bin/python)
2026-04-02 16:36:33.102842 | instance |   jinja version = 3.1.6
2026-04-02 16:36:33.102851 | instance |   libyaml = True
2026-04-02 16:36:33.102859 | instance | No config file found; using defaults
2026-04-02 16:36:33.102868 | instance | : Collection ansible.netcommon does not support Ansible version
2026-04-02 16:36:33.102877 | instance | 2.17.12
2026-04-02 16:36:33.102886 | instance | Found installed collection ansible.netcommon:1.2.0 at '/home/zuul/.ansible/collections/ansible_collections/ansible/netcommon'
2026-04-02 16:36:33.102910 | instance | Opened /home/zuul/.ansible/galaxy_token
2026-04-02 16:36:33.102920 | instance | ERROR! Failed to download collection tar from 'default' due to the following unforeseen error: HTTP Error 502: Bad Gateway. HTTP Error 502: Bad Gateway
2026-04-02 16:36:33.102930 | instance |
2026-04-02 16:36:33.103872 | instance | Traceback (most recent call last):
2026-04-02 16:36:33.104211 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/bin/molecule", line 10, in <module>
2026-04-02 16:36:33.104740 | instance |     sys.exit(main())
2026-04-02 16:36:33.105050 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/click/core.py", line 1442, in __call__
2026-04-02 16:36:33.105942 | instance |     return self.main(*args, **kwargs)
2026-04-02 16:36:33.106241 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/click/core.py", line 1363, in main
2026-04-02 16:36:33.106689 | instance |     rv = self.invoke(ctx)
2026-04-02 16:36:33.106865 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/click/core.py", line 1830, in invoke
2026-04-02 16:36:33.107386 | instance |     return _process_result(sub_ctx.command.invoke(sub_ctx))
2026-04-02 16:36:33.107569 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/click/core.py", line 1226, in invoke
2026-04-02 16:36:33.107984 | instance |     return ctx.invoke(self.callback, **ctx.params)
2026-04-02 16:36:33.108164 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/click/core.py", line 794, in invoke
2026-04-02 16:36:33.108528 | instance |     return callback(*args, **kwargs)
2026-04-02 16:36:33.108710 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/molecule/click_cfg.py", line 583, in wrapper
2026-04-02 16:36:33.109080 | instance |     return func(*args, **kwargs)
2026-04-02 16:36:33.109285 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/click/decorators.py", line 34, in new_func
2026-04-02 16:36:33.109527 | instance |     return f(get_current_context(), *args, **kwargs)
2026-04-02 16:36:33.109684 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/molecule/click_cfg.py", line 418, in wrapper
2026-04-02 16:36:33.110017 | instance |     return func(ctx)
2026-04-02 16:36:33.110151 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/molecule/command/test.py", line 81, in test
2026-04-02 16:36:33.110460 | instance |     base.execute_cmdline_scenarios(scenario_name, args, command_args, ansible_args, exclude)
2026-04-02 16:36:33.110601 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/molecule/command/base.py", line 169, in execute_cmdline_scenarios
2026-04-02 16:36:33.110857 | instance |     _run_scenarios(scenarios, command_args, default_config)
2026-04-02 16:36:33.111091 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/molecule/command/base.py", line 240, in _run_scenarios
2026-04-02 16:36:33.111360 | instance |     scenario.config.runtime.prepare_environment(
2026-04-02 16:36:33.111486 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/ansible_compat/runtime.py", line 694, in prepare_environment
2026-04-02 16:36:33.111817 | instance |     self.install_collection(
2026-04-02 16:36:33.111991 | instance |   File "/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/ansible_compat/runtime.py", line 552, in install_collection
2026-04-02 16:36:33.112303 | instance |     raise InvalidPrerequisiteError(msg)
2026-04-02 16:36:33.113371 | instance | ansible_compat.errors.InvalidPrerequisiteError: Command ansible-galaxy collection install -vvv ansible.posix:1.6.0, returned 1 code:
2026-04-02 16:36:33.113383 | instance | Starting galaxy collection install process
2026-04-02 16:36:33.113393 | instance | Process install dependency map
2026-04-02 16:36:33.113403 | instance | Starting collection install process
2026-04-02 16:36:33.113413 | instance | Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/ansible-posix-1.6.0.tar.gz to /home/zuul/.ansible/tmp/ansible-local-1294l1ykk_5j/tmps0hn0s3t/ansible-posix-1.6.0-vsmcxal8
2026-04-02 16:36:33.113434 | instance |
2026-04-02 16:36:33.113444 | instance | ansible-galaxy [core 2.17.12]
2026-04-02 16:36:33.113454 | instance |   config file = None
2026-04-02 16:36:33.113464 | instance |   configured module search path = ['/home/zuul/src/github.com/vexxhost/atmosphere/plugins/modules', '/home/zuul/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
2026-04-02 16:36:33.113475 | instance |   ansible python module location = /home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages/ansible
2026-04-02 16:36:33.113486 | instance |   ansible collection location = /home/zuul/.ansible/collections:/usr/share/ansible/collections:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/dist-packages:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3/dist-packages:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/local/lib/python3.10/dist-packages:/home/zuul/src/github.com/vexxhost/atmosphere/.venv/lib/python3.10/site-packages
2026-04-02 16:36:33.113496 | instance |   executable location = /home/zuul/src/github.com/vexxhost/atmosphere/.venv/bin/ansible-galaxy
2026-04-02 16:36:33.113523 | instance |   python version = 3.10.12 (main, Mar  3 2026, 11:56:32) [GCC 11.4.0] (/home/zuul/src/github.com/vexxhost/atmosphere/.venv/bin/python)
2026-04-02 16:36:33.113534 | instance |   jinja version = 3.1.6
2026-04-02 16:36:33.113544 | instance |   libyaml = True
2026-04-02 16:36:33.113553 | instance | No config file found; using defaults
2026-04-02 16:36:33.113563 | instance | [WARNING]: Collection ansible.netcommon does not support Ansible version
2026-04-02 16:36:33.113572 | instance | 2.17.12
2026-04-02 16:36:33.113607 | instance | Found installed collection ansible.netcommon:1.2.0 at '/home/zuul/.ansible/collections/ansible_collections/ansible/netcommon'
2026-04-02 16:36:33.113618 | instance | Opened /home/zuul/.ansible/galaxy_token
2026-04-02 16:36:33.113628 | instance | ERROR! Failed to download collection tar from 'default' due to the following unforeseen error: HTTP Error 502: Bad Gateway. HTTP Error 502: Bad Gateway
2026-04-02 16:36:33.113644 | instance |
2026-04-02 16:36:33.412509 | instance | ERROR
2026-04-02 16:36:33.412759 | instance | {
2026-04-02 16:36:33.412802 | instance |   "delta": "0:00:10.525346",
2026-04-02 16:36:33.412832 | instance |   "end": "2026-04-02 16:36:33.222993",
2026-04-02 16:36:33.412860 | instance |   "msg": "non-zero return code",
2026-04-02 16:36:33.412886 | instance |   "rc": 1,
2026-04-02 16:36:33.412914 | instance |   "start": "2026-04-02 16:36:22.697647"
2026-04-02 16:36:33.413092 | instance | }
failure
2026-04-02 16:36:33.415724 | 
2026-04-02 16:36:33.415786 | PLAY RECAP
2026-04-02 16:36:33.415847 | instance | ok: 2 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0
2026-04-02 16:36:33.415912 | 
2026-04-02 16:36:33.557532 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main]
2026-04-02 16:36:33.629147 | POST-RUN START: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-02 16:36:34.297325 | 
2026-04-02 16:36:34.297843 | PLAY [all]
2026-04-02 16:36:34.312022 | 
2026-04-02 16:36:34.312102 | TASK [gather-host-logs : creating directory for system status]
2026-04-02 16:36:34.693223 | instance | changed
2026-04-02 16:36:34.702308 | 
2026-04-02 16:36:34.702525 | TASK [gather-host-logs : Get logs for each host]
2026-04-02 16:36:35.090248 | instance | + systemd-cgls --full --all --no-pager
2026-04-02 16:36:35.101383 | instance | + ip addr
2026-04-02 16:36:35.103754 | instance | + ip route
2026-04-02 16:36:35.105977 | instance | + lsblk
2026-04-02 16:36:35.108926 | instance | + mount
2026-04-02 16:36:35.110651 | instance | + docker images
2026-04-02 16:36:35.111117 | instance | /bin/bash: line 7: docker: command not found
2026-04-02 16:36:35.111416 | instance | + brctl show
2026-04-02 16:36:35.111846 | instance | /bin/bash: line 8: brctl: command not found
2026-04-02 16:36:35.112061 | instance | + ps aux --sort=-%mem
2026-04-02 16:36:35.124475 | instance | + dpkg -l
2026-04-02 16:36:35.131317 | instance | + CONTAINERS=($(docker ps -a --format '{{ .Names }}' --filter label=zuul))
2026-04-02 16:36:35.131800 | instance | ++ docker ps -a --format '{{ .Names }}' --filter label=zuul
2026-04-02 16:36:35.131824 | instance | /bin/bash: line 11: docker: command not found
2026-04-02 16:36:35.132265 | instance | + '[' '!' -z '' ']'
2026-04-02 16:36:35.243326 | instance | ok: Runtime: 0:00:00.049391
2026-04-02 16:36:35.466028 | 
2026-04-02 16:36:35.527446 | TASK [gather-host-logs : Downloads logs to executor]
2026-04-02 16:36:36.247280 | instance | changed:
2026-04-02 16:36:36.247460 | instance | created directory /var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/work/logs/instance
2026-04-02 16:36:36.247492 | instance | cd+++++++++ system/
2026-04-02 16:36:36.247517 | instance | >f+++++++++ system/brctl-show.txt
2026-04-02 16:36:36.247538 | instance | >f+++++++++ system/docker-images.txt
2026-04-02 16:36:36.247559 | instance | >f+++++++++ system/ip-addr.txt
2026-04-02 16:36:36.247582 | instance | >f+++++++++ system/ip-route.txt
2026-04-02 16:36:36.247604 | instance | >f+++++++++ system/lsblk.txt
2026-04-02 16:36:36.247624 | instance | >f+++++++++ system/mount.txt
2026-04-02 16:36:36.247647 | instance | >f+++++++++ system/packages.txt
2026-04-02 16:36:36.247668 | instance | >f+++++++++ system/ps.txt
2026-04-02 16:36:36.247687 | instance | >f+++++++++ system/systemd-cgls.txt
2026-04-02 16:36:36.259778 | 
2026-04-02 16:36:36.259857 | LOOP [helm-release-status : creating directory for helm release status]
2026-04-02 16:36:36.456004 | instance | changed: "values"
2026-04-02 16:36:36.627007 | instance | changed: "releases"
2026-04-02 16:36:36.723577 | 
2026-04-02 16:36:36.723762 | TASK [helm-release-status : Gather get release status for helm charts]
2026-04-02 16:36:36.946248 | instance | /bin/bash: line 3: kubectl: command not found
2026-04-02 16:36:37.260457 | instance | ok: Runtime: 0:00:00.006046
2026-04-02 16:36:37.267655 | 
2026-04-02 16:36:37.267791 | TASK [helm-release-status : Downloads logs to executor]
2026-04-02 16:36:37.768202 | instance | changed:
2026-04-02 16:36:37.768407 | instance | cd+++++++++ helm/
2026-04-02 16:36:37.768454 | instance | cd+++++++++ helm/releases/
2026-04-02 16:36:37.768487 | instance | cd+++++++++ helm/values/
2026-04-02 16:36:37.788043 | 
2026-04-02 16:36:37.788176 | TASK [describe-kubernetes-objects : creating directory for cluster scoped objects]
2026-04-02 16:36:37.984137 | instance | changed
2026-04-02 16:36:38.202810 | 
2026-04-02 16:36:38.202912 | TASK [describe-kubernetes-objects : Gathering descriptions for cluster scoped objects]
2026-04-02 16:36:38.416363 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-02 16:36:38.416703 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-02 16:36:38.421974 | instance | environment: line 1: kubectl: command not found
2026-04-02 16:36:38.422899 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-02 16:36:38.423500 | instance | environment: line 1: kubectl: command not found
2026-04-02 16:36:38.425460 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-02 16:36:38.425648 | instance | environment: line 1: kubectl: command not found
2026-04-02 16:36:38.426936 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-02 16:36:38.430063 | instance | environment: line 1: kubectl: command not found
2026-04-02 16:36:38.431173 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-02 16:36:38.431185 | instance | environment: line 1: kubectl: command not found
2026-04-02 16:36:38.431874 | instance | xargs: warning: options --max-lines and --replace/-I/-i are mutually exclusive, ignoring previous --max-lines value
2026-04-02 16:36:38.732439 | instance | ok: Runtime: 0:00:00.024528
2026-04-02 16:36:38.739445 | 
2026-04-02 16:36:38.739525 | TASK [describe-kubernetes-objects : creating directory for namespace scoped objects]
2026-04-02 16:36:38.954577 | instance | changed
2026-04-02 16:36:39.175516 | 
2026-04-02 16:36:39.175649 | TASK [describe-kubernetes-objects : Gathering descriptions for namespace scoped objects]
2026-04-02 16:36:39.428924 | instance | environment: line 5: kubectl: command not found
2026-04-02 16:36:39.430157 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-02 16:36:39.430235 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-02 16:36:39.430379 | instance | xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
2026-04-02 16:36:39.740910 | instance | ok: Runtime: 0:00:00.009208
2026-04-02 16:36:39.746847 | 
2026-04-02 16:36:39.746916 | TASK [describe-kubernetes-objects : Downloads logs to executor]
2026-04-02 16:36:40.285697 | instance | changed:
2026-04-02 16:36:40.383228 | instance | cd+++++++++ objects/
2026-04-02 16:36:40.383360 | instance | cd+++++++++ objects/cluster/
2026-04-02 16:36:40.383425 | instance | cd+++++++++ objects/namespaced/
2026-04-02 16:36:40.402336 | 
2026-04-02 16:36:40.402458 | TASK [gather-pod-logs : creating directory for pod logs]
2026-04-02 16:36:40.597076 | instance | changed
2026-04-02 16:36:40.749502 | 
2026-04-02 16:36:40.749686 | TASK [gather-pod-logs : creating directory for failed pod logs]
2026-04-02 16:36:40.966453 | instance | changed
2026-04-02 16:36:40.972148 | 
2026-04-02 16:36:40.972229 | TASK [gather-pod-logs : retrieve all kubernetes logs, current and previous (if they exist)]
2026-04-02 16:36:41.196589 | instance | environment: line 3: kubectl: command not found
2026-04-02 16:36:41.611452 | instance | ok: Runtime: 0:00:00.008406
2026-04-02 16:36:41.616907 | 
2026-04-02 16:36:41.616999 | TASK [gather-pod-logs : Downloads pod logs to executor]
2026-04-02 16:36:42.207844 | instance | changed:
2026-04-02 16:36:42.252566 | instance | cd+++++++++ pod-logs/
2026-04-02 16:36:42.252885 | instance | cd+++++++++ pod-logs/failed-pods/
2026-04-02 16:36:42.272867 | 
2026-04-02 16:36:42.272984 | TASK [gather-prom-metrics : creating directory for helm release descriptions]
2026-04-02 16:36:42.471575 | instance | changed
2026-04-02 16:36:42.520306 | 
2026-04-02 16:36:42.520476 | TASK [gather-prom-metrics : Get metrics from exporter services in all namespaces]
2026-04-02 16:36:42.732351 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-02 16:36:43.241241 | instance | ok: Runtime: 0:00:00.037905
2026-04-02 16:36:43.248552 | 
2026-04-02 16:36:43.248648 | TASK [gather-prom-metrics : Get ceph metrics from ceph-mgr]
2026-04-02 16:36:43.460505 | instance | /bin/bash: line 2: kubectl: command not found
2026-04-02 16:36:43.491216 | instance | ceph-mgr endpoints:
2026-04-02 16:36:43.785411 | instance | ok: Runtime: 0:00:00.036865
2026-04-02 16:36:43.791037 | 
2026-04-02 16:36:43.791142 | TASK [gather-prom-metrics : Get metrics from fluentd pods]
2026-04-02 16:36:43.996735 | instance | /bin/bash: line 4: kubectl: command not found
2026-04-02 16:36:44.336010 | instance | ok: Runtime: 0:00:00.033589
2026-04-02 16:36:44.344443 | 
2026-04-02 16:36:44.344571 | TASK [gather-prom-metrics : Downloads logs to executor]
2026-04-02 16:36:44.811400 | instance | changed: cd+++++++++ prometheus/
2026-04-02 16:36:44.822208 | 
2026-04-02 16:36:44.822272 | TASK [gather-selenium-data : creating directory for helm release descriptions]
2026-04-02 16:36:45.007638 | instance | changed
2026-04-02 16:36:45.049438 | 
2026-04-02 16:36:45.049582 | TASK [gather-selenium-data : Get selenium data]
2026-04-02 16:36:45.261518 | instance | + cp '/tmp/artifacts/*' /tmp/logs/selenium/.
2026-04-02 16:36:45.263240 | instance | cp: cannot stat '/tmp/artifacts/*': No such file or directory
2026-04-02 16:36:45.587572 | instance | ERROR
2026-04-02 16:36:45.587799 | instance | {
2026-04-02 16:36:45.587844 | instance |   "delta": "0:00:00.006756",
2026-04-02 16:36:45.587875 | instance |   "end": "2026-04-02 16:36:45.263682",
2026-04-02 16:36:45.587901 | instance |   "msg": "non-zero return code",
2026-04-02 16:36:45.587927 | instance |   "rc": 1,
2026-04-02 16:36:45.587952 | instance |   "start": "2026-04-02 16:36:45.256926"
2026-04-02 16:36:45.587976 | instance | }
2026-04-02 16:36:45.588009 | instance | ERROR: Ignoring Errors
2026-04-02 16:36:45.595296 | 
2026-04-02 16:36:45.595410 | TASK [gather-selenium-data : Downloads logs to executor]
2026-04-02 16:36:46.090313 | instance | changed: cd+++++++++ selenium/
2026-04-02 16:36:46.096767 | 
2026-04-02 16:36:46.096817 | PLAY RECAP
2026-04-02 16:36:46.096898 | instance | ok: 23 changed: 23 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 1
2026-04-02 16:36:46.096927 | 
2026-04-02 16:36:46.204384 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/atmosphere/test-playbooks/molecule/post.yml@main]
2026-04-02 16:36:46.439214 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-02 16:36:47.107460 | 
2026-04-02 16:36:47.107644 | PLAY [all]
2026-04-02 16:36:47.119859 | 
2026-04-02 16:36:47.119961 | TASK [fetch-output : Set log path for multiple nodes]
2026-04-02 16:36:47.164465 | instance | skipping: Conditional result was False
2026-04-02 16:36:47.169621 | 
2026-04-02 16:36:47.169706 | TASK [fetch-output : Set log path for single node]
2026-04-02 16:36:47.207645 | instance | ok
2026-04-02 16:36:47.371420 | 
2026-04-02 16:36:47.371568 | LOOP [fetch-output : Ensure local output dirs]
2026-04-02 16:36:47.783614 | instance -> localhost | ok: "/var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/work/logs"
2026-04-02 16:36:48.195934 | instance -> localhost | changed: "/var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/work/artifacts"
2026-04-02 16:36:48.423272 | instance -> localhost | changed: "/var/lib/zuul/builds/9ff37aa0e09d462cbded0aaa2a6c5801/work/docs"
2026-04-02 16:36:48.443269 | 
2026-04-02 16:36:48.443382 | LOOP [fetch-output : Collect logs, artifacts and docs]
2026-04-02 16:36:49.065216 | instance | changed: .d..t...... ./
2026-04-02 16:36:49.102415 | instance | changed: All items complete
2026-04-02 16:36:49.102635 | 
2026-04-02 16:36:49.551818 | instance | changed: .d..t...... ./
2026-04-02 16:36:50.003968 | instance | changed: .d..t...... ./
2026-04-02 16:36:50.022507 | 
2026-04-02 16:36:50.022667 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir]
2026-04-02 16:36:50.468539 | instance -> localhost | ok: Item: artifacts Runtime: 0:00:00.021803
2026-04-02 16:36:50.813210 | instance -> localhost | ok: Item: docs Runtime: 0:00:00.008011
2026-04-02 16:36:50.832791 | 
2026-04-02 16:36:50.832943 | PLAY [all]
2026-04-02 16:36:50.840323 | 
2026-04-02 16:36:50.840516 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes]
2026-04-02 16:36:51.294870 | instance | changed
2026-04-02 16:36:51.307621 | 
2026-04-02 16:36:51.307781 | PLAY RECAP
2026-04-02 16:36:51.307848 | instance | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0
2026-04-02 16:36:51.307875 | 
2026-04-02 16:36:51.498154 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main]
2026-04-02 16:36:51.517891 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main]
2026-04-02 16:36:52.163983 | 
2026-04-02 16:36:52.164115 | PLAY [localhost]
2026-04-02 16:36:52.174355 | 
2026-04-02 16:36:52.174465 | TASK [Generate Zuul manifest]
2026-04-02 16:36:52.194509 | localhost | ok
2026-04-02 16:36:52.210843 | 
2026-04-02 16:36:52.211005 | TASK [generate-zuul-manifest : Generate Zuul manifest]
2026-04-02 16:36:52.707235 | localhost | changed
2026-04-02 16:36:52.722132 | 
2026-04-02 16:36:52.722246 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul]
2026-04-02 16:36:52.855942 | localhost | ok
2026-04-02 16:36:52.865725 | 
2026-04-02 16:36:52.865850 | TASK [Upload logs]
2026-04-02 16:36:52.896465 | localhost | ok
2026-04-02 16:36:52.956039 | 
2026-04-02 16:36:52.956195 | TASK [Set zuul-log-path fact]
2026-04-02 16:36:52.975025 | localhost | ok
2026-04-02 16:36:52.989778 | 
2026-04-02 16:36:52.989855 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-04-02 16:36:53.020261 | localhost | ok
2026-04-02 16:36:53.029627 | 
2026-04-02 16:36:53.029719 | TASK [upload-logs : Create log directories]
2026-04-02 16:36:53.677692 | localhost | changed
2026-04-02 16:36:53.683456 | 
2026-04-02 16:36:53.683531 | TASK [upload-logs : Ensure logs are readable before uploading]
2026-04-02 16:36:54.131809 | localhost -> localhost | ok: Runtime: 0:00:00.038178
2026-04-02 16:36:54.139213 | 
2026-04-02 16:36:54.139527 | TASK [upload-logs : Upload logs to log server]
2026-04-02 16:36:54.721710 | localhost | Output suppressed because no_log was given
2026-04-02 16:36:54.726389 | 
2026-04-02 16:36:54.726474 | LOOP [upload-logs : Compress console log and json output]
2026-04-02 16:36:54.769893 | localhost | skipping: Conditional result was False
2026-04-02 16:36:54.928529 | localhost | skipping: Conditional result was False
2026-04-02 16:36:54.942674 | 
2026-04-02 16:36:54.942918 | LOOP [upload-logs : Upload compressed console log and json output]
2026-04-02 16:36:54.983121 | localhost | skipping: Conditional result was False
2026-04-02 16:36:54.983506 | 
2026-04-02 16:36:54.987086 | localhost | skipping: Conditional result was False
2026-04-02 16:36:54.996406 | 
2026-04-02 16:36:54.996566 | LOOP [upload-logs : Upload console log and json output]