2026-08-18 14:19:12.258945 | Job console starting 2026-08-18 14:19:12.279489 | Updating git repos 2026-08-18 14:19:12.306171 | Cloning repos into workspace 2026-08-18 14:19:12.915627 | Restoring repo states 2026-08-18 14:19:12.923851 | Merging changes 2026-08-18 14:19:13.793487 | Checking out repos 2026-08-18 14:19:13.925854 | Preparing playbooks 2026-08-18 14:19:17.010380 | Running Ansible setup 2026-08-18 14:19:22.805241 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 14:19:23.471346 | 2026-08-18 14:19:23.471927 | PLAY [localhost] 2026-08-18 14:19:23.479207 | 2026-08-18 14:19:23.479274 | TASK [Gathering Facts] 2026-08-18 14:19:24.347978 | localhost | ok 2026-08-18 14:19:24.374109 | 2026-08-18 14:19:24.374285 | TASK [Setup log path fact] 2026-08-18 14:19:24.393035 | localhost | ok 2026-08-18 14:19:24.406411 | 2026-08-18 14:19:24.406488 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 14:19:24.437357 | localhost | ok 2026-08-18 14:19:24.447376 | 2026-08-18 14:19:24.447439 | TASK [emit-job-header : Print job information] 2026-08-18 14:19:24.488341 | # Job Information 2026-08-18 14:19:24.488553 | Ansible Version: 2.16.18 2026-08-18 14:19:24.488628 | Job: ansible-collection-ceph-molecule-ha-ubuntu-noble 2026-08-18 14:19:24.488679 | Pipeline: check 2026-08-18 14:19:24.488727 | Executor: 2d72f0692154 2026-08-18 14:19:24.488774 | Triggered by: https://github.com/vexxhost/ansible-collection-ceph/pull/123 2026-08-18 14:19:24.488827 | Event ID: 5c848740-9b0f-11f1-99bf-ae0b2ea94fad 2026-08-18 14:19:24.494114 | 2026-08-18 14:19:24.494173 | LOOP [emit-job-header : Print node information] 2026-08-18 14:19:24.632468 | localhost | ok: 2026-08-18 14:19:24.632783 | localhost | # Node Information 2026-08-18 14:19:24.632915 | localhost | Inventory Hostname: node1 2026-08-18 14:19:24.633034 | localhost | Hostname: np0000190012 2026-08-18 14:19:24.633108 | localhost | Username: zuul 2026-08-18 14:19:24.633183 | localhost | Distro: Ubuntu 24.04 2026-08-18 14:19:24.633250 | localhost | Provider: yul1 2026-08-18 14:19:24.633314 | localhost | Region: ca-ymq-1 2026-08-18 14:19:24.633365 | localhost | Label: ubuntu-noble 2026-08-18 14:19:24.633414 | localhost | Product Name: OpenStack Nova 2026-08-18 14:19:24.633463 | localhost | Interface IP: 199.204.45.109 2026-08-18 14:19:24.678965 | localhost | ok: 2026-08-18 14:19:24.679077 | localhost | # Node Information 2026-08-18 14:19:24.679134 | localhost | Inventory Hostname: node2 2026-08-18 14:19:24.679181 | localhost | Hostname: np0000190013 2026-08-18 14:19:24.679224 | localhost | Username: zuul 2026-08-18 14:19:24.679265 | localhost | Distro: Ubuntu 24.04 2026-08-18 14:19:24.679306 | localhost | Provider: yul1 2026-08-18 14:19:24.679346 | localhost | Region: ca-ymq-1 2026-08-18 14:19:24.679386 | localhost | Label: ubuntu-noble 2026-08-18 14:19:24.679425 | localhost | Product Name: OpenStack Nova 2026-08-18 14:19:24.679466 | localhost | Interface IP: 199.204.45.113 2026-08-18 14:19:24.726574 | localhost | ok: 2026-08-18 14:19:24.726680 | localhost | # Node Information 2026-08-18 14:19:24.726732 | localhost | Inventory Hostname: node3 2026-08-18 14:19:24.726777 | localhost | Hostname: np0000190014 2026-08-18 14:19:24.726834 | localhost | Username: zuul 2026-08-18 14:19:24.726885 | localhost | Distro: Ubuntu 24.04 2026-08-18 14:19:24.726934 | localhost | Provider: yul1 2026-08-18 14:19:24.726981 | localhost | Region: ca-ymq-1 2026-08-18 14:19:24.727028 | localhost | Label: ubuntu-noble 2026-08-18 14:19:24.727076 | localhost | Product Name: OpenStack Nova 2026-08-18 14:19:24.727123 | localhost | Interface IP: 199.204.45.177 2026-08-18 14:19:24.734310 | 2026-08-18 14:19:24.734437 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-08-18 14:19:25.103297 | localhost -> localhost | changed 2026-08-18 14:19:25.109359 | 2026-08-18 14:19:25.109426 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-08-18 14:19:25.981059 | localhost -> localhost | changed 2026-08-18 14:19:25.989239 | 2026-08-18 14:19:25.989298 | PLAY [all] 2026-08-18 14:19:26.000714 | 2026-08-18 14:19:26.000787 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-08-18 14:19:26.207121 | node1 -> localhost | ok 2026-08-18 14:19:26.219003 | 2026-08-18 14:19:26.219070 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-08-18 14:19:26.260530 | node1 | ok 2026-08-18 14:19:26.312221 | node2 | ok 2026-08-18 14:19:26.316930 | node3 | ok 2026-08-18 14:19:26.338376 | node1 | included: /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:19:26.338428 | node2 | included: /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:19:26.338452 | node3 | included: /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 14:19:26.345580 | 2026-08-18 14:19:26.345640 | TASK [add-build-sshkey : Create Temp SSH key] 2026-08-18 14:19:27.058838 | node1 -> localhost | Generating public/private rsa key pair. 2026-08-18 14:19:27.059026 | node1 -> localhost | Your identification has been saved in /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/ce2dfe3a2c0b4e649eddedf3a76e04ba_id_rsa 2026-08-18 14:19:27.059070 | node1 -> localhost | Your public key has been saved in /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/ce2dfe3a2c0b4e649eddedf3a76e04ba_id_rsa.pub 2026-08-18 14:19:27.059104 | node1 -> localhost | The key fingerprint is: 2026-08-18 14:19:27.059136 | node1 -> localhost | SHA256:rTcmRp8J9LbPiqmJ0rEQhkkhD43DoT75f9cqmOmfqVw zuul-build-sshkey 2026-08-18 14:19:27.059167 | node1 -> localhost | The key's randomart image is: 2026-08-18 14:19:27.059197 | node1 -> localhost | +---[RSA 3072]----+ 2026-08-18 14:19:27.059227 | node1 -> localhost | |+=o | 2026-08-18 14:19:27.059258 | node1 -> localhost | |==. | 2026-08-18 14:19:27.059288 | node1 -> localhost | |o+. . | 2026-08-18 14:19:27.059317 | node1 -> localhost | |+ + . o | 2026-08-18 14:19:27.059346 | node1 -> localhost | | = . S + | 2026-08-18 14:19:27.059375 | node1 -> localhost | | + . . = + | 2026-08-18 14:19:27.059403 | node1 -> localhost | | + o+E+ X | 2026-08-18 14:19:27.059431 | node1 -> localhost | | . ==.=oB = | 2026-08-18 14:19:27.059460 | node1 -> localhost | | .o=*==.o.o | 2026-08-18 14:19:27.059489 | node1 -> localhost | +----[SHA256]-----+ 2026-08-18 14:19:27.059551 | node1 -> localhost | ok: Runtime: 0:00:00.317841 2026-08-18 14:19:27.067585 | 2026-08-18 14:19:27.067669 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-08-18 14:19:27.101073 | node1 | ok 2026-08-18 14:19:27.120727 | node2 | ok 2026-08-18 14:19:27.127027 | node3 | ok 2026-08-18 14:19:27.145245 | node1 | included: /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:19:27.145292 | node2 | included: /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:19:27.145316 | node3 | included: /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 14:19:27.157981 | 2026-08-18 14:19:27.158040 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-08-18 14:19:27.186540 | node1 | skipping: Conditional result was False 2026-08-18 14:19:27.197766 | node2 | skipping: Conditional result was False 2026-08-18 14:19:27.198063 | node3 | skipping: Conditional result was False 2026-08-18 14:19:27.206239 | 2026-08-18 14:19:27.206333 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-08-18 14:19:27.773700 | node2 | changed 2026-08-18 14:19:27.778871 | node1 | changed 2026-08-18 14:19:27.799087 | node3 | changed 2026-08-18 14:19:27.804532 | 2026-08-18 14:19:27.804595 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-08-18 14:19:28.033865 | node1 | ok 2026-08-18 14:19:28.041037 | node2 | ok 2026-08-18 14:19:28.045642 | node3 | ok 2026-08-18 14:19:28.101239 | 2026-08-18 14:19:28.101302 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-08-18 14:19:28.693993 | node1 | changed 2026-08-18 14:19:28.699562 | node2 | changed 2026-08-18 14:19:28.704226 | node3 | changed 2026-08-18 14:19:28.710228 | 2026-08-18 14:19:28.710293 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-08-18 14:19:29.264078 | node2 | changed 2026-08-18 14:19:29.279385 | node1 | changed 2026-08-18 14:19:29.284852 | node3 | changed 2026-08-18 14:19:29.290067 | 2026-08-18 14:19:29.290128 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-08-18 14:19:29.319105 | node1 | skipping: Conditional result was False 2026-08-18 14:19:29.330276 | node2 | skipping: Conditional result was False 2026-08-18 14:19:29.340612 | node3 | skipping: Conditional result was False 2026-08-18 14:19:29.346630 | 2026-08-18 14:19:29.346695 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-08-18 14:19:29.706426 | node1 -> localhost | changed 2026-08-18 14:19:29.732157 | 2026-08-18 14:19:29.732232 | TASK [add-build-sshkey : Add back temp key] 2026-08-18 14:19:29.998080 | node1 -> localhost | Identity added: /var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/ce2dfe3a2c0b4e649eddedf3a76e04ba_id_rsa (zuul-build-sshkey) 2026-08-18 14:19:29.998288 | node1 -> localhost | ok: Runtime: 0:00:00.015606 2026-08-18 14:19:30.005536 | 2026-08-18 14:19:30.005619 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-08-18 14:19:30.358725 | node1 | ok 2026-08-18 14:19:30.363811 | node2 | ok 2026-08-18 14:19:30.368085 | node3 | ok 2026-08-18 14:19:30.373026 | 2026-08-18 14:19:30.373091 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-08-18 14:19:30.452898 | node1 | skipping: Conditional result was False 2026-08-18 14:19:30.453825 | node2 | skipping: Conditional result was False 2026-08-18 14:19:30.474316 | node3 | skipping: Conditional result was False 2026-08-18 14:19:30.499163 | 2026-08-18 14:19:30.499275 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-08-18 14:19:30.830423 | node3 | ok 2026-08-18 14:19:30.836008 | node1 | ok 2026-08-18 14:19:30.840304 | node2 | ok 2026-08-18 14:19:30.846313 | 2026-08-18 14:19:30.846378 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-08-18 14:19:31.562305 | node1 | Output suppressed because no_log was given 2026-08-18 14:19:31.568207 | node3 | Output suppressed because no_log was given 2026-08-18 14:19:31.608990 | node2 | Output suppressed because no_log was given 2026-08-18 14:19:31.630648 | 2026-08-18 14:19:31.630717 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-08-18 14:19:31.842021 | node1 | ok: "logs" 2026-08-18 14:19:31.842296 | node1 | ok: All items complete 2026-08-18 14:19:31.842325 | 2026-08-18 14:19:31.884565 | node3 | ok: "logs" 2026-08-18 14:19:31.889525 | node2 | ok: "logs" 2026-08-18 14:19:32.041628 | node1 | ok: "artifacts" 2026-08-18 14:19:32.087183 | node3 | ok: "artifacts" 2026-08-18 14:19:32.094701 | node2 | ok: "artifacts" 2026-08-18 14:19:32.240206 | node1 | ok: "docs" 2026-08-18 14:19:32.279883 | node3 | ok: "docs" 2026-08-18 14:19:32.285549 | node2 | ok: "docs" 2026-08-18 14:19:32.299030 | 2026-08-18 14:19:32.299126 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-08-18 14:19:32.535155 | node1 | changed: "logs" 2026-08-18 14:19:32.535558 | node1 | ok: All items complete 2026-08-18 14:19:32.535622 | 2026-08-18 14:19:32.542891 | node2 | changed: "logs" 2026-08-18 14:19:32.550498 | node3 | changed: "logs" 2026-08-18 14:19:32.743420 | node2 | changed: "artifacts" 2026-08-18 14:19:32.748252 | node3 | changed: "artifacts" 2026-08-18 14:19:32.753150 | node1 | changed: "artifacts" 2026-08-18 14:19:32.933234 | node2 | changed: "docs" 2026-08-18 14:19:32.947348 | node1 | changed: "docs" 2026-08-18 14:19:32.959751 | node3 | changed: "docs" 2026-08-18 14:19:32.985802 | 2026-08-18 14:19:32.985913 | PLAY RECAP 2026-08-18 14:19:32.985961 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:19:32.985991 | node1 | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:19:32.986017 | node2 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:19:32.986040 | node3 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 14:19:32.986058 | 2026-08-18 14:19:33.127487 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 14:19:33.142895 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 14:19:33.770833 | 2026-08-18 14:19:33.771313 | PLAY [all] 2026-08-18 14:19:33.791464 | 2026-08-18 14:19:33.791549 | TASK [setup-uv : Extract archive] 2026-08-18 14:19:36.217286 | node1 | changed 2026-08-18 14:19:36.247833 | node3 | changed 2026-08-18 14:19:36.440713 | node2 | changed 2026-08-18 14:19:36.447047 | 2026-08-18 14:19:36.447143 | TASK [setup-uv : Print version] 2026-08-18 14:19:36.854641 | node2 | uv 0.8.13 2026-08-18 14:19:36.867650 | node3 | uv 0.8.13 2026-08-18 14:19:36.849474 | node1 | uv 0.8.13 2026-08-18 14:19:37.067949 | node1 | ok: Runtime: 0:00:00.015369 2026-08-18 14:19:37.079921 | node2 | ok: Runtime: 0:00:00.014699 2026-08-18 14:19:37.089238 | node3 | ok: Runtime: 0:00:00.016775 2026-08-18 14:19:37.108675 | 2026-08-18 14:19:37.108743 | TASK [Install Ansible collection siblings] 2026-08-18 14:19:37.479173 | node1 | ok: 2026-08-18 14:19:37.479366 | node1 | Skipping community.crypto: no checked-out sibling 2026-08-18 14:19:37.479409 | node1 | Skipping community.docker: no checked-out sibling 2026-08-18 14:19:37.479440 | node1 | Skipping community.general: no checked-out sibling 2026-08-18 14:19:37.479468 | node1 | Skipping ansible.utils: no checked-out sibling 2026-08-18 14:19:37.479499 | node1 | Skipping vexxhost.containers: no checked-out sibling 2026-08-18 14:19:37.482290 | 2026-08-18 14:19:37.482352 | PLAY RECAP 2026-08-18 14:19:37.482409 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:19:37.482447 | node2 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:19:37.482481 | node3 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:19:37.482507 | 2026-08-18 14:19:37.613352 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 14:19:37.624554 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 14:19:38.293459 | 2026-08-18 14:19:38.293632 | PLAY [all] 2026-08-18 14:19:38.307326 | 2026-08-18 14:19:38.307412 | LOOP [Check inventory directories] 2026-08-18 14:19:38.669020 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 14:19:38.854774 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/molecule/default/inventory" 2026-08-18 14:19:39.043127 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/inventory" 2026-08-18 14:19:39.058636 | 2026-08-18 14:19:39.058872 | LOOP [Copy inventory file for Zuul] 2026-08-18 14:19:39.948161 | node1 | changed: 2026-08-18 14:19:39.948448 | node1 | { 2026-08-18 14:19:39.948509 | node1 | "ansible_loop_var": "item", 2026-08-18 14:19:39.948560 | node1 | "changed": false, 2026-08-18 14:19:39.948604 | node1 | "failed": false, 2026-08-18 14:19:39.948650 | node1 | "invocation": { 2026-08-18 14:19:39.948692 | node1 | "module_args": { 2026-08-18 14:19:39.948733 | node1 | "checksum_algorithm": "sha1", 2026-08-18 14:19:39.948789 | node1 | "follow": true, 2026-08-18 14:19:39.948831 | node1 | "get_attributes": true, 2026-08-18 14:19:39.948873 | node1 | "get_checksum": true, 2026-08-18 14:19:39.948912 | node1 | "get_mime": true, 2026-08-18 14:19:39.948982 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 14:19:39.949031 | node1 | } 2026-08-18 14:19:39.949072 | node1 | }, 2026-08-18 14:19:39.949110 | node1 | "item": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 14:19:39.949148 | node1 | "stat": { 2026-08-18 14:19:39.949186 | node1 | "atime": 1787062771.4534628, 2026-08-18 14:19:39.949225 | node1 | "attr_flags": "e", 2026-08-18 14:19:39.949263 | node1 | "attributes": [ 2026-08-18 14:19:39.949300 | node1 | "extents" 2026-08-18 14:19:39.949339 | node1 | ], 2026-08-18 14:19:39.949376 | node1 | "block_size": 4096, 2026-08-18 14:19:39.949413 | node1 | "blocks": 8, 2026-08-18 14:19:39.949451 | node1 | "charset": "unknown", 2026-08-18 14:19:39.949492 | node1 | "ctime": 1787062771.4674652, 2026-08-18 14:19:39.949531 | node1 | "dev": 64769, 2026-08-18 14:19:39.949616 | node1 | "device_type": 0, 2026-08-18 14:19:39.949657 | node1 | "executable": true, 2026-08-18 14:19:39.949694 | node1 | "exists": true, 2026-08-18 14:19:39.949731 | node1 | "gid": 1000, 2026-08-18 14:19:39.949770 | node1 | "gr_name": "zuul", 2026-08-18 14:19:39.949807 | node1 | "inode": 1034272, 2026-08-18 14:19:39.949844 | node1 | "isblk": false, 2026-08-18 14:19:39.949883 | node1 | "ischr": false, 2026-08-18 14:19:39.949919 | node1 | "isdir": true, 2026-08-18 14:19:39.949956 | node1 | "isfifo": false, 2026-08-18 14:19:39.949994 | node1 | "isgid": false, 2026-08-18 14:19:39.950030 | node1 | "islnk": false, 2026-08-18 14:19:39.950068 | node1 | "isreg": false, 2026-08-18 14:19:39.950104 | node1 | "issock": false, 2026-08-18 14:19:39.950142 | node1 | "isuid": false, 2026-08-18 14:19:39.950178 | node1 | "mimetype": "unknown", 2026-08-18 14:19:39.950215 | node1 | "mode": "0755", 2026-08-18 14:19:39.950252 | node1 | "mtime": 1787062753.2694123, 2026-08-18 14:19:39.950289 | node1 | "nlink": 10, 2026-08-18 14:19:39.950326 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 14:19:39.950364 | node1 | "pw_name": "zuul", 2026-08-18 14:19:39.950404 | node1 | "readable": true, 2026-08-18 14:19:39.950442 | node1 | "rgrp": true, 2026-08-18 14:19:39.950481 | node1 | "roth": true, 2026-08-18 14:19:39.950519 | node1 | "rusr": true, 2026-08-18 14:19:39.950557 | node1 | "size": 4096, 2026-08-18 14:19:39.950595 | node1 | "uid": 1000, 2026-08-18 14:19:39.950633 | node1 | "version": "337165165", 2026-08-18 14:19:39.950672 | node1 | "wgrp": false, 2026-08-18 14:19:39.950713 | node1 | "woth": false, 2026-08-18 14:19:39.950752 | node1 | "writeable": true, 2026-08-18 14:19:39.950789 | node1 | "wusr": true, 2026-08-18 14:19:39.950828 | node1 | "xgrp": true, 2026-08-18 14:19:39.950865 | node1 | "xoth": true, 2026-08-18 14:19:39.950901 | node1 | "xusr": true 2026-08-18 14:19:39.950939 | node1 | } 2026-08-18 14:19:39.950976 | node1 | } 2026-08-18 14:19:39.951046 | node1 | ok: All items complete 2026-08-18 14:19:39.951091 | 2026-08-18 14:19:39.953628 | node1 | skipping: Conditional result was False 2026-08-18 14:19:39.955004 | node1 | skipping: Conditional result was False 2026-08-18 14:19:39.978020 | 2026-08-18 14:19:39.978166 | LOOP [Switch "ansible_host" to private IP] 2026-08-18 14:19:40.350279 | node1 | changed: 3 replacements made 2026-08-18 14:19:40.351516 | node1 | skipping: Conditional result was False 2026-08-18 14:19:40.352106 | node1 | skipping: Conditional result was False 2026-08-18 14:19:40.368480 | 2026-08-18 14:19:40.368602 | TASK [Run Molecule scenario] 2026-08-18 14:19:40.852962 | node1 | Using CPython 3.12.3 interpreter at: /usr/bin/python3 2026-08-18 14:19:40.853126 | node1 | Creating virtual environment at: .venv 2026-08-18 14:19:40.903778 | node1 | Downloading ansible-core (2.1MiB) 2026-08-18 14:19:40.907470 | node1 | Downloading netaddr (2.2MiB) 2026-08-18 14:19:40.909007 | node1 | Downloading cryptography (4.3MiB) 2026-08-18 14:19:40.909258 | node1 | Downloading pygments (1.2MiB) 2026-08-18 14:19:41.139539 | node1 | Downloading netaddr 2026-08-18 14:19:41.149700 | node1 | Downloading cryptography 2026-08-18 14:19:41.200400 | node1 | Downloading pygments 2026-08-18 14:19:41.347901 | node1 | Downloading ansible-core 2026-08-18 14:19:41.397966 | node1 | Installed 43 packages in 49ms 2026-08-18 14:19:42.142718 | node1 | INFO Collection 'vexxhost.ceph' detected. 2026-08-18 14:19:42.142824 | node1 | INFO Scenarios will be used from 'extensions/molecule' 2026-08-18 14:19:42.785494 | node1 | INFO [default > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy 2026-08-18 14:19:42.785582 | node1 | INFO [default > prerun] Performing prerun with role_name_check=0... 2026-08-18 14:20:17.635711 | node1 | INFO [default > dependency] Executing 2026-08-18 14:20:17.636062 | node1 | WARNING [default > dependency] Missing roles requirements file: requirements.yml 2026-08-18 14:20:17.636353 | node1 | WARNING [default > dependency] Missing collections requirements file: collections.yml 2026-08-18 14:20:17.636567 | node1 | WARNING [default > dependency] Executed: 2 missing (Remove from test_sequence to suppress) 2026-08-18 14:20:17.643905 | node1 | INFO [default > cleanup] Executing 2026-08-18 14:20:17.644135 | node1 | WARNING [default > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 14:20:17.649836 | node1 | INFO [default > destroy] Executing 2026-08-18 14:20:17.649877 | node1 | WARNING [default > destroy] Skipping, '--destroy=never' requested. 2026-08-18 14:20:17.649897 | node1 | INFO [default > destroy] Executed: Successful 2026-08-18 14:20:17.655571 | node1 | INFO [default > syntax] Executing 2026-08-18 14:20:18.977269 | node1 | 2026-08-18 14:20:18.977547 | node1 | playbook: /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:20:19.066208 | node1 | INFO [default > syntax] Executed: Successful 2026-08-18 14:20:19.077807 | node1 | INFO [default > create] Executing 2026-08-18 14:20:19.079279 | node1 | WARNING [default > create] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 14:20:19.085196 | node1 | INFO [default > prepare] Executing 2026-08-18 14:20:19.936954 | node1 | 2026-08-18 14:20:19.937253 | node1 | PLAY [Prepare] ***************************************************************** 2026-08-18 14:20:19.937405 | node1 | 2026-08-18 14:20:19.937674 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:20:19.937918 | node1 | Tuesday 18 August 2026 14:20:19 +0000 (0:00:00.036) 0:00:00.036 ******** 2026-08-18 14:20:21.197153 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 14:20:21.197344 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:20:21.197639 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:20:21.197904 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:20:21.198169 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:20:21.257061 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 14:20:21.257399 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:20:21.257810 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:20:21.258148 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:20:21.258570 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:20:22.236522 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 14:20:22.236738 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:20:22.237067 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:20:22.237368 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:20:22.237698 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:20:22.245720 | node1 | ok: [node1] 2026-08-18 14:20:22.245859 | node1 | ok: [node3] 2026-08-18 14:20:22.246059 | node1 | ok: [node2] 2026-08-18 14:20:22.246269 | node1 | 2026-08-18 14:20:22.246470 | node1 | TASK [Configure short hostname] ************************************************ 2026-08-18 14:20:22.246669 | node1 | Tuesday 18 August 2026 14:20:22 +0000 (0:00:02.309) 0:00:02.346 ******** 2026-08-18 14:20:23.048912 | node1 | changed: [node1] 2026-08-18 14:20:23.049277 | node1 | changed: [node2] 2026-08-18 14:20:23.050236 | node1 | changed: [node3] 2026-08-18 14:20:23.050584 | node1 | 2026-08-18 14:20:23.050861 | node1 | TASK [Ensure hostname inside hosts file] *************************************** 2026-08-18 14:20:23.051156 | node1 | Tuesday 18 August 2026 14:20:23 +0000 (0:00:00.801) 0:00:03.148 ******** 2026-08-18 14:20:23.380589 | node1 | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created 2026-08-18 14:20:23.380932 | node1 | with a mode of 0700, this may cause issues when running as another user. To 2026-08-18 14:20:23.381272 | node1 | avoid this, create the remote_tmp dir with the correct permissions manually 2026-08-18 14:20:23.403665 | node1 | changed: [node1] 2026-08-18 14:20:23.404025 | node1 | changed: [node2] 2026-08-18 14:20:23.404307 | node1 | changed: [node3] 2026-08-18 14:20:23.404570 | node1 | 2026-08-18 14:20:23.404902 | node1 | TASK [Update apt cache] ******************************************************** 2026-08-18 14:20:23.405189 | node1 | Tuesday 18 August 2026 14:20:23 +0000 (0:00:00.356) 0:00:03.504 ******** 2026-08-18 14:20:39.999395 | node1 | changed: [node3] 2026-08-18 14:20:39.999594 | node1 | changed: [node2] 2026-08-18 14:20:39.999947 | node1 | changed: [node1] 2026-08-18 14:20:40.000105 | node1 | 2026-08-18 14:20:40.000345 | node1 | PLAY [Create devices for Ceph] ************************************************* 2026-08-18 14:20:40.000560 | node1 | 2026-08-18 14:20:40.000842 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:20:40.001079 | node1 | Tuesday 18 August 2026 14:20:39 +0000 (0:00:16.595) 0:00:20.100 ******** 2026-08-18 14:20:41.840080 | node1 | ok: [node3] 2026-08-18 14:20:41.840226 | node1 | ok: [node1] 2026-08-18 14:20:41.840517 | node1 | ok: [node2] 2026-08-18 14:20:41.840797 | node1 | 2026-08-18 14:20:41.841034 | node1 | TASK [Install depedencies] ***************************************************** 2026-08-18 14:20:41.841279 | node1 | Tuesday 18 August 2026 14:20:41 +0000 (0:00:01.840) 0:00:21.940 ******** 2026-08-18 14:21:24.192801 | node1 | changed: [node3] 2026-08-18 14:21:24.192915 | node1 | changed: [node1] 2026-08-18 14:21:24.193164 | node1 | changed: [node2] 2026-08-18 14:21:24.193437 | node1 | 2026-08-18 14:21:24.193798 | node1 | TASK [Start up service] ******************************************************** 2026-08-18 14:21:24.194082 | node1 | Tuesday 18 August 2026 14:21:24 +0000 (0:00:42.352) 0:01:04.292 ******** 2026-08-18 14:21:24.839821 | node1 | ok: [node3] 2026-08-18 14:21:24.840075 | node1 | ok: [node2] 2026-08-18 14:21:24.840326 | node1 | ok: [node1] 2026-08-18 14:21:24.840571 | node1 | 2026-08-18 14:21:24.840884 | node1 | TASK [Generate lvm.conf] ******************************************************* 2026-08-18 14:21:24.841201 | node1 | Tuesday 18 August 2026 14:21:24 +0000 (0:00:00.647) 0:01:04.940 ******** 2026-08-18 14:21:25.210958 | node1 | ok: [node2] 2026-08-18 14:21:25.211142 | node1 | ok: [node1] 2026-08-18 14:21:25.211372 | node1 | ok: [node3] 2026-08-18 14:21:25.211658 | node1 | 2026-08-18 14:21:25.211882 | node1 | TASK [Write /etc/lvm/lvm.conf] ************************************************* 2026-08-18 14:21:25.212122 | node1 | Tuesday 18 August 2026 14:21:25 +0000 (0:00:00.371) 0:01:05.311 ******** 2026-08-18 14:21:25.970766 | node1 | changed: [node1] 2026-08-18 14:21:25.970901 | node1 | changed: [node3] 2026-08-18 14:21:25.971053 | node1 | changed: [node2] 2026-08-18 14:21:25.971244 | node1 | 2026-08-18 14:21:25.971401 | node1 | TASK [Get list of all loopback devices] **************************************** 2026-08-18 14:21:25.971561 | node1 | Tuesday 18 August 2026 14:21:25 +0000 (0:00:00.760) 0:01:06.071 ******** 2026-08-18 14:21:26.332538 | node1 | ok: [node1] 2026-08-18 14:21:26.332757 | node1 | ok: [node3] 2026-08-18 14:21:26.332975 | node1 | ok: [node2] 2026-08-18 14:21:26.333160 | node1 | 2026-08-18 14:21:26.333371 | node1 | TASK [Fail if there is any existing loopback devices] ************************** 2026-08-18 14:21:26.333561 | node1 | Tuesday 18 August 2026 14:21:26 +0000 (0:00:00.361) 0:01:06.433 ******** 2026-08-18 14:21:26.376471 | node1 | skipping: [node1] 2026-08-18 14:21:26.376760 | node1 | skipping: [node2] 2026-08-18 14:21:26.384683 | node1 | skipping: [node3] 2026-08-18 14:21:26.384706 | node1 | 2026-08-18 14:21:26.384719 | node1 | TASK [Create devices for Ceph] ************************************************* 2026-08-18 14:21:26.384729 | node1 | Tuesday 18 August 2026 14:21:26 +0000 (0:00:00.044) 0:01:06.477 ******** 2026-08-18 14:21:27.089122 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:21:27.089246 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:21:27.089425 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:21:27.089585 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:21:27.089769 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:21:27.089968 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:21:27.090172 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:21:27.090351 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:21:27.090594 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:21:27.090810 | node1 | 2026-08-18 14:21:27.091023 | node1 | TASK [Set permissions on loopback devices] ************************************* 2026-08-18 14:21:27.091221 | node1 | Tuesday 18 August 2026 14:21:27 +0000 (0:00:00.712) 0:01:07.190 ******** 2026-08-18 14:21:27.913770 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:21:27.913815 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:21:27.913825 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:21:27.913832 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:21:27.913840 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:21:27.913852 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:21:27.914092 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:21:27.914354 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:21:27.914638 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:21:27.914885 | node1 | 2026-08-18 14:21:27.915152 | node1 | TASK [Start loop devices] ****************************************************** 2026-08-18 14:21:27.915417 | node1 | Tuesday 18 August 2026 14:21:27 +0000 (0:00:00.822) 0:01:08.013 ******** 2026-08-18 14:21:29.743056 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:21:29.743138 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:21:29.743325 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:21:29.743508 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:21:29.743704 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:21:29.743879 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:21:29.744034 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:21:29.744184 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:21:29.744343 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:21:29.744494 | node1 | 2026-08-18 14:21:29.744689 | node1 | TASK [Create a volume group for each loop device] ****************************** 2026-08-18 14:21:29.744872 | node1 | Tuesday 18 August 2026 14:21:29 +0000 (0:00:01.830) 0:01:09.843 ******** 2026-08-18 14:21:31.639746 | node1 | changed: [node3] => (item=osd0) 2026-08-18 14:21:31.639990 | node1 | changed: [node2] => (item=osd0) 2026-08-18 14:21:31.640243 | node1 | changed: [node1] => (item=osd0) 2026-08-18 14:21:31.640527 | node1 | changed: [node3] => (item=osd1) 2026-08-18 14:21:31.640904 | node1 | changed: [node2] => (item=osd1) 2026-08-18 14:21:31.641178 | node1 | changed: [node1] => (item=osd1) 2026-08-18 14:21:31.641432 | node1 | changed: [node3] => (item=osd2) 2026-08-18 14:21:31.641714 | node1 | changed: [node2] => (item=osd2) 2026-08-18 14:21:31.641955 | node1 | changed: [node1] => (item=osd2) 2026-08-18 14:21:31.642184 | node1 | 2026-08-18 14:21:31.642435 | node1 | TASK [Create a logical volume for each loop device] **************************** 2026-08-18 14:21:31.642715 | node1 | Tuesday 18 August 2026 14:21:31 +0000 (0:00:01.896) 0:01:11.739 ******** 2026-08-18 14:21:34.480686 | node1 | changed: [node2] => (item=ceph-node2-osd0) 2026-08-18 14:21:34.480752 | node1 | changed: [node1] => (item=ceph-node1-osd0) 2026-08-18 14:21:34.480789 | node1 | changed: [node2] => (item=ceph-node2-osd1) 2026-08-18 14:21:34.480798 | node1 | changed: [node1] => (item=ceph-node1-osd1) 2026-08-18 14:21:34.480806 | node1 | changed: [node3] => (item=ceph-node3-osd0) 2026-08-18 14:21:34.480814 | node1 | changed: [node2] => (item=ceph-node2-osd2) 2026-08-18 14:21:34.480823 | node1 | changed: [node3] => (item=ceph-node3-osd1) 2026-08-18 14:21:34.480831 | node1 | changed: [node3] => (item=ceph-node3-osd2) 2026-08-18 14:21:34.481231 | node1 | changed: [node1] => (item=ceph-node1-osd2) 2026-08-18 14:21:34.481431 | node1 | 2026-08-18 14:21:34.481651 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 14:21:34.481877 | node1 | node1 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:21:34.482086 | node1 | node2 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:21:34.482307 | node1 | node3 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 14:21:34.482486 | node1 | 2026-08-18 14:21:34.482706 | node1 | 2026-08-18 14:21:34.482910 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 14:21:34.483116 | node1 | Tuesday 18 August 2026 14:21:34 +0000 (0:00:02.839) 0:01:14.579 ******** 2026-08-18 14:21:34.483314 | node1 | =============================================================================== 2026-08-18 14:21:34.483508 | node1 | Install depedencies ---------------------------------------------------- 42.35s 2026-08-18 14:21:34.483737 | node1 | Update apt cache ------------------------------------------------------- 16.60s 2026-08-18 14:21:34.483940 | node1 | Create a logical volume for each loop device ---------------------------- 2.84s 2026-08-18 14:21:34.484139 | node1 | Gathering Facts --------------------------------------------------------- 2.31s 2026-08-18 14:21:34.484342 | node1 | Create a volume group for each loop device ------------------------------ 1.90s 2026-08-18 14:21:34.484532 | node1 | Gathering Facts --------------------------------------------------------- 1.84s 2026-08-18 14:21:34.484750 | node1 | Start loop devices ------------------------------------------------------ 1.83s 2026-08-18 14:21:34.484986 | node1 | Set permissions on loopback devices ------------------------------------- 0.82s 2026-08-18 14:21:34.485195 | node1 | Configure short hostname ------------------------------------------------ 0.80s 2026-08-18 14:21:34.485393 | node1 | Write /etc/lvm/lvm.conf ------------------------------------------------- 0.76s 2026-08-18 14:21:34.485585 | node1 | Create devices for Ceph ------------------------------------------------- 0.71s 2026-08-18 14:21:34.485818 | node1 | Start up service -------------------------------------------------------- 0.65s 2026-08-18 14:21:34.486020 | node1 | Generate lvm.conf ------------------------------------------------------- 0.37s 2026-08-18 14:21:34.486226 | node1 | Get list of all loopback devices ---------------------------------------- 0.36s 2026-08-18 14:21:34.486423 | node1 | Ensure hostname inside hosts file --------------------------------------- 0.36s 2026-08-18 14:21:34.486639 | node1 | Fail if there is any existing loopback devices -------------------------- 0.04s 2026-08-18 14:21:34.565332 | node1 | INFO [default > prepare] Executed: Successful 2026-08-18 14:21:34.579841 | node1 | INFO [default > converge] Executing 2026-08-18 14:21:35.792453 | node1 | 2026-08-18 14:21:35.792688 | node1 | PLAY [Deploy Ceph monitors & managers] ***************************************** 2026-08-18 14:21:35.792854 | node1 | 2026-08-18 14:21:35.793023 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:21:35.793195 | node1 | Tuesday 18 August 2026 14:21:35 +0000 (0:00:00.059) 0:00:00.059 ******** 2026-08-18 14:21:37.028931 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 14:21:37.029290 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:21:37.029595 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:21:37.029971 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:21:37.030307 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:21:37.090941 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 14:21:37.091309 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:21:37.091656 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:21:37.091976 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:21:37.092312 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:21:37.095680 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 14:21:37.096815 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 14:21:37.096841 | node1 | interpreter could change the meaning of that path. See 2026-08-18 14:21:37.096851 | node1 | https://docs.ansible.com/ansible- 2026-08-18 14:21:37.096859 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 14:21:37.155823 | node1 | ok: [node1] 2026-08-18 14:21:37.155879 | node1 | ok: [node3] 2026-08-18 14:21:37.155889 | node1 | ok: [node2] 2026-08-18 14:21:37.155898 | node1 | 2026-08-18 14:21:37.155908 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:21:37.156044 | node1 | Tuesday 18 August 2026 14:21:37 +0000 (0:00:01.362) 0:00:01.421 ******** 2026-08-18 14:21:37.507289 | node1 | ok: [node3] 2026-08-18 14:21:37.507454 | node1 | ok: [node1] 2026-08-18 14:21:37.507731 | node1 | ok: [node2] 2026-08-18 14:21:37.507987 | node1 | 2026-08-18 14:21:37.508247 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:21:37.508492 | node1 | Tuesday 18 August 2026 14:21:37 +0000 (0:00:00.352) 0:00:01.773 ******** 2026-08-18 14:21:37.600019 | node1 | skipping: [node1] 2026-08-18 14:21:37.600183 | node1 | skipping: [node2] 2026-08-18 14:21:37.600476 | node1 | skipping: [node3] 2026-08-18 14:21:37.600739 | node1 | 2026-08-18 14:21:37.601142 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:21:37.601410 | node1 | Tuesday 18 August 2026 14:21:37 +0000 (0:00:00.092) 0:00:01.866 ******** 2026-08-18 14:21:37.971150 | node1 | changed: [node1] 2026-08-18 14:21:37.971348 | node1 | changed: [node2] 2026-08-18 14:21:37.971585 | node1 | changed: [node3] 2026-08-18 14:21:37.971881 | node1 | 2026-08-18 14:21:37.972140 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:21:37.972416 | node1 | Tuesday 18 August 2026 14:21:37 +0000 (0:00:00.371) 0:00:02.237 ******** 2026-08-18 14:21:38.074222 | node1 | ok: [node1] => { 2026-08-18 14:21:38.074445 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:21:38.074722 | node1 | } 2026-08-18 14:21:38.074949 | node1 | ok: [node2] => { 2026-08-18 14:21:38.075200 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:21:38.075469 | node1 | } 2026-08-18 14:21:38.075758 | node1 | ok: [node3] => { 2026-08-18 14:21:38.076004 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:21:38.076234 | node1 | } 2026-08-18 14:21:38.076456 | node1 | 2026-08-18 14:21:38.077055 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:21:38.077436 | node1 | Tuesday 18 August 2026 14:21:38 +0000 (0:00:00.103) 0:00:02.340 ******** 2026-08-18 14:21:39.257793 | node1 | changed: [node3] 2026-08-18 14:21:39.259128 | node1 | changed: [node1] 2026-08-18 14:21:39.259185 | node1 | changed: [node2] 2026-08-18 14:21:39.259195 | node1 | 2026-08-18 14:21:39.259205 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:21:39.259214 | node1 | Tuesday 18 August 2026 14:21:39 +0000 (0:00:01.183) 0:00:03.524 ******** 2026-08-18 14:21:39.380692 | node1 | skipping: [node1] 2026-08-18 14:21:39.380746 | node1 | skipping: [node2] 2026-08-18 14:21:39.380756 | node1 | skipping: [node3] 2026-08-18 14:21:39.380765 | node1 | 2026-08-18 14:21:39.380774 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:21:39.380783 | node1 | Tuesday 18 August 2026 14:21:39 +0000 (0:00:00.120) 0:00:03.645 ******** 2026-08-18 14:21:39.496573 | node1 | skipping: [node1] 2026-08-18 14:21:39.496668 | node1 | skipping: [node2] 2026-08-18 14:21:39.496678 | node1 | skipping: [node3] 2026-08-18 14:21:39.496686 | node1 | 2026-08-18 14:21:39.496695 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:21:39.496704 | node1 | Tuesday 18 August 2026 14:21:39 +0000 (0:00:00.116) 0:00:03.761 ******** 2026-08-18 14:21:39.808682 | node1 | ok: [node1] 2026-08-18 14:21:39.808719 | node1 | ok: [node2] 2026-08-18 14:21:39.809397 | node1 | ok: [node3] 2026-08-18 14:21:39.809458 | node1 | 2026-08-18 14:21:39.809465 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:21:39.809471 | node1 | Tuesday 18 August 2026 14:21:39 +0000 (0:00:00.312) 0:00:04.073 ******** 2026-08-18 14:21:41.880920 | node1 | ok: [node3] 2026-08-18 14:21:41.880960 | node1 | ok: [node2] 2026-08-18 14:21:41.880968 | node1 | ok: [node1] 2026-08-18 14:21:41.880974 | node1 | 2026-08-18 14:21:41.880981 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:21:41.880988 | node1 | Tuesday 18 August 2026 14:21:41 +0000 (0:00:02.072) 0:00:06.146 ******** 2026-08-18 14:21:42.133660 | node1 | ok: [node1] => { 2026-08-18 14:21:42.133717 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:21:42.133731 | node1 | } 2026-08-18 14:21:42.133743 | node1 | ok: [node2] => { 2026-08-18 14:21:42.133754 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:21:42.133766 | node1 | } 2026-08-18 14:21:42.133777 | node1 | ok: [node3] => { 2026-08-18 14:21:42.133788 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:21:42.133799 | node1 | } 2026-08-18 14:21:42.133811 | node1 | 2026-08-18 14:21:42.133950 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:21:42.134263 | node1 | Tuesday 18 August 2026 14:21:42 +0000 (0:00:00.251) 0:00:06.397 ******** 2026-08-18 14:21:43.255322 | node1 | changed: [node2] 2026-08-18 14:21:43.255731 | node1 | changed: [node3] 2026-08-18 14:21:43.256052 | node1 | changed: [node1] 2026-08-18 14:21:43.256346 | node1 | 2026-08-18 14:21:43.256705 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:21:43.257053 | node1 | Tuesday 18 August 2026 14:21:43 +0000 (0:00:01.122) 0:00:07.520 ******** 2026-08-18 14:21:46.455984 | node1 | changed: [node1] 2026-08-18 14:21:46.456164 | node1 | changed: [node2] 2026-08-18 14:21:46.456307 | node1 | changed: [node3] 2026-08-18 14:21:46.456442 | node1 | 2026-08-18 14:21:46.456589 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:21:46.456755 | node1 | Tuesday 18 August 2026 14:21:46 +0000 (0:00:03.201) 0:00:10.722 ******** 2026-08-18 14:21:46.532378 | node1 | skipping: [node1] 2026-08-18 14:21:46.532811 | node1 | skipping: [node2] 2026-08-18 14:21:46.533181 | node1 | skipping: [node3] 2026-08-18 14:21:46.533512 | node1 | 2026-08-18 14:21:46.533906 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:21:46.534252 | node1 | Tuesday 18 August 2026 14:21:46 +0000 (0:00:00.076) 0:00:10.798 ******** 2026-08-18 14:21:46.605058 | node1 | skipping: [node1] 2026-08-18 14:21:46.605114 | node1 | skipping: [node2] 2026-08-18 14:21:46.605561 | node1 | skipping: [node3] 2026-08-18 14:21:46.605612 | node1 | 2026-08-18 14:21:46.605622 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:21:46.605628 | node1 | Tuesday 18 August 2026 14:21:46 +0000 (0:00:00.073) 0:00:10.871 ******** 2026-08-18 14:21:46.683055 | node1 | skipping: [node1] 2026-08-18 14:21:46.683144 | node1 | skipping: [node2] 2026-08-18 14:21:46.683272 | node1 | skipping: [node3] 2026-08-18 14:21:46.683367 | node1 | 2026-08-18 14:21:46.683484 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:21:46.683590 | node1 | Tuesday 18 August 2026 14:21:46 +0000 (0:00:00.077) 0:00:10.949 ******** 2026-08-18 14:21:54.931331 | node1 | changed: [node3] 2026-08-18 14:21:54.931405 | node1 | changed: [node1] 2026-08-18 14:21:54.931413 | node1 | changed: [node2] 2026-08-18 14:21:54.931419 | node1 | 2026-08-18 14:21:54.931426 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:21:54.931434 | node1 | Tuesday 18 August 2026 14:21:54 +0000 (0:00:08.246) 0:00:19.195 ******** 2026-08-18 14:21:55.578638 | node1 | changed: [node1] 2026-08-18 14:21:55.578713 | node1 | changed: [node2] 2026-08-18 14:21:55.578723 | node1 | changed: [node3] 2026-08-18 14:21:55.578732 | node1 | 2026-08-18 14:21:55.578741 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:21:55.578750 | node1 | Tuesday 18 August 2026 14:21:55 +0000 (0:00:00.648) 0:00:19.843 ******** 2026-08-18 14:21:56.760670 | node1 | changed: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:21:56.760815 | node1 | changed: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:21:56.760826 | node1 | changed: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:21:56.760836 | node1 | changed: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:21:56.760846 | node1 | changed: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:21:56.760855 | node1 | changed: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:21:56.760864 | node1 | changed: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:21:56.761239 | node1 | changed: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:21:56.761657 | node1 | changed: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:21:56.762065 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:21:56.762461 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:21:56.762857 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:21:56.763248 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:21:56.763654 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:21:56.764025 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:21:56.764338 | node1 | 2026-08-18 14:21:56.764781 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:21:56.765142 | node1 | Tuesday 18 August 2026 14:21:56 +0000 (0:00:01.180) 0:00:21.024 ******** 2026-08-18 14:21:57.548349 | node1 | changed: [node1] 2026-08-18 14:21:57.548755 | node1 | changed: [node2] 2026-08-18 14:21:57.549070 | node1 | changed: [node3] 2026-08-18 14:21:57.549427 | node1 | 2026-08-18 14:21:57.549772 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:21:57.550093 | node1 | Tuesday 18 August 2026 14:21:57 +0000 (0:00:00.766) 0:00:21.790 ******** 2026-08-18 14:21:57.550376 | node1 | 2026-08-18 14:21:57.550879 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:21:57.551197 | node1 | Tuesday 18 August 2026 14:21:57 +0000 (0:00:00.008) 0:00:21.798 ******** 2026-08-18 14:21:57.551479 | node1 | 2026-08-18 14:21:57.551811 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:21:57.552121 | node1 | Tuesday 18 August 2026 14:21:57 +0000 (0:00:00.008) 0:00:21.806 ******** 2026-08-18 14:21:57.552399 | node1 | 2026-08-18 14:21:57.552736 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 14:21:57.553044 | node1 | Tuesday 18 August 2026 14:21:57 +0000 (0:00:00.007) 0:00:21.814 ******** 2026-08-18 14:21:58.681277 | node1 | ok: [node3] 2026-08-18 14:21:58.681674 | node1 | ok: [node2] 2026-08-18 14:21:58.682045 | node1 | ok: [node1] 2026-08-18 14:21:58.682332 | node1 | 2026-08-18 14:21:58.682707 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] ********** 2026-08-18 14:21:58.683030 | node1 | Tuesday 18 August 2026 14:21:58 +0000 (0:00:01.132) 0:00:22.947 ******** 2026-08-18 14:21:59.272142 | node1 | changed: [node1] 2026-08-18 14:21:59.272532 | node1 | changed: [node2] 2026-08-18 14:21:59.272871 | node1 | changed: [node3] 2026-08-18 14:21:59.273169 | node1 | 2026-08-18 14:21:59.273574 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:21:59.273961 | node1 | Tuesday 18 August 2026 14:21:59 +0000 (0:00:00.591) 0:00:23.538 ******** 2026-08-18 14:22:00.039739 | node1 | changed: [node1] 2026-08-18 14:22:00.040108 | node1 | changed: [node2] 2026-08-18 14:22:00.040417 | node1 | changed: [node3] 2026-08-18 14:22:00.040752 | node1 | 2026-08-18 14:22:00.041351 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:22:00.041369 | node1 | Tuesday 18 August 2026 14:22:00 +0000 (0:00:00.767) 0:00:24.306 ******** 2026-08-18 14:22:00.345801 | node1 | ok: [node1] 2026-08-18 14:22:00.345860 | node1 | ok: [node2] 2026-08-18 14:22:00.345877 | node1 | ok: [node3] 2026-08-18 14:22:00.346160 | node1 | 2026-08-18 14:22:00.346438 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:22:00.346815 | node1 | Tuesday 18 August 2026 14:22:00 +0000 (0:00:00.305) 0:00:24.611 ******** 2026-08-18 14:22:00.502978 | node1 | ok: [node1] => { 2026-08-18 14:22:00.503180 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:22:00.503438 | node1 | } 2026-08-18 14:22:00.503774 | node1 | ok: [node2] => { 2026-08-18 14:22:00.504041 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:22:00.504276 | node1 | } 2026-08-18 14:22:00.504544 | node1 | ok: [node3] => { 2026-08-18 14:22:00.504871 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:22:00.505103 | node1 | } 2026-08-18 14:22:00.505326 | node1 | 2026-08-18 14:22:00.505597 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:22:00.505912 | node1 | Tuesday 18 August 2026 14:22:00 +0000 (0:00:00.157) 0:00:24.769 ******** 2026-08-18 14:22:01.769769 | node1 | changed: [node1] 2026-08-18 14:22:01.770012 | node1 | changed: [node3] 2026-08-18 14:22:01.770269 | node1 | changed: [node2] 2026-08-18 14:22:01.770514 | node1 | 2026-08-18 14:22:01.770906 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:22:01.771196 | node1 | Tuesday 18 August 2026 14:22:01 +0000 (0:00:01.266) 0:00:26.035 ******** 2026-08-18 14:22:06.571083 | node1 | changed: [node2] 2026-08-18 14:22:06.571265 | node1 | changed: [node3] 2026-08-18 14:22:06.571507 | node1 | changed: [node1] 2026-08-18 14:22:06.571779 | node1 | 2026-08-18 14:22:06.572026 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:22:06.572282 | node1 | Tuesday 18 August 2026 14:22:06 +0000 (0:00:04.801) 0:00:30.837 ******** 2026-08-18 14:22:07.637158 | node1 | ok: [node1] 2026-08-18 14:22:07.637363 | node1 | ok: [node2] 2026-08-18 14:22:07.637663 | node1 | ok: [node3] 2026-08-18 14:22:07.637880 | node1 | 2026-08-18 14:22:07.638140 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:22:07.638398 | node1 | Tuesday 18 August 2026 14:22:07 +0000 (0:00:01.065) 0:00:31.903 ******** 2026-08-18 14:22:08.060517 | node1 | changed: [node3] 2026-08-18 14:22:08.060795 | node1 | changed: [node1] 2026-08-18 14:22:08.061042 | node1 | changed: [node2] 2026-08-18 14:22:08.061282 | node1 | 2026-08-18 14:22:08.061555 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:22:08.061899 | node1 | Tuesday 18 August 2026 14:22:08 +0000 (0:00:00.423) 0:00:32.326 ******** 2026-08-18 14:22:08.698058 | node1 | changed: [node1] 2026-08-18 14:22:08.698124 | node1 | changed: [node2] 2026-08-18 14:22:08.698219 | node1 | changed: [node3] 2026-08-18 14:22:08.698565 | node1 | 2026-08-18 14:22:08.698615 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:22:08.698625 | node1 | Tuesday 18 August 2026 14:22:08 +0000 (0:00:00.637) 0:00:32.964 ******** 2026-08-18 14:22:09.433082 | node1 | changed: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:22:09.433159 | node1 | changed: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:22:09.433262 | node1 | changed: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:22:09.433369 | node1 | changed: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:22:09.433484 | node1 | changed: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:22:09.433629 | node1 | changed: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:22:09.433730 | node1 | changed: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:22:09.433844 | node1 | changed: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:22:09.433974 | node1 | changed: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:22:09.434333 | node1 | 2026-08-18 14:22:09.434359 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:22:09.434365 | node1 | Tuesday 18 August 2026 14:22:09 +0000 (0:00:00.735) 0:00:33.699 ******** 2026-08-18 14:22:10.062654 | node1 | changed: [node1] 2026-08-18 14:22:10.062705 | node1 | changed: [node2] 2026-08-18 14:22:10.062714 | node1 | changed: [node3] 2026-08-18 14:22:10.062723 | node1 | 2026-08-18 14:22:10.062733 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:22:10.062742 | node1 | Tuesday 18 August 2026 14:22:10 +0000 (0:00:00.627) 0:00:34.326 ******** 2026-08-18 14:22:10.718741 | node1 | changed: [node1] 2026-08-18 14:22:10.718792 | node1 | changed: [node2] 2026-08-18 14:22:10.720667 | node1 | changed: [node3] 2026-08-18 14:22:10.720723 | node1 | 2026-08-18 14:22:10.720733 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:22:10.720743 | node1 | Tuesday 18 August 2026 14:22:10 +0000 (0:00:00.631) 0:00:34.958 ******** 2026-08-18 14:22:10.720752 | node1 | 2026-08-18 14:22:10.720760 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:22:10.720769 | node1 | Tuesday 18 August 2026 14:22:10 +0000 (0:00:00.008) 0:00:34.966 ******** 2026-08-18 14:22:10.720837 | node1 | 2026-08-18 14:22:10.721154 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:22:10.721464 | node1 | Tuesday 18 August 2026 14:22:10 +0000 (0:00:00.008) 0:00:34.974 ******** 2026-08-18 14:22:10.721779 | node1 | 2026-08-18 14:22:10.722106 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 14:22:10.722415 | node1 | Tuesday 18 August 2026 14:22:10 +0000 (0:00:00.009) 0:00:34.984 ******** 2026-08-18 14:22:11.635340 | node1 | ok: [node2] 2026-08-18 14:22:11.635761 | node1 | ok: [node3] 2026-08-18 14:22:11.636083 | node1 | ok: [node1] 2026-08-18 14:22:11.636376 | node1 | 2026-08-18 14:22:11.636727 | node1 | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ****************** 2026-08-18 14:22:11.637138 | node1 | Tuesday 18 August 2026 14:22:11 +0000 (0:00:00.916) 0:00:35.901 ******** 2026-08-18 14:22:15.428555 | node1 | changed: [node3] 2026-08-18 14:22:15.428990 | node1 | changed: [node2] 2026-08-18 14:22:15.429310 | node1 | changed: [node1] 2026-08-18 14:22:15.429629 | node1 | 2026-08-18 14:22:15.429961 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:22:15.430282 | node1 | Tuesday 18 August 2026 14:22:15 +0000 (0:00:03.793) 0:00:39.694 ******** 2026-08-18 14:22:16.176405 | node1 | changed: [node1] 2026-08-18 14:22:16.176897 | node1 | changed: [node3] 2026-08-18 14:22:16.177260 | node1 | changed: [node2] 2026-08-18 14:22:16.177581 | node1 | 2026-08-18 14:22:16.177979 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:22:16.178409 | node1 | Tuesday 18 August 2026 14:22:16 +0000 (0:00:00.747) 0:00:40.442 ******** 2026-08-18 14:22:16.264306 | node1 | ok: [node1] => { 2026-08-18 14:22:16.264432 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:22:16.264538 | node1 | } 2026-08-18 14:22:16.265956 | node1 | ok: [node2] => { 2026-08-18 14:22:16.266024 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:22:16.266035 | node1 | } 2026-08-18 14:22:16.266044 | node1 | ok: [node3] => { 2026-08-18 14:22:16.266052 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:22:16.266080 | node1 | } 2026-08-18 14:22:16.266086 | node1 | 2026-08-18 14:22:16.266091 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:22:16.266097 | node1 | Tuesday 18 August 2026 14:22:16 +0000 (0:00:00.088) 0:00:40.530 ******** 2026-08-18 14:22:18.950864 | node1 | changed: [node1] 2026-08-18 14:22:18.950919 | node1 | changed: [node2] 2026-08-18 14:22:18.950929 | node1 | changed: [node3] 2026-08-18 14:22:18.950939 | node1 | 2026-08-18 14:22:18.950949 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:22:18.950961 | node1 | Tuesday 18 August 2026 14:22:18 +0000 (0:00:02.685) 0:00:43.215 ******** 2026-08-18 14:22:19.046392 | node1 | skipping: [node1] 2026-08-18 14:22:19.046425 | node1 | skipping: [node2] 2026-08-18 14:22:19.046434 | node1 | skipping: [node3] 2026-08-18 14:22:19.046443 | node1 | 2026-08-18 14:22:19.046453 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:22:19.046461 | node1 | Tuesday 18 August 2026 14:22:19 +0000 (0:00:00.095) 0:00:43.311 ******** 2026-08-18 14:22:19.145427 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:22:19.145458 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:22:19.145468 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:22:19.145477 | node1 | 2026-08-18 14:22:19.145486 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:22:19.145495 | node1 | Tuesday 18 August 2026 14:22:19 +0000 (0:00:00.098) 0:00:43.410 ******** 2026-08-18 14:22:20.279460 | node1 | ok: [node2] 2026-08-18 14:22:20.279518 | node1 | ok: [node3] 2026-08-18 14:22:20.279532 | node1 | ok: [node1] 2026-08-18 14:22:20.279544 | node1 | 2026-08-18 14:22:20.279557 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:22:20.279570 | node1 | Tuesday 18 August 2026 14:22:20 +0000 (0:00:01.132) 0:00:44.543 ******** 2026-08-18 14:22:21.111700 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 14:22:21.111756 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:22:21.111769 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:22:21.111780 | node1 | ok: [node1] => (item=ssh) 2026-08-18 14:22:21.111792 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:22:21.111809 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:22:21.112578 | node1 | 2026-08-18 14:22:21.112664 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:22:21.112679 | node1 | Tuesday 18 August 2026 14:22:21 +0000 (0:00:00.832) 0:00:45.375 ******** 2026-08-18 14:22:21.383753 | node1 | ok: [node1] 2026-08-18 14:22:21.383825 | node1 | ok: [node2] 2026-08-18 14:22:21.383839 | node1 | ok: [node3] 2026-08-18 14:22:21.383853 | node1 | 2026-08-18 14:22:21.383866 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:22:21.383879 | node1 | Tuesday 18 August 2026 14:22:21 +0000 (0:00:00.272) 0:00:45.648 ******** 2026-08-18 14:22:22.364321 | node1 | changed: [node3] 2026-08-18 14:22:22.364385 | node1 | changed: [node1] 2026-08-18 14:22:22.364397 | node1 | changed: [node2] 2026-08-18 14:22:22.364409 | node1 | 2026-08-18 14:22:22.364422 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:22:22.364452 | node1 | Tuesday 18 August 2026 14:22:22 +0000 (0:00:00.980) 0:00:46.629 ******** 2026-08-18 14:22:23.018636 | node1 | changed: [node1] 2026-08-18 14:22:23.018672 | node1 | changed: [node2] 2026-08-18 14:22:23.018678 | node1 | changed: [node3] 2026-08-18 14:22:23.018687 | node1 | 2026-08-18 14:22:23.018800 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:22:23.018981 | node1 | Tuesday 18 August 2026 14:22:23 +0000 (0:00:00.655) 0:00:47.284 ******** 2026-08-18 14:22:23.095242 | node1 | skipping: [node1] 2026-08-18 14:22:23.095455 | node1 | skipping: [node2] 2026-08-18 14:22:23.095719 | node1 | skipping: [node3] 2026-08-18 14:22:23.095870 | node1 | 2026-08-18 14:22:23.096034 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:22:23.096201 | node1 | Tuesday 18 August 2026 14:22:23 +0000 (0:00:00.077) 0:00:47.361 ******** 2026-08-18 14:23:08.747628 | node1 | changed: [node3] 2026-08-18 14:23:08.747853 | node1 | changed: [node2] 2026-08-18 14:23:08.748092 | node1 | changed: [node1] 2026-08-18 14:23:08.748310 | node1 | 2026-08-18 14:23:08.748574 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:23:08.748858 | node1 | Tuesday 18 August 2026 14:23:08 +0000 (0:00:45.651) 0:01:33.013 ******** 2026-08-18 14:23:09.443236 | node1 | changed: [node1] 2026-08-18 14:23:09.443439 | node1 | changed: [node3] 2026-08-18 14:23:09.443729 | node1 | changed: [node2] 2026-08-18 14:23:09.443932 | node1 | 2026-08-18 14:23:09.444193 | node1 | TASK [vexxhost.ceph.mon : Find this monitor's addresses on the public network] *** 2026-08-18 14:23:09.444444 | node1 | Tuesday 18 August 2026 14:23:09 +0000 (0:00:00.696) 0:01:33.709 ******** 2026-08-18 14:23:09.835951 | node1 | ok: [node2] 2026-08-18 14:23:09.836243 | node1 | ok: [node1] 2026-08-18 14:23:09.836566 | node1 | ok: [node3] 2026-08-18 14:23:09.836893 | node1 | 2026-08-18 14:23:09.837095 | node1 | TASK [vexxhost.ceph.mon : Assert exactly one address matches the public network] *** 2026-08-18 14:23:09.837337 | node1 | Tuesday 18 August 2026 14:23:09 +0000 (0:00:00.392) 0:01:34.102 ******** 2026-08-18 14:23:09.919969 | node1 | ok: [node1] => { 2026-08-18 14:23:09.920167 | node1 | "changed": false, 2026-08-18 14:23:09.920398 | node1 | "msg": "All assertions passed" 2026-08-18 14:23:09.920812 | node1 | } 2026-08-18 14:23:09.921036 | node1 | ok: [node2] => { 2026-08-18 14:23:09.921285 | node1 | "changed": false, 2026-08-18 14:23:09.921670 | node1 | "msg": "All assertions passed" 2026-08-18 14:23:09.922567 | node1 | } 2026-08-18 14:23:09.922951 | node1 | ok: [node3] => { 2026-08-18 14:23:09.923203 | node1 | "changed": false, 2026-08-18 14:23:09.923630 | node1 | "msg": "All assertions passed" 2026-08-18 14:23:09.923870 | node1 | } 2026-08-18 14:23:09.924088 | node1 | 2026-08-18 14:23:09.924336 | node1 | TASK [vexxhost.ceph.mon : Set this monitor's Ceph public IP] ******************* 2026-08-18 14:23:09.924582 | node1 | Tuesday 18 August 2026 14:23:09 +0000 (0:00:00.084) 0:01:34.186 ******** 2026-08-18 14:23:10.011266 | node1 | ok: [node1] 2026-08-18 14:23:10.011497 | node1 | ok: [node2] 2026-08-18 14:23:10.011782 | node1 | ok: [node3] 2026-08-18 14:23:10.011996 | node1 | 2026-08-18 14:23:10.012249 | node1 | TASK [vexxhost.ceph.mon : Check whether any monitor is already bootstrapped] *** 2026-08-18 14:23:10.012496 | node1 | Tuesday 18 August 2026 14:23:10 +0000 (0:00:00.090) 0:01:34.277 ******** 2026-08-18 14:23:10.710132 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:23:10.710365 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:23:10.710660 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:23:10.710911 | node1 | 2026-08-18 14:23:10.711237 | node1 | TASK [vexxhost.ceph.mon : Select the bootstrap node (existing one, else the first monitor)] *** 2026-08-18 14:23:10.711535 | node1 | Tuesday 18 August 2026 14:23:10 +0000 (0:00:00.698) 0:01:34.976 ******** 2026-08-18 14:23:10.799901 | node1 | ok: [node1] 2026-08-18 14:23:10.800103 | node1 | ok: [node2] 2026-08-18 14:23:10.800322 | node1 | ok: [node3] 2026-08-18 14:23:10.800550 | node1 | 2026-08-18 14:23:10.800851 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 14:23:10.801081 | node1 | Tuesday 18 August 2026 14:23:10 +0000 (0:00:00.089) 0:01:35.066 ******** 2026-08-18 14:23:10.958995 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap.yml for node1, node2, node3 2026-08-18 14:23:10.959202 | node1 | 2026-08-18 14:23:10.959488 | node1 | TASK [vexxhost.ceph.mon : Ensure /etc/ceph exists] ***************************** 2026-08-18 14:23:10.959801 | node1 | Tuesday 18 August 2026 14:23:10 +0000 (0:00:00.158) 0:01:35.225 ******** 2026-08-18 14:23:11.212267 | node1 | skipping: [node2] 2026-08-18 14:23:11.212492 | node1 | skipping: [node3] 2026-08-18 14:23:11.212795 | node1 | changed: [node1] 2026-08-18 14:23:11.213032 | node1 | 2026-08-18 14:23:11.213286 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH private key] *********** 2026-08-18 14:23:11.213533 | node1 | Tuesday 18 August 2026 14:23:11 +0000 (0:00:00.253) 0:01:35.478 ******** 2026-08-18 14:23:11.296246 | node1 | skipping: [node1] 2026-08-18 14:23:11.296455 | node1 | skipping: [node2] 2026-08-18 14:23:11.296734 | node1 | skipping: [node3] 2026-08-18 14:23:11.296954 | node1 | 2026-08-18 14:23:11.297208 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH public key] ************ 2026-08-18 14:23:11.297454 | node1 | Tuesday 18 August 2026 14:23:11 +0000 (0:00:00.083) 0:01:35.562 ******** 2026-08-18 14:23:11.425866 | node1 | skipping: [node1] 2026-08-18 14:23:11.426040 | node1 | skipping: [node2] 2026-08-18 14:23:11.426275 | node1 | skipping: [node3] 2026-08-18 14:23:11.426495 | node1 | 2026-08-18 14:23:11.426802 | node1 | TASK [vexxhost.ceph.mon : Generate a cephadm SSH keypair when none was provided] *** 2026-08-18 14:23:11.427068 | node1 | Tuesday 18 August 2026 14:23:11 +0000 (0:00:00.129) 0:01:35.692 ******** 2026-08-18 14:23:13.890735 | node1 | skipping: [node2] 2026-08-18 14:23:13.890825 | node1 | skipping: [node3] 2026-08-18 14:23:13.890832 | node1 | changed: [node1] 2026-08-18 14:23:13.890838 | node1 | 2026-08-18 14:23:13.890845 | node1 | TASK [vexxhost.ceph.mon : Create a temporary ceph.conf for bootstrap overrides] *** 2026-08-18 14:23:13.890851 | node1 | Tuesday 18 August 2026 14:23:13 +0000 (0:00:02.463) 0:01:38.156 ******** 2026-08-18 14:23:14.265859 | node1 | skipping: [node2] 2026-08-18 14:23:14.265954 | node1 | skipping: [node3] 2026-08-18 14:23:14.265961 | node1 | changed: [node1] 2026-08-18 14:23:14.265967 | node1 | 2026-08-18 14:23:14.265981 | node1 | TASK [vexxhost.ceph.mon : Populate bootstrap ceph.conf overrides] ************** 2026-08-18 14:23:14.266224 | node1 | Tuesday 18 August 2026 14:23:14 +0000 (0:00:00.375) 0:01:38.531 ******** 2026-08-18 14:23:14.336899 | node1 | skipping: [node1] 2026-08-18 14:23:14.337008 | node1 | skipping: [node2] 2026-08-18 14:23:14.337050 | node1 | skipping: [node3] 2026-08-18 14:23:14.337210 | node1 | 2026-08-18 14:23:14.337423 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 14:23:14.337650 | node1 | Tuesday 18 August 2026 14:23:14 +0000 (0:00:00.071) 0:01:38.602 ******** 2026-08-18 14:24:05.577986 | node1 | skipping: [node2] 2026-08-18 14:24:05.578051 | node1 | skipping: [node3] 2026-08-18 14:24:05.578072 | node1 | changed: [node1] 2026-08-18 14:24:05.578262 | node1 | 2026-08-18 14:24:05.578532 | node1 | TASK [vexxhost.ceph.mon : Remove the temporary ceph.conf] ********************** 2026-08-18 14:24:05.578856 | node1 | Tuesday 18 August 2026 14:24:05 +0000 (0:00:51.240) 0:02:29.843 ******** 2026-08-18 14:24:05.838166 | node1 | skipping: [node2] 2026-08-18 14:24:05.838364 | node1 | skipping: [node3] 2026-08-18 14:24:05.838629 | node1 | changed: [node1] 2026-08-18 14:24:05.838914 | node1 | 2026-08-18 14:24:05.839163 | node1 | TASK [Register this monitor host] ********************************************** 2026-08-18 14:24:05.839415 | node1 | Tuesday 18 August 2026 14:24:05 +0000 (0:00:00.261) 0:02:30.104 ******** 2026-08-18 14:24:05.983337 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:24:05.983514 | node1 | 2026-08-18 14:24:05.983810 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:24:05.984044 | node1 | Tuesday 18 August 2026 14:24:05 +0000 (0:00:00.145) 0:02:30.249 ******** 2026-08-18 14:24:08.007493 | node1 | ok: [node1] 2026-08-18 14:24:08.007557 | node1 | 2026-08-18 14:24:08.007567 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:24:08.007575 | node1 | Tuesday 18 August 2026 14:24:08 +0000 (0:00:02.023) 0:02:32.273 ******** 2026-08-18 14:24:08.091021 | node1 | ok: [node1] 2026-08-18 14:24:08.091088 | node1 | ok: [node2] 2026-08-18 14:24:08.091192 | node1 | ok: [node3] 2026-08-18 14:24:08.091294 | node1 | 2026-08-18 14:24:08.091409 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:24:08.091522 | node1 | Tuesday 18 August 2026 14:24:08 +0000 (0:00:00.083) 0:02:32.356 ******** 2026-08-18 14:24:08.178041 | node1 | ok: [node1] => { 2026-08-18 14:24:08.178117 | node1 | "changed": false, 2026-08-18 14:24:08.178321 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:08.178350 | node1 | } 2026-08-18 14:24:08.178429 | node1 | ok: [node2] => { 2026-08-18 14:24:08.178528 | node1 | "changed": false, 2026-08-18 14:24:08.178653 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:08.178748 | node1 | } 2026-08-18 14:24:08.178854 | node1 | ok: [node3] => { 2026-08-18 14:24:08.178960 | node1 | "changed": false, 2026-08-18 14:24:08.179069 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:08.179172 | node1 | } 2026-08-18 14:24:08.179273 | node1 | 2026-08-18 14:24:08.179405 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:24:08.179505 | node1 | Tuesday 18 August 2026 14:24:08 +0000 (0:00:00.087) 0:02:32.444 ******** 2026-08-18 14:24:08.623134 | node1 | ok: [node1] 2026-08-18 14:24:08.623249 | node1 | changed: [node2] 2026-08-18 14:24:08.623404 | node1 | changed: [node3] 2026-08-18 14:24:08.623552 | node1 | 2026-08-18 14:24:08.623761 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:24:08.623917 | node1 | Tuesday 18 August 2026 14:24:08 +0000 (0:00:00.440) 0:02:32.885 ******** 2026-08-18 14:24:08.720354 | node1 | ok: [node1] 2026-08-18 14:24:08.720416 | node1 | ok: [node2] 2026-08-18 14:24:08.720545 | node1 | ok: [node3] 2026-08-18 14:24:08.720697 | node1 | 2026-08-18 14:24:08.720800 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:24:08.720924 | node1 | Tuesday 18 August 2026 14:24:08 +0000 (0:00:00.101) 0:02:32.987 ******** 2026-08-18 14:24:08.835860 | node1 | ok: [node2] 2026-08-18 14:24:08.835961 | node1 | ok: [node1] 2026-08-18 14:24:08.836123 | node1 | ok: [node3] 2026-08-18 14:24:08.836370 | node1 | 2026-08-18 14:24:08.836473 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:24:08.836637 | node1 | Tuesday 18 August 2026 14:24:08 +0000 (0:00:00.115) 0:02:33.102 ******** 2026-08-18 14:24:08.923026 | node1 | ok: [node1] => { 2026-08-18 14:24:08.923065 | node1 | "changed": false, 2026-08-18 14:24:08.923182 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:08.923288 | node1 | } 2026-08-18 14:24:08.923397 | node1 | ok: [node2] => { 2026-08-18 14:24:08.923505 | node1 | "changed": false, 2026-08-18 14:24:08.923625 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:08.923727 | node1 | } 2026-08-18 14:24:08.923832 | node1 | ok: [node3] => { 2026-08-18 14:24:08.923937 | node1 | "changed": false, 2026-08-18 14:24:08.924044 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:08.924147 | node1 | } 2026-08-18 14:24:08.924246 | node1 | 2026-08-18 14:24:08.924400 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:24:08.924504 | node1 | Tuesday 18 August 2026 14:24:08 +0000 (0:00:00.084) 0:02:33.187 ******** 2026-08-18 14:24:30.143013 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:30.143088 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:30.143210 | node1 | changed: [node1] 2026-08-18 14:24:30.143308 | node1 | 2026-08-18 14:24:30.143423 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:24:30.143537 | node1 | Tuesday 18 August 2026 14:24:30 +0000 (0:00:21.221) 0:02:54.409 ******** 2026-08-18 14:24:30.257656 | node1 | skipping: [node1] 2026-08-18 14:24:30.257695 | node1 | skipping: [node2] 2026-08-18 14:24:30.257707 | node1 | skipping: [node3] 2026-08-18 14:24:30.257713 | node1 | 2026-08-18 14:24:30.257719 | node1 | TASK [vexxhost.ceph.mon : Apply the monitor service spec] ********************** 2026-08-18 14:24:30.257728 | node1 | Tuesday 18 August 2026 14:24:30 +0000 (0:00:00.112) 0:02:54.522 ******** 2026-08-18 14:24:33.606073 | node1 | changed: [node1] 2026-08-18 14:24:33.606135 | node1 | 2026-08-18 14:24:33.606367 | node1 | TASK [vexxhost.ceph.mon : Wait for this monitor's daemon to be present] ******** 2026-08-18 14:24:33.606399 | node1 | Tuesday 18 August 2026 14:24:33 +0000 (0:00:03.349) 0:02:57.871 ******** 2026-08-18 14:24:40.065901 | node1 | ok: [node1] 2026-08-18 14:24:40.065975 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:40.065989 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:40.066001 | node1 | 2026-08-18 14:24:40.066013 | node1 | TASK [vexxhost.ceph.mgr : Locate the cluster admin host] *********************** 2026-08-18 14:24:40.066055 | node1 | Tuesday 18 August 2026 14:24:40 +0000 (0:00:06.458) 0:03:04.330 ******** 2026-08-18 14:24:40.801875 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:24:40.801939 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:24:40.801950 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:24:40.801960 | node1 | 2026-08-18 14:24:40.801970 | node1 | TASK [vexxhost.ceph.mgr : Select the admin host] ******************************* 2026-08-18 14:24:40.801980 | node1 | Tuesday 18 August 2026 14:24:40 +0000 (0:00:00.735) 0:03:05.066 ******** 2026-08-18 14:24:40.883770 | node1 | ok: [node1] 2026-08-18 14:24:40.883817 | node1 | ok: [node2] 2026-08-18 14:24:40.883833 | node1 | ok: [node3] 2026-08-18 14:24:40.884571 | node1 | 2026-08-18 14:24:40.884639 | node1 | TASK [Register this manager host] ********************************************** 2026-08-18 14:24:40.884663 | node1 | Tuesday 18 August 2026 14:24:40 +0000 (0:00:00.082) 0:03:05.149 ******** 2026-08-18 14:24:41.015263 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:24:41.015320 | node1 | 2026-08-18 14:24:41.015331 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:24:41.015341 | node1 | Tuesday 18 August 2026 14:24:41 +0000 (0:00:00.131) 0:03:05.280 ******** 2026-08-18 14:24:42.760214 | node1 | ok: [node1] 2026-08-18 14:24:42.760422 | node1 | 2026-08-18 14:24:42.760688 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:24:42.760961 | node1 | Tuesday 18 August 2026 14:24:42 +0000 (0:00:01.745) 0:03:07.026 ******** 2026-08-18 14:24:42.850913 | node1 | ok: [node1] 2026-08-18 14:24:42.851136 | node1 | ok: [node2] 2026-08-18 14:24:42.851383 | node1 | ok: [node3] 2026-08-18 14:24:42.851645 | node1 | 2026-08-18 14:24:42.851906 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:24:42.852164 | node1 | Tuesday 18 August 2026 14:24:42 +0000 (0:00:00.091) 0:03:07.117 ******** 2026-08-18 14:24:42.941073 | node1 | ok: [node1] => { 2026-08-18 14:24:42.941170 | node1 | "changed": false, 2026-08-18 14:24:42.941279 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:42.941385 | node1 | } 2026-08-18 14:24:42.941494 | node1 | ok: [node2] => { 2026-08-18 14:24:42.941593 | node1 | "changed": false, 2026-08-18 14:24:42.941725 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:42.941828 | node1 | } 2026-08-18 14:24:42.941935 | node1 | ok: [node3] => { 2026-08-18 14:24:42.942041 | node1 | "changed": false, 2026-08-18 14:24:42.942149 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:42.942290 | node1 | } 2026-08-18 14:24:42.942376 | node1 | 2026-08-18 14:24:42.942585 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:24:42.942718 | node1 | Tuesday 18 August 2026 14:24:42 +0000 (0:00:00.090) 0:03:07.207 ******** 2026-08-18 14:24:43.332978 | node1 | ok: [node1] 2026-08-18 14:24:43.333206 | node1 | ok: [node2] 2026-08-18 14:24:43.333475 | node1 | ok: [node3] 2026-08-18 14:24:43.333758 | node1 | 2026-08-18 14:24:43.334060 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:24:43.334337 | node1 | Tuesday 18 August 2026 14:24:43 +0000 (0:00:00.391) 0:03:07.599 ******** 2026-08-18 14:24:43.434342 | node1 | ok: [node1] 2026-08-18 14:24:43.434532 | node1 | ok: [node2] 2026-08-18 14:24:43.434894 | node1 | ok: [node3] 2026-08-18 14:24:43.435148 | node1 | 2026-08-18 14:24:43.435442 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:24:43.435726 | node1 | Tuesday 18 August 2026 14:24:43 +0000 (0:00:00.101) 0:03:07.700 ******** 2026-08-18 14:24:43.554844 | node1 | ok: [node1] 2026-08-18 14:24:43.555088 | node1 | ok: [node2] 2026-08-18 14:24:43.555327 | node1 | ok: [node3] 2026-08-18 14:24:43.555552 | node1 | 2026-08-18 14:24:43.555850 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:24:43.556101 | node1 | Tuesday 18 August 2026 14:24:43 +0000 (0:00:00.120) 0:03:07.821 ******** 2026-08-18 14:24:43.640130 | node1 | ok: [node1] => { 2026-08-18 14:24:43.640422 | node1 | "changed": false, 2026-08-18 14:24:43.640729 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:43.640965 | node1 | } 2026-08-18 14:24:43.641220 | node1 | ok: [node2] => { 2026-08-18 14:24:43.641486 | node1 | "changed": false, 2026-08-18 14:24:43.641813 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:43.642077 | node1 | } 2026-08-18 14:24:43.642347 | node1 | ok: [node3] => { 2026-08-18 14:24:43.642649 | node1 | "changed": false, 2026-08-18 14:24:43.643006 | node1 | "msg": "All assertions passed" 2026-08-18 14:24:43.643271 | node1 | } 2026-08-18 14:24:43.643492 | node1 | 2026-08-18 14:24:43.643843 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:24:43.644121 | node1 | Tuesday 18 August 2026 14:24:43 +0000 (0:00:00.085) 0:03:07.906 ******** 2026-08-18 14:24:45.520147 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:45.520259 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:45.520382 | node1 | ok: [node1] 2026-08-18 14:24:45.520483 | node1 | 2026-08-18 14:24:45.520592 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:24:45.520738 | node1 | Tuesday 18 August 2026 14:24:45 +0000 (0:00:01.879) 0:03:09.785 ******** 2026-08-18 14:24:45.654232 | node1 | skipping: [node1] 2026-08-18 14:24:45.654357 | node1 | skipping: [node2] 2026-08-18 14:24:45.654506 | node1 | skipping: [node3] 2026-08-18 14:24:45.654676 | node1 | 2026-08-18 14:24:45.654830 | node1 | TASK [vexxhost.ceph.mgr : Apply the manager service spec] ********************** 2026-08-18 14:24:45.654973 | node1 | Tuesday 18 August 2026 14:24:45 +0000 (0:00:00.135) 0:03:09.920 ******** 2026-08-18 14:24:49.003046 | node1 | changed: [node1] 2026-08-18 14:24:49.003135 | node1 | 2026-08-18 14:24:49.003224 | node1 | TASK [vexxhost.ceph.mgr : Wait for this manager's daemon to be present] ******** 2026-08-18 14:24:49.006647 | node1 | Tuesday 18 August 2026 14:24:49 +0000 (0:00:03.348) 0:03:13.268 ******** 2026-08-18 14:24:55.418433 | node1 | ok: [node1] 2026-08-18 14:24:55.418692 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:55.418932 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:24:55.419158 | node1 | 2026-08-18 14:24:55.419413 | node1 | TASK [vexxhost.ceph.mgr : Configure ceph-mgr modules] ************************** 2026-08-18 14:24:55.419691 | node1 | Tuesday 18 August 2026 14:24:55 +0000 (0:00:06.415) 0:03:19.684 ******** 2026-08-18 14:25:08.117950 | node1 | changed: [node1] => (item={'name': 'diskprediction_local'}) 2026-08-18 14:25:08.118023 | node1 | changed: [node1] => (item={'name': 'dashboard'}) 2026-08-18 14:25:08.118034 | node1 | ok: [node1] => (item={'name': 'iostat'}) 2026-08-18 14:25:08.118044 | node1 | changed: [node1] => (item={'name': 'prometheus'}) 2026-08-18 14:25:08.118053 | node1 | changed: [node1] => (item={'name': 'rgw'}) 2026-08-18 14:25:08.118063 | node1 | 2026-08-18 14:25:08.118072 | node1 | PLAY [Deploy Ceph OSDs] ******************************************************** 2026-08-18 14:25:08.118082 | node1 | 2026-08-18 14:25:08.118092 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:25:08.118101 | node1 | Tuesday 18 August 2026 14:25:08 +0000 (0:00:12.698) 0:03:32.383 ******** 2026-08-18 14:25:10.188248 | node1 | ok: [node2] 2026-08-18 14:25:10.188395 | node1 | ok: [node1] 2026-08-18 14:25:10.189317 | node1 | ok: [node3] 2026-08-18 14:25:10.189340 | node1 | 2026-08-18 14:25:10.189349 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:25:10.189358 | node1 | Tuesday 18 August 2026 14:25:10 +0000 (0:00:02.070) 0:03:34.454 ******** 2026-08-18 14:25:10.491123 | node1 | ok: [node1] 2026-08-18 14:25:10.491188 | node1 | ok: [node2] 2026-08-18 14:25:10.491272 | node1 | ok: [node3] 2026-08-18 14:25:10.491372 | node1 | 2026-08-18 14:25:10.491489 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:25:10.491637 | node1 | Tuesday 18 August 2026 14:25:10 +0000 (0:00:00.302) 0:03:34.756 ******** 2026-08-18 14:25:10.530032 | node1 | skipping: [node1] 2026-08-18 14:25:10.580049 | node1 | skipping: [node2] 2026-08-18 14:25:10.580111 | node1 | skipping: [node3] 2026-08-18 14:25:10.580218 | node1 | 2026-08-18 14:25:10.580334 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:25:10.580447 | node1 | Tuesday 18 August 2026 14:25:10 +0000 (0:00:00.089) 0:03:34.846 ******** 2026-08-18 14:25:10.877018 | node1 | ok: [node1] 2026-08-18 14:25:10.877107 | node1 | ok: [node2] 2026-08-18 14:25:10.877205 | node1 | ok: [node3] 2026-08-18 14:25:10.877306 | node1 | 2026-08-18 14:25:10.877456 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:25:10.877575 | node1 | Tuesday 18 August 2026 14:25:10 +0000 (0:00:00.296) 0:03:35.142 ******** 2026-08-18 14:25:10.963678 | node1 | ok: [node1] => { 2026-08-18 14:25:10.963774 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:25:10.963890 | node1 | } 2026-08-18 14:25:10.963997 | node1 | ok: [node2] => { 2026-08-18 14:25:10.964136 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:25:10.964209 | node1 | } 2026-08-18 14:25:10.964320 | node1 | ok: [node3] => { 2026-08-18 14:25:10.964430 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:25:10.964530 | node1 | } 2026-08-18 14:25:10.964646 | node1 | 2026-08-18 14:25:10.964761 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:25:10.964870 | node1 | Tuesday 18 August 2026 14:25:10 +0000 (0:00:00.088) 0:03:35.230 ******** 2026-08-18 14:25:11.377939 | node1 | ok: [node1] 2026-08-18 14:25:11.377978 | node1 | ok: [node2] 2026-08-18 14:25:11.377985 | node1 | ok: [node3] 2026-08-18 14:25:11.377991 | node1 | 2026-08-18 14:25:11.377996 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:25:11.378002 | node1 | Tuesday 18 August 2026 14:25:11 +0000 (0:00:00.410) 0:03:35.641 ******** 2026-08-18 14:25:11.490490 | node1 | skipping: [node1] 2026-08-18 14:25:11.490917 | node1 | skipping: [node2] 2026-08-18 14:25:11.491256 | node1 | skipping: [node3] 2026-08-18 14:25:11.492143 | node1 | 2026-08-18 14:25:11.492172 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:25:11.492183 | node1 | Tuesday 18 August 2026 14:25:11 +0000 (0:00:00.114) 0:03:35.755 ******** 2026-08-18 14:25:11.592589 | node1 | skipping: [node1] 2026-08-18 14:25:11.592664 | node1 | skipping: [node2] 2026-08-18 14:25:11.592674 | node1 | skipping: [node3] 2026-08-18 14:25:11.592682 | node1 | 2026-08-18 14:25:11.592692 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:25:11.592701 | node1 | Tuesday 18 August 2026 14:25:11 +0000 (0:00:00.101) 0:03:35.857 ******** 2026-08-18 14:25:11.875051 | node1 | ok: [node2] 2026-08-18 14:25:11.875108 | node1 | ok: [node1] 2026-08-18 14:25:11.875232 | node1 | ok: [node3] 2026-08-18 14:25:11.875378 | node1 | 2026-08-18 14:25:11.875517 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:25:11.875672 | node1 | Tuesday 18 August 2026 14:25:11 +0000 (0:00:00.281) 0:03:36.138 ******** 2026-08-18 14:25:13.025802 | node1 | ok: [node2] 2026-08-18 14:25:13.025874 | node1 | ok: [node3] 2026-08-18 14:25:13.026291 | node1 | ok: [node1] 2026-08-18 14:25:13.026312 | node1 | 2026-08-18 14:25:13.026316 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:25:13.026321 | node1 | Tuesday 18 August 2026 14:25:13 +0000 (0:00:01.153) 0:03:37.292 ******** 2026-08-18 14:25:13.136796 | node1 | ok: [node1] => { 2026-08-18 14:25:13.136851 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:25:13.136862 | node1 | } 2026-08-18 14:25:13.136876 | node1 | ok: [node2] => { 2026-08-18 14:25:13.138126 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:25:13.138169 | node1 | } 2026-08-18 14:25:13.138175 | node1 | ok: [node3] => { 2026-08-18 14:25:13.138181 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:25:13.138187 | node1 | } 2026-08-18 14:25:13.138193 | node1 | 2026-08-18 14:25:13.138198 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:25:13.138204 | node1 | Tuesday 18 August 2026 14:25:13 +0000 (0:00:00.110) 0:03:37.402 ******** 2026-08-18 14:25:13.575712 | node1 | ok: [node1] 2026-08-18 14:25:13.575770 | node1 | ok: [node2] 2026-08-18 14:25:13.576720 | node1 | ok: [node3] 2026-08-18 14:25:13.576753 | node1 | 2026-08-18 14:25:13.576763 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:25:13.576773 | node1 | Tuesday 18 August 2026 14:25:13 +0000 (0:00:00.438) 0:03:37.840 ******** 2026-08-18 14:25:16.757041 | node1 | ok: [node2] 2026-08-18 14:25:16.757117 | node1 | ok: [node1] 2026-08-18 14:25:16.757184 | node1 | ok: [node3] 2026-08-18 14:25:16.757284 | node1 | 2026-08-18 14:25:16.757405 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:25:16.757519 | node1 | Tuesday 18 August 2026 14:25:16 +0000 (0:00:03.180) 0:03:41.021 ******** 2026-08-18 14:25:16.826110 | node1 | skipping: [node1] 2026-08-18 14:25:16.826169 | node1 | skipping: [node2] 2026-08-18 14:25:16.826260 | node1 | skipping: [node3] 2026-08-18 14:25:16.826363 | node1 | 2026-08-18 14:25:16.826494 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:25:16.826643 | node1 | Tuesday 18 August 2026 14:25:16 +0000 (0:00:00.071) 0:03:41.092 ******** 2026-08-18 14:25:16.894990 | node1 | skipping: [node1] 2026-08-18 14:25:16.895044 | node1 | skipping: [node2] 2026-08-18 14:25:16.895053 | node1 | skipping: [node3] 2026-08-18 14:25:16.895062 | node1 | 2026-08-18 14:25:16.895071 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:25:16.895094 | node1 | Tuesday 18 August 2026 14:25:16 +0000 (0:00:00.066) 0:03:41.159 ******** 2026-08-18 14:25:16.953786 | node1 | skipping: [node1] 2026-08-18 14:25:16.954035 | node1 | skipping: [node2] 2026-08-18 14:25:16.954302 | node1 | skipping: [node3] 2026-08-18 14:25:16.954517 | node1 | 2026-08-18 14:25:16.954808 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:25:16.955046 | node1 | Tuesday 18 August 2026 14:25:16 +0000 (0:00:00.060) 0:03:41.220 ******** 2026-08-18 14:25:18.083508 | node1 | ok: [node2] 2026-08-18 14:25:18.084242 | node1 | ok: [node3] 2026-08-18 14:25:18.084518 | node1 | ok: [node1] 2026-08-18 14:25:18.085177 | node1 | 2026-08-18 14:25:18.085204 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:25:18.085379 | node1 | Tuesday 18 August 2026 14:25:18 +0000 (0:00:01.129) 0:03:42.349 ******** 2026-08-18 14:25:18.657722 | node1 | ok: [node1] 2026-08-18 14:25:18.657780 | node1 | ok: [node2] 2026-08-18 14:25:18.657790 | node1 | ok: [node3] 2026-08-18 14:25:18.657799 | node1 | 2026-08-18 14:25:18.657808 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:25:18.657818 | node1 | Tuesday 18 August 2026 14:25:18 +0000 (0:00:00.573) 0:03:42.923 ******** 2026-08-18 14:25:19.824192 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:25:19.824441 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:25:19.824759 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:25:19.825039 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:25:19.825299 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:25:19.825564 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:25:19.825865 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:25:19.826121 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:25:19.826454 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:25:19.826726 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:25:19.826892 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:25:19.827055 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:25:19.827223 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:25:19.827377 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:25:19.827544 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:25:19.827791 | node1 | 2026-08-18 14:25:19.827959 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:25:19.828125 | node1 | Tuesday 18 August 2026 14:25:19 +0000 (0:00:01.167) 0:03:44.090 ******** 2026-08-18 14:25:20.536988 | node1 | ok: [node1] 2026-08-18 14:25:20.537198 | node1 | ok: [node3] 2026-08-18 14:25:20.537433 | node1 | ok: [node2] 2026-08-18 14:25:20.537657 | node1 | 2026-08-18 14:25:20.537903 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:25:20.538141 | node1 | Tuesday 18 August 2026 14:25:20 +0000 (0:00:00.693) 0:03:44.784 ******** 2026-08-18 14:25:20.538353 | node1 | 2026-08-18 14:25:20.538702 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:25:20.539021 | node1 | Tuesday 18 August 2026 14:25:20 +0000 (0:00:00.007) 0:03:44.791 ******** 2026-08-18 14:25:20.539241 | node1 | 2026-08-18 14:25:20.539483 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:25:20.539754 | node1 | Tuesday 18 August 2026 14:25:20 +0000 (0:00:00.006) 0:03:44.797 ******** 2026-08-18 14:25:20.539971 | node1 | 2026-08-18 14:25:20.540216 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:25:20.540459 | node1 | Tuesday 18 August 2026 14:25:20 +0000 (0:00:00.006) 0:03:44.803 ******** 2026-08-18 14:25:21.209019 | node1 | ok: [node1] 2026-08-18 14:25:21.209097 | node1 | ok: [node3] 2026-08-18 14:25:21.209195 | node1 | ok: [node2] 2026-08-18 14:25:21.209287 | node1 | 2026-08-18 14:25:21.209409 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:25:21.209512 | node1 | Tuesday 18 August 2026 14:25:21 +0000 (0:00:00.671) 0:03:45.475 ******** 2026-08-18 14:25:21.512334 | node1 | ok: [node1] 2026-08-18 14:25:21.512553 | node1 | ok: [node2] 2026-08-18 14:25:21.512862 | node1 | ok: [node3] 2026-08-18 14:25:21.513100 | node1 | 2026-08-18 14:25:21.513373 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:25:21.513670 | node1 | Tuesday 18 August 2026 14:25:21 +0000 (0:00:00.303) 0:03:45.778 ******** 2026-08-18 14:25:21.606189 | node1 | ok: [node1] => { 2026-08-18 14:25:21.606486 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:25:21.606778 | node1 | } 2026-08-18 14:25:21.607028 | node1 | ok: [node2] => { 2026-08-18 14:25:21.607375 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:25:21.607635 | node1 | } 2026-08-18 14:25:21.607924 | node1 | ok: [node3] => { 2026-08-18 14:25:21.608186 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:25:21.608450 | node1 | } 2026-08-18 14:25:21.608710 | node1 | 2026-08-18 14:25:21.608975 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:25:21.609251 | node1 | Tuesday 18 August 2026 14:25:21 +0000 (0:00:00.093) 0:03:45.872 ******** 2026-08-18 14:25:22.053980 | node1 | ok: [node1] 2026-08-18 14:25:22.054177 | node1 | ok: [node2] 2026-08-18 14:25:22.054494 | node1 | ok: [node3] 2026-08-18 14:25:22.054805 | node1 | 2026-08-18 14:25:22.055084 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:25:22.055338 | node1 | Tuesday 18 August 2026 14:25:22 +0000 (0:00:00.447) 0:03:46.320 ******** 2026-08-18 14:25:25.368095 | node1 | ok: [node1] 2026-08-18 14:25:25.368158 | node1 | ok: [node3] 2026-08-18 14:25:25.368192 | node1 | ok: [node2] 2026-08-18 14:25:25.368205 | node1 | 2026-08-18 14:25:25.368217 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:25:25.368228 | node1 | Tuesday 18 August 2026 14:25:25 +0000 (0:00:03.311) 0:03:49.632 ******** 2026-08-18 14:25:26.474869 | node1 | ok: [node1] 2026-08-18 14:25:26.474918 | node1 | ok: [node2] 2026-08-18 14:25:26.474929 | node1 | ok: [node3] 2026-08-18 14:25:26.474940 | node1 | 2026-08-18 14:25:26.474952 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:25:26.474990 | node1 | Tuesday 18 August 2026 14:25:26 +0000 (0:00:01.107) 0:03:50.739 ******** 2026-08-18 14:25:26.746934 | node1 | ok: [node1] 2026-08-18 14:25:26.746982 | node1 | ok: [node2] 2026-08-18 14:25:26.746992 | node1 | ok: [node3] 2026-08-18 14:25:26.747001 | node1 | 2026-08-18 14:25:26.747010 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:25:26.747020 | node1 | Tuesday 18 August 2026 14:25:26 +0000 (0:00:00.271) 0:03:51.010 ******** 2026-08-18 14:25:27.397817 | node1 | ok: [node2] 2026-08-18 14:25:27.397882 | node1 | ok: [node1] 2026-08-18 14:25:27.398963 | node1 | ok: [node3] 2026-08-18 14:25:27.399061 | node1 | 2026-08-18 14:25:27.399073 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:25:27.399085 | node1 | Tuesday 18 August 2026 14:25:27 +0000 (0:00:00.644) 0:03:51.655 ******** 2026-08-18 14:25:28.103079 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:25:28.103143 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:25:28.103248 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:25:28.103371 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:25:28.103488 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:25:28.103636 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:25:28.103731 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:25:28.103857 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:25:28.103955 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:25:28.104057 | node1 | 2026-08-18 14:25:28.104171 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:25:28.104281 | node1 | Tuesday 18 August 2026 14:25:28 +0000 (0:00:00.711) 0:03:52.366 ******** 2026-08-18 14:25:28.673752 | node1 | ok: [node1] 2026-08-18 14:25:28.673869 | node1 | ok: [node3] 2026-08-18 14:25:28.674059 | node1 | ok: [node2] 2026-08-18 14:25:28.674225 | node1 | 2026-08-18 14:25:28.674598 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:25:28.674663 | node1 | Tuesday 18 August 2026 14:25:28 +0000 (0:00:00.573) 0:03:52.940 ******** 2026-08-18 14:25:29.292666 | node1 | ok: [node1] 2026-08-18 14:25:29.314962 | node1 | ok: [node2] 2026-08-18 14:25:29.315006 | node1 | ok: [node3] 2026-08-18 14:25:29.315019 | node1 | 2026-08-18 14:25:29.315031 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:25:29.315043 | node1 | Tuesday 18 August 2026 14:25:29 +0000 (0:00:00.597) 0:03:53.537 ******** 2026-08-18 14:25:29.315054 | node1 | 2026-08-18 14:25:29.315066 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:25:29.315077 | node1 | Tuesday 18 August 2026 14:25:29 +0000 (0:00:00.008) 0:03:53.545 ******** 2026-08-18 14:25:29.315088 | node1 | 2026-08-18 14:25:29.315099 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:25:29.315110 | node1 | Tuesday 18 August 2026 14:25:29 +0000 (0:00:00.006) 0:03:53.552 ******** 2026-08-18 14:25:29.315121 | node1 | 2026-08-18 14:25:29.315132 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:25:29.315142 | node1 | Tuesday 18 August 2026 14:25:29 +0000 (0:00:00.006) 0:03:53.558 ******** 2026-08-18 14:25:29.789089 | node1 | ok: [node1] 2026-08-18 14:25:29.789161 | node1 | ok: [node2] 2026-08-18 14:25:29.789178 | node1 | ok: [node3] 2026-08-18 14:25:29.789310 | node1 | 2026-08-18 14:25:29.789549 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:25:29.789579 | node1 | Tuesday 18 August 2026 14:25:29 +0000 (0:00:00.496) 0:03:54.055 ******** 2026-08-18 14:25:29.869735 | node1 | ok: [node1] => { 2026-08-18 14:25:29.869778 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:25:29.869787 | node1 | } 2026-08-18 14:25:29.869793 | node1 | ok: [node2] => { 2026-08-18 14:25:29.869813 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:25:29.869834 | node1 | } 2026-08-18 14:25:29.869838 | node1 | ok: [node3] => { 2026-08-18 14:25:29.869843 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:25:29.869847 | node1 | } 2026-08-18 14:25:29.869852 | node1 | 2026-08-18 14:25:29.869857 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:25:29.869862 | node1 | Tuesday 18 August 2026 14:25:29 +0000 (0:00:00.078) 0:03:54.133 ******** 2026-08-18 14:25:30.236051 | node1 | ok: [node1] 2026-08-18 14:25:30.236104 | node1 | ok: [node2] 2026-08-18 14:25:30.236217 | node1 | ok: [node3] 2026-08-18 14:25:30.236319 | node1 | 2026-08-18 14:25:30.236434 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:25:30.236547 | node1 | Tuesday 18 August 2026 14:25:30 +0000 (0:00:00.366) 0:03:54.500 ******** 2026-08-18 14:25:30.321723 | node1 | skipping: [node1] 2026-08-18 14:25:30.321775 | node1 | skipping: [node2] 2026-08-18 14:25:30.321887 | node1 | skipping: [node3] 2026-08-18 14:25:30.321983 | node1 | 2026-08-18 14:25:30.322096 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:25:30.322207 | node1 | Tuesday 18 August 2026 14:25:30 +0000 (0:00:00.088) 0:03:54.588 ******** 2026-08-18 14:25:30.403685 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:25:30.403734 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:25:30.403744 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:25:30.403753 | node1 | 2026-08-18 14:25:30.403762 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:25:30.403772 | node1 | Tuesday 18 August 2026 14:25:30 +0000 (0:00:00.079) 0:03:54.668 ******** 2026-08-18 14:25:31.583187 | node1 | ok: [node2] 2026-08-18 14:25:31.583270 | node1 | ok: [node1] 2026-08-18 14:25:31.583344 | node1 | ok: [node3] 2026-08-18 14:25:31.583456 | node1 | 2026-08-18 14:25:31.583576 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:25:31.583718 | node1 | Tuesday 18 August 2026 14:25:31 +0000 (0:00:01.179) 0:03:55.848 ******** 2026-08-18 14:25:32.433683 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 14:25:32.433750 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:25:32.433763 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:25:32.433775 | node1 | ok: [node1] => (item=ssh) 2026-08-18 14:25:32.433787 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:25:32.433798 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:25:32.434061 | node1 | 2026-08-18 14:25:32.434381 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:25:32.434725 | node1 | Tuesday 18 August 2026 14:25:32 +0000 (0:00:00.849) 0:03:56.697 ******** 2026-08-18 14:25:32.704713 | node1 | ok: [node1] 2026-08-18 14:25:32.705090 | node1 | ok: [node2] 2026-08-18 14:25:32.705437 | node1 | ok: [node3] 2026-08-18 14:25:32.705778 | node1 | 2026-08-18 14:25:32.706115 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:25:32.706444 | node1 | Tuesday 18 August 2026 14:25:32 +0000 (0:00:00.273) 0:03:56.971 ******** 2026-08-18 14:25:33.040289 | node1 | ok: [node1] 2026-08-18 14:25:33.040701 | node1 | ok: [node2] 2026-08-18 14:25:33.041028 | node1 | ok: [node3] 2026-08-18 14:25:33.041364 | node1 | 2026-08-18 14:25:33.041738 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:25:33.042068 | node1 | Tuesday 18 August 2026 14:25:33 +0000 (0:00:00.335) 0:03:57.306 ******** 2026-08-18 14:25:33.599783 | node1 | ok: [node1] 2026-08-18 14:25:33.600021 | node1 | ok: [node2] 2026-08-18 14:25:33.600219 | node1 | ok: [node3] 2026-08-18 14:25:33.600406 | node1 | 2026-08-18 14:25:33.600629 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:25:33.600839 | node1 | Tuesday 18 August 2026 14:25:33 +0000 (0:00:00.559) 0:03:57.866 ******** 2026-08-18 14:25:33.674522 | node1 | skipping: [node1] 2026-08-18 14:25:33.674950 | node1 | skipping: [node2] 2026-08-18 14:25:33.675751 | node1 | skipping: [node3] 2026-08-18 14:25:33.675808 | node1 | 2026-08-18 14:25:33.675825 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:25:33.676043 | node1 | Tuesday 18 August 2026 14:25:33 +0000 (0:00:00.074) 0:03:57.940 ******** 2026-08-18 14:25:34.155800 | node1 | ok: [node1] 2026-08-18 14:25:34.155883 | node1 | ok: [node2] 2026-08-18 14:25:34.155893 | node1 | ok: [node3] 2026-08-18 14:25:34.155937 | node1 | 2026-08-18 14:25:34.156509 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:25:34.156559 | node1 | Tuesday 18 August 2026 14:25:34 +0000 (0:00:00.480) 0:03:58.421 ******** 2026-08-18 14:25:34.738686 | node1 | ok: [node1] 2026-08-18 14:25:34.740036 | node1 | ok: [node2] 2026-08-18 14:25:34.740096 | node1 | ok: [node3] 2026-08-18 14:25:34.740108 | node1 | 2026-08-18 14:25:34.740118 | node1 | TASK [vexxhost.ceph.osd : Locate the cluster admin host] *********************** 2026-08-18 14:25:34.740128 | node1 | Tuesday 18 August 2026 14:25:34 +0000 (0:00:00.582) 0:03:59.004 ******** 2026-08-18 14:25:35.452680 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:25:35.452727 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:25:35.452737 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:25:35.452747 | node1 | 2026-08-18 14:25:35.452756 | node1 | TASK [vexxhost.ceph.osd : Select the admin host and its mon config path] ******* 2026-08-18 14:25:35.452765 | node1 | Tuesday 18 August 2026 14:25:35 +0000 (0:00:00.712) 0:03:59.716 ******** 2026-08-18 14:25:35.538706 | node1 | ok: [node1] 2026-08-18 14:25:35.538763 | node1 | ok: [node2] 2026-08-18 14:25:35.539797 | node1 | ok: [node3] 2026-08-18 14:25:35.539856 | node1 | 2026-08-18 14:25:35.539869 | node1 | TASK [vexxhost.ceph.osd : Set the admin mon config path] *********************** 2026-08-18 14:25:35.539879 | node1 | Tuesday 18 August 2026 14:25:35 +0000 (0:00:00.087) 0:03:59.804 ******** 2026-08-18 14:25:35.627780 | node1 | ok: [node1] 2026-08-18 14:25:35.627839 | node1 | ok: [node2] 2026-08-18 14:25:35.627850 | node1 | ok: [node3] 2026-08-18 14:25:35.627865 | node1 | 2026-08-18 14:25:35.627906 | node1 | TASK [Register this OSD host] ************************************************** 2026-08-18 14:25:35.628066 | node1 | Tuesday 18 August 2026 14:25:35 +0000 (0:00:00.087) 0:03:59.892 ******** 2026-08-18 14:25:35.749061 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:25:35.749144 | node1 | 2026-08-18 14:25:35.749241 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:25:35.749355 | node1 | Tuesday 18 August 2026 14:25:35 +0000 (0:00:00.122) 0:04:00.015 ******** 2026-08-18 14:25:37.457294 | node1 | ok: [node1] 2026-08-18 14:25:37.457365 | node1 | 2026-08-18 14:25:37.457380 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:25:37.457410 | node1 | Tuesday 18 August 2026 14:25:37 +0000 (0:00:01.707) 0:04:01.722 ******** 2026-08-18 14:25:37.531228 | node1 | ok: [node1] 2026-08-18 14:25:37.531276 | node1 | ok: [node2] 2026-08-18 14:25:37.531287 | node1 | ok: [node3] 2026-08-18 14:25:37.531297 | node1 | 2026-08-18 14:25:37.531307 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:25:37.531316 | node1 | Tuesday 18 August 2026 14:25:37 +0000 (0:00:00.073) 0:04:01.796 ******** 2026-08-18 14:25:37.605968 | node1 | ok: [node1] => { 2026-08-18 14:25:37.606008 | node1 | "changed": false, 2026-08-18 14:25:37.606015 | node1 | "msg": "All assertions passed" 2026-08-18 14:25:37.606022 | node1 | } 2026-08-18 14:25:37.606029 | node1 | ok: [node2] => { 2026-08-18 14:25:37.606035 | node1 | "changed": false, 2026-08-18 14:25:37.606041 | node1 | "msg": "All assertions passed" 2026-08-18 14:25:37.606047 | node1 | } 2026-08-18 14:25:37.606053 | node1 | ok: [node3] => { 2026-08-18 14:25:37.606059 | node1 | "changed": false, 2026-08-18 14:25:37.606066 | node1 | "msg": "All assertions passed" 2026-08-18 14:25:37.606072 | node1 | } 2026-08-18 14:25:37.606078 | node1 | 2026-08-18 14:25:37.606095 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:25:37.606102 | node1 | Tuesday 18 August 2026 14:25:37 +0000 (0:00:00.074) 0:04:01.870 ******** 2026-08-18 14:25:37.954688 | node1 | ok: [node1] 2026-08-18 14:25:37.954742 | node1 | ok: [node2] 2026-08-18 14:25:37.954753 | node1 | ok: [node3] 2026-08-18 14:25:37.954762 | node1 | 2026-08-18 14:25:37.954771 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:25:37.954786 | node1 | Tuesday 18 August 2026 14:25:37 +0000 (0:00:00.348) 0:04:02.219 ******** 2026-08-18 14:25:38.025799 | node1 | ok: [node1] 2026-08-18 14:25:38.025888 | node1 | ok: [node2] 2026-08-18 14:25:38.025917 | node1 | ok: [node3] 2026-08-18 14:25:38.025933 | node1 | 2026-08-18 14:25:38.025944 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:25:38.025954 | node1 | Tuesday 18 August 2026 14:25:38 +0000 (0:00:00.070) 0:04:02.289 ******** 2026-08-18 14:25:38.134108 | node1 | ok: [node1] 2026-08-18 14:25:38.134217 | node1 | ok: [node2] 2026-08-18 14:25:38.134227 | node1 | ok: [node3] 2026-08-18 14:25:38.134319 | node1 | 2026-08-18 14:25:38.134441 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:25:38.134556 | node1 | Tuesday 18 August 2026 14:25:38 +0000 (0:00:00.110) 0:04:02.399 ******** 2026-08-18 14:25:38.203063 | node1 | ok: [node1] => { 2026-08-18 14:25:38.203162 | node1 | "changed": false, 2026-08-18 14:25:38.203199 | node1 | "msg": "All assertions passed" 2026-08-18 14:25:38.203307 | node1 | } 2026-08-18 14:25:38.203417 | node1 | ok: [node2] => { 2026-08-18 14:25:38.203525 | node1 | "changed": false, 2026-08-18 14:25:38.204671 | node1 | "msg": "All assertions passed" 2026-08-18 14:25:38.204716 | node1 | } 2026-08-18 14:25:38.204722 | node1 | ok: [node3] => { 2026-08-18 14:25:38.204726 | node1 | "changed": false, 2026-08-18 14:25:38.204731 | node1 | "msg": "All assertions passed" 2026-08-18 14:25:38.204735 | node1 | } 2026-08-18 14:25:38.204739 | node1 | 2026-08-18 14:25:38.204743 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:25:38.204748 | node1 | Tuesday 18 August 2026 14:25:38 +0000 (0:00:00.069) 0:04:02.468 ******** 2026-08-18 14:25:40.302697 | node1 | ok: [node1] 2026-08-18 14:25:40.302771 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:25:40.302787 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:25:40.302801 | node1 | 2026-08-18 14:25:40.302816 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:25:40.302830 | node1 | Tuesday 18 August 2026 14:25:40 +0000 (0:00:02.098) 0:04:04.567 ******** 2026-08-18 14:25:40.395568 | node1 | skipping: [node1] 2026-08-18 14:25:40.395658 | node1 | skipping: [node2] 2026-08-18 14:25:40.395675 | node1 | skipping: [node3] 2026-08-18 14:25:40.395687 | node1 | 2026-08-18 14:25:40.395700 | node1 | TASK [vexxhost.ceph.osd : Apply OSD service specs (drive groups)] ************** 2026-08-18 14:25:40.395712 | node1 | Tuesday 18 August 2026 14:25:40 +0000 (0:00:00.093) 0:04:04.660 ******** 2026-08-18 14:25:40.428711 | node1 | skipping: [node1] 2026-08-18 14:25:40.428774 | node1 | 2026-08-18 14:25:40.429186 | node1 | TASK [vexxhost.ceph.osd : List existing ceph-volume OSDs on this host] ********* 2026-08-18 14:25:40.429234 | node1 | Tuesday 18 August 2026 14:25:40 +0000 (0:00:00.033) 0:04:04.694 ******** 2026-08-18 14:25:41.955071 | node1 | ok: [node1] 2026-08-18 14:25:41.955162 | node1 | ok: [node3] 2026-08-18 14:25:41.955180 | node1 | ok: [node2] 2026-08-18 14:25:41.955294 | node1 | 2026-08-18 14:25:41.955407 | node1 | TASK [vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices] ******** 2026-08-18 14:25:41.955521 | node1 | Tuesday 18 August 2026 14:25:41 +0000 (0:00:01.527) 0:04:06.221 ******** 2026-08-18 14:29:41.953121 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd0/data) 2026-08-18 14:29:41.953725 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd0/data) 2026-08-18 14:29:41.953785 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd0/data) 2026-08-18 14:29:41.953799 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd1/data) 2026-08-18 14:29:41.953815 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd1/data) 2026-08-18 14:29:41.953849 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd1/data) 2026-08-18 14:29:41.954507 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd2/data) 2026-08-18 14:29:41.954520 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd2/data) 2026-08-18 14:29:41.954525 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd2/data) 2026-08-18 14:29:41.954530 | node1 | 2026-08-18 14:29:41.954536 | node1 | TASK [vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host] *** 2026-08-18 14:29:41.954540 | node1 | Tuesday 18 August 2026 14:29:41 +0000 (0:03:59.997) 0:08:06.218 ******** 2026-08-18 14:29:58.123688 | node1 | FAILED - RETRYING: [node1]: Wait for explicitly requested OSD daemons to be running on this host (120 retries left). 2026-08-18 14:29:58.123736 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:29:58.124662 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:29:58.124695 | node1 | FAILED - RETRYING: [node1]: Wait for explicitly requested OSD daemons to be running on this host (119 retries left). 2026-08-18 14:29:58.124701 | node1 | ok: [node1] 2026-08-18 14:29:58.125380 | node1 | 2026-08-18 14:29:58.125413 | node1 | TASK [vexxhost.ceph.osd : Wait for OSD drive-group services to deploy at least one daemon] *** 2026-08-18 14:29:58.125427 | node1 | Tuesday 18 August 2026 14:29:58 +0000 (0:00:16.170) 0:08:22.388 ******** 2026-08-18 14:29:58.206141 | node1 | skipping: [node1] 2026-08-18 14:29:58.206206 | node1 | 2026-08-18 14:29:58.206313 | node1 | PLAY [Apply Ceph configuration] ************************************************ 2026-08-18 14:29:58.206402 | node1 | 2026-08-18 14:29:58.206520 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:29:58.206681 | node1 | Tuesday 18 August 2026 14:29:58 +0000 (0:00:00.082) 0:08:22.471 ******** 2026-08-18 14:30:00.379100 | node1 | ok: [node1] 2026-08-18 14:30:00.379177 | node1 | ok: [node3] 2026-08-18 14:30:00.379228 | node1 | ok: [node2] 2026-08-18 14:30:00.379332 | node1 | 2026-08-18 14:30:00.379449 | node1 | TASK [vexxhost.ceph.configure : Locate the cluster admin host] ***************** 2026-08-18 14:30:00.379564 | node1 | Tuesday 18 August 2026 14:30:00 +0000 (0:00:02.174) 0:08:24.645 ******** 2026-08-18 14:30:01.115530 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:30:01.115569 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:30:01.115576 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:30:01.115582 | node1 | 2026-08-18 14:30:01.115587 | node1 | TASK [vexxhost.ceph.configure : Select the admin host] ************************* 2026-08-18 14:30:01.115593 | node1 | Tuesday 18 August 2026 14:30:01 +0000 (0:00:00.734) 0:08:25.380 ******** 2026-08-18 14:30:01.188952 | node1 | ok: [node1] 2026-08-18 14:30:01.188999 | node1 | ok: [node2] 2026-08-18 14:30:01.189007 | node1 | ok: [node3] 2026-08-18 14:30:01.189015 | node1 | 2026-08-18 14:30:01.189023 | node1 | TASK [vexxhost.ceph.configure : Set the admin mon config path] ***************** 2026-08-18 14:30:01.189031 | node1 | Tuesday 18 August 2026 14:30:01 +0000 (0:00:00.072) 0:08:25.452 ******** 2026-08-18 14:30:01.260145 | node1 | ok: [node1] 2026-08-18 14:30:01.260201 | node1 | ok: [node2] 2026-08-18 14:30:01.260214 | node1 | ok: [node3] 2026-08-18 14:30:01.260226 | node1 | 2026-08-18 14:30:01.260239 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 14:30:01.260251 | node1 | Tuesday 18 August 2026 14:30:01 +0000 (0:00:00.071) 0:08:25.524 ******** 2026-08-18 14:30:01.342330 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/config.yml for node1, node2, node3 2026-08-18 14:30:01.342679 | node1 | 2026-08-18 14:30:01.343091 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 14:30:01.343121 | node1 | Tuesday 18 August 2026 14:30:01 +0000 (0:00:00.083) 0:08:25.608 ******** 2026-08-18 14:30:28.881779 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/standby_behaviour) 2026-08-18 14:30:28.881878 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/RGW_API_SSL_VERIFY) 2026-08-18 14:30:28.882700 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_CEPHFS) 2026-08-18 14:30:28.882762 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_ISCSI) 2026-08-18 14:30:28.882772 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_MIRRORING) 2026-08-18 14:30:28.882787 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_NFS) 2026-08-18 14:30:28.883546 | node1 | 2026-08-18 14:30:28.883571 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 14:30:28.883582 | node1 | Tuesday 18 August 2026 14:30:28 +0000 (0:00:27.539) 0:08:53.147 ******** 2026-08-18 14:30:28.953859 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/ec_profiles.yml for node1, node2, node3 2026-08-18 14:30:28.953917 | node1 | 2026-08-18 14:30:28.954213 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 14:30:28.954584 | node1 | Tuesday 18 August 2026 14:30:28 +0000 (0:00:00.071) 0:08:53.219 ******** 2026-08-18 14:30:33.674318 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 14:30:33.674655 | node1 | 2026-08-18 14:30:33.675376 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 14:30:33.675405 | node1 | Tuesday 18 August 2026 14:30:33 +0000 (0:00:04.721) 0:08:57.940 ******** 2026-08-18 14:30:33.756799 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/crush_rules.yml for node1, node2, node3 2026-08-18 14:30:33.756877 | node1 | 2026-08-18 14:30:33.757320 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 14:30:33.757350 | node1 | Tuesday 18 August 2026 14:30:33 +0000 (0:00:00.082) 0:08:58.023 ******** 2026-08-18 14:30:46.993212 | node1 | changed: [node1] => (item=rep_hdd) 2026-08-18 14:30:46.993263 | node1 | changed: [node1] => (item=rep_lab) 2026-08-18 14:30:46.993269 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 14:30:46.993275 | node1 | 2026-08-18 14:30:46.993280 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 14:30:46.993286 | node1 | Tuesday 18 August 2026 14:30:46 +0000 (0:00:13.236) 0:09:11.259 ******** 2026-08-18 14:30:47.071532 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/pools.yml for node1, node2, node3 2026-08-18 14:30:47.071653 | node1 | 2026-08-18 14:30:47.071671 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 14:30:47.071685 | node1 | Tuesday 18 August 2026 14:30:47 +0000 (0:00:00.076) 0:09:11.335 ******** 2026-08-18 14:31:23.963127 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.data) 2026-08-18 14:31:23.963202 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.index) 2026-08-18 14:31:23.963298 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.non-ec) 2026-08-18 14:31:23.963402 | node1 | changed: [node1] => (item=molecule-1.rgw.control) 2026-08-18 14:31:23.963512 | node1 | changed: [node1] => (item=molecule-1.rgw.log) 2026-08-18 14:31:23.963650 | node1 | changed: [node1] => (item=molecule-1.rgw.meta) 2026-08-18 14:31:23.963972 | node1 | 2026-08-18 14:31:23.963988 | node1 | TASK [vexxhost.ceph.configure : Configure CephX keys] ************************** 2026-08-18 14:31:23.963995 | node1 | Tuesday 18 August 2026 14:31:23 +0000 (0:00:36.892) 0:09:48.228 ******** 2026-08-18 14:31:24.032454 | node1 | skipping: [node1] 2026-08-18 14:31:24.032499 | node1 | skipping: [node2] 2026-08-18 14:31:24.032511 | node1 | skipping: [node3] 2026-08-18 14:31:24.032523 | node1 | 2026-08-18 14:31:24.032535 | node1 | TASK [vexxhost.ceph.configure : Enroll the dashboard TLS certificate] ********** 2026-08-18 14:31:24.032547 | node1 | Tuesday 18 August 2026 14:31:24 +0000 (0:00:00.069) 0:09:48.297 ******** 2026-08-18 14:31:24.128180 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard.yml for node1, node2, node3 2026-08-18 14:31:24.128238 | node1 | 2026-08-18 14:31:24.128252 | node1 | TASK [vexxhost.ceph.configure : Check whether the dashboard mgr module is enabled] *** 2026-08-18 14:31:24.128264 | node1 | Tuesday 18 August 2026 14:31:24 +0000 (0:00:00.096) 0:09:48.393 ******** 2026-08-18 14:31:25.986838 | node1 | ok: [node1] 2026-08-18 14:31:25.986898 | node1 | 2026-08-18 14:31:25.987355 | node1 | TASK [vexxhost.ceph.configure : Get the current dashboard certificate] ********* 2026-08-18 14:31:25.987414 | node1 | Tuesday 18 August 2026 14:31:25 +0000 (0:00:01.857) 0:09:50.250 ******** 2026-08-18 14:31:27.853253 | node1 | ok: [node1] 2026-08-18 14:31:27.853316 | node1 | 2026-08-18 14:31:27.853325 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate] ***************** 2026-08-18 14:31:27.853334 | node1 | Tuesday 18 August 2026 14:31:27 +0000 (0:00:01.868) 0:09:52.119 ******** 2026-08-18 14:31:27.897539 | node1 | skipping: [node1] 2026-08-18 14:31:27.897653 | node1 | 2026-08-18 14:31:27.897679 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate key] ************* 2026-08-18 14:31:27.897691 | node1 | Tuesday 18 August 2026 14:31:27 +0000 (0:00:00.044) 0:09:52.164 ******** 2026-08-18 14:31:27.940841 | node1 | skipping: [node1] 2026-08-18 14:31:27.941072 | node1 | 2026-08-18 14:31:27.941349 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 14:31:27.942113 | node1 | Tuesday 18 August 2026 14:31:27 +0000 (0:00:00.043) 0:09:52.207 ******** 2026-08-18 14:31:27.987088 | node1 | skipping: [node1] 2026-08-18 14:31:27.987294 | node1 | 2026-08-18 14:31:27.987584 | node1 | TASK [vexxhost.ceph.configure : Create a self-signed certificate] ************** 2026-08-18 14:31:27.987960 | node1 | Tuesday 18 August 2026 14:31:27 +0000 (0:00:00.046) 0:09:52.253 ******** 2026-08-18 14:31:30.072388 | node1 | changed: [node1] 2026-08-18 14:31:30.072489 | node1 | 2026-08-18 14:31:30.072869 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 14:31:30.073179 | node1 | Tuesday 18 August 2026 14:31:30 +0000 (0:00:02.084) 0:09:54.338 ******** 2026-08-18 14:31:30.122084 | node1 | ok: [node1] 2026-08-18 14:31:30.122175 | node1 | 2026-08-18 14:31:30.122297 | node1 | TASK [vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate] *** 2026-08-18 14:31:30.122411 | node1 | Tuesday 18 August 2026 14:31:30 +0000 (0:00:00.049) 0:09:54.388 ******** 2026-08-18 14:31:37.436266 | node1 | changed: [node1] => (item=ceph mgr module disable dashboard) 2026-08-18 14:31:37.436340 | node1 | changed: [node1] => (item=ceph mgr module enable dashboard) 2026-08-18 14:31:37.436424 | node1 | changed: [node1] => (item=ceph orch reconfig mgr) 2026-08-18 14:31:37.436521 | node1 | 2026-08-18 14:31:37.437633 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 14:31:37.437651 | node1 | Tuesday 18 August 2026 14:31:37 +0000 (0:00:07.314) 0:10:01.702 ******** 2026-08-18 14:31:37.532633 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard_users.yml for node1, node2, node3 2026-08-18 14:31:37.532680 | node1 | 2026-08-18 14:31:37.532690 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 14:31:37.532698 | node1 | Tuesday 18 August 2026 14:31:37 +0000 (0:00:00.095) 0:10:01.798 ******** 2026-08-18 14:31:48.949731 | node1 | changed: [node1] => (item=None) 2026-08-18 14:31:48.949784 | node1 | changed: [node1] => (item=None) 2026-08-18 14:31:48.949791 | node1 | changed: [node1 -> {{ ceph_configure_admin_host }}] 2026-08-18 14:31:48.949797 | node1 | 2026-08-18 14:31:48.949804 | node1 | PLAY [Deploy Ceph RADOS Gateways] ********************************************** 2026-08-18 14:31:48.949811 | node1 | 2026-08-18 14:31:48.949816 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:31:48.949826 | node1 | Tuesday 18 August 2026 14:31:48 +0000 (0:00:11.417) 0:10:13.215 ******** 2026-08-18 14:31:51.179051 | node1 | ok: [node1] 2026-08-18 14:31:51.182070 | node1 | ok: [node2] 2026-08-18 14:31:51.182094 | node1 | ok: [node3] 2026-08-18 14:31:51.182107 | node1 | 2026-08-18 14:31:51.182120 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:31:51.182132 | node1 | Tuesday 18 August 2026 14:31:51 +0000 (0:00:02.228) 0:10:15.444 ******** 2026-08-18 14:31:51.479026 | node1 | ok: [node1] 2026-08-18 14:31:51.479114 | node1 | ok: [node2] 2026-08-18 14:31:51.479180 | node1 | ok: [node3] 2026-08-18 14:31:51.479307 | node1 | 2026-08-18 14:31:51.479413 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:31:51.479527 | node1 | Tuesday 18 August 2026 14:31:51 +0000 (0:00:00.300) 0:10:15.744 ******** 2026-08-18 14:31:51.570590 | node1 | skipping: [node1] 2026-08-18 14:31:51.570682 | node1 | skipping: [node2] 2026-08-18 14:31:51.570774 | node1 | skipping: [node3] 2026-08-18 14:31:51.570873 | node1 | 2026-08-18 14:31:51.570989 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:31:51.571102 | node1 | Tuesday 18 August 2026 14:31:51 +0000 (0:00:00.092) 0:10:15.837 ******** 2026-08-18 14:31:51.888642 | node1 | ok: [node1] 2026-08-18 14:31:51.888704 | node1 | ok: [node2] 2026-08-18 14:31:51.888714 | node1 | ok: [node3] 2026-08-18 14:31:51.888723 | node1 | 2026-08-18 14:31:51.888732 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:31:51.888741 | node1 | Tuesday 18 August 2026 14:31:51 +0000 (0:00:00.315) 0:10:16.153 ******** 2026-08-18 14:31:51.993703 | node1 | ok: [node1] => { 2026-08-18 14:31:51.993772 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:31:51.994654 | node1 | } 2026-08-18 14:31:51.994715 | node1 | ok: [node2] => { 2026-08-18 14:31:51.994727 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:31:51.995689 | node1 | } 2026-08-18 14:31:51.995739 | node1 | ok: [node3] => { 2026-08-18 14:31:51.995750 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:31:51.995760 | node1 | } 2026-08-18 14:31:51.996019 | node1 | 2026-08-18 14:31:51.996346 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:31:51.996714 | node1 | Tuesday 18 August 2026 14:31:51 +0000 (0:00:00.105) 0:10:16.258 ******** 2026-08-18 14:31:52.415980 | node1 | ok: [node1] 2026-08-18 14:31:52.416068 | node1 | ok: [node2] 2026-08-18 14:31:52.416232 | node1 | ok: [node3] 2026-08-18 14:31:52.416373 | node1 | 2026-08-18 14:31:52.416534 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:31:52.416786 | node1 | Tuesday 18 August 2026 14:31:52 +0000 (0:00:00.423) 0:10:16.682 ******** 2026-08-18 14:31:52.581118 | node1 | skipping: [node1] 2026-08-18 14:31:52.581195 | node1 | skipping: [node2] 2026-08-18 14:31:52.581244 | node1 | skipping: [node3] 2026-08-18 14:31:52.581350 | node1 | 2026-08-18 14:31:52.581465 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:31:52.581595 | node1 | Tuesday 18 August 2026 14:31:52 +0000 (0:00:00.164) 0:10:16.847 ******** 2026-08-18 14:31:52.698147 | node1 | skipping: [node1] 2026-08-18 14:31:52.698217 | node1 | skipping: [node2] 2026-08-18 14:31:52.698232 | node1 | skipping: [node3] 2026-08-18 14:31:52.698352 | node1 | 2026-08-18 14:31:52.698459 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:31:52.698569 | node1 | Tuesday 18 August 2026 14:31:52 +0000 (0:00:00.117) 0:10:16.964 ******** 2026-08-18 14:31:53.006694 | node1 | ok: [node1] 2026-08-18 14:31:53.006748 | node1 | ok: [node2] 2026-08-18 14:31:53.006758 | node1 | ok: [node3] 2026-08-18 14:31:53.006767 | node1 | 2026-08-18 14:31:53.006777 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:31:53.006786 | node1 | Tuesday 18 August 2026 14:31:53 +0000 (0:00:00.307) 0:10:17.271 ******** 2026-08-18 14:31:54.242107 | node1 | ok: [node2] 2026-08-18 14:31:54.242185 | node1 | ok: [node1] 2026-08-18 14:31:54.242199 | node1 | ok: [node3] 2026-08-18 14:31:54.242314 | node1 | 2026-08-18 14:31:54.242426 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:31:54.242539 | node1 | Tuesday 18 August 2026 14:31:54 +0000 (0:00:01.235) 0:10:18.506 ******** 2026-08-18 14:31:54.353466 | node1 | ok: [node1] => { 2026-08-18 14:31:54.353560 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:31:54.353699 | node1 | } 2026-08-18 14:31:54.354697 | node1 | ok: [node2] => { 2026-08-18 14:31:54.354736 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:31:54.354754 | node1 | } 2026-08-18 14:31:54.354758 | node1 | ok: [node3] => { 2026-08-18 14:31:54.354769 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:31:54.354773 | node1 | } 2026-08-18 14:31:54.354777 | node1 | 2026-08-18 14:31:54.354781 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:31:54.354785 | node1 | Tuesday 18 August 2026 14:31:54 +0000 (0:00:00.113) 0:10:18.620 ******** 2026-08-18 14:31:54.812226 | node1 | ok: [node2] 2026-08-18 14:31:54.812299 | node1 | ok: [node1] 2026-08-18 14:31:54.812392 | node1 | ok: [node3] 2026-08-18 14:31:54.812521 | node1 | 2026-08-18 14:31:54.812680 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:31:54.812799 | node1 | Tuesday 18 August 2026 14:31:54 +0000 (0:00:00.454) 0:10:19.074 ******** 2026-08-18 14:31:56.996716 | node1 | ok: [node2] 2026-08-18 14:31:56.996791 | node1 | ok: [node3] 2026-08-18 14:31:56.996868 | node1 | ok: [node1] 2026-08-18 14:31:56.996965 | node1 | 2026-08-18 14:31:56.997080 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:31:56.997191 | node1 | Tuesday 18 August 2026 14:31:56 +0000 (0:00:02.188) 0:10:21.263 ******** 2026-08-18 14:31:57.058797 | node1 | skipping: [node1] 2026-08-18 14:31:57.059004 | node1 | skipping: [node2] 2026-08-18 14:31:57.059015 | node1 | skipping: [node3] 2026-08-18 14:31:57.059027 | node1 | 2026-08-18 14:31:57.059297 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:31:57.059349 | node1 | Tuesday 18 August 2026 14:31:57 +0000 (0:00:00.062) 0:10:21.325 ******** 2026-08-18 14:31:57.134155 | node1 | skipping: [node1] 2026-08-18 14:31:57.134241 | node1 | skipping: [node2] 2026-08-18 14:31:57.134416 | node1 | skipping: [node3] 2026-08-18 14:31:57.134927 | node1 | 2026-08-18 14:31:57.134968 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:31:57.134976 | node1 | Tuesday 18 August 2026 14:31:57 +0000 (0:00:00.074) 0:10:21.400 ******** 2026-08-18 14:31:57.213793 | node1 | skipping: [node1] 2026-08-18 14:31:57.213868 | node1 | skipping: [node2] 2026-08-18 14:31:57.213884 | node1 | skipping: [node3] 2026-08-18 14:31:57.214628 | node1 | 2026-08-18 14:31:57.214697 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:31:57.214708 | node1 | Tuesday 18 August 2026 14:31:57 +0000 (0:00:00.077) 0:10:21.477 ******** 2026-08-18 14:31:58.430700 | node1 | ok: [node2] 2026-08-18 14:31:58.430764 | node1 | ok: [node1] 2026-08-18 14:31:58.430774 | node1 | ok: [node3] 2026-08-18 14:31:58.430783 | node1 | 2026-08-18 14:31:58.430896 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:31:58.430910 | node1 | Tuesday 18 August 2026 14:31:58 +0000 (0:00:01.217) 0:10:22.695 ******** 2026-08-18 14:31:59.009983 | node1 | ok: [node1] 2026-08-18 14:31:59.010031 | node1 | ok: [node2] 2026-08-18 14:31:59.010037 | node1 | ok: [node3] 2026-08-18 14:31:59.010043 | node1 | 2026-08-18 14:31:59.010049 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:31:59.010055 | node1 | Tuesday 18 August 2026 14:31:59 +0000 (0:00:00.579) 0:10:23.275 ******** 2026-08-18 14:32:00.155729 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 14:32:00.155803 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:32:00.155813 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:32:00.155823 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:32:00.155833 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:32:00.155849 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:32:00.156680 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:32:00.156748 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:32:00.156758 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:32:00.156999 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:32:00.157328 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:32:00.157695 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:32:00.158054 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:32:00.158382 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:32:00.158735 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:32:00.159035 | node1 | 2026-08-18 14:32:00.159385 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:32:00.159723 | node1 | Tuesday 18 August 2026 14:32:00 +0000 (0:00:01.145) 0:10:24.420 ******** 2026-08-18 14:32:00.925761 | node1 | ok: [node1] 2026-08-18 14:32:00.926254 | node1 | ok: [node2] 2026-08-18 14:32:00.928646 | node1 | ok: [node3] 2026-08-18 14:32:00.928678 | node1 | 2026-08-18 14:32:00.928692 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:32:00.928705 | node1 | Tuesday 18 August 2026 14:32:00 +0000 (0:00:00.716) 0:10:25.137 ******** 2026-08-18 14:32:00.928716 | node1 | 2026-08-18 14:32:00.928728 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:32:00.928740 | node1 | Tuesday 18 August 2026 14:32:00 +0000 (0:00:00.016) 0:10:25.153 ******** 2026-08-18 14:32:00.928847 | node1 | 2026-08-18 14:32:00.929211 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:32:00.929700 | node1 | Tuesday 18 August 2026 14:32:00 +0000 (0:00:00.018) 0:10:25.172 ******** 2026-08-18 14:32:00.930087 | node1 | 2026-08-18 14:32:00.930363 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:32:00.930592 | node1 | Tuesday 18 August 2026 14:32:00 +0000 (0:00:00.019) 0:10:25.191 ******** 2026-08-18 14:32:01.419308 | node1 | ok: [node1] 2026-08-18 14:32:01.419746 | node1 | ok: [node2] 2026-08-18 14:32:01.423091 | node1 | ok: [node3] 2026-08-18 14:32:01.423865 | node1 | 2026-08-18 14:32:01.424225 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:32:01.424541 | node1 | Tuesday 18 August 2026 14:32:01 +0000 (0:00:00.493) 0:10:25.685 ******** 2026-08-18 14:32:01.729388 | node1 | ok: [node1] 2026-08-18 14:32:01.729799 | node1 | ok: [node2] 2026-08-18 14:32:01.730104 | node1 | ok: [node3] 2026-08-18 14:32:01.730381 | node1 | 2026-08-18 14:32:01.730719 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:32:01.731030 | node1 | Tuesday 18 August 2026 14:32:01 +0000 (0:00:00.310) 0:10:25.995 ******** 2026-08-18 14:32:01.835486 | node1 | ok: [node1] => { 2026-08-18 14:32:01.835931 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:32:01.836244 | node1 | } 2026-08-18 14:32:01.836552 | node1 | ok: [node2] => { 2026-08-18 14:32:01.836919 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:32:01.837250 | node1 | } 2026-08-18 14:32:01.837554 | node1 | ok: [node3] => { 2026-08-18 14:32:01.837909 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:32:01.838213 | node1 | } 2026-08-18 14:32:01.838536 | node1 | 2026-08-18 14:32:01.838964 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:32:01.839293 | node1 | Tuesday 18 August 2026 14:32:01 +0000 (0:00:00.106) 0:10:26.101 ******** 2026-08-18 14:32:02.302256 | node1 | ok: [node3] 2026-08-18 14:32:02.303698 | node1 | ok: [node2] 2026-08-18 14:32:02.303759 | node1 | ok: [node1] 2026-08-18 14:32:02.303770 | node1 | 2026-08-18 14:32:02.303779 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:32:02.303789 | node1 | Tuesday 18 August 2026 14:32:02 +0000 (0:00:00.466) 0:10:26.568 ******** 2026-08-18 14:32:05.670004 | node1 | ok: [node2] 2026-08-18 14:32:05.670249 | node1 | ok: [node3] 2026-08-18 14:32:05.670457 | node1 | ok: [node1] 2026-08-18 14:32:05.670728 | node1 | 2026-08-18 14:32:05.670973 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:32:05.671228 | node1 | Tuesday 18 August 2026 14:32:05 +0000 (0:00:03.367) 0:10:29.936 ******** 2026-08-18 14:32:06.782690 | node1 | ok: [node2] 2026-08-18 14:32:06.782877 | node1 | ok: [node1] 2026-08-18 14:32:06.783151 | node1 | ok: [node3] 2026-08-18 14:32:06.783377 | node1 | 2026-08-18 14:32:06.783689 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:32:06.783935 | node1 | Tuesday 18 August 2026 14:32:06 +0000 (0:00:01.112) 0:10:31.048 ******** 2026-08-18 14:32:07.064488 | node1 | ok: [node1] 2026-08-18 14:32:07.064739 | node1 | ok: [node2] 2026-08-18 14:32:07.064977 | node1 | ok: [node3] 2026-08-18 14:32:07.065255 | node1 | 2026-08-18 14:32:07.065523 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:32:07.065822 | node1 | Tuesday 18 August 2026 14:32:07 +0000 (0:00:00.281) 0:10:31.329 ******** 2026-08-18 14:32:07.641381 | node1 | ok: [node1] 2026-08-18 14:32:07.641582 | node1 | ok: [node2] 2026-08-18 14:32:07.641872 | node1 | ok: [node3] 2026-08-18 14:32:07.642090 | node1 | 2026-08-18 14:32:07.642348 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:32:07.642647 | node1 | Tuesday 18 August 2026 14:32:07 +0000 (0:00:00.577) 0:10:31.907 ******** 2026-08-18 14:32:08.358282 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 14:32:08.358320 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:32:08.358324 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:32:08.358331 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:32:08.358654 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:32:08.359680 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:32:08.359737 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:32:08.359747 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:32:08.359756 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:32:08.359966 | node1 | 2026-08-18 14:32:08.360364 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:32:08.360725 | node1 | Tuesday 18 August 2026 14:32:08 +0000 (0:00:00.715) 0:10:32.623 ******** 2026-08-18 14:32:08.921046 | node1 | ok: [node2] 2026-08-18 14:32:08.921123 | node1 | ok: [node1] 2026-08-18 14:32:08.921196 | node1 | ok: [node3] 2026-08-18 14:32:08.921295 | node1 | 2026-08-18 14:32:08.921409 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:32:08.921521 | node1 | Tuesday 18 August 2026 14:32:08 +0000 (0:00:00.563) 0:10:33.187 ******** 2026-08-18 14:32:09.540078 | node1 | ok: [node1] 2026-08-18 14:32:09.540158 | node1 | ok: [node2] 2026-08-18 14:32:09.540225 | node1 | ok: [node3] 2026-08-18 14:32:09.540324 | node1 | 2026-08-18 14:32:09.541548 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:32:09.541587 | node1 | Tuesday 18 August 2026 14:32:09 +0000 (0:00:00.576) 0:10:33.764 ******** 2026-08-18 14:32:09.541592 | node1 | 2026-08-18 14:32:09.541657 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:32:09.541666 | node1 | Tuesday 18 August 2026 14:32:09 +0000 (0:00:00.015) 0:10:33.779 ******** 2026-08-18 14:32:09.541670 | node1 | 2026-08-18 14:32:09.541674 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:32:09.541678 | node1 | Tuesday 18 August 2026 14:32:09 +0000 (0:00:00.013) 0:10:33.792 ******** 2026-08-18 14:32:09.541682 | node1 | 2026-08-18 14:32:09.541685 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:32:09.541689 | node1 | Tuesday 18 August 2026 14:32:09 +0000 (0:00:00.013) 0:10:33.806 ******** 2026-08-18 14:32:10.033748 | node1 | ok: [node1] 2026-08-18 14:32:10.033828 | node1 | ok: [node2] 2026-08-18 14:32:10.033864 | node1 | ok: [node3] 2026-08-18 14:32:10.033874 | node1 | 2026-08-18 14:32:10.033891 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:32:10.034189 | node1 | Tuesday 18 August 2026 14:32:10 +0000 (0:00:00.493) 0:10:34.299 ******** 2026-08-18 14:32:10.104157 | node1 | ok: [node1] => { 2026-08-18 14:32:10.105847 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:32:10.105886 | node1 | } 2026-08-18 14:32:10.105891 | node1 | ok: [node2] => { 2026-08-18 14:32:10.105895 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:32:10.105899 | node1 | } 2026-08-18 14:32:10.105903 | node1 | ok: [node3] => { 2026-08-18 14:32:10.105906 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:32:10.105910 | node1 | } 2026-08-18 14:32:10.105914 | node1 | 2026-08-18 14:32:10.105918 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:32:10.105922 | node1 | Tuesday 18 August 2026 14:32:10 +0000 (0:00:00.071) 0:10:34.370 ******** 2026-08-18 14:32:10.504745 | node1 | ok: [node1] 2026-08-18 14:32:10.504798 | node1 | ok: [node2] 2026-08-18 14:32:10.504809 | node1 | ok: [node3] 2026-08-18 14:32:10.504818 | node1 | 2026-08-18 14:32:10.504827 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:32:10.504850 | node1 | Tuesday 18 August 2026 14:32:10 +0000 (0:00:00.398) 0:10:34.769 ******** 2026-08-18 14:32:10.668565 | node1 | skipping: [node1] 2026-08-18 14:32:10.668643 | node1 | skipping: [node2] 2026-08-18 14:32:10.668658 | node1 | skipping: [node3] 2026-08-18 14:32:10.668667 | node1 | 2026-08-18 14:32:10.668676 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:32:10.668685 | node1 | Tuesday 18 August 2026 14:32:10 +0000 (0:00:00.164) 0:10:34.933 ******** 2026-08-18 14:32:10.756832 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:32:10.756913 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:32:10.757086 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:32:10.757342 | node1 | 2026-08-18 14:32:10.757632 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:32:10.757916 | node1 | Tuesday 18 August 2026 14:32:10 +0000 (0:00:00.089) 0:10:35.023 ******** 2026-08-18 14:32:11.895370 | node1 | ok: [node2] 2026-08-18 14:32:11.895582 | node1 | ok: [node3] 2026-08-18 14:32:11.895871 | node1 | ok: [node1] 2026-08-18 14:32:11.896124 | node1 | 2026-08-18 14:32:11.896392 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:32:11.896739 | node1 | Tuesday 18 August 2026 14:32:11 +0000 (0:00:01.138) 0:10:36.161 ******** 2026-08-18 14:32:12.738573 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 14:32:12.738851 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:32:12.739099 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:32:12.739340 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:32:12.739578 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:32:12.739855 | node1 | ok: [node1] => (item=ssh) 2026-08-18 14:32:12.740075 | node1 | 2026-08-18 14:32:12.740377 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:32:12.740677 | node1 | Tuesday 18 August 2026 14:32:12 +0000 (0:00:00.843) 0:10:37.004 ******** 2026-08-18 14:32:13.039767 | node1 | ok: [node1] 2026-08-18 14:32:13.039826 | node1 | ok: [node2] 2026-08-18 14:32:13.040749 | node1 | ok: [node3] 2026-08-18 14:32:13.040806 | node1 | 2026-08-18 14:32:13.040817 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:32:13.040827 | node1 | Tuesday 18 August 2026 14:32:13 +0000 (0:00:00.301) 0:10:37.305 ******** 2026-08-18 14:32:13.377949 | node1 | ok: [node1] 2026-08-18 14:32:13.378017 | node1 | ok: [node2] 2026-08-18 14:32:13.378113 | node1 | ok: [node3] 2026-08-18 14:32:13.378212 | node1 | 2026-08-18 14:32:13.378331 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:32:13.378445 | node1 | Tuesday 18 August 2026 14:32:13 +0000 (0:00:00.336) 0:10:37.641 ******** 2026-08-18 14:32:13.958708 | node1 | ok: [node1] 2026-08-18 14:32:13.958768 | node1 | ok: [node3] 2026-08-18 14:32:13.959242 | node1 | ok: [node2] 2026-08-18 14:32:13.959265 | node1 | 2026-08-18 14:32:13.959270 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:32:13.959275 | node1 | Tuesday 18 August 2026 14:32:13 +0000 (0:00:00.583) 0:10:38.225 ******** 2026-08-18 14:32:14.036440 | node1 | skipping: [node1] 2026-08-18 14:32:14.036495 | node1 | skipping: [node2] 2026-08-18 14:32:14.036500 | node1 | skipping: [node3] 2026-08-18 14:32:14.036504 | node1 | 2026-08-18 14:32:14.036513 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:32:14.036833 | node1 | Tuesday 18 August 2026 14:32:14 +0000 (0:00:00.075) 0:10:38.301 ******** 2026-08-18 14:32:14.537593 | node1 | ok: [node2] 2026-08-18 14:32:14.537669 | node1 | ok: [node1] 2026-08-18 14:32:14.537678 | node1 | ok: [node3] 2026-08-18 14:32:14.537688 | node1 | 2026-08-18 14:32:14.537697 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:32:14.537707 | node1 | Tuesday 18 August 2026 14:32:14 +0000 (0:00:00.500) 0:10:38.802 ******** 2026-08-18 14:32:15.160747 | node1 | ok: [node2] 2026-08-18 14:32:15.160802 | node1 | ok: [node1] 2026-08-18 14:32:15.161937 | node1 | ok: [node3] 2026-08-18 14:32:15.161997 | node1 | 2026-08-18 14:32:15.162012 | node1 | TASK [vexxhost.ceph.rgw : Locate the cluster admin host] *********************** 2026-08-18 14:32:15.162024 | node1 | Tuesday 18 August 2026 14:32:15 +0000 (0:00:00.623) 0:10:39.425 ******** 2026-08-18 14:32:15.883279 | node1 | ok: [node1] => (item=node1) 2026-08-18 14:32:15.883341 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 14:32:15.883354 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:32:15.883366 | node1 | 2026-08-18 14:32:15.883378 | node1 | TASK [vexxhost.ceph.rgw : Select the admin host] ******************************* 2026-08-18 14:32:15.883389 | node1 | Tuesday 18 August 2026 14:32:15 +0000 (0:00:00.721) 0:10:40.147 ******** 2026-08-18 14:32:15.968382 | node1 | ok: [node1] 2026-08-18 14:32:15.968415 | node1 | ok: [node2] 2026-08-18 14:32:15.968425 | node1 | ok: [node3] 2026-08-18 14:32:15.968434 | node1 | 2026-08-18 14:32:15.968442 | node1 | TASK [vexxhost.ceph.rgw : Set the admin mon config path] *********************** 2026-08-18 14:32:15.968451 | node1 | Tuesday 18 August 2026 14:32:15 +0000 (0:00:00.086) 0:10:40.233 ******** 2026-08-18 14:32:16.063696 | node1 | ok: [node1] 2026-08-18 14:32:16.063745 | node1 | ok: [node2] 2026-08-18 14:32:16.063755 | node1 | ok: [node3] 2026-08-18 14:32:16.063764 | node1 | 2026-08-18 14:32:16.063774 | node1 | TASK [Register this RGW host] ************************************************** 2026-08-18 14:32:16.063783 | node1 | Tuesday 18 August 2026 14:32:16 +0000 (0:00:00.095) 0:10:40.328 ******** 2026-08-18 14:32:16.189155 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 14:32:16.189204 | node1 | 2026-08-18 14:32:16.189215 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 14:32:16.189225 | node1 | Tuesday 18 August 2026 14:32:16 +0000 (0:00:00.125) 0:10:40.454 ******** 2026-08-18 14:32:17.881771 | node1 | ok: [node1] 2026-08-18 14:32:17.881843 | node1 | 2026-08-18 14:32:17.882447 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 14:32:17.882500 | node1 | Tuesday 18 August 2026 14:32:17 +0000 (0:00:01.693) 0:10:42.147 ******** 2026-08-18 14:32:17.957082 | node1 | ok: [node1] 2026-08-18 14:32:17.957147 | node1 | ok: [node2] 2026-08-18 14:32:17.957246 | node1 | ok: [node3] 2026-08-18 14:32:17.957348 | node1 | 2026-08-18 14:32:17.957467 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 14:32:17.957582 | node1 | Tuesday 18 August 2026 14:32:17 +0000 (0:00:00.075) 0:10:42.223 ******** 2026-08-18 14:32:18.038210 | node1 | ok: [node1] => { 2026-08-18 14:32:18.038231 | node1 | "changed": false, 2026-08-18 14:32:18.038237 | node1 | "msg": "All assertions passed" 2026-08-18 14:32:18.038242 | node1 | } 2026-08-18 14:32:18.038257 | node1 | ok: [node2] => { 2026-08-18 14:32:18.038261 | node1 | "changed": false, 2026-08-18 14:32:18.038271 | node1 | "msg": "All assertions passed" 2026-08-18 14:32:18.038275 | node1 | } 2026-08-18 14:32:18.038278 | node1 | ok: [node3] => { 2026-08-18 14:32:18.038282 | node1 | "changed": false, 2026-08-18 14:32:18.038286 | node1 | "msg": "All assertions passed" 2026-08-18 14:32:18.038290 | node1 | } 2026-08-18 14:32:18.038294 | node1 | 2026-08-18 14:32:18.038298 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 14:32:18.038302 | node1 | Tuesday 18 August 2026 14:32:18 +0000 (0:00:00.079) 0:10:42.302 ******** 2026-08-18 14:32:18.430096 | node1 | ok: [node1] 2026-08-18 14:32:18.430143 | node1 | ok: [node2] 2026-08-18 14:32:18.430151 | node1 | ok: [node3] 2026-08-18 14:32:18.430158 | node1 | 2026-08-18 14:32:18.430164 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 14:32:18.430170 | node1 | Tuesday 18 August 2026 14:32:18 +0000 (0:00:00.391) 0:10:42.693 ******** 2026-08-18 14:32:18.507697 | node1 | ok: [node1] 2026-08-18 14:32:18.507752 | node1 | ok: [node2] 2026-08-18 14:32:18.507762 | node1 | ok: [node3] 2026-08-18 14:32:18.507771 | node1 | 2026-08-18 14:32:18.507780 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 14:32:18.507789 | node1 | Tuesday 18 August 2026 14:32:18 +0000 (0:00:00.078) 0:10:42.772 ******** 2026-08-18 14:32:18.625669 | node1 | ok: [node1] 2026-08-18 14:32:18.625719 | node1 | ok: [node2] 2026-08-18 14:32:18.625729 | node1 | ok: [node3] 2026-08-18 14:32:18.625738 | node1 | 2026-08-18 14:32:18.625748 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 14:32:18.625757 | node1 | Tuesday 18 August 2026 14:32:18 +0000 (0:00:00.118) 0:10:42.890 ******** 2026-08-18 14:32:18.704681 | node1 | ok: [node1] => { 2026-08-18 14:32:18.704738 | node1 | "changed": false, 2026-08-18 14:32:18.704748 | node1 | "msg": "All assertions passed" 2026-08-18 14:32:18.704756 | node1 | } 2026-08-18 14:32:18.704765 | node1 | ok: [node2] => { 2026-08-18 14:32:18.704773 | node1 | "changed": false, 2026-08-18 14:32:18.704782 | node1 | "msg": "All assertions passed" 2026-08-18 14:32:18.704790 | node1 | } 2026-08-18 14:32:18.704798 | node1 | ok: [node3] => { 2026-08-18 14:32:18.704806 | node1 | "changed": false, 2026-08-18 14:32:18.704814 | node1 | "msg": "All assertions passed" 2026-08-18 14:32:18.704822 | node1 | } 2026-08-18 14:32:18.704831 | node1 | 2026-08-18 14:32:18.704840 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 14:32:18.704854 | node1 | Tuesday 18 August 2026 14:32:18 +0000 (0:00:00.077) 0:10:42.968 ******** 2026-08-18 14:32:20.517079 | node1 | ok: [node1] 2026-08-18 14:32:20.517318 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:32:20.517586 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:32:20.517845 | node1 | 2026-08-18 14:32:20.518109 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 14:32:20.518364 | node1 | Tuesday 18 August 2026 14:32:20 +0000 (0:00:01.815) 0:10:44.783 ******** 2026-08-18 14:32:20.619517 | node1 | skipping: [node1] 2026-08-18 14:32:20.619786 | node1 | skipping: [node2] 2026-08-18 14:32:20.620047 | node1 | skipping: [node3] 2026-08-18 14:32:20.620260 | node1 | 2026-08-18 14:32:20.620552 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW multisite (realm, zonegroup, zone)] **** 2026-08-18 14:32:20.620833 | node1 | Tuesday 18 August 2026 14:32:20 +0000 (0:00:00.102) 0:10:44.885 ******** 2026-08-18 14:32:20.680104 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/multisite.yml for node1 2026-08-18 14:32:20.680153 | node1 | 2026-08-18 14:32:20.680307 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW realms] ******************************** 2026-08-18 14:32:20.680481 | node1 | Tuesday 18 August 2026 14:32:20 +0000 (0:00:00.060) 0:10:44.946 ******** 2026-08-18 14:32:24.493752 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:32:24.493824 | node1 | 2026-08-18 14:32:24.494307 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zonegroups] **************************** 2026-08-18 14:32:24.494354 | node1 | Tuesday 18 August 2026 14:32:24 +0000 (0:00:03.813) 0:10:48.759 ******** 2026-08-18 14:32:27.264693 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:32:27.264777 | node1 | 2026-08-18 14:32:27.265295 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zones] ********************************* 2026-08-18 14:32:27.265393 | node1 | Tuesday 18 August 2026 14:32:27 +0000 (0:00:02.770) 0:10:51.530 ******** 2026-08-18 14:32:29.755414 | node1 | changed: [node1] => (item=None) 2026-08-18 14:32:29.755451 | node1 | changed: [node1 -> {{ ceph_rgw_admin_host }}] 2026-08-18 14:32:29.755457 | node1 | 2026-08-18 14:32:29.755463 | node1 | TASK [vexxhost.ceph.rgw : Commit the RGW period] ******************************* 2026-08-18 14:32:29.755469 | node1 | Tuesday 18 August 2026 14:32:29 +0000 (0:00:02.490) 0:10:54.020 ******** 2026-08-18 14:32:31.555103 | node1 | changed: [node1] 2026-08-18 14:32:31.555204 | node1 | 2026-08-18 14:32:31.555412 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 14:32:31.555570 | node1 | Tuesday 18 August 2026 14:32:31 +0000 (0:00:01.798) 0:10:55.818 ******** 2026-08-18 14:32:31.614853 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/services.yml for node1 2026-08-18 14:32:31.614919 | node1 | 2026-08-18 14:32:31.615002 | node1 | TASK [vexxhost.ceph.rgw : Snapshot current RGW service specs] ****************** 2026-08-18 14:32:31.615112 | node1 | Tuesday 18 August 2026 14:32:31 +0000 (0:00:00.063) 0:10:55.881 ******** 2026-08-18 14:32:33.338439 | node1 | ok: [node1] 2026-08-18 14:32:33.338478 | node1 | 2026-08-18 14:32:33.338484 | node1 | TASK [vexxhost.ceph.rgw : Index current RGW specs by service name] ************* 2026-08-18 14:32:33.338490 | node1 | Tuesday 18 August 2026 14:32:33 +0000 (0:00:01.723) 0:10:57.604 ******** 2026-08-18 14:32:33.394107 | node1 | ok: [node1] 2026-08-18 14:32:33.394133 | node1 | 2026-08-18 14:32:33.394139 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 14:32:33.394144 | node1 | Tuesday 18 August 2026 14:32:33 +0000 (0:00:00.055) 0:10:57.660 ******** 2026-08-18 14:32:36.726562 | node1 | changed: [node1] => (item=molecule) 2026-08-18 14:32:36.727018 | node1 | 2026-08-18 14:32:36.727389 | node1 | TASK [vexxhost.ceph.rgw : Redeploy RGW services whose redeploy-triggering keys changed] *** 2026-08-18 14:32:36.727770 | node1 | Tuesday 18 August 2026 14:32:36 +0000 (0:00:03.331) 0:11:00.991 ******** 2026-08-18 14:32:38.446774 | node1 | failed: [node1] (item=rgw.molecule) => {"ansible_loop_var": "item", "changed": true, "cmd": ["cephadm", "shell", "--fsid", "1dff0e0f-3c44-48da-81cd-4f3c6e8722b2", "--config", "/var/lib/ceph/1dff0e0f-3c44-48da-81cd-4f3c6e8722b2/mon.node1/config", "--", "ceph", "orch", "redeploy", "rgw.molecule"], "delta": "0:00:01.449807", "end": "2026-08-18 14:32:38.413150", "item": {"count_per_host": 1, "id": "molecule", "spec": {"generate_cert": true, "rgw_frontend_port": 8080, "rgw_realm": "molecule", "rgw_zone": "molecule-1", "rgw_zonegroup": "molecule", "ssl": true}}, "msg": "non-zero return code", "rc": 22, "start": "2026-08-18 14:32:36.963343", "stderr": "Error EINVAL: No daemons exist under service name \"rgw.molecule\". View currently running services using \"ceph orch ls\"", "stderr_lines": ["Error EINVAL: No daemons exist under service name \"rgw.molecule\". View currently running services using \"ceph orch ls\""], "stdout": "", "stdout_lines": []} 2026-08-18 14:32:38.467220 | node1 | 2026-08-18 14:32:38.467290 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone / zonegroup / pools] ********* 2026-08-18 14:32:38.467304 | node1 | Tuesday 18 August 2026 14:32:38 +0000 (0:00:01.740) 0:11:02.732 ******** 2026-08-18 14:32:38.534978 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/cleanup.yml for node2 2026-08-18 14:32:38.535042 | node1 | 2026-08-18 14:32:38.535056 | node1 | TASK [vexxhost.ceph.rgw : List RGW zones] ************************************** 2026-08-18 14:32:38.535077 | node1 | Tuesday 18 August 2026 14:32:38 +0000 (0:00:00.066) 0:11:02.798 ******** 2026-08-18 14:32:39.883128 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 14:32:39.883196 | node1 | 2026-08-18 14:32:39.883230 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zonegroup] ************************ 2026-08-18 14:32:39.883240 | node1 | Tuesday 18 August 2026 14:32:39 +0000 (0:00:01.349) 0:11:04.147 ******** 2026-08-18 14:32:39.931806 | node1 | skipping: [node2] 2026-08-18 14:32:39.931883 | node1 | 2026-08-18 14:32:39.931902 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone] ***************************** 2026-08-18 14:32:39.932056 | node1 | Tuesday 18 August 2026 14:32:39 +0000 (0:00:00.049) 0:11:04.197 ******** 2026-08-18 14:32:39.985113 | node1 | skipping: [node2] 2026-08-18 14:32:39.985158 | node1 | 2026-08-18 14:32:39.985288 | node1 | TASK [vexxhost.ceph.rgw : Get the current pool deletion setting] *************** 2026-08-18 14:32:39.985426 | node1 | Tuesday 18 August 2026 14:32:39 +0000 (0:00:00.053) 0:11:04.250 ******** 2026-08-18 14:32:40.041688 | node1 | skipping: [node2] 2026-08-18 14:32:40.041738 | node1 | 2026-08-18 14:32:40.042211 | node1 | TASK [vexxhost.ceph.rgw : Allow pool deletion] ********************************* 2026-08-18 14:32:40.042235 | node1 | Tuesday 18 August 2026 14:32:40 +0000 (0:00:00.055) 0:11:04.306 ******** 2026-08-18 14:32:40.095764 | node1 | skipping: [node2] 2026-08-18 14:32:40.095839 | node1 | 2026-08-18 14:32:40.096343 | node1 | TASK [vexxhost.ceph.rgw : Remove the default RGW pools] ************************ 2026-08-18 14:32:40.096384 | node1 | Tuesday 18 August 2026 14:32:40 +0000 (0:00:00.054) 0:11:04.361 ******** 2026-08-18 14:32:40.176665 | node1 | skipping: [node2] => (item=default.rgw.control) 2026-08-18 14:32:40.176720 | node1 | skipping: [node2] => (item=default.rgw.log) 2026-08-18 14:32:40.176730 | node1 | skipping: [node2] => (item=default.rgw.meta) 2026-08-18 14:32:40.176739 | node1 | skipping: [node2] 2026-08-18 14:32:40.191093 | node1 | 2026-08-18 14:32:40.191385 | node1 | TASK [vexxhost.ceph.rgw : Restore the configured pool deletion setting] ******** 2026-08-18 14:32:40.191534 | node1 | Tuesday 18 August 2026 14:32:40 +0000 (0:00:00.078) 0:11:04.440 ******** 2026-08-18 14:32:40.231959 | node1 | skipping: [node2] 2026-08-18 14:32:40.232284 | node1 | 2026-08-18 14:32:40.232638 | node1 | TASK [vexxhost.ceph.rgw : Remove the temporary pool deletion setting] ********** 2026-08-18 14:32:40.232965 | node1 | Tuesday 18 August 2026 14:32:40 +0000 (0:00:00.058) 0:11:04.498 ******** 2026-08-18 14:32:40.326444 | node1 | skipping: [node2] 2026-08-18 14:32:40.326805 | node1 | 2026-08-18 14:32:40.327134 | node1 | PLAY [Deploy Ceph monitoring stack] ******************************************** 2026-08-18 14:32:40.327451 | node1 | 2026-08-18 14:32:40.327800 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 14:32:40.328122 | node1 | Tuesday 18 August 2026 14:32:40 +0000 (0:00:00.094) 0:11:04.593 ******** 2026-08-18 14:32:41.343239 | node1 | ok: [node2] 2026-08-18 14:32:41.343645 | node1 | ok: [node3] 2026-08-18 14:32:41.343957 | node1 | 2026-08-18 14:32:41.344283 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:32:41.344621 | node1 | Tuesday 18 August 2026 14:32:41 +0000 (0:00:01.016) 0:11:05.609 ******** 2026-08-18 14:32:41.637575 | node1 | ok: [node2] 2026-08-18 14:32:41.637756 | node1 | ok: [node3] 2026-08-18 14:32:41.637929 | node1 | 2026-08-18 14:32:41.638113 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:32:41.638285 | node1 | Tuesday 18 August 2026 14:32:41 +0000 (0:00:00.293) 0:11:05.903 ******** 2026-08-18 14:32:41.743617 | node1 | skipping: [node2] 2026-08-18 14:32:41.743871 | node1 | skipping: [node3] 2026-08-18 14:32:41.744093 | node1 | 2026-08-18 14:32:41.744341 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 14:32:41.744579 | node1 | Tuesday 18 August 2026 14:32:41 +0000 (0:00:00.106) 0:11:06.010 ******** 2026-08-18 14:32:42.036721 | node1 | ok: [node2] 2026-08-18 14:32:42.036985 | node1 | ok: [node3] 2026-08-18 14:32:42.037211 | node1 | 2026-08-18 14:32:42.037466 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:32:42.037752 | node1 | Tuesday 18 August 2026 14:32:42 +0000 (0:00:00.292) 0:11:06.302 ******** 2026-08-18 14:32:42.107085 | node1 | ok: [node2] => { 2026-08-18 14:32:42.107348 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:32:42.107583 | node1 | } 2026-08-18 14:32:42.107850 | node1 | ok: [node3] => { 2026-08-18 14:32:42.108103 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 14:32:42.108332 | node1 | } 2026-08-18 14:32:42.108554 | node1 | 2026-08-18 14:32:42.108827 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:32:42.109068 | node1 | Tuesday 18 August 2026 14:32:42 +0000 (0:00:00.070) 0:11:06.373 ******** 2026-08-18 14:32:42.519369 | node1 | ok: [node2] 2026-08-18 14:32:42.519624 | node1 | ok: [node3] 2026-08-18 14:32:42.519864 | node1 | 2026-08-18 14:32:42.520133 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:32:42.520431 | node1 | Tuesday 18 August 2026 14:32:42 +0000 (0:00:00.411) 0:11:06.785 ******** 2026-08-18 14:32:42.600583 | node1 | skipping: [node2] 2026-08-18 14:32:42.600857 | node1 | skipping: [node3] 2026-08-18 14:32:42.601080 | node1 | 2026-08-18 14:32:42.601363 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:32:42.601630 | node1 | Tuesday 18 August 2026 14:32:42 +0000 (0:00:00.081) 0:11:06.867 ******** 2026-08-18 14:32:42.682339 | node1 | skipping: [node2] 2026-08-18 14:32:42.682578 | node1 | skipping: [node3] 2026-08-18 14:32:42.682914 | node1 | 2026-08-18 14:32:42.683184 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:32:42.683431 | node1 | Tuesday 18 August 2026 14:32:42 +0000 (0:00:00.081) 0:11:06.948 ******** 2026-08-18 14:32:42.976642 | node1 | ok: [node2] 2026-08-18 14:32:42.976676 | node1 | ok: [node3] 2026-08-18 14:32:42.976681 | node1 | 2026-08-18 14:32:42.976685 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 14:32:42.976689 | node1 | Tuesday 18 August 2026 14:32:42 +0000 (0:00:00.292) 0:11:07.240 ******** 2026-08-18 14:32:43.923679 | node1 | ok: [node3] 2026-08-18 14:32:43.923933 | node1 | ok: [node2] 2026-08-18 14:32:43.924161 | node1 | 2026-08-18 14:32:43.924419 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:32:43.924776 | node1 | Tuesday 18 August 2026 14:32:43 +0000 (0:00:00.949) 0:11:08.190 ******** 2026-08-18 14:32:44.015406 | node1 | ok: [node2] => { 2026-08-18 14:32:44.015558 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:32:44.015744 | node1 | } 2026-08-18 14:32:44.015904 | node1 | ok: [node3] => { 2026-08-18 14:32:44.016078 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 14:32:44.016226 | node1 | } 2026-08-18 14:32:44.016568 | node1 | 2026-08-18 14:32:44.016784 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:32:44.016921 | node1 | Tuesday 18 August 2026 14:32:44 +0000 (0:00:00.091) 0:11:08.281 ******** 2026-08-18 14:32:44.470434 | node1 | ok: [node2] 2026-08-18 14:32:44.470691 | node1 | ok: [node3] 2026-08-18 14:32:44.470916 | node1 | 2026-08-18 14:32:44.471169 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:32:44.471416 | node1 | Tuesday 18 August 2026 14:32:44 +0000 (0:00:00.454) 0:11:08.736 ******** 2026-08-18 14:32:46.665032 | node1 | ok: [node2] 2026-08-18 14:32:46.665119 | node1 | ok: [node3] 2026-08-18 14:32:46.665390 | node1 | 2026-08-18 14:32:46.665682 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 14:32:46.665997 | node1 | Tuesday 18 August 2026 14:32:46 +0000 (0:00:02.193) 0:11:10.930 ******** 2026-08-18 14:32:46.716480 | node1 | skipping: [node2] 2026-08-18 14:32:46.716741 | node1 | skipping: [node3] 2026-08-18 14:32:46.716967 | node1 | 2026-08-18 14:32:46.717301 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 14:32:46.717566 | node1 | Tuesday 18 August 2026 14:32:46 +0000 (0:00:00.051) 0:11:10.982 ******** 2026-08-18 14:32:46.767931 | node1 | skipping: [node2] 2026-08-18 14:32:46.768232 | node1 | skipping: [node3] 2026-08-18 14:32:46.768568 | node1 | 2026-08-18 14:32:46.768867 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 14:32:46.769127 | node1 | Tuesday 18 August 2026 14:32:46 +0000 (0:00:00.052) 0:11:11.034 ******** 2026-08-18 14:32:46.822437 | node1 | skipping: [node2] 2026-08-18 14:32:46.822667 | node1 | skipping: [node3] 2026-08-18 14:32:46.822895 | node1 | 2026-08-18 14:32:46.823143 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 14:32:46.823384 | node1 | Tuesday 18 August 2026 14:32:46 +0000 (0:00:00.054) 0:11:11.088 ******** 2026-08-18 14:32:47.797830 | node1 | ok: [node2] 2026-08-18 14:32:47.798068 | node1 | ok: [node3] 2026-08-18 14:32:47.798299 | node1 | 2026-08-18 14:32:47.798573 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 14:32:47.798879 | node1 | Tuesday 18 August 2026 14:32:47 +0000 (0:00:00.974) 0:11:12.063 ******** 2026-08-18 14:32:48.383651 | node1 | ok: [node2] 2026-08-18 14:32:48.383860 | node1 | ok: [node3] 2026-08-18 14:32:48.384083 | node1 | 2026-08-18 14:32:48.384338 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 14:32:48.384584 | node1 | Tuesday 18 August 2026 14:32:48 +0000 (0:00:00.586) 0:11:12.649 ******** 2026-08-18 14:32:49.600249 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 14:32:49.600500 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 14:32:49.600877 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:32:49.601266 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 14:32:49.601508 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:32:49.601822 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 14:32:49.602174 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:32:49.602469 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 14:32:49.602796 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:32:49.603124 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 14:32:49.603371 | node1 | 2026-08-18 14:32:49.603693 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 14:32:49.603958 | node1 | Tuesday 18 August 2026 14:32:49 +0000 (0:00:01.216) 0:11:13.866 ******** 2026-08-18 14:32:50.312040 | node1 | ok: [node2] 2026-08-18 14:32:50.312265 | node1 | ok: [node3] 2026-08-18 14:32:50.312510 | node1 | 2026-08-18 14:32:50.312807 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:32:50.319473 | node1 | Tuesday 18 August 2026 14:32:50 +0000 (0:00:00.697) 0:11:14.563 ******** 2026-08-18 14:32:50.319499 | node1 | 2026-08-18 14:32:50.319513 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 14:32:50.319521 | node1 | Tuesday 18 August 2026 14:32:50 +0000 (0:00:00.007) 0:11:14.571 ******** 2026-08-18 14:32:50.319528 | node1 | 2026-08-18 14:32:50.319534 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 14:32:50.319539 | node1 | Tuesday 18 August 2026 14:32:50 +0000 (0:00:00.007) 0:11:14.578 ******** 2026-08-18 14:32:50.800040 | node1 | ok: [node2] 2026-08-18 14:32:50.800305 | node1 | ok: [node3] 2026-08-18 14:32:50.800573 | node1 | 2026-08-18 14:32:50.800923 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 14:32:50.801181 | node1 | Tuesday 18 August 2026 14:32:50 +0000 (0:00:00.487) 0:11:15.066 ******** 2026-08-18 14:32:51.096178 | node1 | ok: [node2] 2026-08-18 14:32:51.096222 | node1 | ok: [node3] 2026-08-18 14:32:51.096350 | node1 | 2026-08-18 14:32:51.096533 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:32:51.096767 | node1 | Tuesday 18 August 2026 14:32:51 +0000 (0:00:00.295) 0:11:15.361 ******** 2026-08-18 14:32:51.191404 | node1 | ok: [node2] => { 2026-08-18 14:32:51.192304 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:32:51.192561 | node1 | } 2026-08-18 14:32:51.192868 | node1 | ok: [node3] => { 2026-08-18 14:32:51.193148 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 14:32:51.193393 | node1 | } 2026-08-18 14:32:51.193674 | node1 | 2026-08-18 14:32:51.194043 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:32:51.194366 | node1 | Tuesday 18 August 2026 14:32:51 +0000 (0:00:00.094) 0:11:15.456 ******** 2026-08-18 14:32:51.651043 | node1 | ok: [node3] 2026-08-18 14:32:51.651180 | node1 | ok: [node2] 2026-08-18 14:32:51.651350 | node1 | 2026-08-18 14:32:51.651492 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:32:51.651686 | node1 | Tuesday 18 August 2026 14:32:51 +0000 (0:00:00.461) 0:11:15.917 ******** 2026-08-18 14:32:54.975647 | node1 | ok: [node2] 2026-08-18 14:32:54.975871 | node1 | ok: [node3] 2026-08-18 14:32:54.976101 | node1 | 2026-08-18 14:32:54.976348 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 14:32:54.976678 | node1 | Tuesday 18 August 2026 14:32:54 +0000 (0:00:03.324) 0:11:19.241 ******** 2026-08-18 14:32:55.896765 | node1 | ok: [node3] 2026-08-18 14:32:55.896826 | node1 | ok: [node2] 2026-08-18 14:32:55.896937 | node1 | 2026-08-18 14:32:55.897048 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 14:32:55.897174 | node1 | Tuesday 18 August 2026 14:32:55 +0000 (0:00:00.921) 0:11:20.162 ******** 2026-08-18 14:32:56.156462 | node1 | ok: [node2] 2026-08-18 14:32:56.156511 | node1 | ok: [node3] 2026-08-18 14:32:56.156521 | node1 | 2026-08-18 14:32:56.156530 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 14:32:56.156540 | node1 | Tuesday 18 August 2026 14:32:56 +0000 (0:00:00.258) 0:11:20.421 ******** 2026-08-18 14:32:56.726666 | node1 | ok: [node3] 2026-08-18 14:32:56.726876 | node1 | ok: [node2] 2026-08-18 14:32:56.727134 | node1 | 2026-08-18 14:32:56.727420 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 14:32:56.727761 | node1 | Tuesday 18 August 2026 14:32:56 +0000 (0:00:00.571) 0:11:20.993 ******** 2026-08-18 14:32:57.435832 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 14:32:57.436025 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 14:32:57.436354 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:32:57.436549 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 14:32:57.436849 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:32:57.437083 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 14:32:57.437303 | node1 | 2026-08-18 14:32:57.437555 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 14:32:57.437898 | node1 | Tuesday 18 August 2026 14:32:57 +0000 (0:00:00.709) 0:11:21.702 ******** 2026-08-18 14:32:57.993767 | node1 | ok: [node2] 2026-08-18 14:32:57.993880 | node1 | ok: [node3] 2026-08-18 14:32:57.993891 | node1 | 2026-08-18 14:32:57.994060 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 14:32:57.994378 | node1 | Tuesday 18 August 2026 14:32:57 +0000 (0:00:00.557) 0:11:22.259 ******** 2026-08-18 14:32:58.584040 | node1 | ok: [node2] 2026-08-18 14:32:58.584152 | node1 | ok: [node3] 2026-08-18 14:32:58.584168 | node1 | 2026-08-18 14:32:58.584314 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:32:58.584596 | node1 | Tuesday 18 August 2026 14:32:58 +0000 (0:00:00.566) 0:11:22.825 ******** 2026-08-18 14:32:58.584669 | node1 | 2026-08-18 14:32:58.584684 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 14:32:58.584790 | node1 | Tuesday 18 August 2026 14:32:58 +0000 (0:00:00.018) 0:11:22.843 ******** 2026-08-18 14:32:58.584891 | node1 | 2026-08-18 14:32:58.585005 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 14:32:58.585148 | node1 | Tuesday 18 August 2026 14:32:58 +0000 (0:00:00.006) 0:11:22.850 ******** 2026-08-18 14:32:59.092039 | node1 | ok: [node2] 2026-08-18 14:32:59.092230 | node1 | ok: [node3] 2026-08-18 14:32:59.092457 | node1 | 2026-08-18 14:32:59.092751 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 14:32:59.092997 | node1 | Tuesday 18 August 2026 14:32:59 +0000 (0:00:00.507) 0:11:23.358 ******** 2026-08-18 14:32:59.145446 | node1 | ok: [node2] => { 2026-08-18 14:32:59.145696 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:32:59.145939 | node1 | } 2026-08-18 14:32:59.146250 | node1 | ok: [node3] => { 2026-08-18 14:32:59.146636 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 14:32:59.146928 | node1 | } 2026-08-18 14:32:59.147153 | node1 | 2026-08-18 14:32:59.147401 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 14:32:59.147687 | node1 | Tuesday 18 August 2026 14:32:59 +0000 (0:00:00.053) 0:11:23.411 ******** 2026-08-18 14:32:59.499468 | node1 | ok: [node2] 2026-08-18 14:32:59.499687 | node1 | ok: [node3] 2026-08-18 14:32:59.499900 | node1 | 2026-08-18 14:32:59.500153 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 14:32:59.500415 | node1 | Tuesday 18 August 2026 14:32:59 +0000 (0:00:00.353) 0:11:23.765 ******** 2026-08-18 14:32:59.561844 | node1 | skipping: [node2] 2026-08-18 14:32:59.562064 | node1 | skipping: [node3] 2026-08-18 14:32:59.562288 | node1 | 2026-08-18 14:32:59.562541 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 14:32:59.562842 | node1 | Tuesday 18 August 2026 14:32:59 +0000 (0:00:00.063) 0:11:23.828 ******** 2026-08-18 14:32:59.634107 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:32:59.634368 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 14:32:59.634650 | node1 | 2026-08-18 14:32:59.634944 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 14:32:59.635421 | node1 | Tuesday 18 August 2026 14:32:59 +0000 (0:00:00.071) 0:11:23.900 ******** 2026-08-18 14:33:00.773362 | node1 | ok: [node2] 2026-08-18 14:33:00.773475 | node1 | ok: [node3] 2026-08-18 14:33:00.773826 | node1 | 2026-08-18 14:33:00.774029 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 14:33:00.774319 | node1 | Tuesday 18 August 2026 14:33:00 +0000 (0:00:01.139) 0:11:25.039 ******** 2026-08-18 14:33:02.617859 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 14:33:02.618104 | node1 | ok: [node3] => (item=ssh) 2026-08-18 14:33:02.618452 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 14:33:02.618775 | node1 | ok: [node2] => (item=ssh) 2026-08-18 14:33:02.619149 | node1 | 2026-08-18 14:33:02.619416 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 14:33:02.619792 | node1 | Tuesday 18 August 2026 14:33:02 +0000 (0:00:01.844) 0:11:26.883 ******** 2026-08-18 14:33:02.892830 | node1 | ok: [node2] 2026-08-18 14:33:02.893122 | node1 | ok: [node3] 2026-08-18 14:33:02.893436 | node1 | 2026-08-18 14:33:02.893837 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 14:33:02.894213 | node1 | Tuesday 18 August 2026 14:33:02 +0000 (0:00:00.275) 0:11:27.159 ******** 2026-08-18 14:33:03.223563 | node1 | ok: [node3] 2026-08-18 14:33:03.223970 | node1 | ok: [node2] 2026-08-18 14:33:03.224301 | node1 | 2026-08-18 14:33:03.224741 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 14:33:03.225113 | node1 | Tuesday 18 August 2026 14:33:03 +0000 (0:00:00.330) 0:11:27.489 ******** 2026-08-18 14:33:03.773142 | node1 | ok: [node2] 2026-08-18 14:33:03.773223 | node1 | ok: [node3] 2026-08-18 14:33:03.773285 | node1 | 2026-08-18 14:33:03.773411 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 14:33:03.773541 | node1 | Tuesday 18 August 2026 14:33:03 +0000 (0:00:00.549) 0:11:28.039 ******** 2026-08-18 14:33:03.822398 | node1 | skipping: [node2] 2026-08-18 14:33:03.822501 | node1 | skipping: [node3] 2026-08-18 14:33:03.822681 | node1 | 2026-08-18 14:33:03.822834 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 14:33:03.822988 | node1 | Tuesday 18 August 2026 14:33:03 +0000 (0:00:00.049) 0:11:28.088 ******** 2026-08-18 14:33:04.285247 | node1 | ok: [node2] 2026-08-18 14:33:04.286106 | node1 | ok: [node3] 2026-08-18 14:33:04.286216 | node1 | 2026-08-18 14:33:04.286532 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 14:33:04.286861 | node1 | Tuesday 18 August 2026 14:33:04 +0000 (0:00:00.462) 0:11:28.551 ******** 2026-08-18 14:33:04.891066 | node1 | ok: [node2] 2026-08-18 14:33:04.891279 | node1 | ok: [node3] 2026-08-18 14:33:04.891509 | node1 | 2026-08-18 14:33:04.891814 | node1 | TASK [vexxhost.ceph.monitoring : Locate the cluster admin host] **************** 2026-08-18 14:33:04.892067 | node1 | Tuesday 18 August 2026 14:33:04 +0000 (0:00:00.605) 0:11:29.157 ******** 2026-08-18 14:33:05.606088 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=node1) 2026-08-18 14:33:05.606261 | node1 | ok: [node2] => (item=node2) 2026-08-18 14:33:05.606399 | node1 | ok: [node2 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 14:33:05.606543 | node1 | 2026-08-18 14:33:05.606684 | node1 | TASK [vexxhost.ceph.monitoring : Select the admin host] ************************ 2026-08-18 14:33:05.606822 | node1 | Tuesday 18 August 2026 14:33:05 +0000 (0:00:00.715) 0:11:29.872 ******** 2026-08-18 14:33:05.676323 | node1 | ok: [node2] 2026-08-18 14:33:05.676557 | node1 | ok: [node3] 2026-08-18 14:33:05.676834 | node1 | 2026-08-18 14:33:05.677097 | node1 | TASK [vexxhost.ceph.monitoring : Set the admin mon config path] **************** 2026-08-18 14:33:05.677380 | node1 | Tuesday 18 August 2026 14:33:05 +0000 (0:00:00.070) 0:11:29.942 ******** 2026-08-18 14:33:05.734978 | node1 | ok: [node2] 2026-08-18 14:33:05.735194 | node1 | ok: [node3] 2026-08-18 14:33:05.735490 | node1 | 2026-08-18 14:33:05.735781 | node1 | TASK [vexxhost.ceph.monitoring : Pin monitoring container images] ************** 2026-08-18 14:33:05.736019 | node1 | Tuesday 18 August 2026 14:33:05 +0000 (0:00:00.058) 0:11:30.001 ******** 2026-08-18 14:33:05.768256 | node1 | skipping: [node2] 2026-08-18 14:33:05.768487 | node1 | 2026-08-18 14:33:05.768801 | node1 | TASK [vexxhost.ceph.monitoring : Apply the monitoring service specs] *********** 2026-08-18 14:33:05.769049 | node1 | Tuesday 18 August 2026 14:33:05 +0000 (0:00:00.033) 0:11:30.034 ******** 2026-08-18 14:33:09.167779 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=ceph-exporter) 2026-08-18 14:33:09.167859 | node1 | 2026-08-18 14:33:09.167874 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 14:33:09.167992 | node1 | node1 : ok=204 changed=46 unreachable=0 failed=1 skipped=37 rescued=0 ignored=0 2026-08-18 14:33:09.168260 | node1 | node2 : ok=210 changed=28 unreachable=0 failed=0 skipped=53 rescued=0 ignored=0 2026-08-18 14:33:09.168529 | node1 | node3 : ok=206 changed=27 unreachable=0 failed=0 skipped=45 rescued=0 ignored=0 2026-08-18 14:33:09.168795 | node1 | 2026-08-18 14:33:09.169014 | node1 | 2026-08-18 14:33:09.169251 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 14:33:09.169495 | node1 | Tuesday 18 August 2026 14:33:09 +0000 (0:00:03.399) 0:11:33.433 ******** 2026-08-18 14:33:09.169767 | node1 | =============================================================================== 2026-08-18 14:33:09.170023 | node1 | vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices ------ 240.00s 2026-08-18 14:33:09.170273 | node1 | vexxhost.ceph.mon : Bootstrap the cluster ------------------------------ 51.24s 2026-08-18 14:33:09.170525 | node1 | vexxhost.ceph.cephadm : Pre-pull the Ceph image ------------------------ 45.65s 2026-08-18 14:33:09.170842 | node1 | vexxhost.ceph.configure : Configure pools ------------------------------ 36.89s 2026-08-18 14:33:09.171168 | node1 | vexxhost.ceph.configure : Apply central Ceph configuration ------------- 27.54s 2026-08-18 14:33:09.171353 | node1 | vexxhost.ceph.cephadm_host : Register the host (labels) ---------------- 21.22s 2026-08-18 14:33:09.171526 | node1 | vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host -- 16.17s 2026-08-18 14:33:09.171725 | node1 | vexxhost.ceph.configure : Configure CRUSH rules ------------------------ 13.24s 2026-08-18 14:33:09.171879 | node1 | vexxhost.ceph.mgr : Configure ceph-mgr modules ------------------------- 12.70s 2026-08-18 14:33:09.172037 | node1 | vexxhost.ceph.configure : Configure dashboard users -------------------- 11.42s 2026-08-18 14:33:09.172224 | node1 | vexxhost.containers.containerd : Install AppArmor packages -------------- 8.25s 2026-08-18 14:33:09.172443 | node1 | vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate --- 7.31s 2026-08-18 14:33:09.172647 | node1 | vexxhost.ceph.mon : Wait for this monitor's daemon to be present -------- 6.46s 2026-08-18 14:33:09.172793 | node1 | vexxhost.ceph.mgr : Wait for this manager's daemon to be present -------- 6.42s 2026-08-18 14:33:09.172956 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 4.80s 2026-08-18 14:33:09.173170 | node1 | vexxhost.ceph.configure : Configure erasure-code profiles --------------- 4.72s 2026-08-18 14:33:09.173348 | node1 | vexxhost.ceph.rgw : Configure RGW realms -------------------------------- 3.81s 2026-08-18 14:33:09.173510 | node1 | vexxhost.containers.docker : Restart docker ----------------------------- 3.79s 2026-08-18 14:33:09.173701 | node1 | vexxhost.ceph.monitoring : Apply the monitoring service specs ----------- 3.40s 2026-08-18 14:33:09.173925 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.37s 2026-08-18 14:33:09.275579 | node1 | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:33:09.275699 | node1 | ERROR [default > converge] Executed: Failed 2026-08-18 14:33:09.275715 | node1 | ERROR Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 14:33:19.586156 | node1 | ERROR 2026-08-18 14:33:19.586395 | node1 | { 2026-08-18 14:33:19.586438 | node1 | "delta": "0:13:28.569318", 2026-08-18 14:33:19.586469 | node1 | "end": "2026-08-18 14:33:09.346191", 2026-08-18 14:33:19.586497 | node1 | "msg": "non-zero return code", 2026-08-18 14:33:19.586524 | node1 | "rc": 2, 2026-08-18 14:33:19.586550 | node1 | "start": "2026-08-18 14:19:40.776873" 2026-08-18 14:33:19.586576 | node1 | } failure 2026-08-18 14:33:19.607652 | 2026-08-18 14:33:19.607729 | PLAY RECAP 2026-08-18 14:33:19.607795 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0 2026-08-18 14:33:19.607819 | 2026-08-18 14:33:19.728465 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 14:33:19.742325 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 14:33:20.389501 | 2026-08-18 14:33:20.389617 | PLAY [all] 2026-08-18 14:33:20.402033 | 2026-08-18 14:33:20.402108 | TASK [fetch-output : Set log path for multiple nodes] 2026-08-18 14:33:20.453698 | node1 | ok 2026-08-18 14:33:20.459532 | node2 | ok 2026-08-18 14:33:20.465247 | node3 | ok 2026-08-18 14:33:20.469976 | 2026-08-18 14:33:20.470063 | TASK [fetch-output : Set log path for single node] 2026-08-18 14:33:20.502574 | node1 | skipping: Conditional result was False 2026-08-18 14:33:20.503748 | node2 | skipping: Conditional result was False 2026-08-18 14:33:20.514342 | node3 | skipping: Conditional result was False 2026-08-18 14:33:20.523384 | 2026-08-18 14:33:20.523473 | LOOP [fetch-output : Ensure local output dirs] 2026-08-18 14:33:20.907735 | node2 -> localhost | changed: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/logs/node2" 2026-08-18 14:33:20.910551 | node1 -> localhost | changed: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/logs/node1" 2026-08-18 14:33:20.927268 | node3 -> localhost | changed: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/logs/node3" 2026-08-18 14:33:21.109036 | node2 -> localhost | changed: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/artifacts" 2026-08-18 14:33:21.127727 | node3 -> localhost | ok: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/artifacts" 2026-08-18 14:33:21.132545 | node1 -> localhost | ok: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/artifacts" 2026-08-18 14:33:21.326401 | node3 -> localhost | ok: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/docs" 2026-08-18 14:33:21.328051 | node2 -> localhost | changed: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/docs" 2026-08-18 14:33:21.343674 | node1 -> localhost | ok: "/var/lib/zuul/builds/ce2dfe3a2c0b4e649eddedf3a76e04ba/work/docs" 2026-08-18 14:33:21.343803 | node1 -> localhost | changed: All items complete 2026-08-18 14:33:21.343858 | 2026-08-18 14:33:21.358497 | 2026-08-18 14:33:21.359304 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-08-18 14:33:21.957607 | node1 | changed: .d..t...... ./ 2026-08-18 14:33:21.966542 | node2 | changed: .d..t...... ./ 2026-08-18 14:33:21.971810 | node3 | changed: .d..t...... ./ 2026-08-18 14:33:22.438042 | node3 | changed: .d..t...... ./ 2026-08-18 14:33:22.444667 | node1 | ok 2026-08-18 14:33:22.478316 | node2 | ok 2026-08-18 14:33:22.909107 | node1 | changed: .d..t...... ./ 2026-08-18 14:33:22.950600 | node3 | ok 2026-08-18 14:33:22.950691 | node3 | changed: All items complete 2026-08-18 14:33:22.950726 | 2026-08-18 14:33:22.991736 | node2 | ok 2026-08-18 14:33:23.019419 | 2026-08-18 14:33:23.019551 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-08-18 14:33:23.382111 | node1 -> localhost | ok: Item: artifacts Runtime: 0:00:00.006124 2026-08-18 14:33:23.382464 | node1 -> localhost | changed: All items complete 2026-08-18 14:33:23.382511 | 2026-08-18 14:33:23.595875 | node1 -> localhost | ok: Item: docs Runtime: 0:00:00.007655 2026-08-18 14:33:23.626570 | 2026-08-18 14:33:23.626684 | PLAY [all] 2026-08-18 14:33:23.633920 | 2026-08-18 14:33:23.633983 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes] 2026-08-18 14:33:24.135522 | node1 | changed 2026-08-18 14:33:24.308396 | node3 | changed 2026-08-18 14:33:24.712205 | node2 | changed 2026-08-18 14:33:24.728254 | 2026-08-18 14:33:24.728304 | PLAY RECAP 2026-08-18 14:33:24.728348 | node1 | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:33:24.728375 | node2 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:33:24.728413 | node3 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 14:33:24.728432 | 2026-08-18 14:33:24.837831 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 14:33:24.853772 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main] 2026-08-18 14:33:25.460709 | 2026-08-18 14:33:25.460825 | PLAY [localhost] 2026-08-18 14:33:25.470291 | 2026-08-18 14:33:25.470360 | TASK [Generate Zuul manifest] 2026-08-18 14:33:25.491464 | localhost | ok 2026-08-18 14:33:25.507279 | 2026-08-18 14:33:25.507356 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-08-18 14:33:25.873500 | localhost | changed 2026-08-18 14:33:25.885449 | 2026-08-18 14:33:25.885524 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-08-18 14:33:25.917830 | localhost | ok 2026-08-18 14:33:25.926183 | 2026-08-18 14:33:25.926246 | TASK [Upload logs] 2026-08-18 14:33:25.947187 | localhost | ok 2026-08-18 14:33:26.038718 | 2026-08-18 14:33:26.038865 | TASK [Set zuul-log-path fact] 2026-08-18 14:33:26.059368 | localhost | ok 2026-08-18 14:33:26.073134 | 2026-08-18 14:33:26.073219 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 14:33:26.105738 | localhost | ok 2026-08-18 14:33:26.114429 | 2026-08-18 14:33:26.114494 | TASK [upload-logs : Create log directories] 2026-08-18 14:33:26.450925 | localhost | changed 2026-08-18 14:33:26.457348 | 2026-08-18 14:33:26.457438 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-08-18 14:33:26.823658 | localhost -> localhost | ok: Runtime: 0:00:00.005479 2026-08-18 14:33:26.830370 | 2026-08-18 14:33:26.830449 | TASK [upload-logs : Upload logs to log server] 2026-08-18 14:33:27.309367 | localhost | Output suppressed because no_log was given 2026-08-18 14:33:27.316401 | 2026-08-18 14:33:27.316494 | LOOP [upload-logs : Compress console log and json output] 2026-08-18 14:33:27.361280 | localhost | skipping: Conditional result was False 2026-08-18 14:33:27.368519 | localhost | skipping: Conditional result was False 2026-08-18 14:33:27.380773 | 2026-08-18 14:33:27.380923 | LOOP [upload-logs : Upload compressed console log and json output] 2026-08-18 14:33:27.425262 | localhost | skipping: Conditional result was False 2026-08-18 14:33:27.425674 | 2026-08-18 14:33:27.429382 | localhost | skipping: Conditional result was False 2026-08-18 14:33:27.445307 | 2026-08-18 14:33:27.445477 | LOOP [upload-logs : Upload console log and json output]