2026-08-18 13:25:13.152811 | Job console starting 2026-08-18 13:25:13.172948 | Updating git repos 2026-08-18 13:25:13.200184 | Cloning repos into workspace 2026-08-18 13:25:13.240995 | Restoring repo states 2026-08-18 13:25:13.248542 | Merging changes 2026-08-18 13:25:13.592829 | Checking out repos 2026-08-18 13:25:13.639894 | Preparing playbooks 2026-08-18 13:25:14.566346 | Running Ansible setup 2026-08-18 13:25:20.354411 | PRE-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 13:25:20.985150 | 2026-08-18 13:25:20.985266 | PLAY [localhost] 2026-08-18 13:25:20.992726 | 2026-08-18 13:25:20.992799 | TASK [Gathering Facts] 2026-08-18 13:25:21.921650 | localhost | ok 2026-08-18 13:25:21.931622 | 2026-08-18 13:25:21.931695 | TASK [Setup log path fact] 2026-08-18 13:25:21.951225 | localhost | ok 2026-08-18 13:25:21.962739 | 2026-08-18 13:25:21.962802 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 13:25:21.992068 | localhost | ok 2026-08-18 13:25:21.998483 | 2026-08-18 13:25:21.998549 | TASK [emit-job-header : Print job information] 2026-08-18 13:25:22.038984 | # Job Information 2026-08-18 13:25:22.039192 | Ansible Version: 2.16.18 2026-08-18 13:25:22.039260 | Job: ansible-collection-ceph-molecule-ha-ubuntu-noble 2026-08-18 13:25:22.039309 | Pipeline: check 2026-08-18 13:25:22.039353 | Executor: 2d72f0692154 2026-08-18 13:25:22.039396 | Triggered by: https://github.com/vexxhost/ansible-collection-ceph/pull/123 2026-08-18 13:25:22.039444 | Event ID: b9670990-9b07-11f1-9c6d-2490f072e427 2026-08-18 13:25:22.043658 | 2026-08-18 13:25:22.043738 | LOOP [emit-job-header : Print node information] 2026-08-18 13:25:22.177789 | localhost | ok: 2026-08-18 13:25:22.178013 | localhost | # Node Information 2026-08-18 13:25:22.178054 | localhost | Inventory Hostname: node1 2026-08-18 13:25:22.178084 | localhost | Hostname: np0000189993 2026-08-18 13:25:22.178111 | localhost | Username: zuul 2026-08-18 13:25:22.178141 | localhost | Distro: Ubuntu 24.04 2026-08-18 13:25:22.178168 | localhost | Provider: yul1 2026-08-18 13:25:22.178194 | localhost | Region: ca-ymq-1 2026-08-18 13:25:22.178220 | localhost | Label: ubuntu-noble 2026-08-18 13:25:22.178245 | localhost | Product Name: OpenStack Nova 2026-08-18 13:25:22.178270 | localhost | Interface IP: 199.204.45.77 2026-08-18 13:25:22.223899 | localhost | ok: 2026-08-18 13:25:22.224010 | localhost | # Node Information 2026-08-18 13:25:22.224071 | localhost | Inventory Hostname: node2 2026-08-18 13:25:22.224118 | localhost | Hostname: np0000189994 2026-08-18 13:25:22.224164 | localhost | Username: zuul 2026-08-18 13:25:22.224207 | localhost | Distro: Ubuntu 24.04 2026-08-18 13:25:22.224250 | localhost | Provider: yul1 2026-08-18 13:25:22.224293 | localhost | Region: ca-ymq-1 2026-08-18 13:25:22.224336 | localhost | Label: ubuntu-noble 2026-08-18 13:25:22.224378 | localhost | Product Name: OpenStack Nova 2026-08-18 13:25:22.224423 | localhost | Interface IP: 199.204.45.169 2026-08-18 13:25:22.271816 | localhost | ok: 2026-08-18 13:25:22.271915 | localhost | # Node Information 2026-08-18 13:25:22.271981 | localhost | Inventory Hostname: node3 2026-08-18 13:25:22.272034 | localhost | Hostname: np0000189995 2026-08-18 13:25:22.272085 | localhost | Username: zuul 2026-08-18 13:25:22.272134 | localhost | Distro: Ubuntu 24.04 2026-08-18 13:25:22.272183 | localhost | Provider: yul1 2026-08-18 13:25:22.272231 | localhost | Region: ca-ymq-1 2026-08-18 13:25:22.272278 | localhost | Label: ubuntu-noble 2026-08-18 13:25:22.272325 | localhost | Product Name: OpenStack Nova 2026-08-18 13:25:22.272372 | localhost | Interface IP: 199.204.45.35 2026-08-18 13:25:22.283454 | 2026-08-18 13:25:22.283619 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-08-18 13:25:22.666047 | localhost -> localhost | changed 2026-08-18 13:25:22.672979 | 2026-08-18 13:25:22.673078 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-08-18 13:25:23.577164 | localhost -> localhost | changed 2026-08-18 13:25:23.586706 | 2026-08-18 13:25:23.586777 | PLAY [all] 2026-08-18 13:25:23.598263 | 2026-08-18 13:25:23.598328 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-08-18 13:25:23.844405 | node1 -> localhost | ok 2026-08-18 13:25:23.851631 | 2026-08-18 13:25:23.851698 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-08-18 13:25:23.894388 | node1 | ok 2026-08-18 13:25:23.914382 | node2 | ok 2026-08-18 13:25:23.989772 | node3 | ok 2026-08-18 13:25:24.012563 | node1 | included: /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 13:25:24.012634 | node2 | included: /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 13:25:24.012658 | node3 | included: /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-08-18 13:25:24.019623 | 2026-08-18 13:25:24.019684 | TASK [add-build-sshkey : Create Temp SSH key] 2026-08-18 13:25:24.807744 | node1 -> localhost | Generating public/private rsa key pair. 2026-08-18 13:25:24.807936 | node1 -> localhost | Your identification has been saved in /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/b7fff044e3b543c1a2af2431bf46dcc5_id_rsa 2026-08-18 13:25:24.807990 | node1 -> localhost | Your public key has been saved in /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/b7fff044e3b543c1a2af2431bf46dcc5_id_rsa.pub 2026-08-18 13:25:24.808034 | node1 -> localhost | The key fingerprint is: 2026-08-18 13:25:24.808074 | node1 -> localhost | SHA256:ZxcHxsKXTjMiRNzszkoM0EcMf8w5OIEfDie0P9bydYM zuul-build-sshkey 2026-08-18 13:25:24.808117 | node1 -> localhost | The key's randomart image is: 2026-08-18 13:25:24.808158 | node1 -> localhost | +---[RSA 3072]----+ 2026-08-18 13:25:24.808198 | node1 -> localhost | | ooO=+ .o. | 2026-08-18 13:25:24.808241 | node1 -> localhost | | . =oB=*oB. | 2026-08-18 13:25:24.808277 | node1 -> localhost | | ..B++**.o. | 2026-08-18 13:25:24.808306 | node1 -> localhost | | ..o+...+ | 2026-08-18 13:25:24.808336 | node1 -> localhost | | oSo+ E o | 2026-08-18 13:25:24.808375 | node1 -> localhost | | .o*oo . . | 2026-08-18 13:25:24.808411 | node1 -> localhost | | . .. | 2026-08-18 13:25:24.808441 | node1 -> localhost | | . | 2026-08-18 13:25:24.808469 | node1 -> localhost | | | 2026-08-18 13:25:24.808497 | node1 -> localhost | +----[SHA256]-----+ 2026-08-18 13:25:24.808556 | node1 -> localhost | ok: Runtime: 0:00:00.356883 2026-08-18 13:25:24.816811 | 2026-08-18 13:25:24.816900 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-08-18 13:25:24.849938 | node1 | ok 2026-08-18 13:25:24.857247 | node2 | ok 2026-08-18 13:25:24.861701 | node3 | ok 2026-08-18 13:25:24.879229 | node1 | included: /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 13:25:24.879278 | node2 | included: /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 13:25:24.879303 | node3 | included: /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-08-18 13:25:24.893288 | 2026-08-18 13:25:24.895258 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-08-18 13:25:24.923510 | node1 | skipping: Conditional result was False 2026-08-18 13:25:24.925948 | node2 | skipping: Conditional result was False 2026-08-18 13:25:24.946478 | node3 | skipping: Conditional result was False 2026-08-18 13:25:24.956845 | 2026-08-18 13:25:24.956968 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-08-18 13:25:25.510925 | node3 | changed 2026-08-18 13:25:25.516184 | node1 | changed 2026-08-18 13:25:25.542097 | node2 | changed 2026-08-18 13:25:25.549523 | 2026-08-18 13:25:25.549676 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-08-18 13:25:25.775025 | node1 | ok 2026-08-18 13:25:25.780292 | node2 | ok 2026-08-18 13:25:25.810582 | node3 | ok 2026-08-18 13:25:25.817170 | 2026-08-18 13:25:25.817232 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-08-18 13:25:26.362345 | node1 | changed 2026-08-18 13:25:26.394570 | node2 | changed 2026-08-18 13:25:26.409654 | node3 | changed 2026-08-18 13:25:26.416635 | 2026-08-18 13:25:26.416717 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-08-18 13:25:27.006108 | node2 | changed 2026-08-18 13:25:27.013364 | node1 | changed 2026-08-18 13:25:27.020568 | node3 | changed 2026-08-18 13:25:27.027227 | 2026-08-18 13:25:27.027297 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-08-18 13:25:27.061920 | node1 | skipping: Conditional result was False 2026-08-18 13:25:27.064976 | node2 | skipping: Conditional result was False 2026-08-18 13:25:27.075439 | node3 | skipping: Conditional result was False 2026-08-18 13:25:27.086900 | 2026-08-18 13:25:27.086997 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-08-18 13:25:27.447656 | node1 -> localhost | changed 2026-08-18 13:25:27.476772 | 2026-08-18 13:25:27.476847 | TASK [add-build-sshkey : Add back temp key] 2026-08-18 13:25:27.740672 | node1 -> localhost | Identity added: /var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/b7fff044e3b543c1a2af2431bf46dcc5_id_rsa (zuul-build-sshkey) 2026-08-18 13:25:27.740912 | node1 -> localhost | ok: Runtime: 0:00:00.013291 2026-08-18 13:25:27.748616 | 2026-08-18 13:25:27.748678 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-08-18 13:25:28.085730 | node2 | ok 2026-08-18 13:25:28.104195 | node1 | ok 2026-08-18 13:25:28.134158 | node3 | ok 2026-08-18 13:25:28.141162 | 2026-08-18 13:25:28.141229 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-08-18 13:25:28.168912 | node1 | skipping: Conditional result was False 2026-08-18 13:25:28.180058 | node2 | skipping: Conditional result was False 2026-08-18 13:25:28.190586 | node3 | skipping: Conditional result was False 2026-08-18 13:25:28.252115 | 2026-08-18 13:25:28.252246 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-08-18 13:25:28.582355 | node2 | ok 2026-08-18 13:25:28.591410 | node3 | ok 2026-08-18 13:25:28.601034 | node1 | ok 2026-08-18 13:25:28.609911 | 2026-08-18 13:25:28.610042 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-08-18 13:25:29.339776 | node3 | Output suppressed because no_log was given 2026-08-18 13:25:29.345049 | node2 | Output suppressed because no_log was given 2026-08-18 13:25:29.384985 | node1 | Output suppressed because no_log was given 2026-08-18 13:25:29.415419 | 2026-08-18 13:25:29.415628 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-08-18 13:25:29.660813 | node1 | ok: "logs" 2026-08-18 13:25:29.661190 | node1 | ok: All items complete 2026-08-18 13:25:29.661231 | 2026-08-18 13:25:29.671041 | node3 | ok: "logs" 2026-08-18 13:25:29.678598 | node2 | ok: "logs" 2026-08-18 13:25:29.852838 | node1 | ok: "artifacts" 2026-08-18 13:25:29.868564 | node3 | ok: "artifacts" 2026-08-18 13:25:29.876601 | node2 | ok: "artifacts" 2026-08-18 13:25:30.047627 | node1 | ok: "docs" 2026-08-18 13:25:30.066174 | node3 | ok: "docs" 2026-08-18 13:25:30.082283 | node2 | ok: "docs" 2026-08-18 13:25:30.082365 | node2 | ok: All items complete 2026-08-18 13:25:30.082400 | 2026-08-18 13:25:30.102443 | 2026-08-18 13:25:30.102589 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-08-18 13:25:30.331541 | node1 | changed: "logs" 2026-08-18 13:25:30.337053 | node2 | changed: "logs" 2026-08-18 13:25:30.409508 | node3 | changed: "logs" 2026-08-18 13:25:30.524580 | node2 | changed: "artifacts" 2026-08-18 13:25:30.532166 | node1 | changed: "artifacts" 2026-08-18 13:25:30.615925 | node3 | changed: "artifacts" 2026-08-18 13:25:30.716396 | node2 | changed: "docs" 2026-08-18 13:25:30.725262 | node1 | changed: "docs" 2026-08-18 13:25:30.824178 | node3 | changed: "docs" 2026-08-18 13:25:30.824275 | node3 | changed: All items complete 2026-08-18 13:25:30.824316 | 2026-08-18 13:25:30.849068 | 2026-08-18 13:25:30.849117 | PLAY RECAP 2026-08-18 13:25:30.849162 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 13:25:30.849190 | node1 | ok: 15 changed: 8 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 13:25:30.849215 | node2 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 13:25:30.849238 | node3 | ok: 11 changed: 5 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-08-18 13:25:30.849256 | 2026-08-18 13:25:31.005182 | PRE-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/pre.yaml@main] 2026-08-18 13:25:31.013559 | PRE-RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 13:25:31.638383 | 2026-08-18 13:25:31.638507 | PLAY [all] 2026-08-18 13:25:31.650890 | 2026-08-18 13:25:31.650999 | TASK [setup-uv : Extract archive] 2026-08-18 13:25:34.125292 | node2 | changed 2026-08-18 13:25:34.130472 | node1 | changed 2026-08-18 13:25:34.231388 | node3 | changed 2026-08-18 13:25:34.238088 | 2026-08-18 13:25:34.238182 | TASK [setup-uv : Print version] 2026-08-18 13:25:34.654530 | node1 | uv 0.8.13 2026-08-18 13:25:34.657762 | node3 | uv 0.8.13 2026-08-18 13:25:34.640703 | node2 | uv 0.8.13 2026-08-18 13:25:34.861902 | node2 | ok: Runtime: 0:00:00.009708 2026-08-18 13:25:34.871154 | node3 | ok: Runtime: 0:00:00.014316 2026-08-18 13:25:34.879714 | node1 | ok: Runtime: 0:00:00.014786 2026-08-18 13:25:34.899197 | 2026-08-18 13:25:34.899266 | TASK [Install Ansible collection siblings] 2026-08-18 13:25:35.253277 | node1 | ok: 2026-08-18 13:25:35.253449 | node1 | Skipping community.crypto: no checked-out sibling 2026-08-18 13:25:35.253480 | node1 | Skipping community.docker: no checked-out sibling 2026-08-18 13:25:35.253503 | node1 | Skipping community.general: no checked-out sibling 2026-08-18 13:25:35.253524 | node1 | Skipping ansible.utils: no checked-out sibling 2026-08-18 13:25:35.253548 | node1 | Skipping vexxhost.containers: no checked-out sibling 2026-08-18 13:25:35.255452 | 2026-08-18 13:25:35.255498 | PLAY RECAP 2026-08-18 13:25:35.255546 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 13:25:35.255574 | node2 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 13:25:35.255599 | node3 | ok: 2 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-08-18 13:25:35.255617 | 2026-08-18 13:25:35.397062 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/pre.yaml@main] 2026-08-18 13:25:35.428609 | RUN START: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 13:25:36.022381 | 2026-08-18 13:25:36.022512 | PLAY [all] 2026-08-18 13:25:36.035072 | 2026-08-18 13:25:36.035147 | LOOP [Check inventory directories] 2026-08-18 13:25:36.422749 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 13:25:36.612154 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/molecule/default/inventory" 2026-08-18 13:25:36.806632 | node1 | ok: "src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/inventory" 2026-08-18 13:25:36.825439 | 2026-08-18 13:25:36.825536 | LOOP [Copy inventory file for Zuul] 2026-08-18 13:25:37.770255 | node1 | changed: 2026-08-18 13:25:37.770479 | node1 | { 2026-08-18 13:25:37.770517 | node1 | "ansible_loop_var": "item", 2026-08-18 13:25:37.770548 | node1 | "changed": false, 2026-08-18 13:25:37.770576 | node1 | "failed": false, 2026-08-18 13:25:37.770607 | node1 | "invocation": { 2026-08-18 13:25:37.770632 | node1 | "module_args": { 2026-08-18 13:25:37.770673 | node1 | "checksum_algorithm": "sha1", 2026-08-18 13:25:37.770699 | node1 | "follow": true, 2026-08-18 13:25:37.770723 | node1 | "get_attributes": true, 2026-08-18 13:25:37.770749 | node1 | "get_checksum": true, 2026-08-18 13:25:37.770774 | node1 | "get_mime": true, 2026-08-18 13:25:37.770799 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph" 2026-08-18 13:25:37.770843 | node1 | } 2026-08-18 13:25:37.770869 | node1 | }, 2026-08-18 13:25:37.770892 | node1 | "item": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 13:25:37.770916 | node1 | "stat": { 2026-08-18 13:25:37.770939 | node1 | "atime": 1787059529.2647343, 2026-08-18 13:25:37.770963 | node1 | "attr_flags": "e", 2026-08-18 13:25:37.770986 | node1 | "attributes": [ 2026-08-18 13:25:37.771010 | node1 | "extents" 2026-08-18 13:25:37.771034 | node1 | ], 2026-08-18 13:25:37.771057 | node1 | "block_size": 4096, 2026-08-18 13:25:37.771080 | node1 | "blocks": 8, 2026-08-18 13:25:37.771103 | node1 | "charset": "unknown", 2026-08-18 13:25:37.771127 | node1 | "ctime": 1787059529.2777345, 2026-08-18 13:25:37.771152 | node1 | "dev": 64769, 2026-08-18 13:25:37.771176 | node1 | "device_type": 0, 2026-08-18 13:25:37.771200 | node1 | "executable": true, 2026-08-18 13:25:37.771222 | node1 | "exists": true, 2026-08-18 13:25:37.771245 | node1 | "gid": 1000, 2026-08-18 13:25:37.771268 | node1 | "gr_name": "zuul", 2026-08-18 13:25:37.771291 | node1 | "inode": 1034272, 2026-08-18 13:25:37.771313 | node1 | "isblk": false, 2026-08-18 13:25:37.771337 | node1 | "ischr": false, 2026-08-18 13:25:37.771361 | node1 | "isdir": true, 2026-08-18 13:25:37.771384 | node1 | "isfifo": false, 2026-08-18 13:25:37.771407 | node1 | "isgid": false, 2026-08-18 13:25:37.771430 | node1 | "islnk": false, 2026-08-18 13:25:37.771452 | node1 | "isreg": false, 2026-08-18 13:25:37.771475 | node1 | "issock": false, 2026-08-18 13:25:37.771497 | node1 | "isuid": false, 2026-08-18 13:25:37.771519 | node1 | "mimetype": "unknown", 2026-08-18 13:25:37.771541 | node1 | "mode": "0755", 2026-08-18 13:25:37.771563 | node1 | "mtime": 1787059513.5505686, 2026-08-18 13:25:37.771586 | node1 | "nlink": 10, 2026-08-18 13:25:37.771608 | node1 | "path": "src/github.com/vexxhost/ansible-collection-ceph", 2026-08-18 13:25:37.771631 | node1 | "pw_name": "zuul", 2026-08-18 13:25:37.771655 | node1 | "readable": true, 2026-08-18 13:25:37.771677 | node1 | "rgrp": true, 2026-08-18 13:25:37.771700 | node1 | "roth": true, 2026-08-18 13:25:37.771724 | node1 | "rusr": true, 2026-08-18 13:25:37.771747 | node1 | "size": 4096, 2026-08-18 13:25:37.771769 | node1 | "uid": 1000, 2026-08-18 13:25:37.771793 | node1 | "version": "1896886773", 2026-08-18 13:25:37.771816 | node1 | "wgrp": false, 2026-08-18 13:25:37.771841 | node1 | "woth": false, 2026-08-18 13:25:37.771864 | node1 | "writeable": true, 2026-08-18 13:25:37.771888 | node1 | "wusr": true, 2026-08-18 13:25:37.771910 | node1 | "xgrp": true, 2026-08-18 13:25:37.771933 | node1 | "xoth": true, 2026-08-18 13:25:37.771956 | node1 | "xusr": true 2026-08-18 13:25:37.771979 | node1 | } 2026-08-18 13:25:37.772002 | node1 | } 2026-08-18 13:25:37.772047 | node1 | ok: All items complete 2026-08-18 13:25:37.772076 | 2026-08-18 13:25:37.773864 | node1 | skipping: Conditional result was False 2026-08-18 13:25:37.774664 | node1 | skipping: Conditional result was False 2026-08-18 13:25:37.789559 | 2026-08-18 13:25:37.789726 | LOOP [Switch "ansible_host" to private IP] 2026-08-18 13:25:38.193114 | node1 | changed: 3 replacements made 2026-08-18 13:25:38.195064 | node1 | skipping: Conditional result was False 2026-08-18 13:25:38.195630 | node1 | skipping: Conditional result was False 2026-08-18 13:25:38.215229 | 2026-08-18 13:25:38.215375 | TASK [Run Molecule scenario] 2026-08-18 13:25:38.695570 | node1 | Using CPython 3.12.3 interpreter at: /usr/bin/python3 2026-08-18 13:25:38.695687 | node1 | Creating virtual environment at: .venv 2026-08-18 13:25:38.746897 | node1 | Downloading pygments (1.2MiB) 2026-08-18 13:25:38.749427 | node1 | Downloading netaddr (2.2MiB) 2026-08-18 13:25:38.749629 | node1 | Downloading cryptography (4.3MiB) 2026-08-18 13:25:38.751337 | node1 | Downloading ansible-core (2.1MiB) 2026-08-18 13:25:38.983213 | node1 | Downloading netaddr 2026-08-18 13:25:38.993717 | node1 | Downloading cryptography 2026-08-18 13:25:39.050048 | node1 | Downloading pygments 2026-08-18 13:25:39.206468 | node1 | Downloading ansible-core 2026-08-18 13:25:39.252071 | node1 | Installed 43 packages in 44ms 2026-08-18 13:25:40.035153 | node1 | INFO Collection 'vexxhost.ceph' detected. 2026-08-18 13:25:40.035227 | node1 | INFO Scenarios will be used from 'extensions/molecule' 2026-08-18 13:25:40.657697 | node1 | INFO [default > discovery] scenario test matrix: dependency, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy 2026-08-18 13:25:40.657752 | node1 | INFO [default > prerun] Performing prerun with role_name_check=0... 2026-08-18 13:26:17.323509 | node1 | INFO [default > dependency] Executing 2026-08-18 13:26:17.323612 | node1 | WARNING [default > dependency] Missing roles requirements file: requirements.yml 2026-08-18 13:26:17.323803 | node1 | WARNING [default > dependency] Missing collections requirements file: collections.yml 2026-08-18 13:26:17.323922 | node1 | WARNING [default > dependency] Executed: 2 missing (Remove from test_sequence to suppress) 2026-08-18 13:26:17.329748 | node1 | INFO [default > cleanup] Executing 2026-08-18 13:26:17.330104 | node1 | WARNING [default > cleanup] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 13:26:17.335676 | node1 | INFO [default > destroy] Executing 2026-08-18 13:26:17.335760 | node1 | WARNING [default > destroy] Skipping, '--destroy=never' requested. 2026-08-18 13:26:17.335831 | node1 | INFO [default > destroy] Executed: Successful 2026-08-18 13:26:17.341439 | node1 | INFO [default > syntax] Executing 2026-08-18 13:26:18.637061 | node1 | 2026-08-18 13:26:18.637135 | node1 | playbook: /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/converge.yml 2026-08-18 13:26:18.704130 | node1 | INFO [default > syntax] Executed: Successful 2026-08-18 13:26:18.714212 | node1 | INFO [default > create] Executing 2026-08-18 13:26:18.715806 | node1 | WARNING [default > create] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 13:26:18.721827 | node1 | INFO [default > prepare] Executing 2026-08-18 13:26:19.520564 | node1 | 2026-08-18 13:26:19.520759 | node1 | PLAY [Prepare] ***************************************************************** 2026-08-18 13:26:19.521025 | node1 | 2026-08-18 13:26:19.521272 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:26:19.521517 | node1 | Tuesday 18 August 2026 13:26:19 +0000 (0:00:00.032) 0:00:00.032 ******** 2026-08-18 13:26:20.771834 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 13:26:20.771989 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:26:20.772144 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:26:20.774998 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:26:20.775065 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:26:20.776532 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 13:26:20.776685 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:26:20.776835 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:26:20.777014 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:26:20.777166 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:26:20.800506 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 13:26:20.800609 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:26:20.800757 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:26:20.800908 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:26:20.801088 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:26:20.811367 | node1 | ok: [node2] 2026-08-18 13:26:20.811515 | node1 | ok: [node1] 2026-08-18 13:26:20.811631 | node1 | ok: [node3] 2026-08-18 13:26:20.811768 | node1 | 2026-08-18 13:26:20.811956 | node1 | TASK [Configure short hostname] ************************************************ 2026-08-18 13:26:20.812111 | node1 | Tuesday 18 August 2026 13:26:20 +0000 (0:00:01.291) 0:00:01.324 ******** 2026-08-18 13:26:21.617161 | node1 | changed: [node2] 2026-08-18 13:26:21.617345 | node1 | changed: [node1] 2026-08-18 13:26:21.617659 | node1 | changed: [node3] 2026-08-18 13:26:21.617897 | node1 | 2026-08-18 13:26:21.618252 | node1 | TASK [Ensure hostname inside hosts file] *************************************** 2026-08-18 13:26:21.618528 | node1 | Tuesday 18 August 2026 13:26:21 +0000 (0:00:00.804) 0:00:02.129 ******** 2026-08-18 13:26:21.932156 | node1 | [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created 2026-08-18 13:26:21.932467 | node1 | with a mode of 0700, this may cause issues when running as another user. To 2026-08-18 13:26:21.932764 | node1 | avoid this, create the remote_tmp dir with the correct permissions manually 2026-08-18 13:26:21.948223 | node1 | changed: [node1] 2026-08-18 13:26:21.948482 | node1 | changed: [node3] 2026-08-18 13:26:21.948751 | node1 | changed: [node2] 2026-08-18 13:26:21.949025 | node1 | 2026-08-18 13:26:21.949259 | node1 | TASK [Update apt cache] ******************************************************** 2026-08-18 13:26:21.949516 | node1 | Tuesday 18 August 2026 13:26:21 +0000 (0:00:00.331) 0:00:02.461 ******** 2026-08-18 13:30:34.542253 | node1 | changed: [node2] 2026-08-18 13:30:34.542475 | node1 | changed: [node1] 2026-08-18 13:30:34.542739 | node1 | changed: [node3] 2026-08-18 13:30:34.543020 | node1 | 2026-08-18 13:30:34.543271 | node1 | PLAY [Create devices for Ceph] ************************************************* 2026-08-18 13:30:34.543521 | node1 | 2026-08-18 13:30:34.543766 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:30:34.544056 | node1 | Tuesday 18 August 2026 13:30:34 +0000 (0:04:12.594) 0:04:15.055 ******** 2026-08-18 13:30:36.511346 | node1 | ok: [node3] 2026-08-18 13:30:36.511554 | node1 | ok: [node2] 2026-08-18 13:30:36.511788 | node1 | ok: [node1] 2026-08-18 13:30:36.512100 | node1 | 2026-08-18 13:30:36.512495 | node1 | TASK [Install depedencies] ***************************************************** 2026-08-18 13:30:36.512826 | node1 | Tuesday 18 August 2026 13:30:36 +0000 (0:00:01.969) 0:04:17.024 ******** 2026-08-18 13:30:56.497156 | node1 | changed: [node2] 2026-08-18 13:30:56.497217 | node1 | changed: [node1] 2026-08-18 13:30:56.497223 | node1 | changed: [node3] 2026-08-18 13:30:56.497233 | node1 | 2026-08-18 13:30:56.497427 | node1 | TASK [Start up service] ******************************************************** 2026-08-18 13:30:56.497671 | node1 | Tuesday 18 August 2026 13:30:56 +0000 (0:00:19.984) 0:04:37.008 ******** 2026-08-18 13:30:57.144977 | node1 | ok: [node1] 2026-08-18 13:30:57.145035 | node1 | ok: [node3] 2026-08-18 13:30:57.145048 | node1 | ok: [node2] 2026-08-18 13:30:57.145059 | node1 | 2026-08-18 13:30:57.145071 | node1 | TASK [Generate lvm.conf] ******************************************************* 2026-08-18 13:30:57.145083 | node1 | Tuesday 18 August 2026 13:30:57 +0000 (0:00:00.646) 0:04:37.655 ******** 2026-08-18 13:30:57.505709 | node1 | ok: [node1] 2026-08-18 13:30:57.505736 | node1 | ok: [node2] 2026-08-18 13:30:57.505742 | node1 | ok: [node3] 2026-08-18 13:30:57.505747 | node1 | 2026-08-18 13:30:57.505753 | node1 | TASK [Write /etc/lvm/lvm.conf] ************************************************* 2026-08-18 13:30:57.505758 | node1 | Tuesday 18 August 2026 13:30:57 +0000 (0:00:00.363) 0:04:38.018 ******** 2026-08-18 13:30:58.274794 | node1 | changed: [node1] 2026-08-18 13:30:58.274853 | node1 | changed: [node2] 2026-08-18 13:30:58.274866 | node1 | changed: [node3] 2026-08-18 13:30:58.274878 | node1 | 2026-08-18 13:30:58.274891 | node1 | TASK [Get list of all loopback devices] **************************************** 2026-08-18 13:30:58.274917 | node1 | Tuesday 18 August 2026 13:30:58 +0000 (0:00:00.767) 0:04:38.786 ******** 2026-08-18 13:30:58.544038 | node1 | ok: [node1] 2026-08-18 13:30:58.544100 | node1 | ok: [node2] 2026-08-18 13:30:58.544106 | node1 | ok: [node3] 2026-08-18 13:30:58.544112 | node1 | 2026-08-18 13:30:58.544118 | node1 | TASK [Fail if there is any existing loopback devices] ************************** 2026-08-18 13:30:58.544125 | node1 | Tuesday 18 August 2026 13:30:58 +0000 (0:00:00.269) 0:04:39.055 ******** 2026-08-18 13:30:58.588437 | node1 | skipping: [node1] 2026-08-18 13:30:58.588641 | node1 | skipping: [node2] 2026-08-18 13:30:58.588876 | node1 | skipping: [node3] 2026-08-18 13:30:58.589133 | node1 | 2026-08-18 13:30:58.589377 | node1 | TASK [Create devices for Ceph] ************************************************* 2026-08-18 13:30:58.589697 | node1 | Tuesday 18 August 2026 13:30:58 +0000 (0:00:00.045) 0:04:39.101 ******** 2026-08-18 13:30:59.346336 | node1 | changed: [node1] => (item=osd0) 2026-08-18 13:30:59.346547 | node1 | changed: [node2] => (item=osd0) 2026-08-18 13:30:59.346797 | node1 | changed: [node3] => (item=osd0) 2026-08-18 13:30:59.347100 | node1 | changed: [node2] => (item=osd1) 2026-08-18 13:30:59.347325 | node1 | changed: [node1] => (item=osd1) 2026-08-18 13:30:59.347556 | node1 | changed: [node3] => (item=osd1) 2026-08-18 13:30:59.347811 | node1 | changed: [node2] => (item=osd2) 2026-08-18 13:30:59.348082 | node1 | changed: [node1] => (item=osd2) 2026-08-18 13:30:59.348332 | node1 | changed: [node3] => (item=osd2) 2026-08-18 13:30:59.348546 | node1 | 2026-08-18 13:30:59.348791 | node1 | TASK [Set permissions on loopback devices] ************************************* 2026-08-18 13:30:59.349072 | node1 | Tuesday 18 August 2026 13:30:59 +0000 (0:00:00.757) 0:04:39.859 ******** 2026-08-18 13:31:00.142813 | node1 | changed: [node2] => (item=osd0) 2026-08-18 13:31:00.143073 | node1 | changed: [node1] => (item=osd0) 2026-08-18 13:31:00.143335 | node1 | changed: [node3] => (item=osd0) 2026-08-18 13:31:00.143606 | node1 | changed: [node2] => (item=osd1) 2026-08-18 13:31:00.143859 | node1 | changed: [node1] => (item=osd1) 2026-08-18 13:31:00.144138 | node1 | changed: [node3] => (item=osd1) 2026-08-18 13:31:00.144392 | node1 | changed: [node2] => (item=osd2) 2026-08-18 13:31:00.144669 | node1 | changed: [node1] => (item=osd2) 2026-08-18 13:31:00.144915 | node1 | changed: [node3] => (item=osd2) 2026-08-18 13:31:00.145184 | node1 | 2026-08-18 13:31:00.145438 | node1 | TASK [Start loop devices] ****************************************************** 2026-08-18 13:31:00.145707 | node1 | Tuesday 18 August 2026 13:31:00 +0000 (0:00:00.796) 0:04:40.655 ******** 2026-08-18 13:31:00.958378 | node1 | changed: [node2] => (item=osd0) 2026-08-18 13:31:00.958444 | node1 | changed: [node3] => (item=osd0) 2026-08-18 13:31:00.958592 | node1 | changed: [node1] => (item=osd0) 2026-08-18 13:31:00.958708 | node1 | changed: [node2] => (item=osd1) 2026-08-18 13:31:00.958842 | node1 | changed: [node3] => (item=osd1) 2026-08-18 13:31:00.959000 | node1 | changed: [node1] => (item=osd1) 2026-08-18 13:31:00.959157 | node1 | changed: [node3] => (item=osd2) 2026-08-18 13:31:00.959414 | node1 | changed: [node1] => (item=osd2) 2026-08-18 13:31:00.959674 | node1 | changed: [node2] => (item=osd2) 2026-08-18 13:31:00.959909 | node1 | 2026-08-18 13:31:00.960209 | node1 | TASK [Create a volume group for each loop device] ****************************** 2026-08-18 13:31:00.960499 | node1 | Tuesday 18 August 2026 13:31:00 +0000 (0:00:00.815) 0:04:41.471 ******** 2026-08-18 13:31:02.433036 | node1 | changed: [node1] => (item=osd0) 2026-08-18 13:31:02.433132 | node1 | changed: [node3] => (item=osd0) 2026-08-18 13:31:02.433141 | node1 | changed: [node2] => (item=osd0) 2026-08-18 13:31:02.433148 | node1 | changed: [node3] => (item=osd1) 2026-08-18 13:31:02.433156 | node1 | changed: [node2] => (item=osd1) 2026-08-18 13:31:02.433164 | node1 | changed: [node1] => (item=osd1) 2026-08-18 13:31:02.433171 | node1 | changed: [node2] => (item=osd2) 2026-08-18 13:31:02.433178 | node1 | changed: [node3] => (item=osd2) 2026-08-18 13:31:02.433185 | node1 | changed: [node1] => (item=osd2) 2026-08-18 13:31:02.433193 | node1 | 2026-08-18 13:31:02.435104 | node1 | TASK [Create a logical volume for each loop device] **************************** 2026-08-18 13:31:02.435644 | node1 | Tuesday 18 August 2026 13:31:02 +0000 (0:00:01.471) 0:04:42.943 ******** 2026-08-18 13:31:03.817994 | node1 | changed: [node1] => (item=ceph-node1-osd0) 2026-08-18 13:31:03.819021 | node1 | changed: [node2] => (item=ceph-node2-osd0) 2026-08-18 13:31:03.819088 | node1 | changed: [node3] => (item=ceph-node3-osd0) 2026-08-18 13:31:03.819094 | node1 | changed: [node3] => (item=ceph-node3-osd1) 2026-08-18 13:31:03.819100 | node1 | changed: [node2] => (item=ceph-node2-osd1) 2026-08-18 13:31:03.819264 | node1 | changed: [node1] => (item=ceph-node1-osd1) 2026-08-18 13:31:03.819523 | node1 | changed: [node3] => (item=ceph-node3-osd2) 2026-08-18 13:31:03.819784 | node1 | changed: [node1] => (item=ceph-node1-osd2) 2026-08-18 13:31:03.822058 | node1 | changed: [node2] => (item=ceph-node2-osd2) 2026-08-18 13:31:03.822109 | node1 | 2026-08-18 13:31:03.822116 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 13:31:03.822123 | node1 | node1 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 13:31:03.822129 | node1 | node2 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 13:31:03.822134 | node1 | node3 : ok=15 changed=10 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 2026-08-18 13:31:03.822139 | node1 | 2026-08-18 13:31:03.822149 | node1 | 2026-08-18 13:31:03.822988 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 13:31:03.823033 | node1 | Tuesday 18 August 2026 13:31:03 +0000 (0:00:01.387) 0:04:44.330 ******** 2026-08-18 13:31:03.823047 | node1 | =============================================================================== 2026-08-18 13:31:03.823286 | node1 | Update apt cache ------------------------------------------------------ 252.59s 2026-08-18 13:31:03.823622 | node1 | Install depedencies ---------------------------------------------------- 19.98s 2026-08-18 13:31:03.823993 | node1 | Gathering Facts --------------------------------------------------------- 1.97s 2026-08-18 13:31:03.824528 | node1 | Create a volume group for each loop device ------------------------------ 1.47s 2026-08-18 13:31:03.824873 | node1 | Create a logical volume for each loop device ---------------------------- 1.39s 2026-08-18 13:31:03.825216 | node1 | Gathering Facts --------------------------------------------------------- 1.29s 2026-08-18 13:31:03.825355 | node1 | Start loop devices ------------------------------------------------------ 0.82s 2026-08-18 13:31:03.825490 | node1 | Configure short hostname ------------------------------------------------ 0.80s 2026-08-18 13:31:03.825626 | node1 | Set permissions on loopback devices ------------------------------------- 0.80s 2026-08-18 13:31:03.825761 | node1 | Write /etc/lvm/lvm.conf ------------------------------------------------- 0.77s 2026-08-18 13:31:03.825895 | node1 | Create devices for Ceph ------------------------------------------------- 0.76s 2026-08-18 13:31:03.826051 | node1 | Start up service -------------------------------------------------------- 0.65s 2026-08-18 13:31:03.826193 | node1 | Generate lvm.conf ------------------------------------------------------- 0.36s 2026-08-18 13:31:03.826327 | node1 | Ensure hostname inside hosts file --------------------------------------- 0.33s 2026-08-18 13:31:03.826462 | node1 | Get list of all loopback devices ---------------------------------------- 0.27s 2026-08-18 13:31:03.826606 | node1 | Fail if there is any existing loopback devices -------------------------- 0.05s 2026-08-18 13:31:03.906402 | node1 | INFO [default > prepare] Executed: Successful 2026-08-18 13:31:03.918486 | node1 | INFO [default > converge] Executing 2026-08-18 13:31:05.109823 | node1 | 2026-08-18 13:31:05.111000 | node1 | PLAY [Deploy Ceph monitors & managers] ***************************************** 2026-08-18 13:31:05.111027 | node1 | 2026-08-18 13:31:05.111038 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:31:05.111046 | node1 | Tuesday 18 August 2026 13:31:05 +0000 (0:00:00.061) 0:00:00.061 ******** 2026-08-18 13:31:06.380610 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 13:31:06.380649 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:31:06.380655 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:31:06.380776 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:31:06.381162 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:31:06.465149 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 13:31:06.465219 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:31:06.465232 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:31:06.465315 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:31:06.465610 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:31:06.470328 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 13:31:06.470530 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:31:06.470761 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:31:06.471116 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:31:06.471346 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:31:06.531259 | node1 | ok: [node1] 2026-08-18 13:31:06.531448 | node1 | ok: [node2] 2026-08-18 13:31:06.531678 | node1 | ok: [node3] 2026-08-18 13:31:06.531896 | node1 | 2026-08-18 13:31:06.532198 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:31:06.532427 | node1 | Tuesday 18 August 2026 13:31:06 +0000 (0:00:01.421) 0:00:01.482 ******** 2026-08-18 13:31:06.891412 | node1 | ok: [node1] 2026-08-18 13:31:06.891610 | node1 | ok: [node2] 2026-08-18 13:31:06.891841 | node1 | ok: [node3] 2026-08-18 13:31:06.892170 | node1 | 2026-08-18 13:31:06.892461 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:31:06.892714 | node1 | Tuesday 18 August 2026 13:31:06 +0000 (0:00:00.359) 0:00:01.842 ******** 2026-08-18 13:31:06.991999 | node1 | skipping: [node1] 2026-08-18 13:31:06.992305 | node1 | skipping: [node2] 2026-08-18 13:31:06.992593 | node1 | skipping: [node3] 2026-08-18 13:31:06.992859 | node1 | 2026-08-18 13:31:06.993237 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:31:06.993548 | node1 | Tuesday 18 August 2026 13:31:06 +0000 (0:00:00.100) 0:00:01.943 ******** 2026-08-18 13:31:07.393135 | node1 | changed: [node1] 2026-08-18 13:31:07.393196 | node1 | changed: [node3] 2026-08-18 13:31:07.393207 | node1 | changed: [node2] 2026-08-18 13:31:07.393217 | node1 | 2026-08-18 13:31:07.393233 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:31:07.393446 | node1 | Tuesday 18 August 2026 13:31:07 +0000 (0:00:00.400) 0:00:02.343 ******** 2026-08-18 13:31:07.502740 | node1 | ok: [node2] => { 2026-08-18 13:31:07.502781 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:31:07.502972 | node1 | } 2026-08-18 13:31:07.503123 | node1 | ok: [node3] => { 2026-08-18 13:31:07.503288 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:31:07.503433 | node1 | } 2026-08-18 13:31:07.503582 | node1 | ok: [node1] => { 2026-08-18 13:31:07.503733 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:31:07.503876 | node1 | } 2026-08-18 13:31:07.504050 | node1 | 2026-08-18 13:31:07.504391 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:31:07.504558 | node1 | Tuesday 18 August 2026 13:31:07 +0000 (0:00:00.110) 0:00:02.453 ******** 2026-08-18 13:31:08.410961 | node1 | changed: [node2] 2026-08-18 13:31:08.411174 | node1 | changed: [node3] 2026-08-18 13:31:08.411411 | node1 | changed: [node1] 2026-08-18 13:31:08.411638 | node1 | 2026-08-18 13:31:08.411893 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:31:08.412194 | node1 | Tuesday 18 August 2026 13:31:08 +0000 (0:00:00.908) 0:00:03.362 ******** 2026-08-18 13:31:08.529466 | node1 | skipping: [node1] 2026-08-18 13:31:08.529656 | node1 | skipping: [node2] 2026-08-18 13:31:08.529897 | node1 | skipping: [node3] 2026-08-18 13:31:08.530175 | node1 | 2026-08-18 13:31:08.530416 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:31:08.530699 | node1 | Tuesday 18 August 2026 13:31:08 +0000 (0:00:00.118) 0:00:03.480 ******** 2026-08-18 13:31:08.649634 | node1 | skipping: [node1] 2026-08-18 13:31:08.649824 | node1 | skipping: [node2] 2026-08-18 13:31:08.650117 | node1 | skipping: [node3] 2026-08-18 13:31:08.650335 | node1 | 2026-08-18 13:31:08.650610 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:31:08.650879 | node1 | Tuesday 18 August 2026 13:31:08 +0000 (0:00:00.120) 0:00:03.600 ******** 2026-08-18 13:31:08.954131 | node1 | ok: [node1] 2026-08-18 13:31:08.954295 | node1 | ok: [node2] 2026-08-18 13:31:08.954527 | node1 | ok: [node3] 2026-08-18 13:31:08.954791 | node1 | 2026-08-18 13:31:08.955123 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:31:08.955374 | node1 | Tuesday 18 August 2026 13:31:08 +0000 (0:00:00.304) 0:00:03.905 ******** 2026-08-18 13:31:10.127157 | node1 | ok: [node1] 2026-08-18 13:31:10.127269 | node1 | ok: [node3] 2026-08-18 13:31:10.128298 | node1 | ok: [node2] 2026-08-18 13:31:10.128365 | node1 | 2026-08-18 13:31:10.128376 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:31:10.128386 | node1 | Tuesday 18 August 2026 13:31:10 +0000 (0:00:01.172) 0:00:05.077 ******** 2026-08-18 13:31:10.353136 | node1 | ok: [node1] => { 2026-08-18 13:31:10.353290 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:31:10.353303 | node1 | } 2026-08-18 13:31:10.353312 | node1 | ok: [node3] => { 2026-08-18 13:31:10.353331 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:31:10.353340 | node1 | } 2026-08-18 13:31:10.354031 | node1 | ok: [node2] => { 2026-08-18 13:31:10.354118 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:31:10.354295 | node1 | } 2026-08-18 13:31:10.354578 | node1 | 2026-08-18 13:31:10.354893 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:31:10.355242 | node1 | Tuesday 18 August 2026 13:31:10 +0000 (0:00:00.224) 0:00:05.302 ******** 2026-08-18 13:31:11.362743 | node1 | changed: [node2] 2026-08-18 13:31:11.364257 | node1 | changed: [node1] 2026-08-18 13:31:11.364285 | node1 | changed: [node3] 2026-08-18 13:31:11.364295 | node1 | 2026-08-18 13:31:11.364304 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:31:11.364313 | node1 | Tuesday 18 August 2026 13:31:11 +0000 (0:00:01.010) 0:00:06.313 ******** 2026-08-18 13:31:14.648123 | node1 | changed: [node3] 2026-08-18 13:31:14.648195 | node1 | changed: [node1] 2026-08-18 13:31:14.648208 | node1 | changed: [node2] 2026-08-18 13:31:14.648220 | node1 | 2026-08-18 13:31:14.648232 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:31:14.648244 | node1 | Tuesday 18 August 2026 13:31:14 +0000 (0:00:03.284) 0:00:09.597 ******** 2026-08-18 13:31:14.718873 | node1 | skipping: [node1] 2026-08-18 13:31:14.718923 | node1 | skipping: [node2] 2026-08-18 13:31:14.718978 | node1 | skipping: [node3] 2026-08-18 13:31:14.718991 | node1 | 2026-08-18 13:31:14.719003 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:31:14.719015 | node1 | Tuesday 18 August 2026 13:31:14 +0000 (0:00:00.071) 0:00:09.668 ******** 2026-08-18 13:31:14.790133 | node1 | skipping: [node1] 2026-08-18 13:31:14.790267 | node1 | skipping: [node2] 2026-08-18 13:31:14.791300 | node1 | skipping: [node3] 2026-08-18 13:31:14.791369 | node1 | 2026-08-18 13:31:14.791383 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:31:14.791398 | node1 | Tuesday 18 August 2026 13:31:14 +0000 (0:00:00.071) 0:00:09.739 ******** 2026-08-18 13:31:14.871839 | node1 | skipping: [node1] 2026-08-18 13:31:14.871878 | node1 | skipping: [node2] 2026-08-18 13:31:14.871892 | node1 | skipping: [node3] 2026-08-18 13:31:14.871905 | node1 | 2026-08-18 13:31:14.871918 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:31:14.871967 | node1 | Tuesday 18 August 2026 13:31:14 +0000 (0:00:00.081) 0:00:09.821 ******** 2026-08-18 13:31:49.268707 | node1 | changed: [node2] 2026-08-18 13:31:49.268818 | node1 | changed: [node3] 2026-08-18 13:31:49.268825 | node1 | changed: [node1] 2026-08-18 13:31:49.268831 | node1 | 2026-08-18 13:31:49.268845 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:31:49.268853 | node1 | Tuesday 18 August 2026 13:31:49 +0000 (0:00:34.394) 0:00:44.216 ******** 2026-08-18 13:31:49.972100 | node1 | changed: [node1] 2026-08-18 13:31:49.972167 | node1 | changed: [node3] 2026-08-18 13:31:49.973299 | node1 | changed: [node2] 2026-08-18 13:31:49.973362 | node1 | 2026-08-18 13:31:49.973373 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:31:49.973383 | node1 | Tuesday 18 August 2026 13:31:49 +0000 (0:00:00.705) 0:00:44.922 ******** 2026-08-18 13:31:51.181003 | node1 | changed: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:31:51.181053 | node1 | changed: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:31:51.181063 | node1 | changed: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:31:51.181072 | node1 | changed: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:31:51.181081 | node1 | changed: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:31:51.181090 | node1 | changed: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:31:51.181098 | node1 | changed: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:31:51.181106 | node1 | changed: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:31:51.181245 | node1 | changed: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:31:51.181586 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:31:51.181992 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:31:51.182352 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:31:51.182685 | node1 | changed: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:31:51.183053 | node1 | changed: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:31:51.183395 | node1 | changed: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:31:51.183688 | node1 | 2026-08-18 13:31:51.184048 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:31:51.184398 | node1 | Tuesday 18 August 2026 13:31:51 +0000 (0:00:01.207) 0:00:46.129 ******** 2026-08-18 13:31:52.016682 | node1 | changed: [node1] 2026-08-18 13:31:52.017085 | node1 | changed: [node2] 2026-08-18 13:31:52.017399 | node1 | changed: [node3] 2026-08-18 13:31:52.017687 | node1 | 2026-08-18 13:31:52.018033 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:31:52.018354 | node1 | Tuesday 18 August 2026 13:31:51 +0000 (0:00:00.813) 0:00:46.942 ******** 2026-08-18 13:31:52.018646 | node1 | 2026-08-18 13:31:52.019081 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:31:52.019414 | node1 | Tuesday 18 August 2026 13:31:51 +0000 (0:00:00.008) 0:00:46.951 ******** 2026-08-18 13:31:52.019718 | node1 | 2026-08-18 13:31:52.020066 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:31:52.020381 | node1 | Tuesday 18 August 2026 13:31:52 +0000 (0:00:00.008) 0:00:46.959 ******** 2026-08-18 13:31:52.020661 | node1 | 2026-08-18 13:31:52.021002 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 13:31:52.021319 | node1 | Tuesday 18 August 2026 13:31:52 +0000 (0:00:00.008) 0:00:46.967 ******** 2026-08-18 13:31:53.182371 | node1 | ok: [node2] 2026-08-18 13:31:53.182436 | node1 | ok: [node1] 2026-08-18 13:31:53.182538 | node1 | ok: [node3] 2026-08-18 13:31:53.182881 | node1 | 2026-08-18 13:31:53.182906 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Restart containerd] ********** 2026-08-18 13:31:53.182912 | node1 | Tuesday 18 August 2026 13:31:53 +0000 (0:00:01.165) 0:00:48.132 ******** 2026-08-18 13:31:53.807893 | node1 | changed: [node2] 2026-08-18 13:31:53.807995 | node1 | changed: [node1] 2026-08-18 13:31:53.808002 | node1 | changed: [node3] 2026-08-18 13:31:53.808007 | node1 | 2026-08-18 13:31:53.808014 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:31:53.808020 | node1 | Tuesday 18 August 2026 13:31:53 +0000 (0:00:00.623) 0:00:48.756 ******** 2026-08-18 13:31:54.607212 | node1 | changed: [node1] 2026-08-18 13:31:54.607284 | node1 | changed: [node2] 2026-08-18 13:31:54.607294 | node1 | changed: [node3] 2026-08-18 13:31:54.607304 | node1 | 2026-08-18 13:31:54.607314 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:31:54.607332 | node1 | Tuesday 18 August 2026 13:31:54 +0000 (0:00:00.799) 0:00:49.556 ******** 2026-08-18 13:31:54.910042 | node1 | ok: [node1] 2026-08-18 13:31:54.911485 | node1 | ok: [node2] 2026-08-18 13:31:54.911512 | node1 | ok: [node3] 2026-08-18 13:31:54.911521 | node1 | 2026-08-18 13:31:54.911531 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:31:54.911540 | node1 | Tuesday 18 August 2026 13:31:54 +0000 (0:00:00.303) 0:00:49.860 ******** 2026-08-18 13:31:55.062150 | node1 | ok: [node1] => { 2026-08-18 13:31:55.062236 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:31:55.062246 | node1 | } 2026-08-18 13:31:55.062393 | node1 | ok: [node2] => { 2026-08-18 13:31:55.062652 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:31:55.062895 | node1 | } 2026-08-18 13:31:55.063204 | node1 | ok: [node3] => { 2026-08-18 13:31:55.063471 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:31:55.063714 | node1 | } 2026-08-18 13:31:55.063977 | node1 | 2026-08-18 13:31:55.064255 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:31:55.064521 | node1 | Tuesday 18 August 2026 13:31:55 +0000 (0:00:00.152) 0:00:50.012 ******** 2026-08-18 13:31:56.361490 | node1 | changed: [node2] 2026-08-18 13:31:56.361704 | node1 | changed: [node3] 2026-08-18 13:31:56.361970 | node1 | changed: [node1] 2026-08-18 13:31:56.362196 | node1 | 2026-08-18 13:31:56.362448 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:31:56.362691 | node1 | Tuesday 18 August 2026 13:31:56 +0000 (0:00:01.299) 0:00:51.312 ******** 2026-08-18 13:32:01.194152 | node1 | changed: [node2] 2026-08-18 13:32:01.194213 | node1 | changed: [node3] 2026-08-18 13:32:01.194223 | node1 | changed: [node1] 2026-08-18 13:32:01.194239 | node1 | 2026-08-18 13:32:01.194416 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:32:01.194653 | node1 | Tuesday 18 August 2026 13:32:01 +0000 (0:00:04.832) 0:00:56.144 ******** 2026-08-18 13:32:02.120611 | node1 | ok: [node2] 2026-08-18 13:32:02.120792 | node1 | ok: [node1] 2026-08-18 13:32:02.121066 | node1 | ok: [node3] 2026-08-18 13:32:02.121278 | node1 | 2026-08-18 13:32:02.121530 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:32:02.121775 | node1 | Tuesday 18 August 2026 13:32:02 +0000 (0:00:00.927) 0:00:57.071 ******** 2026-08-18 13:32:02.542293 | node1 | changed: [node3] 2026-08-18 13:32:02.542527 | node1 | changed: [node1] 2026-08-18 13:32:02.542972 | node1 | changed: [node2] 2026-08-18 13:32:02.543248 | node1 | 2026-08-18 13:32:02.543523 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:32:02.543802 | node1 | Tuesday 18 August 2026 13:32:02 +0000 (0:00:00.421) 0:00:57.493 ******** 2026-08-18 13:32:03.153385 | node1 | changed: [node1] 2026-08-18 13:32:03.153616 | node1 | changed: [node3] 2026-08-18 13:32:03.153870 | node1 | changed: [node2] 2026-08-18 13:32:03.154146 | node1 | 2026-08-18 13:32:03.154448 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:32:03.154686 | node1 | Tuesday 18 August 2026 13:32:03 +0000 (0:00:00.610) 0:00:58.104 ******** 2026-08-18 13:32:03.909179 | node1 | changed: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:32:03.909379 | node1 | changed: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:32:03.909637 | node1 | changed: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:32:03.909888 | node1 | changed: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:32:03.910183 | node1 | changed: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:32:03.910441 | node1 | changed: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:32:03.910686 | node1 | changed: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:32:03.910997 | node1 | changed: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:32:03.911343 | node1 | changed: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:32:03.911705 | node1 | 2026-08-18 13:32:03.912108 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:32:03.912388 | node1 | Tuesday 18 August 2026 13:32:03 +0000 (0:00:00.755) 0:00:58.860 ******** 2026-08-18 13:32:04.542521 | node1 | changed: [node2] 2026-08-18 13:32:04.542763 | node1 | changed: [node1] 2026-08-18 13:32:04.543067 | node1 | changed: [node3] 2026-08-18 13:32:04.543310 | node1 | 2026-08-18 13:32:04.543565 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:32:04.543854 | node1 | Tuesday 18 August 2026 13:32:04 +0000 (0:00:00.633) 0:00:59.493 ******** 2026-08-18 13:32:05.198450 | node1 | changed: [node1] 2026-08-18 13:32:05.198718 | node1 | changed: [node2] 2026-08-18 13:32:05.199085 | node1 | changed: [node3] 2026-08-18 13:32:05.199361 | node1 | 2026-08-18 13:32:05.199677 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:32:05.200030 | node1 | Tuesday 18 August 2026 13:32:05 +0000 (0:00:00.626) 0:01:00.119 ******** 2026-08-18 13:32:05.200329 | node1 | 2026-08-18 13:32:05.200623 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:32:05.201123 | node1 | Tuesday 18 August 2026 13:32:05 +0000 (0:00:00.009) 0:01:00.129 ******** 2026-08-18 13:32:05.201338 | node1 | 2026-08-18 13:32:05.201590 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:32:05.201882 | node1 | Tuesday 18 August 2026 13:32:05 +0000 (0:00:00.009) 0:01:00.139 ******** 2026-08-18 13:32:05.202178 | node1 | 2026-08-18 13:32:05.202489 | node1 | RUNNING HANDLER [vexxhost.containers.containerd : Reload systemd] ************** 2026-08-18 13:32:05.203428 | node1 | Tuesday 18 August 2026 13:32:05 +0000 (0:00:00.010) 0:01:00.149 ******** 2026-08-18 13:32:06.143396 | node1 | ok: [node1] 2026-08-18 13:32:06.143450 | node1 | ok: [node3] 2026-08-18 13:32:06.143457 | node1 | ok: [node2] 2026-08-18 13:32:06.143464 | node1 | 2026-08-18 13:32:06.143471 | node1 | RUNNING HANDLER [vexxhost.containers.docker : Restart docker] ****************** 2026-08-18 13:32:06.143486 | node1 | Tuesday 18 August 2026 13:32:06 +0000 (0:00:00.943) 0:01:01.093 ******** 2026-08-18 13:32:08.287711 | node1 | changed: [node3] 2026-08-18 13:32:08.287916 | node1 | changed: [node2] 2026-08-18 13:32:08.288197 | node1 | changed: [node1] 2026-08-18 13:32:08.288424 | node1 | 2026-08-18 13:32:08.288672 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:32:08.288921 | node1 | Tuesday 18 August 2026 13:32:08 +0000 (0:00:02.145) 0:01:03.238 ******** 2026-08-18 13:32:10.004105 | node1 | changed: [node1] 2026-08-18 13:32:10.004173 | node1 | changed: [node3] 2026-08-18 13:32:10.004186 | node1 | changed: [node2] 2026-08-18 13:32:10.004198 | node1 | 2026-08-18 13:32:10.004217 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:32:10.004424 | node1 | Tuesday 18 August 2026 13:32:10 +0000 (0:00:01.715) 0:01:04.954 ******** 2026-08-18 13:32:10.074981 | node1 | ok: [node1] => { 2026-08-18 13:32:10.075020 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:32:10.075033 | node1 | } 2026-08-18 13:32:10.075045 | node1 | ok: [node2] => { 2026-08-18 13:32:10.075057 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:32:10.075070 | node1 | } 2026-08-18 13:32:10.075081 | node1 | ok: [node3] => { 2026-08-18 13:32:10.075093 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:32:10.075124 | node1 | } 2026-08-18 13:32:10.075133 | node1 | 2026-08-18 13:32:10.075142 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:32:10.075450 | node1 | Tuesday 18 August 2026 13:32:10 +0000 (0:00:00.068) 0:01:05.022 ******** 2026-08-18 13:32:11.618322 | node1 | changed: [node1] 2026-08-18 13:32:11.618498 | node1 | changed: [node2] 2026-08-18 13:32:11.618640 | node1 | changed: [node3] 2026-08-18 13:32:11.618772 | node1 | 2026-08-18 13:32:11.618918 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:32:11.620852 | node1 | Tuesday 18 August 2026 13:32:11 +0000 (0:00:01.546) 0:01:06.569 ******** 2026-08-18 13:32:11.657384 | node1 | skipping: [node1] 2026-08-18 13:32:11.721640 | node1 | skipping: [node2] 2026-08-18 13:32:11.721905 | node1 | skipping: [node3] 2026-08-18 13:32:11.722138 | node1 | 2026-08-18 13:32:11.722375 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:32:11.722580 | node1 | Tuesday 18 August 2026 13:32:11 +0000 (0:00:00.103) 0:01:06.672 ******** 2026-08-18 13:32:11.758443 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:32:11.807521 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:32:11.807574 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:32:11.807675 | node1 | 2026-08-18 13:32:11.807787 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:32:11.807901 | node1 | Tuesday 18 August 2026 13:32:11 +0000 (0:00:00.086) 0:01:06.759 ******** 2026-08-18 13:32:12.758881 | node1 | ok: [node1] 2026-08-18 13:32:12.758984 | node1 | ok: [node2] 2026-08-18 13:32:12.759047 | node1 | ok: [node3] 2026-08-18 13:32:12.759404 | node1 | 2026-08-18 13:32:12.759431 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:32:12.759436 | node1 | Tuesday 18 August 2026 13:32:12 +0000 (0:00:00.951) 0:01:07.710 ******** 2026-08-18 13:32:13.645082 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:32:13.645194 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:32:13.645207 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:32:13.645219 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:32:13.645231 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:32:13.645242 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:32:13.645254 | node1 | 2026-08-18 13:32:13.645266 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:32:13.645287 | node1 | Tuesday 18 August 2026 13:32:13 +0000 (0:00:00.883) 0:01:08.593 ******** 2026-08-18 13:32:13.960401 | node1 | ok: [node2] 2026-08-18 13:32:13.960568 | node1 | ok: [node1] 2026-08-18 13:32:13.960584 | node1 | ok: [node3] 2026-08-18 13:32:13.960687 | node1 | 2026-08-18 13:32:13.960809 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:32:13.960904 | node1 | Tuesday 18 August 2026 13:32:13 +0000 (0:00:00.318) 0:01:08.911 ******** 2026-08-18 13:32:14.563094 | node1 | changed: [node1] 2026-08-18 13:32:14.563137 | node1 | changed: [node3] 2026-08-18 13:32:14.563143 | node1 | changed: [node2] 2026-08-18 13:32:14.563149 | node1 | 2026-08-18 13:32:14.563156 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:32:14.563171 | node1 | Tuesday 18 August 2026 13:32:14 +0000 (0:00:00.602) 0:01:09.513 ******** 2026-08-18 13:32:15.215904 | node1 | changed: [node1] 2026-08-18 13:32:15.215990 | node1 | changed: [node2] 2026-08-18 13:32:15.216003 | node1 | changed: [node3] 2026-08-18 13:32:15.216015 | node1 | 2026-08-18 13:32:15.216027 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:32:15.216039 | node1 | Tuesday 18 August 2026 13:32:15 +0000 (0:00:00.651) 0:01:10.165 ******** 2026-08-18 13:32:15.308218 | node1 | skipping: [node1] 2026-08-18 13:32:15.308268 | node1 | skipping: [node2] 2026-08-18 13:32:15.308276 | node1 | skipping: [node3] 2026-08-18 13:32:15.308283 | node1 | 2026-08-18 13:32:15.308313 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:32:15.308319 | node1 | Tuesday 18 August 2026 13:32:15 +0000 (0:00:00.092) 0:01:10.257 ******** 2026-08-18 13:32:57.576890 | node1 | changed: [node1] 2026-08-18 13:32:57.577175 | node1 | changed: [node3] 2026-08-18 13:32:57.577470 | node1 | changed: [node2] 2026-08-18 13:32:57.577718 | node1 | 2026-08-18 13:32:57.578313 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:32:57.578351 | node1 | Tuesday 18 August 2026 13:32:57 +0000 (0:00:42.269) 0:01:52.526 ******** 2026-08-18 13:32:58.212813 | node1 | changed: [node1] 2026-08-18 13:32:58.213092 | node1 | changed: [node2] 2026-08-18 13:32:58.213318 | node1 | changed: [node3] 2026-08-18 13:32:58.213546 | node1 | 2026-08-18 13:32:58.213814 | node1 | TASK [vexxhost.ceph.mon : Find this monitor's addresses on the public network] *** 2026-08-18 13:32:58.214111 | node1 | Tuesday 18 August 2026 13:32:58 +0000 (0:00:00.636) 0:01:53.163 ******** 2026-08-18 13:32:58.347175 | node1 | ok: [node1] 2026-08-18 13:32:58.347405 | node1 | ok: [node2] 2026-08-18 13:32:58.347643 | node1 | ok: [node3] 2026-08-18 13:32:58.347865 | node1 | 2026-08-18 13:32:58.348180 | node1 | TASK [vexxhost.ceph.mon : Assert exactly one address matches the public network] *** 2026-08-18 13:32:58.348485 | node1 | Tuesday 18 August 2026 13:32:58 +0000 (0:00:00.134) 0:01:53.298 ******** 2026-08-18 13:32:58.431124 | node1 | ok: [node1] => { 2026-08-18 13:32:58.431338 | node1 | "changed": false, 2026-08-18 13:32:58.431578 | node1 | "msg": "All assertions passed" 2026-08-18 13:32:58.431808 | node1 | } 2026-08-18 13:32:58.432083 | node1 | ok: [node2] => { 2026-08-18 13:32:58.432349 | node1 | "changed": false, 2026-08-18 13:32:58.432592 | node1 | "msg": "All assertions passed" 2026-08-18 13:32:58.432820 | node1 | } 2026-08-18 13:32:58.433092 | node1 | ok: [node3] => { 2026-08-18 13:32:58.433331 | node1 | "changed": false, 2026-08-18 13:32:58.433688 | node1 | "msg": "All assertions passed" 2026-08-18 13:32:58.433918 | node1 | } 2026-08-18 13:32:58.434171 | node1 | 2026-08-18 13:32:58.434425 | node1 | TASK [vexxhost.ceph.mon : Set this monitor's Ceph public IP] ******************* 2026-08-18 13:32:58.434678 | node1 | Tuesday 18 August 2026 13:32:58 +0000 (0:00:00.084) 0:01:53.382 ******** 2026-08-18 13:32:58.513079 | node1 | ok: [node1] 2026-08-18 13:32:58.513298 | node1 | ok: [node2] 2026-08-18 13:32:58.513534 | node1 | ok: [node3] 2026-08-18 13:32:58.513753 | node1 | 2026-08-18 13:32:58.514056 | node1 | TASK [vexxhost.ceph.mon : Check whether any monitor is already bootstrapped] *** 2026-08-18 13:32:58.514295 | node1 | Tuesday 18 August 2026 13:32:58 +0000 (0:00:00.081) 0:01:53.464 ******** 2026-08-18 13:32:59.214081 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:32:59.214158 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:32:59.214303 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:32:59.214516 | node1 | 2026-08-18 13:32:59.214780 | node1 | TASK [vexxhost.ceph.mon : Select the bootstrap node (existing one, else the first monitor)] *** 2026-08-18 13:32:59.215073 | node1 | Tuesday 18 August 2026 13:32:59 +0000 (0:00:00.700) 0:01:54.164 ******** 2026-08-18 13:32:59.305608 | node1 | ok: [node1] 2026-08-18 13:32:59.305825 | node1 | ok: [node2] 2026-08-18 13:32:59.306103 | node1 | ok: [node3] 2026-08-18 13:32:59.306311 | node1 | 2026-08-18 13:32:59.306559 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 13:32:59.306803 | node1 | Tuesday 18 August 2026 13:32:59 +0000 (0:00:00.091) 0:01:54.256 ******** 2026-08-18 13:32:59.465547 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/mon/tasks/bootstrap.yml for node1, node2, node3 2026-08-18 13:32:59.465765 | node1 | 2026-08-18 13:32:59.466074 | node1 | TASK [vexxhost.ceph.mon : Ensure /etc/ceph exists] ***************************** 2026-08-18 13:32:59.466309 | node1 | Tuesday 18 August 2026 13:32:59 +0000 (0:00:00.160) 0:01:54.416 ******** 2026-08-18 13:32:59.730280 | node1 | skipping: [node2] 2026-08-18 13:32:59.730491 | node1 | skipping: [node3] 2026-08-18 13:32:59.730733 | node1 | changed: [node1] 2026-08-18 13:32:59.730992 | node1 | 2026-08-18 13:32:59.731263 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH private key] *********** 2026-08-18 13:32:59.731508 | node1 | Tuesday 18 August 2026 13:32:59 +0000 (0:00:00.264) 0:01:54.681 ******** 2026-08-18 13:32:59.817354 | node1 | skipping: [node1] 2026-08-18 13:32:59.817495 | node1 | skipping: [node2] 2026-08-18 13:32:59.817742 | node1 | skipping: [node3] 2026-08-18 13:32:59.817996 | node1 | 2026-08-18 13:32:59.818258 | node1 | TASK [vexxhost.ceph.mon : Write the caller-provided SSH public key] ************ 2026-08-18 13:32:59.818504 | node1 | Tuesday 18 August 2026 13:32:59 +0000 (0:00:00.087) 0:01:54.768 ******** 2026-08-18 13:32:59.962510 | node1 | skipping: [node1] 2026-08-18 13:32:59.962758 | node1 | skipping: [node2] 2026-08-18 13:32:59.963120 | node1 | skipping: [node3] 2026-08-18 13:32:59.963431 | node1 | 2026-08-18 13:32:59.963731 | node1 | TASK [vexxhost.ceph.mon : Generate a cephadm SSH keypair when none was provided] *** 2026-08-18 13:32:59.964050 | node1 | Tuesday 18 August 2026 13:32:59 +0000 (0:00:00.144) 0:01:54.913 ******** 2026-08-18 13:33:01.163969 | node1 | skipping: [node2] 2026-08-18 13:33:01.164253 | node1 | skipping: [node3] 2026-08-18 13:33:01.164533 | node1 | changed: [node1] 2026-08-18 13:33:01.164783 | node1 | 2026-08-18 13:33:01.165139 | node1 | TASK [vexxhost.ceph.mon : Create a temporary ceph.conf for bootstrap overrides] *** 2026-08-18 13:33:01.165429 | node1 | Tuesday 18 August 2026 13:33:01 +0000 (0:00:01.201) 0:01:56.115 ******** 2026-08-18 13:33:01.499225 | node1 | skipping: [node2] 2026-08-18 13:33:01.499420 | node1 | skipping: [node3] 2026-08-18 13:33:01.499680 | node1 | changed: [node1] 2026-08-18 13:33:01.499915 | node1 | 2026-08-18 13:33:01.500263 | node1 | TASK [vexxhost.ceph.mon : Populate bootstrap ceph.conf overrides] ************** 2026-08-18 13:33:01.500553 | node1 | Tuesday 18 August 2026 13:33:01 +0000 (0:00:00.335) 0:01:56.450 ******** 2026-08-18 13:33:01.536906 | node1 | skipping: [node1] 2026-08-18 13:33:01.571596 | node1 | skipping: [node2] 2026-08-18 13:33:01.571810 | node1 | skipping: [node3] 2026-08-18 13:33:01.572081 | node1 | 2026-08-18 13:33:01.572324 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 13:33:01.572571 | node1 | Tuesday 18 August 2026 13:33:01 +0000 (0:00:00.072) 0:01:56.523 ******** 2026-08-18 13:33:49.470714 | node1 | skipping: [node2] 2026-08-18 13:33:49.470793 | node1 | skipping: [node3] 2026-08-18 13:33:49.470870 | node1 | changed: [node1] 2026-08-18 13:33:49.471004 | node1 | 2026-08-18 13:33:49.472015 | node1 | TASK [vexxhost.ceph.mon : Remove the temporary ceph.conf] ********************** 2026-08-18 13:33:49.472079 | node1 | Tuesday 18 August 2026 13:33:49 +0000 (0:00:47.898) 0:02:44.421 ******** 2026-08-18 13:33:49.734129 | node1 | skipping: [node2] 2026-08-18 13:33:49.734220 | node1 | skipping: [node3] 2026-08-18 13:33:49.734233 | node1 | changed: [node1] 2026-08-18 13:33:49.734245 | node1 | 2026-08-18 13:33:49.734257 | node1 | TASK [Register this monitor host] ********************************************** 2026-08-18 13:33:49.734270 | node1 | Tuesday 18 August 2026 13:33:49 +0000 (0:00:00.262) 0:02:44.683 ******** 2026-08-18 13:33:49.882550 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:33:49.882626 | node1 | 2026-08-18 13:33:49.882640 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:33:49.882652 | node1 | Tuesday 18 August 2026 13:33:49 +0000 (0:00:00.149) 0:02:44.832 ******** 2026-08-18 13:33:51.733106 | node1 | ok: [node1] 2026-08-18 13:33:51.733172 | node1 | 2026-08-18 13:33:51.733186 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:33:51.733206 | node1 | Tuesday 18 August 2026 13:33:51 +0000 (0:00:01.839) 0:02:46.672 ******** 2026-08-18 13:33:51.808334 | node1 | ok: [node1] 2026-08-18 13:33:51.808434 | node1 | ok: [node2] 2026-08-18 13:33:51.808579 | node1 | ok: [node3] 2026-08-18 13:33:51.808745 | node1 | 2026-08-18 13:33:51.809069 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:33:51.809324 | node1 | Tuesday 18 August 2026 13:33:51 +0000 (0:00:00.087) 0:02:46.759 ******** 2026-08-18 13:33:51.895419 | node1 | ok: [node1] => { 2026-08-18 13:33:51.895646 | node1 | "changed": false, 2026-08-18 13:33:51.895887 | node1 | "msg": "All assertions passed" 2026-08-18 13:33:51.896167 | node1 | } 2026-08-18 13:33:51.896404 | node1 | ok: [node2] => { 2026-08-18 13:33:51.896722 | node1 | "changed": false, 2026-08-18 13:33:51.896999 | node1 | "msg": "All assertions passed" 2026-08-18 13:33:51.897238 | node1 | } 2026-08-18 13:33:51.897488 | node1 | ok: [node3] => { 2026-08-18 13:33:51.897743 | node1 | "changed": false, 2026-08-18 13:33:51.898949 | node1 | "msg": "All assertions passed" 2026-08-18 13:33:51.898961 | node1 | } 2026-08-18 13:33:51.898966 | node1 | 2026-08-18 13:33:51.898970 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:33:51.898998 | node1 | Tuesday 18 August 2026 13:33:51 +0000 (0:00:00.086) 0:02:46.846 ******** 2026-08-18 13:33:52.349057 | node1 | ok: [node1] 2026-08-18 13:33:52.349141 | node1 | changed: [node3] 2026-08-18 13:33:52.349151 | node1 | changed: [node2] 2026-08-18 13:33:52.349287 | node1 | 2026-08-18 13:33:52.349578 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:33:52.349850 | node1 | Tuesday 18 August 2026 13:33:52 +0000 (0:00:00.452) 0:02:47.299 ******** 2026-08-18 13:33:52.430258 | node1 | ok: [node1] 2026-08-18 13:33:52.430441 | node1 | ok: [node2] 2026-08-18 13:33:52.430662 | node1 | ok: [node3] 2026-08-18 13:33:52.430987 | node1 | 2026-08-18 13:33:52.431301 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:33:52.431580 | node1 | Tuesday 18 August 2026 13:33:52 +0000 (0:00:00.081) 0:02:47.381 ******** 2026-08-18 13:33:52.546467 | node1 | ok: [node1] 2026-08-18 13:33:52.546643 | node1 | ok: [node2] 2026-08-18 13:33:52.546845 | node1 | ok: [node3] 2026-08-18 13:33:52.547128 | node1 | 2026-08-18 13:33:52.547383 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:33:52.547629 | node1 | Tuesday 18 August 2026 13:33:52 +0000 (0:00:00.115) 0:02:47.497 ******** 2026-08-18 13:33:52.636424 | node1 | ok: [node1] => { 2026-08-18 13:33:52.636653 | node1 | "changed": false, 2026-08-18 13:33:52.636916 | node1 | "msg": "All assertions passed" 2026-08-18 13:33:52.637184 | node1 | } 2026-08-18 13:33:52.637427 | node1 | ok: [node2] => { 2026-08-18 13:33:52.637663 | node1 | "changed": false, 2026-08-18 13:33:52.637904 | node1 | "msg": "All assertions passed" 2026-08-18 13:33:52.638181 | node1 | } 2026-08-18 13:33:52.638411 | node1 | ok: [node3] => { 2026-08-18 13:33:52.638646 | node1 | "changed": false, 2026-08-18 13:33:52.638883 | node1 | "msg": "All assertions passed" 2026-08-18 13:33:52.639197 | node1 | } 2026-08-18 13:33:52.639418 | node1 | 2026-08-18 13:33:52.639675 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:33:52.639959 | node1 | Tuesday 18 August 2026 13:33:52 +0000 (0:00:00.090) 0:02:47.587 ******** 2026-08-18 13:34:15.111055 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:15.111358 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:15.111667 | node1 | changed: [node1] 2026-08-18 13:34:15.112037 | node1 | 2026-08-18 13:34:15.112394 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:34:15.112701 | node1 | Tuesday 18 August 2026 13:34:15 +0000 (0:00:22.474) 0:03:10.061 ******** 2026-08-18 13:34:15.225277 | node1 | skipping: [node1] 2026-08-18 13:34:15.225498 | node1 | skipping: [node2] 2026-08-18 13:34:15.225754 | node1 | skipping: [node3] 2026-08-18 13:34:15.226009 | node1 | 2026-08-18 13:34:15.226258 | node1 | TASK [vexxhost.ceph.mon : Apply the monitor service spec] ********************** 2026-08-18 13:34:15.226504 | node1 | Tuesday 18 August 2026 13:34:15 +0000 (0:00:00.114) 0:03:10.176 ******** 2026-08-18 13:34:19.245165 | node1 | changed: [node1] 2026-08-18 13:34:19.245270 | node1 | 2026-08-18 13:34:19.245284 | node1 | TASK [vexxhost.ceph.mon : Wait for this monitor's daemon to be present] ******** 2026-08-18 13:34:19.245305 | node1 | Tuesday 18 August 2026 13:34:19 +0000 (0:00:04.019) 0:03:14.195 ******** 2026-08-18 13:34:27.332764 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:27.332860 | node1 | ok: [node1] 2026-08-18 13:34:27.332884 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:27.333212 | node1 | 2026-08-18 13:34:27.333315 | node1 | TASK [vexxhost.ceph.mgr : Locate the cluster admin host] *********************** 2026-08-18 13:34:27.333432 | node1 | Tuesday 18 August 2026 13:34:27 +0000 (0:00:08.086) 0:03:22.282 ******** 2026-08-18 13:34:28.099153 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:34:28.099301 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:34:28.099312 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:34:28.099321 | node1 | 2026-08-18 13:34:28.099330 | node1 | TASK [vexxhost.ceph.mgr : Select the admin host] ******************************* 2026-08-18 13:34:28.099340 | node1 | Tuesday 18 August 2026 13:34:28 +0000 (0:00:00.765) 0:03:23.047 ******** 2026-08-18 13:34:28.192091 | node1 | ok: [node1] 2026-08-18 13:34:28.192182 | node1 | ok: [node2] 2026-08-18 13:34:28.192192 | node1 | ok: [node3] 2026-08-18 13:34:28.192213 | node1 | 2026-08-18 13:34:28.192705 | node1 | TASK [Register this manager host] ********************************************** 2026-08-18 13:34:28.192770 | node1 | Tuesday 18 August 2026 13:34:28 +0000 (0:00:00.094) 0:03:23.141 ******** 2026-08-18 13:34:28.340826 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:34:28.340912 | node1 | 2026-08-18 13:34:28.340923 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:34:28.340995 | node1 | Tuesday 18 August 2026 13:34:28 +0000 (0:00:00.148) 0:03:23.290 ******** 2026-08-18 13:34:30.397716 | node1 | ok: [node1] 2026-08-18 13:34:30.397798 | node1 | 2026-08-18 13:34:30.397813 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:34:30.397826 | node1 | Tuesday 18 August 2026 13:34:30 +0000 (0:00:02.056) 0:03:25.346 ******** 2026-08-18 13:34:30.490013 | node1 | ok: [node1] 2026-08-18 13:34:30.490083 | node1 | ok: [node2] 2026-08-18 13:34:30.490096 | node1 | ok: [node3] 2026-08-18 13:34:30.490108 | node1 | 2026-08-18 13:34:30.490120 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:34:30.490132 | node1 | Tuesday 18 August 2026 13:34:30 +0000 (0:00:00.092) 0:03:25.439 ******** 2026-08-18 13:34:30.589148 | node1 | ok: [node1] => { 2026-08-18 13:34:30.589209 | node1 | "changed": false, 2026-08-18 13:34:30.589230 | node1 | "msg": "All assertions passed" 2026-08-18 13:34:30.589395 | node1 | } 2026-08-18 13:34:30.589631 | node1 | ok: [node2] => { 2026-08-18 13:34:30.589880 | node1 | "changed": false, 2026-08-18 13:34:30.590232 | node1 | "msg": "All assertions passed" 2026-08-18 13:34:30.590400 | node1 | } 2026-08-18 13:34:30.590633 | node1 | ok: [node3] => { 2026-08-18 13:34:30.590911 | node1 | "changed": false, 2026-08-18 13:34:30.591205 | node1 | "msg": "All assertions passed" 2026-08-18 13:34:30.591421 | node1 | } 2026-08-18 13:34:30.591642 | node1 | 2026-08-18 13:34:30.591905 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:34:30.592259 | node1 | Tuesday 18 August 2026 13:34:30 +0000 (0:00:00.099) 0:03:25.539 ******** 2026-08-18 13:34:31.000062 | node1 | ok: [node1] 2026-08-18 13:34:31.000288 | node1 | ok: [node2] 2026-08-18 13:34:31.000523 | node1 | ok: [node3] 2026-08-18 13:34:31.000748 | node1 | 2026-08-18 13:34:31.001103 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:34:31.001429 | node1 | Tuesday 18 August 2026 13:34:30 +0000 (0:00:00.411) 0:03:25.950 ******** 2026-08-18 13:34:31.103109 | node1 | ok: [node1] 2026-08-18 13:34:31.103175 | node1 | ok: [node2] 2026-08-18 13:34:31.103188 | node1 | ok: [node3] 2026-08-18 13:34:31.103206 | node1 | 2026-08-18 13:34:31.103286 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:34:31.103440 | node1 | Tuesday 18 August 2026 13:34:31 +0000 (0:00:00.102) 0:03:26.053 ******** 2026-08-18 13:34:31.233794 | node1 | ok: [node1] 2026-08-18 13:34:31.234084 | node1 | ok: [node2] 2026-08-18 13:34:31.234336 | node1 | ok: [node3] 2026-08-18 13:34:31.234567 | node1 | 2026-08-18 13:34:31.234964 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:34:31.235235 | node1 | Tuesday 18 August 2026 13:34:31 +0000 (0:00:00.130) 0:03:26.184 ******** 2026-08-18 13:34:31.331556 | node1 | ok: [node1] => { 2026-08-18 13:34:31.331631 | node1 | "changed": false, 2026-08-18 13:34:31.331816 | node1 | "msg": "All assertions passed" 2026-08-18 13:34:31.332007 | node1 | } 2026-08-18 13:34:31.332159 | node1 | ok: [node2] => { 2026-08-18 13:34:31.332313 | node1 | "changed": false, 2026-08-18 13:34:31.332465 | node1 | "msg": "All assertions passed" 2026-08-18 13:34:31.332605 | node1 | } 2026-08-18 13:34:31.332757 | node1 | ok: [node3] => { 2026-08-18 13:34:31.332953 | node1 | "changed": false, 2026-08-18 13:34:31.333108 | node1 | "msg": "All assertions passed" 2026-08-18 13:34:31.333331 | node1 | } 2026-08-18 13:34:31.333519 | node1 | 2026-08-18 13:34:31.333713 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:34:31.333994 | node1 | Tuesday 18 August 2026 13:34:31 +0000 (0:00:00.097) 0:03:26.282 ******** 2026-08-18 13:34:33.297987 | node1 | ok: [node1] 2026-08-18 13:34:33.298347 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:33.298710 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:33.299103 | node1 | 2026-08-18 13:34:33.299551 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:34:33.299898 | node1 | Tuesday 18 August 2026 13:34:33 +0000 (0:00:01.966) 0:03:28.248 ******** 2026-08-18 13:34:33.426343 | node1 | skipping: [node1] 2026-08-18 13:34:33.426529 | node1 | skipping: [node2] 2026-08-18 13:34:33.426767 | node1 | skipping: [node3] 2026-08-18 13:34:33.427081 | node1 | 2026-08-18 13:34:33.427332 | node1 | TASK [vexxhost.ceph.mgr : Apply the manager service spec] ********************** 2026-08-18 13:34:33.427676 | node1 | Tuesday 18 August 2026 13:34:33 +0000 (0:00:00.128) 0:03:28.377 ******** 2026-08-18 13:34:41.385124 | node1 | changed: [node1] 2026-08-18 13:34:41.385344 | node1 | 2026-08-18 13:34:41.385627 | node1 | TASK [vexxhost.ceph.mgr : Wait for this manager's daemon to be present] ******** 2026-08-18 13:34:41.385879 | node1 | Tuesday 18 August 2026 13:34:41 +0000 (0:00:07.958) 0:03:36.335 ******** 2026-08-18 13:34:43.626237 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:43.626337 | node1 | ok: [node1] 2026-08-18 13:34:43.626461 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:34:43.626500 | node1 | 2026-08-18 13:34:43.626743 | node1 | TASK [vexxhost.ceph.mgr : Configure ceph-mgr modules] ************************** 2026-08-18 13:34:43.627211 | node1 | Tuesday 18 August 2026 13:34:43 +0000 (0:00:02.239) 0:03:38.575 ******** 2026-08-18 13:34:56.188627 | node1 | changed: [node1] => (item={'name': 'diskprediction_local'}) 2026-08-18 13:34:56.188733 | node1 | changed: [node1] => (item={'name': 'dashboard'}) 2026-08-18 13:34:56.188751 | node1 | ok: [node1] => (item={'name': 'iostat'}) 2026-08-18 13:34:56.188882 | node1 | changed: [node1] => (item={'name': 'prometheus'}) 2026-08-18 13:34:56.189618 | node1 | changed: [node1] => (item={'name': 'rgw'}) 2026-08-18 13:34:56.189674 | node1 | 2026-08-18 13:34:56.189682 | node1 | PLAY [Deploy Ceph OSDs] ******************************************************** 2026-08-18 13:34:56.189688 | node1 | 2026-08-18 13:34:56.189694 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:34:56.189700 | node1 | Tuesday 18 August 2026 13:34:56 +0000 (0:00:12.565) 0:03:51.140 ******** 2026-08-18 13:34:57.331516 | node1 | ok: [node1] 2026-08-18 13:34:57.331610 | node1 | ok: [node2] 2026-08-18 13:34:57.331627 | node1 | ok: [node3] 2026-08-18 13:34:57.331752 | node1 | 2026-08-18 13:34:57.331861 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:34:57.331995 | node1 | Tuesday 18 August 2026 13:34:57 +0000 (0:00:01.139) 0:03:52.279 ******** 2026-08-18 13:34:57.636582 | node1 | ok: [node1] 2026-08-18 13:34:57.636623 | node1 | ok: [node2] 2026-08-18 13:34:57.636629 | node1 | ok: [node3] 2026-08-18 13:34:57.636634 | node1 | 2026-08-18 13:34:57.636640 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:34:57.636645 | node1 | Tuesday 18 August 2026 13:34:57 +0000 (0:00:00.307) 0:03:52.587 ******** 2026-08-18 13:34:57.736316 | node1 | skipping: [node1] 2026-08-18 13:34:57.736499 | node1 | skipping: [node2] 2026-08-18 13:34:57.739423 | node1 | skipping: [node3] 2026-08-18 13:34:57.739437 | node1 | 2026-08-18 13:34:57.739443 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:34:57.739462 | node1 | Tuesday 18 August 2026 13:34:57 +0000 (0:00:00.100) 0:03:52.687 ******** 2026-08-18 13:34:58.073344 | node1 | ok: [node1] 2026-08-18 13:34:58.073404 | node1 | ok: [node2] 2026-08-18 13:34:58.073526 | node1 | ok: [node3] 2026-08-18 13:34:58.073616 | node1 | 2026-08-18 13:34:58.073752 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:34:58.073956 | node1 | Tuesday 18 August 2026 13:34:58 +0000 (0:00:00.336) 0:03:53.024 ******** 2026-08-18 13:34:58.179233 | node1 | ok: [node1] => { 2026-08-18 13:34:58.182258 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:34:58.182272 | node1 | } 2026-08-18 13:34:58.182277 | node1 | ok: [node2] => { 2026-08-18 13:34:58.182281 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:34:58.182284 | node1 | } 2026-08-18 13:34:58.182288 | node1 | ok: [node3] => { 2026-08-18 13:34:58.182292 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:34:58.182295 | node1 | } 2026-08-18 13:34:58.182299 | node1 | 2026-08-18 13:34:58.182304 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:34:58.182308 | node1 | Tuesday 18 August 2026 13:34:58 +0000 (0:00:00.105) 0:03:53.130 ******** 2026-08-18 13:34:58.621506 | node1 | ok: [node2] 2026-08-18 13:34:58.621594 | node1 | ok: [node1] 2026-08-18 13:34:58.621707 | node1 | ok: [node3] 2026-08-18 13:34:58.621810 | node1 | 2026-08-18 13:34:58.621950 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:34:58.622067 | node1 | Tuesday 18 August 2026 13:34:58 +0000 (0:00:00.441) 0:03:53.572 ******** 2026-08-18 13:34:58.731474 | node1 | skipping: [node1] 2026-08-18 13:34:58.731584 | node1 | skipping: [node2] 2026-08-18 13:34:58.731692 | node1 | skipping: [node3] 2026-08-18 13:34:58.731798 | node1 | 2026-08-18 13:34:58.731914 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:34:58.732049 | node1 | Tuesday 18 August 2026 13:34:58 +0000 (0:00:00.109) 0:03:53.681 ******** 2026-08-18 13:34:58.853981 | node1 | skipping: [node1] 2026-08-18 13:34:58.854014 | node1 | skipping: [node2] 2026-08-18 13:34:58.854023 | node1 | skipping: [node3] 2026-08-18 13:34:58.854032 | node1 | 2026-08-18 13:34:58.854041 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:34:58.854050 | node1 | Tuesday 18 August 2026 13:34:58 +0000 (0:00:00.116) 0:03:53.798 ******** 2026-08-18 13:34:59.179147 | node1 | ok: [node1] 2026-08-18 13:34:59.179207 | node1 | ok: [node2] 2026-08-18 13:34:59.179327 | node1 | ok: [node3] 2026-08-18 13:34:59.179427 | node1 | 2026-08-18 13:34:59.179545 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:34:59.179650 | node1 | Tuesday 18 August 2026 13:34:59 +0000 (0:00:00.331) 0:03:54.130 ******** 2026-08-18 13:35:00.418644 | node1 | ok: [node3] 2026-08-18 13:35:00.418725 | node1 | ok: [node2] 2026-08-18 13:35:00.418738 | node1 | ok: [node1] 2026-08-18 13:35:00.418866 | node1 | 2026-08-18 13:35:00.419009 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:35:00.419161 | node1 | Tuesday 18 August 2026 13:35:00 +0000 (0:00:01.239) 0:03:55.369 ******** 2026-08-18 13:35:00.534992 | node1 | ok: [node1] => { 2026-08-18 13:35:00.535307 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:35:00.535316 | node1 | } 2026-08-18 13:35:00.535323 | node1 | ok: [node2] => { 2026-08-18 13:35:00.535330 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:35:00.535336 | node1 | } 2026-08-18 13:35:00.535343 | node1 | ok: [node3] => { 2026-08-18 13:35:00.535349 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:35:00.535356 | node1 | } 2026-08-18 13:35:00.535363 | node1 | 2026-08-18 13:35:00.535370 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:35:00.535610 | node1 | Tuesday 18 August 2026 13:35:00 +0000 (0:00:00.113) 0:03:55.483 ******** 2026-08-18 13:35:00.997205 | node1 | ok: [node1] 2026-08-18 13:35:00.997601 | node1 | ok: [node2] 2026-08-18 13:35:00.997908 | node1 | ok: [node3] 2026-08-18 13:35:00.998235 | node1 | 2026-08-18 13:35:00.998558 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:35:00.999280 | node1 | Tuesday 18 August 2026 13:35:00 +0000 (0:00:00.464) 0:03:55.947 ******** 2026-08-18 13:35:03.281716 | node1 | ok: [node1] 2026-08-18 13:35:03.282145 | node1 | ok: [node3] 2026-08-18 13:35:03.282455 | node1 | ok: [node2] 2026-08-18 13:35:03.282743 | node1 | 2026-08-18 13:35:03.283104 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:35:03.283420 | node1 | Tuesday 18 August 2026 13:35:03 +0000 (0:00:02.285) 0:03:58.232 ******** 2026-08-18 13:35:03.353009 | node1 | skipping: [node1] 2026-08-18 13:35:03.353390 | node1 | skipping: [node2] 2026-08-18 13:35:03.353721 | node1 | skipping: [node3] 2026-08-18 13:35:03.354067 | node1 | 2026-08-18 13:35:03.354413 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:35:03.354747 | node1 | Tuesday 18 August 2026 13:35:03 +0000 (0:00:00.071) 0:03:58.303 ******** 2026-08-18 13:35:03.427991 | node1 | skipping: [node1] 2026-08-18 13:35:03.428342 | node1 | skipping: [node2] 2026-08-18 13:35:03.428682 | node1 | skipping: [node3] 2026-08-18 13:35:03.429009 | node1 | 2026-08-18 13:35:03.429334 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:35:03.429665 | node1 | Tuesday 18 August 2026 13:35:03 +0000 (0:00:00.075) 0:03:58.379 ******** 2026-08-18 13:35:03.504617 | node1 | skipping: [node1] 2026-08-18 13:35:03.505080 | node1 | skipping: [node2] 2026-08-18 13:35:03.505420 | node1 | skipping: [node3] 2026-08-18 13:35:03.505728 | node1 | 2026-08-18 13:35:03.506104 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:35:03.506426 | node1 | Tuesday 18 August 2026 13:35:03 +0000 (0:00:00.076) 0:03:58.455 ******** 2026-08-18 13:35:04.778808 | node1 | ok: [node3] 2026-08-18 13:35:04.780351 | node1 | ok: [node1] 2026-08-18 13:35:04.780383 | node1 | ok: [node2] 2026-08-18 13:35:04.780397 | node1 | 2026-08-18 13:35:04.780410 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:35:04.780423 | node1 | Tuesday 18 August 2026 13:35:04 +0000 (0:00:01.273) 0:03:59.729 ******** 2026-08-18 13:35:05.373441 | node1 | ok: [node1] 2026-08-18 13:35:05.373505 | node1 | ok: [node3] 2026-08-18 13:35:05.373518 | node1 | ok: [node2] 2026-08-18 13:35:05.373530 | node1 | 2026-08-18 13:35:05.373543 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:35:05.373555 | node1 | Tuesday 18 August 2026 13:35:05 +0000 (0:00:00.593) 0:04:00.322 ******** 2026-08-18 13:35:06.618005 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:35:06.618055 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:35:06.618061 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:35:06.618255 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:35:06.618547 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:35:06.618792 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:35:06.619076 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:35:06.619327 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:35:06.619577 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:35:06.619869 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:35:06.620170 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:35:06.620427 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:35:06.620684 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:35:06.620957 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:35:06.621239 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:35:06.621464 | node1 | 2026-08-18 13:35:06.621713 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:35:06.622005 | node1 | Tuesday 18 August 2026 13:35:06 +0000 (0:00:01.245) 0:04:01.568 ******** 2026-08-18 13:35:07.379595 | node1 | ok: [node1] 2026-08-18 13:35:07.379883 | node1 | ok: [node2] 2026-08-18 13:35:07.380269 | node1 | ok: [node3] 2026-08-18 13:35:07.380529 | node1 | 2026-08-18 13:35:07.380833 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:35:07.381155 | node1 | Tuesday 18 August 2026 13:35:07 +0000 (0:00:00.743) 0:04:02.312 ******** 2026-08-18 13:35:07.381398 | node1 | 2026-08-18 13:35:07.381664 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:35:07.382024 | node1 | Tuesday 18 August 2026 13:35:07 +0000 (0:00:00.006) 0:04:02.318 ******** 2026-08-18 13:35:07.382174 | node1 | 2026-08-18 13:35:07.382375 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:35:07.382529 | node1 | Tuesday 18 August 2026 13:35:07 +0000 (0:00:00.006) 0:04:02.324 ******** 2026-08-18 13:35:07.382752 | node1 | 2026-08-18 13:35:07.382919 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:35:07.383097 | node1 | Tuesday 18 August 2026 13:35:07 +0000 (0:00:00.006) 0:04:02.330 ******** 2026-08-18 13:35:07.876976 | node1 | ok: [node3] 2026-08-18 13:35:07.877023 | node1 | ok: [node1] 2026-08-18 13:35:07.877031 | node1 | ok: [node2] 2026-08-18 13:35:07.877254 | node1 | 2026-08-18 13:35:07.877520 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:35:07.877787 | node1 | Tuesday 18 August 2026 13:35:07 +0000 (0:00:00.496) 0:04:02.827 ******** 2026-08-18 13:35:08.205078 | node1 | ok: [node1] 2026-08-18 13:35:08.205311 | node1 | ok: [node2] 2026-08-18 13:35:08.205567 | node1 | ok: [node3] 2026-08-18 13:35:08.205785 | node1 | 2026-08-18 13:35:08.206073 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:35:08.206322 | node1 | Tuesday 18 August 2026 13:35:08 +0000 (0:00:00.328) 0:04:03.156 ******** 2026-08-18 13:35:08.302078 | node1 | ok: [node1] => { 2026-08-18 13:35:08.302324 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:35:08.302549 | node1 | } 2026-08-18 13:35:08.302787 | node1 | ok: [node2] => { 2026-08-18 13:35:08.303094 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:35:08.309843 | node1 | } 2026-08-18 13:35:08.309869 | node1 | ok: [node3] => { 2026-08-18 13:35:08.309878 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:35:08.309886 | node1 | } 2026-08-18 13:35:08.309895 | node1 | 2026-08-18 13:35:08.309904 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:35:08.309913 | node1 | Tuesday 18 August 2026 13:35:08 +0000 (0:00:00.097) 0:04:03.253 ******** 2026-08-18 13:35:08.821609 | node1 | ok: [node1] 2026-08-18 13:35:08.821792 | node1 | ok: [node2] 2026-08-18 13:35:08.822074 | node1 | ok: [node3] 2026-08-18 13:35:08.822311 | node1 | 2026-08-18 13:35:08.822598 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:35:08.822913 | node1 | Tuesday 18 August 2026 13:35:08 +0000 (0:00:00.519) 0:04:03.772 ******** 2026-08-18 13:35:12.237574 | node1 | ok: [node1] 2026-08-18 13:35:12.237699 | node1 | ok: [node3] 2026-08-18 13:35:12.237718 | node1 | ok: [node2] 2026-08-18 13:35:12.237801 | node1 | 2026-08-18 13:35:12.237984 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:35:12.238042 | node1 | Tuesday 18 August 2026 13:35:12 +0000 (0:00:03.416) 0:04:07.188 ******** 2026-08-18 13:35:13.337454 | node1 | ok: [node3] 2026-08-18 13:35:13.337656 | node1 | ok: [node1] 2026-08-18 13:35:13.337981 | node1 | ok: [node2] 2026-08-18 13:35:13.338225 | node1 | 2026-08-18 13:35:13.338478 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:35:13.338728 | node1 | Tuesday 18 August 2026 13:35:13 +0000 (0:00:01.099) 0:04:08.287 ******** 2026-08-18 13:35:13.618647 | node1 | ok: [node1] 2026-08-18 13:35:13.618858 | node1 | ok: [node2] 2026-08-18 13:35:13.619131 | node1 | ok: [node3] 2026-08-18 13:35:13.619347 | node1 | 2026-08-18 13:35:13.619614 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:35:13.619883 | node1 | Tuesday 18 August 2026 13:35:13 +0000 (0:00:00.281) 0:04:08.569 ******** 2026-08-18 13:35:14.315870 | node1 | ok: [node1] 2026-08-18 13:35:14.316051 | node1 | ok: [node2] 2026-08-18 13:35:14.316222 | node1 | ok: [node3] 2026-08-18 13:35:14.316387 | node1 | 2026-08-18 13:35:14.316569 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:35:14.316741 | node1 | Tuesday 18 August 2026 13:35:14 +0000 (0:00:00.696) 0:04:09.266 ******** 2026-08-18 13:35:15.089017 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:35:15.089237 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:35:15.089475 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:35:15.089760 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:35:15.090043 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:35:15.090292 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:35:15.090535 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:35:15.090855 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:35:15.091125 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:35:15.091347 | node1 | 2026-08-18 13:35:15.091594 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:35:15.091867 | node1 | Tuesday 18 August 2026 13:35:15 +0000 (0:00:00.773) 0:04:10.040 ******** 2026-08-18 13:35:15.689386 | node1 | ok: [node1] 2026-08-18 13:35:15.689516 | node1 | ok: [node2] 2026-08-18 13:35:15.689667 | node1 | ok: [node3] 2026-08-18 13:35:15.689810 | node1 | 2026-08-18 13:35:15.689993 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:35:15.690160 | node1 | Tuesday 18 August 2026 13:35:15 +0000 (0:00:00.600) 0:04:10.640 ******** 2026-08-18 13:35:16.307595 | node1 | ok: [node1] 2026-08-18 13:35:16.308951 | node1 | ok: [node3] 2026-08-18 13:35:16.308966 | node1 | ok: [node2] 2026-08-18 13:35:16.308972 | node1 | 2026-08-18 13:35:16.308978 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:35:16.308984 | node1 | Tuesday 18 August 2026 13:35:16 +0000 (0:00:00.595) 0:04:11.236 ******** 2026-08-18 13:35:16.308989 | node1 | 2026-08-18 13:35:16.308994 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:35:16.309172 | node1 | Tuesday 18 August 2026 13:35:16 +0000 (0:00:00.009) 0:04:11.245 ******** 2026-08-18 13:35:16.309395 | node1 | 2026-08-18 13:35:16.309644 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:35:16.309889 | node1 | Tuesday 18 August 2026 13:35:16 +0000 (0:00:00.006) 0:04:11.252 ******** 2026-08-18 13:35:16.310145 | node1 | 2026-08-18 13:35:16.310406 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:35:16.310646 | node1 | Tuesday 18 August 2026 13:35:16 +0000 (0:00:00.006) 0:04:11.258 ******** 2026-08-18 13:35:16.857012 | node1 | ok: [node1] 2026-08-18 13:35:16.865432 | node1 | ok: [node2] 2026-08-18 13:35:16.865470 | node1 | ok: [node3] 2026-08-18 13:35:16.865483 | node1 | 2026-08-18 13:35:16.865495 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:35:16.865508 | node1 | Tuesday 18 August 2026 13:35:16 +0000 (0:00:00.549) 0:04:11.808 ******** 2026-08-18 13:35:16.938306 | node1 | ok: [node1] => { 2026-08-18 13:35:16.938434 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:35:16.938584 | node1 | } 2026-08-18 13:35:16.938740 | node1 | ok: [node2] => { 2026-08-18 13:35:16.938898 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:35:16.939062 | node1 | } 2026-08-18 13:35:16.939214 | node1 | ok: [node3] => { 2026-08-18 13:35:16.939371 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:35:16.939516 | node1 | } 2026-08-18 13:35:16.939658 | node1 | 2026-08-18 13:35:16.939858 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:35:16.940051 | node1 | Tuesday 18 August 2026 13:35:16 +0000 (0:00:00.081) 0:04:11.889 ******** 2026-08-18 13:35:17.325613 | node1 | ok: [node1] 2026-08-18 13:35:17.325646 | node1 | ok: [node2] 2026-08-18 13:35:17.325657 | node1 | ok: [node3] 2026-08-18 13:35:17.325970 | node1 | 2026-08-18 13:35:17.327023 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:35:17.327035 | node1 | Tuesday 18 August 2026 13:35:17 +0000 (0:00:00.386) 0:04:12.275 ******** 2026-08-18 13:35:17.416037 | node1 | skipping: [node1] 2026-08-18 13:35:17.417178 | node1 | skipping: [node2] 2026-08-18 13:35:17.417230 | node1 | skipping: [node3] 2026-08-18 13:35:17.417244 | node1 | 2026-08-18 13:35:17.417257 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:35:17.417270 | node1 | Tuesday 18 August 2026 13:35:17 +0000 (0:00:00.090) 0:04:12.366 ******** 2026-08-18 13:35:17.516545 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:35:17.517154 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:35:17.517197 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:35:17.517203 | node1 | 2026-08-18 13:35:17.517209 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:35:17.517215 | node1 | Tuesday 18 August 2026 13:35:17 +0000 (0:00:00.099) 0:04:12.466 ******** 2026-08-18 13:35:18.602046 | node1 | ok: [node2] 2026-08-18 13:35:18.602104 | node1 | ok: [node3] 2026-08-18 13:35:18.602112 | node1 | ok: [node1] 2026-08-18 13:35:18.602119 | node1 | 2026-08-18 13:35:18.602127 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:35:18.602134 | node1 | Tuesday 18 August 2026 13:35:18 +0000 (0:00:01.085) 0:04:13.552 ******** 2026-08-18 13:35:19.460078 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:35:19.460154 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:35:19.460167 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:35:19.460184 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:35:19.460416 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:35:19.460650 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:35:19.460871 | node1 | 2026-08-18 13:35:19.461234 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:35:19.461426 | node1 | Tuesday 18 August 2026 13:35:19 +0000 (0:00:00.858) 0:04:14.410 ******** 2026-08-18 13:35:19.755137 | node1 | ok: [node1] 2026-08-18 13:35:19.755202 | node1 | ok: [node2] 2026-08-18 13:35:19.755214 | node1 | ok: [node3] 2026-08-18 13:35:19.755227 | node1 | 2026-08-18 13:35:19.755239 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:35:19.755258 | node1 | Tuesday 18 August 2026 13:35:19 +0000 (0:00:00.295) 0:04:14.705 ******** 2026-08-18 13:35:20.113586 | node1 | ok: [node1] 2026-08-18 13:35:20.113649 | node1 | ok: [node2] 2026-08-18 13:35:20.113666 | node1 | ok: [node3] 2026-08-18 13:35:20.113681 | node1 | 2026-08-18 13:35:20.113697 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:35:20.113712 | node1 | Tuesday 18 August 2026 13:35:20 +0000 (0:00:00.357) 0:04:15.062 ******** 2026-08-18 13:35:20.724151 | node1 | ok: [node1] 2026-08-18 13:35:20.724208 | node1 | ok: [node2] 2026-08-18 13:35:20.724221 | node1 | ok: [node3] 2026-08-18 13:35:20.724239 | node1 | 2026-08-18 13:35:20.724665 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:35:20.724714 | node1 | Tuesday 18 August 2026 13:35:20 +0000 (0:00:00.612) 0:04:15.674 ******** 2026-08-18 13:35:20.797521 | node1 | skipping: [node1] 2026-08-18 13:35:20.797617 | node1 | skipping: [node2] 2026-08-18 13:35:20.797631 | node1 | skipping: [node3] 2026-08-18 13:35:20.797774 | node1 | 2026-08-18 13:35:20.797881 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:35:20.798018 | node1 | Tuesday 18 August 2026 13:35:20 +0000 (0:00:00.073) 0:04:15.748 ******** 2026-08-18 13:35:21.282271 | node1 | ok: [node1] 2026-08-18 13:35:21.282486 | node1 | ok: [node2] 2026-08-18 13:35:21.282810 | node1 | ok: [node3] 2026-08-18 13:35:21.283167 | node1 | 2026-08-18 13:35:21.283525 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:35:21.283870 | node1 | Tuesday 18 August 2026 13:35:21 +0000 (0:00:00.483) 0:04:16.231 ******** 2026-08-18 13:35:21.916284 | node1 | ok: [node1] 2026-08-18 13:35:21.916344 | node1 | ok: [node2] 2026-08-18 13:35:21.916421 | node1 | ok: [node3] 2026-08-18 13:35:21.916521 | node1 | 2026-08-18 13:35:21.916664 | node1 | TASK [vexxhost.ceph.osd : Locate the cluster admin host] *********************** 2026-08-18 13:35:21.916780 | node1 | Tuesday 18 August 2026 13:35:21 +0000 (0:00:00.623) 0:04:16.854 ******** 2026-08-18 13:35:22.689545 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:35:22.689774 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:35:22.690072 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:35:22.690294 | node1 | 2026-08-18 13:35:22.690552 | node1 | TASK [vexxhost.ceph.osd : Select the admin host and its mon config path] ******* 2026-08-18 13:35:22.690918 | node1 | Tuesday 18 August 2026 13:35:22 +0000 (0:00:00.785) 0:04:17.639 ******** 2026-08-18 13:35:22.778568 | node1 | ok: [node1] 2026-08-18 13:35:22.778870 | node1 | ok: [node2] 2026-08-18 13:35:22.779129 | node1 | ok: [node3] 2026-08-18 13:35:22.779340 | node1 | 2026-08-18 13:35:22.779628 | node1 | TASK [vexxhost.ceph.osd : Set the admin mon config path] *********************** 2026-08-18 13:35:22.779848 | node1 | Tuesday 18 August 2026 13:35:22 +0000 (0:00:00.089) 0:04:17.729 ******** 2026-08-18 13:35:22.879805 | node1 | ok: [node1] 2026-08-18 13:35:22.880036 | node1 | ok: [node2] 2026-08-18 13:35:22.880268 | node1 | ok: [node3] 2026-08-18 13:35:22.880566 | node1 | 2026-08-18 13:35:22.880747 | node1 | TASK [Register this OSD host] ************************************************** 2026-08-18 13:35:22.881034 | node1 | Tuesday 18 August 2026 13:35:22 +0000 (0:00:00.101) 0:04:17.830 ******** 2026-08-18 13:35:23.013519 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:35:23.013579 | node1 | 2026-08-18 13:35:23.013589 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:35:23.013599 | node1 | Tuesday 18 August 2026 13:35:23 +0000 (0:00:00.132) 0:04:17.963 ******** 2026-08-18 13:35:24.843471 | node1 | ok: [node1] 2026-08-18 13:35:24.843581 | node1 | 2026-08-18 13:35:24.843989 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:35:24.844021 | node1 | Tuesday 18 August 2026 13:35:24 +0000 (0:00:01.829) 0:04:19.793 ******** 2026-08-18 13:35:24.937074 | node1 | ok: [node1] 2026-08-18 13:35:24.937115 | node1 | ok: [node2] 2026-08-18 13:35:24.937121 | node1 | ok: [node3] 2026-08-18 13:35:24.937130 | node1 | 2026-08-18 13:35:24.937592 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:35:24.937648 | node1 | Tuesday 18 August 2026 13:35:24 +0000 (0:00:00.093) 0:04:19.886 ******** 2026-08-18 13:35:25.014579 | node1 | ok: [node1] => { 2026-08-18 13:35:25.014659 | node1 | "changed": false, 2026-08-18 13:35:25.014750 | node1 | "msg": "All assertions passed" 2026-08-18 13:35:25.014857 | node1 | } 2026-08-18 13:35:25.014981 | node1 | ok: [node2] => { 2026-08-18 13:35:25.015101 | node1 | "changed": false, 2026-08-18 13:35:25.015202 | node1 | "msg": "All assertions passed" 2026-08-18 13:35:25.015309 | node1 | } 2026-08-18 13:35:25.015419 | node1 | ok: [node3] => { 2026-08-18 13:35:25.015528 | node1 | "changed": false, 2026-08-18 13:35:25.015639 | node1 | "msg": "All assertions passed" 2026-08-18 13:35:25.015744 | node1 | } 2026-08-18 13:35:25.015845 | node1 | 2026-08-18 13:35:25.016131 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:35:25.016223 | node1 | Tuesday 18 August 2026 13:35:25 +0000 (0:00:00.078) 0:04:19.965 ******** 2026-08-18 13:35:25.393118 | node1 | ok: [node1] 2026-08-18 13:35:25.393187 | node1 | ok: [node2] 2026-08-18 13:35:25.393197 | node1 | ok: [node3] 2026-08-18 13:35:25.393206 | node1 | 2026-08-18 13:35:25.393216 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:35:25.393232 | node1 | Tuesday 18 August 2026 13:35:25 +0000 (0:00:00.376) 0:04:20.341 ******** 2026-08-18 13:35:25.501987 | node1 | ok: [node1] 2026-08-18 13:35:25.502078 | node1 | ok: [node2] 2026-08-18 13:35:25.502092 | node1 | ok: [node3] 2026-08-18 13:35:25.502498 | node1 | 2026-08-18 13:35:25.502532 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:35:25.502539 | node1 | Tuesday 18 August 2026 13:35:25 +0000 (0:00:00.109) 0:04:20.451 ******** 2026-08-18 13:35:25.633091 | node1 | ok: [node1] 2026-08-18 13:35:25.633337 | node1 | ok: [node2] 2026-08-18 13:35:25.633575 | node1 | ok: [node3] 2026-08-18 13:35:25.633801 | node1 | 2026-08-18 13:35:25.634097 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:35:25.634360 | node1 | Tuesday 18 August 2026 13:35:25 +0000 (0:00:00.131) 0:04:20.582 ******** 2026-08-18 13:35:25.705813 | node1 | ok: [node1] => { 2026-08-18 13:35:25.706105 | node1 | "changed": false, 2026-08-18 13:35:25.706350 | node1 | "msg": "All assertions passed" 2026-08-18 13:35:25.706583 | node1 | } 2026-08-18 13:35:25.706826 | node1 | ok: [node2] => { 2026-08-18 13:35:25.707104 | node1 | "changed": false, 2026-08-18 13:35:25.707346 | node1 | "msg": "All assertions passed" 2026-08-18 13:35:25.707579 | node1 | } 2026-08-18 13:35:25.707822 | node1 | ok: [node3] => { 2026-08-18 13:35:25.708481 | node1 | "changed": false, 2026-08-18 13:35:25.708734 | node1 | "msg": "All assertions passed" 2026-08-18 13:35:25.708886 | node1 | } 2026-08-18 13:35:25.709042 | node1 | 2026-08-18 13:35:25.709212 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:35:25.709357 | node1 | Tuesday 18 August 2026 13:35:25 +0000 (0:00:00.074) 0:04:20.657 ******** 2026-08-18 13:35:27.649104 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:35:27.649170 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:35:27.649184 | node1 | ok: [node1] 2026-08-18 13:35:27.649198 | node1 | 2026-08-18 13:35:27.649453 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:35:27.649714 | node1 | Tuesday 18 August 2026 13:35:27 +0000 (0:00:01.941) 0:04:22.598 ******** 2026-08-18 13:35:27.749597 | node1 | skipping: [node1] 2026-08-18 13:35:27.749697 | node1 | skipping: [node2] 2026-08-18 13:35:27.749857 | node1 | skipping: [node3] 2026-08-18 13:35:27.750022 | node1 | 2026-08-18 13:35:27.750189 | node1 | TASK [vexxhost.ceph.osd : Apply OSD service specs (drive groups)] ************** 2026-08-18 13:35:27.750347 | node1 | Tuesday 18 August 2026 13:35:27 +0000 (0:00:00.101) 0:04:22.700 ******** 2026-08-18 13:35:27.785742 | node1 | skipping: [node1] 2026-08-18 13:35:27.785967 | node1 | 2026-08-18 13:35:27.786253 | node1 | TASK [vexxhost.ceph.osd : List existing ceph-volume OSDs on this host] ********* 2026-08-18 13:35:27.786512 | node1 | Tuesday 18 August 2026 13:35:27 +0000 (0:00:00.036) 0:04:22.736 ******** 2026-08-18 13:35:29.636585 | node1 | ok: [node1] 2026-08-18 13:35:29.636784 | node1 | ok: [node3] 2026-08-18 13:35:29.637061 | node1 | ok: [node2] 2026-08-18 13:35:29.637326 | node1 | 2026-08-18 13:35:29.637551 | node1 | TASK [vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices] ******** 2026-08-18 13:35:29.637804 | node1 | Tuesday 18 August 2026 13:35:29 +0000 (0:00:01.850) 0:04:24.587 ******** 2026-08-18 13:41:33.825437 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd0/data) 2026-08-18 13:41:33.825580 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd0/data) 2026-08-18 13:41:33.825763 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd0/data) 2026-08-18 13:41:33.825971 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd1/data) 2026-08-18 13:41:33.826142 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd1/data) 2026-08-18 13:41:33.826355 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd1/data) 2026-08-18 13:41:33.826545 | node1 | changed: [node3 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node3-osd2/data) 2026-08-18 13:41:33.826714 | node1 | changed: [node2 -> node1({{ nodepool.private_ipv4 }})] => (item=/dev/ceph-node2-osd2/data) 2026-08-18 13:41:33.826888 | node1 | changed: [node1] => (item=/dev/ceph-node1-osd2/data) 2026-08-18 13:41:33.828640 | node1 | 2026-08-18 13:41:33.828676 | node1 | TASK [vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host] *** 2026-08-18 13:41:33.828687 | node1 | Tuesday 18 August 2026 13:41:33 +0000 (0:06:04.180) 0:10:28.768 ******** 2026-08-18 13:41:55.734038 | node1 | FAILED - RETRYING: [node2 -> node1]: Wait for explicitly requested OSD daemons to be running on this host (120 retries left). 2026-08-18 13:41:55.734119 | node1 | FAILED - RETRYING: [node1]: Wait for explicitly requested OSD daemons to be running on this host (120 retries left). 2026-08-18 13:41:55.734134 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:41:55.734155 | node1 | FAILED - RETRYING: [node1]: Wait for explicitly requested OSD daemons to be running on this host (119 retries left). 2026-08-18 13:41:55.735991 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:41:55.736032 | node1 | FAILED - RETRYING: [node1]: Wait for explicitly requested OSD daemons to be running on this host (118 retries left). 2026-08-18 13:41:55.736039 | node1 | ok: [node1] 2026-08-18 13:41:55.736045 | node1 | 2026-08-18 13:41:55.736052 | node1 | TASK [vexxhost.ceph.osd : Wait for OSD drive-group services to deploy at least one daemon] *** 2026-08-18 13:41:55.736343 | node1 | Tuesday 18 August 2026 13:41:55 +0000 (0:00:21.916) 0:10:50.684 ******** 2026-08-18 13:41:55.814038 | node1 | skipping: [node1] 2026-08-18 13:41:55.814090 | node1 | 2026-08-18 13:41:55.814104 | node1 | PLAY [Apply Ceph configuration] ************************************************ 2026-08-18 13:41:55.814116 | node1 | 2026-08-18 13:41:55.814127 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:41:55.814139 | node1 | Tuesday 18 August 2026 13:41:55 +0000 (0:00:00.079) 0:10:50.764 ******** 2026-08-18 13:41:57.048676 | node1 | ok: [node1] 2026-08-18 13:41:57.048714 | node1 | ok: [node2] 2026-08-18 13:41:57.048720 | node1 | ok: [node3] 2026-08-18 13:41:57.048726 | node1 | 2026-08-18 13:41:57.048731 | node1 | TASK [vexxhost.ceph.configure : Locate the cluster admin host] ***************** 2026-08-18 13:41:57.048737 | node1 | Tuesday 18 August 2026 13:41:57 +0000 (0:00:01.235) 0:10:51.999 ******** 2026-08-18 13:41:57.796205 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:41:57.796280 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:41:57.796294 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:41:57.796306 | node1 | 2026-08-18 13:41:57.796325 | node1 | TASK [vexxhost.ceph.configure : Select the admin host] ************************* 2026-08-18 13:41:57.796487 | node1 | Tuesday 18 August 2026 13:41:57 +0000 (0:00:00.746) 0:10:52.746 ******** 2026-08-18 13:41:57.868442 | node1 | ok: [node1] 2026-08-18 13:41:57.868710 | node1 | ok: [node2] 2026-08-18 13:41:57.869031 | node1 | ok: [node3] 2026-08-18 13:41:57.869368 | node1 | 2026-08-18 13:41:57.869733 | node1 | TASK [vexxhost.ceph.configure : Set the admin mon config path] ***************** 2026-08-18 13:41:57.870070 | node1 | Tuesday 18 August 2026 13:41:57 +0000 (0:00:00.072) 0:10:52.819 ******** 2026-08-18 13:41:57.941641 | node1 | ok: [node1] 2026-08-18 13:41:57.941835 | node1 | ok: [node2] 2026-08-18 13:41:57.942137 | node1 | ok: [node3] 2026-08-18 13:41:57.942407 | node1 | 2026-08-18 13:41:57.942712 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 13:41:57.943019 | node1 | Tuesday 18 August 2026 13:41:57 +0000 (0:00:00.073) 0:10:52.892 ******** 2026-08-18 13:41:58.022181 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/config.yml for node1, node2, node3 2026-08-18 13:41:58.022383 | node1 | 2026-08-18 13:41:58.022639 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 13:41:58.022912 | node1 | Tuesday 18 August 2026 13:41:58 +0000 (0:00:00.080) 0:10:52.973 ******** 2026-08-18 13:42:17.588917 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/standby_behaviour) 2026-08-18 13:42:17.589020 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/RGW_API_SSL_VERIFY) 2026-08-18 13:42:17.589041 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_CEPHFS) 2026-08-18 13:42:17.589187 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_ISCSI) 2026-08-18 13:42:17.589342 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_MIRRORING) 2026-08-18 13:42:17.589515 | node1 | changed: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_NFS) 2026-08-18 13:42:17.589665 | node1 | 2026-08-18 13:42:17.589826 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 13:42:17.590027 | node1 | Tuesday 18 August 2026 13:42:17 +0000 (0:00:19.564) 0:11:12.537 ******** 2026-08-18 13:42:17.677807 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/ec_profiles.yml for node1, node2, node3 2026-08-18 13:42:17.677899 | node1 | 2026-08-18 13:42:17.678112 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 13:42:17.678258 | node1 | Tuesday 18 August 2026 13:42:17 +0000 (0:00:00.089) 0:11:12.627 ******** 2026-08-18 13:42:21.649386 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 13:42:21.649475 | node1 | 2026-08-18 13:42:21.649564 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 13:42:21.649679 | node1 | Tuesday 18 August 2026 13:42:21 +0000 (0:00:03.972) 0:11:16.600 ******** 2026-08-18 13:42:21.730501 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/crush_rules.yml for node1, node2, node3 2026-08-18 13:42:21.730566 | node1 | 2026-08-18 13:42:21.733113 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 13:42:21.733174 | node1 | Tuesday 18 August 2026 13:42:21 +0000 (0:00:00.081) 0:11:16.682 ******** 2026-08-18 13:42:33.388286 | node1 | changed: [node1] => (item=rep_hdd) 2026-08-18 13:42:33.388386 | node1 | changed: [node1] => (item=rep_lab) 2026-08-18 13:42:33.388399 | node1 | changed: [node1] => (item=ec_hdd_s3) 2026-08-18 13:42:33.389006 | node1 | 2026-08-18 13:42:33.389470 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 13:42:33.389558 | node1 | Tuesday 18 August 2026 13:42:33 +0000 (0:00:11.652) 0:11:28.334 ******** 2026-08-18 13:42:33.477881 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/pools.yml for node1, node2, node3 2026-08-18 13:42:33.478229 | node1 | 2026-08-18 13:42:33.478518 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 13:42:33.478765 | node1 | Tuesday 18 August 2026 13:42:33 +0000 (0:00:00.093) 0:11:28.428 ******** 2026-08-18 13:43:10.442838 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.data) 2026-08-18 13:43:10.443074 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.index) 2026-08-18 13:43:10.443324 | node1 | changed: [node1] => (item=molecule-1.rgw.buckets.non-ec) 2026-08-18 13:43:10.443573 | node1 | changed: [node1] => (item=molecule-1.rgw.control) 2026-08-18 13:43:10.443829 | node1 | changed: [node1] => (item=molecule-1.rgw.log) 2026-08-18 13:43:10.444163 | node1 | changed: [node1] => (item=molecule-1.rgw.meta) 2026-08-18 13:43:10.444385 | node1 | 2026-08-18 13:43:10.444635 | node1 | TASK [vexxhost.ceph.configure : Configure CephX keys] ************************** 2026-08-18 13:43:10.444890 | node1 | Tuesday 18 August 2026 13:43:10 +0000 (0:00:36.965) 0:12:05.393 ******** 2026-08-18 13:43:10.504964 | node1 | skipping: [node1] 2026-08-18 13:43:10.505191 | node1 | skipping: [node2] 2026-08-18 13:43:10.505427 | node1 | skipping: [node3] 2026-08-18 13:43:10.505646 | node1 | 2026-08-18 13:43:10.505899 | node1 | TASK [vexxhost.ceph.configure : Enroll the dashboard TLS certificate] ********** 2026-08-18 13:43:10.506190 | node1 | Tuesday 18 August 2026 13:43:10 +0000 (0:00:00.062) 0:12:05.456 ******** 2026-08-18 13:43:10.604835 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard.yml for node1, node2, node3 2026-08-18 13:43:10.605099 | node1 | 2026-08-18 13:43:10.605387 | node1 | TASK [vexxhost.ceph.configure : Check whether the dashboard mgr module is enabled] *** 2026-08-18 13:43:10.605652 | node1 | Tuesday 18 August 2026 13:43:10 +0000 (0:00:00.099) 0:12:05.555 ******** 2026-08-18 13:43:12.505701 | node1 | ok: [node1] 2026-08-18 13:43:12.505775 | node1 | 2026-08-18 13:43:12.505793 | node1 | TASK [vexxhost.ceph.configure : Get the current dashboard certificate] ********* 2026-08-18 13:43:12.505825 | node1 | Tuesday 18 August 2026 13:43:12 +0000 (0:00:01.899) 0:12:07.455 ******** 2026-08-18 13:43:14.467475 | node1 | ok: [node1] 2026-08-18 13:43:14.467531 | node1 | 2026-08-18 13:43:14.467545 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate] ***************** 2026-08-18 13:43:14.467558 | node1 | Tuesday 18 August 2026 13:43:14 +0000 (0:00:01.961) 0:12:09.417 ******** 2026-08-18 13:43:14.513539 | node1 | skipping: [node1] 2026-08-18 13:43:14.513604 | node1 | 2026-08-18 13:43:14.513618 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate key] ************* 2026-08-18 13:43:14.513631 | node1 | Tuesday 18 August 2026 13:43:14 +0000 (0:00:00.046) 0:12:09.463 ******** 2026-08-18 13:43:14.556381 | node1 | skipping: [node1] 2026-08-18 13:43:14.556508 | node1 | 2026-08-18 13:43:14.556634 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 13:43:14.556783 | node1 | Tuesday 18 August 2026 13:43:14 +0000 (0:00:00.040) 0:12:09.504 ******** 2026-08-18 13:43:14.597338 | node1 | skipping: [node1] 2026-08-18 13:43:14.597427 | node1 | 2026-08-18 13:43:14.597446 | node1 | TASK [vexxhost.ceph.configure : Create a self-signed certificate] ************** 2026-08-18 13:43:14.600108 | node1 | Tuesday 18 August 2026 13:43:14 +0000 (0:00:00.043) 0:12:09.548 ******** 2026-08-18 13:43:16.699509 | node1 | changed: [node1] 2026-08-18 13:43:16.699610 | node1 | 2026-08-18 13:43:16.699745 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 13:43:16.699907 | node1 | Tuesday 18 August 2026 13:43:16 +0000 (0:00:02.102) 0:12:11.650 ******** 2026-08-18 13:43:16.756614 | node1 | ok: [node1] 2026-08-18 13:43:16.756682 | node1 | 2026-08-18 13:43:16.756696 | node1 | TASK [vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate] *** 2026-08-18 13:43:16.756709 | node1 | Tuesday 18 August 2026 13:43:16 +0000 (0:00:00.056) 0:12:11.706 ******** 2026-08-18 13:43:23.951963 | node1 | changed: [node1] => (item=ceph mgr module disable dashboard) 2026-08-18 13:43:23.952018 | node1 | changed: [node1] => (item=ceph mgr module enable dashboard) 2026-08-18 13:43:23.952025 | node1 | changed: [node1] => (item=ceph orch reconfig mgr) 2026-08-18 13:43:23.952031 | node1 | 2026-08-18 13:43:23.952036 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 13:43:23.952042 | node1 | Tuesday 18 August 2026 13:43:23 +0000 (0:00:07.191) 0:12:18.898 ******** 2026-08-18 13:43:24.051214 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard_users.yml for node1, node2, node3 2026-08-18 13:43:24.051248 | node1 | 2026-08-18 13:43:24.051254 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 13:43:24.051260 | node1 | Tuesday 18 August 2026 13:43:24 +0000 (0:00:00.103) 0:12:19.002 ******** 2026-08-18 13:43:34.744193 | node1 | changed: [node1] => (item=None) 2026-08-18 13:43:34.744271 | node1 | changed: [node1] => (item=None) 2026-08-18 13:43:34.745146 | node1 | changed: [node1 -> {{ ceph_configure_admin_host }}] 2026-08-18 13:43:34.745157 | node1 | 2026-08-18 13:43:34.745163 | node1 | PLAY [Deploy Ceph RADOS Gateways] ********************************************** 2026-08-18 13:43:34.745613 | node1 | 2026-08-18 13:43:34.745682 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:43:34.745698 | node1 | Tuesday 18 August 2026 13:43:34 +0000 (0:00:10.691) 0:12:29.693 ******** 2026-08-18 13:43:36.769203 | node1 | ok: [node3] 2026-08-18 13:43:36.769258 | node1 | ok: [node2] 2026-08-18 13:43:36.769274 | node1 | ok: [node1] 2026-08-18 13:43:36.769289 | node1 | 2026-08-18 13:43:36.769384 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:43:36.769396 | node1 | Tuesday 18 August 2026 13:43:36 +0000 (0:00:02.024) 0:12:31.718 ******** 2026-08-18 13:43:37.068345 | node1 | ok: [node2] 2026-08-18 13:43:37.068404 | node1 | ok: [node1] 2026-08-18 13:43:37.068528 | node1 | ok: [node3] 2026-08-18 13:43:37.068628 | node1 | 2026-08-18 13:43:37.068743 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:43:37.068877 | node1 | Tuesday 18 August 2026 13:43:37 +0000 (0:00:00.299) 0:12:32.017 ******** 2026-08-18 13:43:37.168070 | node1 | skipping: [node1] 2026-08-18 13:43:37.168134 | node1 | skipping: [node2] 2026-08-18 13:43:37.168144 | node1 | skipping: [node3] 2026-08-18 13:43:37.168153 | node1 | 2026-08-18 13:43:37.168163 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:43:37.168283 | node1 | Tuesday 18 August 2026 13:43:37 +0000 (0:00:00.099) 0:12:32.117 ******** 2026-08-18 13:43:37.487963 | node1 | ok: [node1] 2026-08-18 13:43:37.488018 | node1 | ok: [node2] 2026-08-18 13:43:37.488027 | node1 | ok: [node3] 2026-08-18 13:43:37.488037 | node1 | 2026-08-18 13:43:37.488046 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:43:37.488055 | node1 | Tuesday 18 August 2026 13:43:37 +0000 (0:00:00.319) 0:12:32.437 ******** 2026-08-18 13:43:37.586452 | node1 | ok: [node1] => { 2026-08-18 13:43:37.586512 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:43:37.586615 | node1 | } 2026-08-18 13:43:37.586726 | node1 | ok: [node2] => { 2026-08-18 13:43:37.586842 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:43:37.586981 | node1 | } 2026-08-18 13:43:37.587078 | node1 | ok: [node3] => { 2026-08-18 13:43:37.587193 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:43:37.587296 | node1 | } 2026-08-18 13:43:37.587428 | node1 | 2026-08-18 13:43:37.587532 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:43:37.587642 | node1 | Tuesday 18 August 2026 13:43:37 +0000 (0:00:00.099) 0:12:32.536 ******** 2026-08-18 13:43:37.996984 | node1 | ok: [node1] 2026-08-18 13:43:37.997037 | node1 | ok: [node3] 2026-08-18 13:43:37.997041 | node1 | ok: [node2] 2026-08-18 13:43:37.997046 | node1 | 2026-08-18 13:43:37.997050 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:43:37.997054 | node1 | Tuesday 18 August 2026 13:43:37 +0000 (0:00:00.407) 0:12:32.944 ******** 2026-08-18 13:43:38.168483 | node1 | skipping: [node1] 2026-08-18 13:43:38.168560 | node1 | skipping: [node2] 2026-08-18 13:43:38.168628 | node1 | skipping: [node3] 2026-08-18 13:43:38.168738 | node1 | 2026-08-18 13:43:38.168853 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:43:38.168988 | node1 | Tuesday 18 August 2026 13:43:38 +0000 (0:00:00.175) 0:12:33.119 ******** 2026-08-18 13:43:38.292391 | node1 | skipping: [node1] 2026-08-18 13:43:38.292468 | node1 | skipping: [node2] 2026-08-18 13:43:38.292568 | node1 | skipping: [node3] 2026-08-18 13:43:38.292678 | node1 | 2026-08-18 13:43:38.292778 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:43:38.292899 | node1 | Tuesday 18 August 2026 13:43:38 +0000 (0:00:00.121) 0:12:33.241 ******** 2026-08-18 13:43:38.594273 | node1 | ok: [node1] 2026-08-18 13:43:38.594468 | node1 | ok: [node2] 2026-08-18 13:43:38.594744 | node1 | ok: [node3] 2026-08-18 13:43:38.594995 | node1 | 2026-08-18 13:43:38.595258 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:43:38.595557 | node1 | Tuesday 18 August 2026 13:43:38 +0000 (0:00:00.304) 0:12:33.545 ******** 2026-08-18 13:43:39.561150 | node1 | ok: [node3] 2026-08-18 13:43:39.561227 | node1 | ok: [node2] 2026-08-18 13:43:39.561240 | node1 | ok: [node1] 2026-08-18 13:43:39.561260 | node1 | 2026-08-18 13:43:39.561830 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:43:39.561898 | node1 | Tuesday 18 August 2026 13:43:39 +0000 (0:00:00.966) 0:12:34.511 ******** 2026-08-18 13:43:39.682070 | node1 | ok: [node1] => { 2026-08-18 13:43:39.682131 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:43:39.682136 | node1 | } 2026-08-18 13:43:39.682141 | node1 | ok: [node2] => { 2026-08-18 13:43:39.682166 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:43:39.683220 | node1 | } 2026-08-18 13:43:39.683253 | node1 | ok: [node3] => { 2026-08-18 13:43:39.683352 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:43:39.683362 | node1 | } 2026-08-18 13:43:39.683368 | node1 | 2026-08-18 13:43:39.683374 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:43:39.683380 | node1 | Tuesday 18 August 2026 13:43:39 +0000 (0:00:00.121) 0:12:34.632 ******** 2026-08-18 13:43:40.153052 | node1 | ok: [node1] 2026-08-18 13:43:40.153136 | node1 | ok: [node3] 2026-08-18 13:43:40.154009 | node1 | ok: [node2] 2026-08-18 13:43:40.154074 | node1 | 2026-08-18 13:43:40.154085 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:43:40.154095 | node1 | Tuesday 18 August 2026 13:43:40 +0000 (0:00:00.470) 0:12:35.102 ******** 2026-08-18 13:43:42.370850 | node1 | ok: [node2] 2026-08-18 13:43:42.371153 | node1 | ok: [node1] 2026-08-18 13:43:42.371224 | node1 | ok: [node3] 2026-08-18 13:43:42.371238 | node1 | 2026-08-18 13:43:42.371248 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:43:42.371266 | node1 | Tuesday 18 August 2026 13:43:42 +0000 (0:00:02.217) 0:12:37.320 ******** 2026-08-18 13:43:42.405747 | node1 | skipping: [node1] 2026-08-18 13:43:42.442093 | node1 | skipping: [node2] 2026-08-18 13:43:42.442127 | node1 | skipping: [node3] 2026-08-18 13:43:42.442134 | node1 | 2026-08-18 13:43:42.442140 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:43:42.442147 | node1 | Tuesday 18 August 2026 13:43:42 +0000 (0:00:00.071) 0:12:37.392 ******** 2026-08-18 13:43:42.483580 | node1 | skipping: [node1] 2026-08-18 13:43:42.516007 | node1 | skipping: [node2] 2026-08-18 13:43:42.516067 | node1 | skipping: [node3] 2026-08-18 13:43:42.516921 | node1 | 2026-08-18 13:43:42.517019 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:43:42.517030 | node1 | Tuesday 18 August 2026 13:43:42 +0000 (0:00:00.073) 0:12:37.466 ******** 2026-08-18 13:43:42.583404 | node1 | skipping: [node1] 2026-08-18 13:43:42.583446 | node1 | skipping: [node2] 2026-08-18 13:43:42.583909 | node1 | skipping: [node3] 2026-08-18 13:43:42.583960 | node1 | 2026-08-18 13:43:42.583965 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:43:42.583970 | node1 | Tuesday 18 August 2026 13:43:42 +0000 (0:00:00.067) 0:12:37.533 ******** 2026-08-18 13:43:43.607367 | node1 | ok: [node2] 2026-08-18 13:43:43.607440 | node1 | ok: [node1] 2026-08-18 13:43:43.607519 | node1 | ok: [node3] 2026-08-18 13:43:43.607621 | node1 | 2026-08-18 13:43:43.607738 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:43:43.607851 | node1 | Tuesday 18 August 2026 13:43:43 +0000 (0:00:01.024) 0:12:38.558 ******** 2026-08-18 13:43:44.213051 | node1 | ok: [node2] 2026-08-18 13:43:44.213108 | node1 | ok: [node1] 2026-08-18 13:43:44.213122 | node1 | ok: [node3] 2026-08-18 13:43:44.213134 | node1 | 2026-08-18 13:43:44.213146 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:43:44.213158 | node1 | Tuesday 18 August 2026 13:43:44 +0000 (0:00:00.602) 0:12:39.160 ******** 2026-08-18 13:43:45.410978 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:43:45.411021 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:43:45.411029 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:43:45.411036 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:43:45.411044 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:43:45.411051 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:43:45.411071 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:43:45.411076 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:43:45.411081 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:43:45.411086 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:43:45.411092 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:43:45.411097 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:43:45.411102 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:43:45.411107 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:43:45.411112 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:43:45.411118 | node1 | 2026-08-18 13:43:45.411281 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:43:45.411475 | node1 | Tuesday 18 August 2026 13:43:45 +0000 (0:00:01.198) 0:12:40.359 ******** 2026-08-18 13:43:46.119394 | node1 | ok: [node1] 2026-08-18 13:43:46.119454 | node1 | ok: [node2] 2026-08-18 13:43:46.119563 | node1 | ok: [node3] 2026-08-18 13:43:46.119664 | node1 | 2026-08-18 13:43:46.119793 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:43:46.119906 | node1 | Tuesday 18 August 2026 13:43:46 +0000 (0:00:00.688) 0:12:41.047 ******** 2026-08-18 13:43:46.120027 | node1 | 2026-08-18 13:43:46.120144 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:43:46.120916 | node1 | Tuesday 18 August 2026 13:43:46 +0000 (0:00:00.007) 0:12:41.055 ******** 2026-08-18 13:43:46.120986 | node1 | 2026-08-18 13:43:46.120991 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:43:46.121002 | node1 | Tuesday 18 August 2026 13:43:46 +0000 (0:00:00.008) 0:12:41.064 ******** 2026-08-18 13:43:46.121006 | node1 | 2026-08-18 13:43:46.121010 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:43:46.121014 | node1 | Tuesday 18 August 2026 13:43:46 +0000 (0:00:00.006) 0:12:41.070 ******** 2026-08-18 13:43:46.609213 | node1 | ok: [node1] 2026-08-18 13:43:46.609309 | node1 | ok: [node2] 2026-08-18 13:43:46.609433 | node1 | ok: [node3] 2026-08-18 13:43:46.609536 | node1 | 2026-08-18 13:43:46.609653 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:43:46.609758 | node1 | Tuesday 18 August 2026 13:43:46 +0000 (0:00:00.487) 0:12:41.557 ******** 2026-08-18 13:43:46.904031 | node1 | ok: [node1] 2026-08-18 13:43:46.904085 | node1 | ok: [node2] 2026-08-18 13:43:46.904095 | node1 | ok: [node3] 2026-08-18 13:43:46.904104 | node1 | 2026-08-18 13:43:46.904113 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:43:46.904122 | node1 | Tuesday 18 August 2026 13:43:46 +0000 (0:00:00.295) 0:12:41.853 ******** 2026-08-18 13:43:47.000052 | node1 | ok: [node1] => { 2026-08-18 13:43:47.000120 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:43:47.000130 | node1 | } 2026-08-18 13:43:47.000139 | node1 | ok: [node2] => { 2026-08-18 13:43:47.000147 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:43:47.000162 | node1 | } 2026-08-18 13:43:47.000306 | node1 | ok: [node3] => { 2026-08-18 13:43:47.000594 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:43:47.000829 | node1 | } 2026-08-18 13:43:47.001120 | node1 | 2026-08-18 13:43:47.001366 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:43:47.001604 | node1 | Tuesday 18 August 2026 13:43:46 +0000 (0:00:00.095) 0:12:41.949 ******** 2026-08-18 13:43:47.442034 | node1 | ok: [node1] 2026-08-18 13:43:47.442123 | node1 | ok: [node2] 2026-08-18 13:43:47.442239 | node1 | ok: [node3] 2026-08-18 13:43:47.442327 | node1 | 2026-08-18 13:43:47.442446 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:43:47.442543 | node1 | Tuesday 18 August 2026 13:43:47 +0000 (0:00:00.444) 0:12:42.393 ******** 2026-08-18 13:43:50.775014 | node1 | ok: [node1] 2026-08-18 13:43:50.775271 | node1 | ok: [node3] 2026-08-18 13:43:50.775514 | node1 | ok: [node2] 2026-08-18 13:43:50.775744 | node1 | 2026-08-18 13:43:50.776027 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:43:50.776287 | node1 | Tuesday 18 August 2026 13:43:50 +0000 (0:00:03.332) 0:12:45.726 ******** 2026-08-18 13:43:51.697944 | node1 | ok: [node1] 2026-08-18 13:43:51.698113 | node1 | ok: [node2] 2026-08-18 13:43:51.698289 | node1 | ok: [node3] 2026-08-18 13:43:51.698453 | node1 | 2026-08-18 13:43:51.698710 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:43:51.698859 | node1 | Tuesday 18 August 2026 13:43:51 +0000 (0:00:00.922) 0:12:46.648 ******** 2026-08-18 13:43:51.965048 | node1 | ok: [node1] 2026-08-18 13:43:51.965086 | node1 | ok: [node2] 2026-08-18 13:43:51.965094 | node1 | ok: [node3] 2026-08-18 13:43:51.965695 | node1 | 2026-08-18 13:43:51.965813 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:43:51.965829 | node1 | Tuesday 18 August 2026 13:43:51 +0000 (0:00:00.266) 0:12:46.915 ******** 2026-08-18 13:43:52.530757 | node1 | ok: [node1] 2026-08-18 13:43:52.531004 | node1 | ok: [node2] 2026-08-18 13:43:52.531395 | node1 | ok: [node3] 2026-08-18 13:43:52.531629 | node1 | 2026-08-18 13:43:52.531885 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:43:52.532184 | node1 | Tuesday 18 August 2026 13:43:52 +0000 (0:00:00.565) 0:12:47.481 ******** 2026-08-18 13:43:53.247358 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:43:53.247580 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:43:53.247835 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:43:53.248146 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:43:53.248388 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:43:53.248639 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:43:53.248890 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:43:53.249188 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:43:53.249424 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:43:53.249649 | node1 | 2026-08-18 13:43:53.249900 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:43:53.250198 | node1 | Tuesday 18 August 2026 13:43:53 +0000 (0:00:00.717) 0:12:48.198 ******** 2026-08-18 13:43:53.804651 | node1 | ok: [node1] 2026-08-18 13:43:53.804793 | node1 | ok: [node3] 2026-08-18 13:43:53.804963 | node1 | ok: [node2] 2026-08-18 13:43:53.805131 | node1 | 2026-08-18 13:43:53.805312 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:43:53.805476 | node1 | Tuesday 18 August 2026 13:43:53 +0000 (0:00:00.557) 0:12:48.755 ******** 2026-08-18 13:43:54.385523 | node1 | ok: [node1] 2026-08-18 13:43:54.385723 | node1 | ok: [node2] 2026-08-18 13:43:54.385991 | node1 | ok: [node3] 2026-08-18 13:43:54.386226 | node1 | 2026-08-18 13:43:54.386484 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:43:54.386731 | node1 | Tuesday 18 August 2026 13:43:54 +0000 (0:00:00.559) 0:12:49.314 ******** 2026-08-18 13:43:54.387001 | node1 | 2026-08-18 13:43:54.387270 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:43:54.387537 | node1 | Tuesday 18 August 2026 13:43:54 +0000 (0:00:00.006) 0:12:49.321 ******** 2026-08-18 13:43:54.387773 | node1 | 2026-08-18 13:43:54.388085 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:43:54.388359 | node1 | Tuesday 18 August 2026 13:43:54 +0000 (0:00:00.008) 0:12:49.330 ******** 2026-08-18 13:43:54.388597 | node1 | 2026-08-18 13:43:54.388863 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:43:54.389263 | node1 | Tuesday 18 August 2026 13:43:54 +0000 (0:00:00.006) 0:12:49.336 ******** 2026-08-18 13:43:54.873256 | node1 | ok: [node1] 2026-08-18 13:43:54.873314 | node1 | ok: [node2] 2026-08-18 13:43:54.873324 | node1 | ok: [node3] 2026-08-18 13:43:54.873333 | node1 | 2026-08-18 13:43:54.873343 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:43:54.873352 | node1 | Tuesday 18 August 2026 13:43:54 +0000 (0:00:00.486) 0:12:49.822 ******** 2026-08-18 13:43:54.948188 | node1 | ok: [node1] => { 2026-08-18 13:43:54.948256 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:43:54.948274 | node1 | } 2026-08-18 13:43:54.948564 | node1 | ok: [node2] => { 2026-08-18 13:43:54.948836 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:43:54.949171 | node1 | } 2026-08-18 13:43:54.949446 | node1 | ok: [node3] => { 2026-08-18 13:43:54.949744 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:43:54.950084 | node1 | } 2026-08-18 13:43:54.950323 | node1 | 2026-08-18 13:43:54.950604 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:43:54.950891 | node1 | Tuesday 18 August 2026 13:43:54 +0000 (0:00:00.076) 0:12:49.899 ******** 2026-08-18 13:43:55.341657 | node1 | ok: [node1] 2026-08-18 13:43:55.341873 | node1 | ok: [node3] 2026-08-18 13:43:55.342182 | node1 | ok: [node2] 2026-08-18 13:43:55.342397 | node1 | 2026-08-18 13:43:55.342675 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:43:55.342956 | node1 | Tuesday 18 August 2026 13:43:55 +0000 (0:00:00.393) 0:12:50.292 ******** 2026-08-18 13:43:55.433022 | node1 | skipping: [node1] 2026-08-18 13:43:55.433092 | node1 | skipping: [node2] 2026-08-18 13:43:55.433101 | node1 | skipping: [node3] 2026-08-18 13:43:55.433117 | node1 | 2026-08-18 13:43:55.433354 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:43:55.433590 | node1 | Tuesday 18 August 2026 13:43:55 +0000 (0:00:00.090) 0:12:50.383 ******** 2026-08-18 13:43:55.519982 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:43:55.520265 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:43:55.520538 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:43:55.520776 | node1 | 2026-08-18 13:43:55.521094 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:43:55.521450 | node1 | Tuesday 18 August 2026 13:43:55 +0000 (0:00:00.087) 0:12:50.471 ******** 2026-08-18 13:43:56.831974 | node1 | ok: [node2] 2026-08-18 13:43:56.832150 | node1 | ok: [node3] 2026-08-18 13:43:56.832456 | node1 | ok: [node1] 2026-08-18 13:43:56.832688 | node1 | 2026-08-18 13:43:56.833151 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:43:56.833236 | node1 | Tuesday 18 August 2026 13:43:56 +0000 (0:00:01.311) 0:12:51.782 ******** 2026-08-18 13:43:57.677735 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:43:57.678117 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:43:57.678509 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:43:57.678844 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:43:57.679212 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:43:57.679462 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:43:57.679689 | node1 | 2026-08-18 13:43:57.679955 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:43:57.680240 | node1 | Tuesday 18 August 2026 13:43:57 +0000 (0:00:00.845) 0:12:52.628 ******** 2026-08-18 13:43:57.960482 | node1 | ok: [node1] 2026-08-18 13:43:57.960701 | node1 | ok: [node2] 2026-08-18 13:43:57.961033 | node1 | ok: [node3] 2026-08-18 13:43:57.961334 | node1 | 2026-08-18 13:43:57.961552 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:43:57.961794 | node1 | Tuesday 18 August 2026 13:43:57 +0000 (0:00:00.283) 0:12:52.911 ******** 2026-08-18 13:43:58.284424 | node1 | ok: [node2] 2026-08-18 13:43:58.284660 | node1 | ok: [node1] 2026-08-18 13:43:58.284895 | node1 | ok: [node3] 2026-08-18 13:43:58.285221 | node1 | 2026-08-18 13:43:58.285485 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:43:58.285730 | node1 | Tuesday 18 August 2026 13:43:58 +0000 (0:00:00.324) 0:12:53.235 ******** 2026-08-18 13:43:58.869133 | node1 | ok: [node1] 2026-08-18 13:43:58.869207 | node1 | ok: [node2] 2026-08-18 13:43:58.869217 | node1 | ok: [node3] 2026-08-18 13:43:58.869226 | node1 | 2026-08-18 13:43:58.869366 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:43:58.869646 | node1 | Tuesday 18 August 2026 13:43:58 +0000 (0:00:00.583) 0:12:53.819 ******** 2026-08-18 13:43:58.960515 | node1 | skipping: [node1] 2026-08-18 13:43:58.960685 | node1 | skipping: [node2] 2026-08-18 13:43:58.960918 | node1 | skipping: [node3] 2026-08-18 13:43:58.961185 | node1 | 2026-08-18 13:43:58.961525 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:43:58.961714 | node1 | Tuesday 18 August 2026 13:43:58 +0000 (0:00:00.092) 0:12:53.911 ******** 2026-08-18 13:43:59.429474 | node1 | ok: [node1] 2026-08-18 13:43:59.429721 | node1 | ok: [node2] 2026-08-18 13:43:59.430013 | node1 | ok: [node3] 2026-08-18 13:43:59.430255 | node1 | 2026-08-18 13:43:59.430529 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:43:59.430788 | node1 | Tuesday 18 August 2026 13:43:59 +0000 (0:00:00.468) 0:12:54.380 ******** 2026-08-18 13:44:00.041749 | node1 | ok: [node1] 2026-08-18 13:44:00.041870 | node1 | ok: [node3] 2026-08-18 13:44:00.042124 | node1 | ok: [node2] 2026-08-18 13:44:00.042263 | node1 | 2026-08-18 13:44:00.042426 | node1 | TASK [vexxhost.ceph.rgw : Locate the cluster admin host] *********************** 2026-08-18 13:44:00.042574 | node1 | Tuesday 18 August 2026 13:44:00 +0000 (0:00:00.612) 0:12:54.993 ******** 2026-08-18 13:44:01.785248 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:44:01.785371 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:44:01.785382 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:44:01.785392 | node1 | 2026-08-18 13:44:01.785401 | node1 | TASK [vexxhost.ceph.rgw : Select the admin host] ******************************* 2026-08-18 13:44:01.785420 | node1 | Tuesday 18 August 2026 13:44:01 +0000 (0:00:01.742) 0:12:56.735 ******** 2026-08-18 13:44:01.884848 | node1 | ok: [node1] 2026-08-18 13:44:01.884905 | node1 | ok: [node2] 2026-08-18 13:44:01.884915 | node1 | ok: [node3] 2026-08-18 13:44:01.884924 | node1 | 2026-08-18 13:44:01.884986 | node1 | TASK [vexxhost.ceph.rgw : Set the admin mon config path] *********************** 2026-08-18 13:44:01.884995 | node1 | Tuesday 18 August 2026 13:44:01 +0000 (0:00:00.098) 0:12:56.834 ******** 2026-08-18 13:44:01.968153 | node1 | ok: [node1] 2026-08-18 13:44:01.968233 | node1 | ok: [node2] 2026-08-18 13:44:01.968250 | node1 | ok: [node3] 2026-08-18 13:44:01.968988 | node1 | 2026-08-18 13:44:01.969047 | node1 | TASK [Register this RGW host] ************************************************** 2026-08-18 13:44:01.969058 | node1 | Tuesday 18 August 2026 13:44:01 +0000 (0:00:00.083) 0:12:56.917 ******** 2026-08-18 13:44:02.098261 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:44:02.098306 | node1 | 2026-08-18 13:44:02.098320 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:44:02.098332 | node1 | Tuesday 18 August 2026 13:44:02 +0000 (0:00:00.130) 0:12:57.048 ******** 2026-08-18 13:44:03.894099 | node1 | ok: [node1] 2026-08-18 13:44:03.894202 | node1 | 2026-08-18 13:44:03.894227 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:44:03.894502 | node1 | Tuesday 18 August 2026 13:44:03 +0000 (0:00:01.794) 0:12:58.843 ******** 2026-08-18 13:44:03.977641 | node1 | ok: [node1] 2026-08-18 13:44:03.977691 | node1 | ok: [node2] 2026-08-18 13:44:03.977817 | node1 | ok: [node3] 2026-08-18 13:44:03.978187 | node1 | 2026-08-18 13:44:03.978246 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:44:03.978284 | node1 | Tuesday 18 August 2026 13:44:03 +0000 (0:00:00.084) 0:12:58.927 ******** 2026-08-18 13:44:04.063058 | node1 | ok: [node1] => { 2026-08-18 13:44:04.063137 | node1 | "changed": false, 2026-08-18 13:44:04.065001 | node1 | "msg": "All assertions passed" 2026-08-18 13:44:04.065057 | node1 | } 2026-08-18 13:44:04.065067 | node1 | ok: [node2] => { 2026-08-18 13:44:04.065075 | node1 | "changed": false, 2026-08-18 13:44:04.065085 | node1 | "msg": "All assertions passed" 2026-08-18 13:44:04.065093 | node1 | } 2026-08-18 13:44:04.065102 | node1 | ok: [node3] => { 2026-08-18 13:44:04.065254 | node1 | "changed": false, 2026-08-18 13:44:04.065561 | node1 | "msg": "All assertions passed" 2026-08-18 13:44:04.065856 | node1 | } 2026-08-18 13:44:04.066177 | node1 | 2026-08-18 13:44:04.066516 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:44:04.066828 | node1 | Tuesday 18 August 2026 13:44:04 +0000 (0:00:00.085) 0:12:59.013 ******** 2026-08-18 13:44:04.418384 | node1 | ok: [node1] 2026-08-18 13:44:04.418448 | node1 | ok: [node2] 2026-08-18 13:44:04.418540 | node1 | ok: [node3] 2026-08-18 13:44:04.418647 | node1 | 2026-08-18 13:44:04.418757 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:44:04.418866 | node1 | Tuesday 18 August 2026 13:44:04 +0000 (0:00:00.356) 0:12:59.369 ******** 2026-08-18 13:44:04.507895 | node1 | ok: [node1] 2026-08-18 13:44:04.507985 | node1 | ok: [node2] 2026-08-18 13:44:04.508627 | node1 | ok: [node3] 2026-08-18 13:44:04.508715 | node1 | 2026-08-18 13:44:04.509001 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:44:04.509207 | node1 | Tuesday 18 August 2026 13:44:04 +0000 (0:00:00.088) 0:12:59.458 ******** 2026-08-18 13:44:04.632181 | node1 | ok: [node1] 2026-08-18 13:44:04.632229 | node1 | ok: [node2] 2026-08-18 13:44:04.632234 | node1 | ok: [node3] 2026-08-18 13:44:04.632239 | node1 | 2026-08-18 13:44:04.632244 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:44:04.632250 | node1 | Tuesday 18 August 2026 13:44:04 +0000 (0:00:00.124) 0:12:59.582 ******** 2026-08-18 13:44:04.717738 | node1 | ok: [node1] => { 2026-08-18 13:44:04.717786 | node1 | "changed": false, 2026-08-18 13:44:04.717792 | node1 | "msg": "All assertions passed" 2026-08-18 13:44:04.717797 | node1 | } 2026-08-18 13:44:04.717802 | node1 | ok: [node2] => { 2026-08-18 13:44:04.717807 | node1 | "changed": false, 2026-08-18 13:44:04.717811 | node1 | "msg": "All assertions passed" 2026-08-18 13:44:04.717816 | node1 | } 2026-08-18 13:44:04.717821 | node1 | ok: [node3] => { 2026-08-18 13:44:04.717825 | node1 | "changed": false, 2026-08-18 13:44:04.717830 | node1 | "msg": "All assertions passed" 2026-08-18 13:44:04.717834 | node1 | } 2026-08-18 13:44:04.717839 | node1 | 2026-08-18 13:44:04.717844 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:44:04.717849 | node1 | Tuesday 18 August 2026 13:44:04 +0000 (0:00:00.083) 0:12:59.666 ******** 2026-08-18 13:44:06.628851 | node1 | ok: [node1] 2026-08-18 13:44:06.628916 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:44:06.628970 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:44:06.628987 | node1 | 2026-08-18 13:44:06.629000 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:44:06.629012 | node1 | Tuesday 18 August 2026 13:44:06 +0000 (0:00:01.910) 0:13:01.576 ******** 2026-08-18 13:44:06.754116 | node1 | skipping: [node1] 2026-08-18 13:44:06.754178 | node1 | skipping: [node2] 2026-08-18 13:44:06.755170 | node1 | skipping: [node3] 2026-08-18 13:44:06.755220 | node1 | 2026-08-18 13:44:06.755234 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW multisite (realm, zonegroup, zone)] **** 2026-08-18 13:44:06.755247 | node1 | Tuesday 18 August 2026 13:44:06 +0000 (0:00:00.127) 0:13:01.704 ******** 2026-08-18 13:44:06.818314 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/multisite.yml for node1 2026-08-18 13:44:06.818346 | node1 | 2026-08-18 13:44:06.818359 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW realms] ******************************** 2026-08-18 13:44:06.818371 | node1 | Tuesday 18 August 2026 13:44:06 +0000 (0:00:00.064) 0:13:01.768 ******** 2026-08-18 13:44:11.537666 | node1 | changed: [node1] => (item=molecule) 2026-08-18 13:44:11.537723 | node1 | 2026-08-18 13:44:11.537738 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zonegroups] **************************** 2026-08-18 13:44:11.537752 | node1 | Tuesday 18 August 2026 13:44:11 +0000 (0:00:04.718) 0:13:06.486 ******** 2026-08-18 13:44:14.230344 | node1 | changed: [node1] => (item=molecule) 2026-08-18 13:44:14.230408 | node1 | 2026-08-18 13:44:14.230419 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zones] ********************************* 2026-08-18 13:44:14.230428 | node1 | Tuesday 18 August 2026 13:44:14 +0000 (0:00:02.693) 0:13:09.180 ******** 2026-08-18 13:44:16.933097 | node1 | changed: [node1] => (item=None) 2026-08-18 13:44:16.934006 | node1 | changed: [node1 -> {{ ceph_rgw_admin_host }}] 2026-08-18 13:44:16.934032 | node1 | 2026-08-18 13:44:16.934043 | node1 | TASK [vexxhost.ceph.rgw : Commit the RGW period] ******************************* 2026-08-18 13:44:16.934057 | node1 | Tuesday 18 August 2026 13:44:16 +0000 (0:00:02.703) 0:13:11.883 ******** 2026-08-18 13:44:18.854285 | node1 | changed: [node1] 2026-08-18 13:44:18.854351 | node1 | 2026-08-18 13:44:18.854468 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 13:44:18.854580 | node1 | Tuesday 18 August 2026 13:44:18 +0000 (0:00:01.921) 0:13:13.805 ******** 2026-08-18 13:44:18.919732 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/services.yml for node1 2026-08-18 13:44:18.919804 | node1 | 2026-08-18 13:44:18.919818 | node1 | TASK [vexxhost.ceph.rgw : Snapshot current RGW service specs] ****************** 2026-08-18 13:44:18.919831 | node1 | Tuesday 18 August 2026 13:44:18 +0000 (0:00:00.064) 0:13:13.869 ******** 2026-08-18 13:44:20.660201 | node1 | ok: [node1] 2026-08-18 13:44:20.660262 | node1 | 2026-08-18 13:44:20.660844 | node1 | TASK [vexxhost.ceph.rgw : Index current RGW specs by service name] ************* 2026-08-18 13:44:20.660913 | node1 | Tuesday 18 August 2026 13:44:20 +0000 (0:00:01.741) 0:13:15.610 ******** 2026-08-18 13:44:20.717539 | node1 | ok: [node1] 2026-08-18 13:44:20.717626 | node1 | 2026-08-18 13:44:20.717752 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 13:44:20.717911 | node1 | Tuesday 18 August 2026 13:44:20 +0000 (0:00:00.057) 0:13:15.668 ******** 2026-08-18 13:44:24.124143 | node1 | changed: [node1] => (item=molecule) 2026-08-18 13:44:24.124221 | node1 | 2026-08-18 13:44:24.124352 | node1 | TASK [vexxhost.ceph.rgw : Redeploy RGW services whose redeploy-triggering keys changed] *** 2026-08-18 13:44:24.124467 | node1 | Tuesday 18 August 2026 13:44:24 +0000 (0:00:03.404) 0:13:19.072 ******** 2026-08-18 13:44:24.178180 | node1 | skipping: [node1] => (item=rgw.molecule) 2026-08-18 13:44:24.178294 | node1 | skipping: [node1] 2026-08-18 13:44:24.178397 | node1 | 2026-08-18 13:44:24.178516 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone / zonegroup / pools] ********* 2026-08-18 13:44:24.178629 | node1 | Tuesday 18 August 2026 13:44:24 +0000 (0:00:00.056) 0:13:19.129 ******** 2026-08-18 13:44:24.249145 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/cleanup.yml for node1 2026-08-18 13:44:24.249246 | node1 | 2026-08-18 13:44:24.249361 | node1 | TASK [vexxhost.ceph.rgw : List RGW zones] ************************************** 2026-08-18 13:44:24.249474 | node1 | Tuesday 18 August 2026 13:44:24 +0000 (0:00:00.071) 0:13:19.200 ******** 2026-08-18 13:44:26.226918 | node1 | ok: [node1] 2026-08-18 13:44:26.227037 | node1 | 2026-08-18 13:44:26.227052 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zonegroup] ************************ 2026-08-18 13:44:26.227189 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:01.964) 0:13:21.165 ******** 2026-08-18 13:44:26.262269 | node1 | skipping: [node1] 2026-08-18 13:44:26.262486 | node1 | 2026-08-18 13:44:26.262774 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone] ***************************** 2026-08-18 13:44:26.263032 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:00.048) 0:13:21.213 ******** 2026-08-18 13:44:26.310241 | node1 | skipping: [node1] 2026-08-18 13:44:26.310493 | node1 | 2026-08-18 13:44:26.310750 | node1 | TASK [vexxhost.ceph.rgw : Get the current pool deletion setting] *************** 2026-08-18 13:44:26.311072 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:00.048) 0:13:21.261 ******** 2026-08-18 13:44:26.362139 | node1 | skipping: [node1] 2026-08-18 13:44:26.362434 | node1 | 2026-08-18 13:44:26.362788 | node1 | TASK [vexxhost.ceph.rgw : Allow pool deletion] ********************************* 2026-08-18 13:44:26.363220 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:00.051) 0:13:21.313 ******** 2026-08-18 13:44:26.417446 | node1 | skipping: [node1] 2026-08-18 13:44:26.417564 | node1 | 2026-08-18 13:44:26.417773 | node1 | TASK [vexxhost.ceph.rgw : Remove the default RGW pools] ************************ 2026-08-18 13:44:26.417979 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:00.055) 0:13:21.368 ******** 2026-08-18 13:44:26.498823 | node1 | skipping: [node1] => (item=default.rgw.control) 2026-08-18 13:44:26.518574 | node1 | skipping: [node1] => (item=default.rgw.log) 2026-08-18 13:44:26.518875 | node1 | skipping: [node1] => (item=default.rgw.meta) 2026-08-18 13:44:26.519004 | node1 | skipping: [node1] 2026-08-18 13:44:26.519137 | node1 | 2026-08-18 13:44:26.519254 | node1 | TASK [vexxhost.ceph.rgw : Restore the configured pool deletion setting] ******** 2026-08-18 13:44:26.519366 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:00.081) 0:13:21.449 ******** 2026-08-18 13:44:26.555389 | node1 | skipping: [node1] 2026-08-18 13:44:26.555673 | node1 | 2026-08-18 13:44:26.556003 | node1 | TASK [vexxhost.ceph.rgw : Remove the temporary pool deletion setting] ********** 2026-08-18 13:44:26.556243 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:00.056) 0:13:21.506 ******** 2026-08-18 13:44:26.662370 | node1 | skipping: [node1] 2026-08-18 13:44:26.662536 | node1 | 2026-08-18 13:44:26.662781 | node1 | PLAY [Deploy Ceph monitoring stack] ******************************************** 2026-08-18 13:44:26.663046 | node1 | 2026-08-18 13:44:26.663277 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:44:26.663513 | node1 | Tuesday 18 August 2026 13:44:26 +0000 (0:00:00.107) 0:13:21.613 ******** 2026-08-18 13:44:28.714911 | node1 | ok: [node2] 2026-08-18 13:44:28.715141 | node1 | ok: [node3] 2026-08-18 13:44:28.715421 | node1 | ok: [node1] 2026-08-18 13:44:28.715601 | node1 | 2026-08-18 13:44:28.715992 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:44:28.716184 | node1 | Tuesday 18 August 2026 13:44:28 +0000 (0:00:02.051) 0:13:23.664 ******** 2026-08-18 13:44:29.050926 | node1 | ok: [node1] 2026-08-18 13:44:29.051169 | node1 | ok: [node2] 2026-08-18 13:44:29.051406 | node1 | ok: [node3] 2026-08-18 13:44:29.051630 | node1 | 2026-08-18 13:44:29.051895 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:44:29.052188 | node1 | Tuesday 18 August 2026 13:44:29 +0000 (0:00:00.336) 0:13:24.001 ******** 2026-08-18 13:44:29.168514 | node1 | skipping: [node1] 2026-08-18 13:44:29.227513 | node1 | skipping: [node2] 2026-08-18 13:44:29.227552 | node1 | skipping: [node3] 2026-08-18 13:44:29.227668 | node1 | 2026-08-18 13:44:29.229060 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:44:29.229128 | node1 | Tuesday 18 August 2026 13:44:29 +0000 (0:00:00.176) 0:13:24.178 ******** 2026-08-18 13:44:29.554263 | node1 | ok: [node1] 2026-08-18 13:44:29.554448 | node1 | ok: [node3] 2026-08-18 13:44:29.554681 | node1 | ok: [node2] 2026-08-18 13:44:29.554903 | node1 | 2026-08-18 13:44:29.555212 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:44:29.555452 | node1 | Tuesday 18 August 2026 13:44:29 +0000 (0:00:00.326) 0:13:24.504 ******** 2026-08-18 13:44:29.662007 | node1 | ok: [node1] => { 2026-08-18 13:44:29.662058 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:44:29.662068 | node1 | } 2026-08-18 13:44:29.662076 | node1 | ok: [node2] => { 2026-08-18 13:44:29.662085 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:44:29.662093 | node1 | } 2026-08-18 13:44:29.662101 | node1 | ok: [node3] => { 2026-08-18 13:44:29.662109 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:44:29.662143 | node1 | } 2026-08-18 13:44:29.662266 | node1 | 2026-08-18 13:44:29.662578 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:44:29.662881 | node1 | Tuesday 18 August 2026 13:44:29 +0000 (0:00:00.104) 0:13:24.609 ******** 2026-08-18 13:44:30.089308 | node1 | ok: [node1] 2026-08-18 13:44:30.089668 | node1 | ok: [node2] 2026-08-18 13:44:30.090004 | node1 | ok: [node3] 2026-08-18 13:44:30.090298 | node1 | 2026-08-18 13:44:30.090652 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:44:30.090991 | node1 | Tuesday 18 August 2026 13:44:30 +0000 (0:00:00.429) 0:13:25.039 ******** 2026-08-18 13:44:30.208555 | node1 | skipping: [node1] 2026-08-18 13:44:30.208913 | node1 | skipping: [node2] 2026-08-18 13:44:30.209276 | node1 | skipping: [node3] 2026-08-18 13:44:30.209565 | node1 | 2026-08-18 13:44:30.209874 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:44:30.210220 | node1 | Tuesday 18 August 2026 13:44:30 +0000 (0:00:00.119) 0:13:25.159 ******** 2026-08-18 13:44:30.323146 | node1 | skipping: [node1] 2026-08-18 13:44:30.323502 | node1 | skipping: [node2] 2026-08-18 13:44:30.323802 | node1 | skipping: [node3] 2026-08-18 13:44:30.324125 | node1 | 2026-08-18 13:44:30.324444 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:44:30.324746 | node1 | Tuesday 18 August 2026 13:44:30 +0000 (0:00:00.114) 0:13:25.274 ******** 2026-08-18 13:44:30.634627 | node1 | ok: [node1] 2026-08-18 13:44:30.634721 | node1 | ok: [node2] 2026-08-18 13:44:30.634824 | node1 | ok: [node3] 2026-08-18 13:44:30.634932 | node1 | 2026-08-18 13:44:30.635071 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:44:30.636850 | node1 | Tuesday 18 August 2026 13:44:30 +0000 (0:00:00.310) 0:13:25.585 ******** 2026-08-18 13:44:31.847167 | node1 | ok: [node2] 2026-08-18 13:44:31.847352 | node1 | ok: [node3] 2026-08-18 13:44:31.847584 | node1 | ok: [node1] 2026-08-18 13:44:31.847802 | node1 | 2026-08-18 13:44:31.848106 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:44:31.848345 | node1 | Tuesday 18 August 2026 13:44:31 +0000 (0:00:01.213) 0:13:26.798 ******** 2026-08-18 13:44:31.961416 | node1 | ok: [node1] => { 2026-08-18 13:44:31.961729 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:44:31.962095 | node1 | } 2026-08-18 13:44:31.962339 | node1 | ok: [node2] => { 2026-08-18 13:44:31.962595 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:44:31.962823 | node1 | } 2026-08-18 13:44:31.963182 | node1 | ok: [node3] => { 2026-08-18 13:44:31.963429 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:44:31.963656 | node1 | } 2026-08-18 13:44:31.963874 | node1 | 2026-08-18 13:44:31.964156 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:44:31.964398 | node1 | Tuesday 18 August 2026 13:44:31 +0000 (0:00:00.114) 0:13:26.912 ******** 2026-08-18 13:44:32.420539 | node1 | ok: [node1] 2026-08-18 13:44:32.420595 | node1 | ok: [node2] 2026-08-18 13:44:32.420761 | node1 | ok: [node3] 2026-08-18 13:44:32.421115 | node1 | 2026-08-18 13:44:32.421376 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:44:32.421695 | node1 | Tuesday 18 August 2026 13:44:32 +0000 (0:00:00.458) 0:13:27.371 ******** 2026-08-18 13:44:34.686028 | node1 | ok: [node2] 2026-08-18 13:44:34.686227 | node1 | ok: [node1] 2026-08-18 13:44:34.686552 | node1 | ok: [node3] 2026-08-18 13:44:34.686841 | node1 | 2026-08-18 13:44:34.687204 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:44:34.687526 | node1 | Tuesday 18 August 2026 13:44:34 +0000 (0:00:02.265) 0:13:29.636 ******** 2026-08-18 13:44:34.720897 | node1 | skipping: [node1] 2026-08-18 13:44:34.752162 | node1 | skipping: [node2] 2026-08-18 13:44:34.752227 | node1 | skipping: [node3] 2026-08-18 13:44:34.752240 | node1 | 2026-08-18 13:44:34.752254 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:44:34.752295 | node1 | Tuesday 18 August 2026 13:44:34 +0000 (0:00:00.065) 0:13:29.702 ******** 2026-08-18 13:44:34.818106 | node1 | skipping: [node1] 2026-08-18 13:44:34.818326 | node1 | skipping: [node2] 2026-08-18 13:44:34.818561 | node1 | skipping: [node3] 2026-08-18 13:44:34.818846 | node1 | 2026-08-18 13:44:34.819175 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:44:34.819414 | node1 | Tuesday 18 August 2026 13:44:34 +0000 (0:00:00.066) 0:13:29.769 ******** 2026-08-18 13:44:34.893427 | node1 | skipping: [node1] 2026-08-18 13:44:34.893584 | node1 | skipping: [node2] 2026-08-18 13:44:34.893813 | node1 | skipping: [node3] 2026-08-18 13:44:34.894084 | node1 | 2026-08-18 13:44:34.894319 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:44:34.894561 | node1 | Tuesday 18 August 2026 13:44:34 +0000 (0:00:00.075) 0:13:29.844 ******** 2026-08-18 13:44:36.019292 | node1 | ok: [node2] 2026-08-18 13:44:36.019511 | node1 | ok: [node1] 2026-08-18 13:44:36.019750 | node1 | ok: [node3] 2026-08-18 13:44:36.020003 | node1 | 2026-08-18 13:44:36.020273 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:44:36.020516 | node1 | Tuesday 18 August 2026 13:44:36 +0000 (0:00:01.125) 0:13:30.970 ******** 2026-08-18 13:44:36.613833 | node1 | ok: [node1] 2026-08-18 13:44:36.614191 | node1 | ok: [node3] 2026-08-18 13:44:36.614480 | node1 | ok: [node2] 2026-08-18 13:44:36.614739 | node1 | 2026-08-18 13:44:36.615077 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:44:36.615362 | node1 | Tuesday 18 August 2026 13:44:36 +0000 (0:00:00.594) 0:13:31.564 ******** 2026-08-18 13:44:37.801002 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:44:37.801072 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:44:37.801078 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:44:37.801194 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:44:37.801455 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:44:37.801716 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:44:37.802017 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:44:37.802308 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:44:37.802608 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:44:37.802907 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:44:37.803299 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:44:37.803531 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:44:37.803875 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:44:37.804228 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:44:37.804555 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:44:37.804795 | node1 | 2026-08-18 13:44:37.805145 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:44:37.805419 | node1 | Tuesday 18 August 2026 13:44:37 +0000 (0:00:01.185) 0:13:32.750 ******** 2026-08-18 13:44:38.546579 | node1 | ok: [node1] 2026-08-18 13:44:38.546679 | node1 | ok: [node2] 2026-08-18 13:44:38.546692 | node1 | ok: [node3] 2026-08-18 13:44:38.546767 | node1 | 2026-08-18 13:44:38.546884 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:44:38.547035 | node1 | Tuesday 18 August 2026 13:44:38 +0000 (0:00:00.723) 0:13:33.474 ******** 2026-08-18 13:44:38.547143 | node1 | 2026-08-18 13:44:38.547319 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:44:38.547411 | node1 | Tuesday 18 August 2026 13:44:38 +0000 (0:00:00.007) 0:13:33.481 ******** 2026-08-18 13:44:38.547514 | node1 | 2026-08-18 13:44:38.547627 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:44:38.547739 | node1 | Tuesday 18 August 2026 13:44:38 +0000 (0:00:00.008) 0:13:33.489 ******** 2026-08-18 13:44:38.547840 | node1 | 2026-08-18 13:44:38.547982 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:44:38.548093 | node1 | Tuesday 18 August 2026 13:44:38 +0000 (0:00:00.006) 0:13:33.496 ******** 2026-08-18 13:44:39.047106 | node1 | ok: [node1] 2026-08-18 13:44:39.047200 | node1 | ok: [node2] 2026-08-18 13:44:39.047261 | node1 | ok: [node3] 2026-08-18 13:44:39.047374 | node1 | 2026-08-18 13:44:39.047495 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:44:39.047609 | node1 | Tuesday 18 August 2026 13:44:39 +0000 (0:00:00.501) 0:13:33.997 ******** 2026-08-18 13:44:39.341103 | node1 | ok: [node1] 2026-08-18 13:44:39.341162 | node1 | ok: [node3] 2026-08-18 13:44:39.342086 | node1 | ok: [node2] 2026-08-18 13:44:39.342145 | node1 | 2026-08-18 13:44:39.342155 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:44:39.342165 | node1 | Tuesday 18 August 2026 13:44:39 +0000 (0:00:00.292) 0:13:34.290 ******** 2026-08-18 13:44:39.438465 | node1 | ok: [node1] => { 2026-08-18 13:44:39.438556 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:44:39.439678 | node1 | } 2026-08-18 13:44:39.439729 | node1 | ok: [node2] => { 2026-08-18 13:44:39.439734 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:44:39.439738 | node1 | } 2026-08-18 13:44:39.439742 | node1 | ok: [node3] => { 2026-08-18 13:44:39.439746 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:44:39.439750 | node1 | } 2026-08-18 13:44:39.439754 | node1 | 2026-08-18 13:44:39.439758 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:44:39.439763 | node1 | Tuesday 18 August 2026 13:44:39 +0000 (0:00:00.098) 0:13:34.389 ******** 2026-08-18 13:44:39.908270 | node1 | ok: [node1] 2026-08-18 13:44:39.908323 | node1 | ok: [node2] 2026-08-18 13:44:39.908333 | node1 | ok: [node3] 2026-08-18 13:44:39.908342 | node1 | 2026-08-18 13:44:39.908352 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:44:39.908361 | node1 | Tuesday 18 August 2026 13:44:39 +0000 (0:00:00.468) 0:13:34.857 ******** 2026-08-18 13:44:43.276654 | node1 | ok: [node3] 2026-08-18 13:44:43.276727 | node1 | ok: [node1] 2026-08-18 13:44:43.276740 | node1 | ok: [node2] 2026-08-18 13:44:43.276752 | node1 | 2026-08-18 13:44:43.276765 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:44:43.276777 | node1 | Tuesday 18 August 2026 13:44:43 +0000 (0:00:03.368) 0:13:38.226 ******** 2026-08-18 13:44:44.375047 | node1 | ok: [node2] 2026-08-18 13:44:44.375105 | node1 | ok: [node3] 2026-08-18 13:44:44.375118 | node1 | ok: [node1] 2026-08-18 13:44:44.375130 | node1 | 2026-08-18 13:44:44.375142 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:44:44.375161 | node1 | Tuesday 18 August 2026 13:44:44 +0000 (0:00:01.098) 0:13:39.324 ******** 2026-08-18 13:44:44.641034 | node1 | ok: [node1] 2026-08-18 13:44:44.642240 | node1 | ok: [node2] 2026-08-18 13:44:44.642335 | node1 | ok: [node3] 2026-08-18 13:44:44.642348 | node1 | 2026-08-18 13:44:44.642360 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:44:44.642373 | node1 | Tuesday 18 August 2026 13:44:44 +0000 (0:00:00.266) 0:13:39.591 ******** 2026-08-18 13:44:45.234138 | node1 | ok: [node1] 2026-08-18 13:44:45.234195 | node1 | ok: [node2] 2026-08-18 13:44:45.234208 | node1 | ok: [node3] 2026-08-18 13:44:45.234220 | node1 | 2026-08-18 13:44:45.234233 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:44:45.234245 | node1 | Tuesday 18 August 2026 13:44:45 +0000 (0:00:00.591) 0:13:40.182 ******** 2026-08-18 13:44:45.951093 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:44:45.953002 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:44:45.953094 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:44:45.953104 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:44:45.953113 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:44:45.953121 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:44:45.953130 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:44:45.953139 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:44:45.953389 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:44:45.953708 | node1 | 2026-08-18 13:44:45.954063 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:44:45.954390 | node1 | Tuesday 18 August 2026 13:44:45 +0000 (0:00:00.718) 0:13:40.900 ******** 2026-08-18 13:44:46.524917 | node1 | ok: [node1] 2026-08-18 13:44:46.525357 | node1 | ok: [node2] 2026-08-18 13:44:46.525663 | node1 | ok: [node3] 2026-08-18 13:44:46.525984 | node1 | 2026-08-18 13:44:46.526322 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:44:46.526716 | node1 | Tuesday 18 August 2026 13:44:46 +0000 (0:00:00.574) 0:13:41.474 ******** 2026-08-18 13:44:47.182816 | node1 | ok: [node1] 2026-08-18 13:44:47.183228 | node1 | ok: [node2] 2026-08-18 13:44:47.183545 | node1 | ok: [node3] 2026-08-18 13:44:47.183895 | node1 | 2026-08-18 13:44:47.184247 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:44:47.184567 | node1 | Tuesday 18 August 2026 13:44:47 +0000 (0:00:00.567) 0:13:42.041 ******** 2026-08-18 13:44:47.184850 | node1 | 2026-08-18 13:44:47.185192 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:44:47.185510 | node1 | Tuesday 18 August 2026 13:44:47 +0000 (0:00:00.007) 0:13:42.049 ******** 2026-08-18 13:44:47.185789 | node1 | 2026-08-18 13:44:47.186127 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:44:47.186442 | node1 | Tuesday 18 August 2026 13:44:47 +0000 (0:00:00.010) 0:13:42.059 ******** 2026-08-18 13:44:47.186720 | node1 | 2026-08-18 13:44:47.187057 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:44:47.187370 | node1 | Tuesday 18 August 2026 13:44:47 +0000 (0:00:00.072) 0:13:42.132 ******** 2026-08-18 13:44:47.706845 | node1 | ok: [node1] 2026-08-18 13:44:47.707270 | node1 | ok: [node2] 2026-08-18 13:44:47.707596 | node1 | ok: [node3] 2026-08-18 13:44:47.708679 | node1 | 2026-08-18 13:44:47.708708 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:44:47.708718 | node1 | Tuesday 18 August 2026 13:44:47 +0000 (0:00:00.524) 0:13:42.656 ******** 2026-08-18 13:44:47.789033 | node1 | ok: [node1] => { 2026-08-18 13:44:47.790092 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:44:47.790164 | node1 | } 2026-08-18 13:44:47.790175 | node1 | ok: [node2] => { 2026-08-18 13:44:47.790184 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:44:47.790199 | node1 | } 2026-08-18 13:44:47.791011 | node1 | ok: [node3] => { 2026-08-18 13:44:47.791067 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:44:47.791077 | node1 | } 2026-08-18 13:44:47.791301 | node1 | 2026-08-18 13:44:47.791632 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:44:47.791981 | node1 | Tuesday 18 August 2026 13:44:47 +0000 (0:00:00.082) 0:13:42.738 ******** 2026-08-18 13:44:48.163844 | node1 | ok: [node1] 2026-08-18 13:44:48.164283 | node1 | ok: [node2] 2026-08-18 13:44:48.164623 | node1 | ok: [node3] 2026-08-18 13:44:48.164977 | node1 | 2026-08-18 13:44:48.165707 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:44:48.165729 | node1 | Tuesday 18 August 2026 13:44:48 +0000 (0:00:00.375) 0:13:43.114 ******** 2026-08-18 13:44:48.256310 | node1 | skipping: [node1] 2026-08-18 13:44:48.256370 | node1 | skipping: [node2] 2026-08-18 13:44:48.256380 | node1 | skipping: [node3] 2026-08-18 13:44:48.256390 | node1 | 2026-08-18 13:44:48.256427 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:44:48.256437 | node1 | Tuesday 18 August 2026 13:44:48 +0000 (0:00:00.090) 0:13:43.205 ******** 2026-08-18 13:44:48.293906 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:44:48.362130 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:44:48.362205 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:44:48.362216 | node1 | 2026-08-18 13:44:48.362226 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:44:48.362328 | node1 | Tuesday 18 August 2026 13:44:48 +0000 (0:00:00.106) 0:13:43.312 ******** 2026-08-18 13:44:49.427085 | node1 | ok: [node3] 2026-08-18 13:44:49.427191 | node1 | ok: [node2] 2026-08-18 13:44:49.427556 | node1 | ok: [node1] 2026-08-18 13:44:49.427800 | node1 | 2026-08-18 13:44:49.428192 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:44:49.428462 | node1 | Tuesday 18 August 2026 13:44:49 +0000 (0:00:01.065) 0:13:44.377 ******** 2026-08-18 13:44:50.276028 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:44:50.276240 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:44:50.276521 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:44:50.276762 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:44:50.277049 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:44:50.277285 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:44:50.277504 | node1 | 2026-08-18 13:44:50.277752 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:44:50.278051 | node1 | Tuesday 18 August 2026 13:44:50 +0000 (0:00:00.848) 0:13:45.226 ******** 2026-08-18 13:44:50.551821 | node1 | ok: [node2] 2026-08-18 13:44:50.552097 | node1 | ok: [node1] 2026-08-18 13:44:50.552338 | node1 | ok: [node3] 2026-08-18 13:44:50.552564 | node1 | 2026-08-18 13:44:50.552910 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:44:50.553207 | node1 | Tuesday 18 August 2026 13:44:50 +0000 (0:00:00.276) 0:13:45.503 ******** 2026-08-18 13:44:50.881865 | node1 | ok: [node2] 2026-08-18 13:44:50.882105 | node1 | ok: [node1] 2026-08-18 13:44:50.882427 | node1 | ok: [node3] 2026-08-18 13:44:50.882730 | node1 | 2026-08-18 13:44:50.883034 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:44:50.883306 | node1 | Tuesday 18 August 2026 13:44:50 +0000 (0:00:00.329) 0:13:45.832 ******** 2026-08-18 13:44:51.474334 | node1 | ok: [node1] 2026-08-18 13:44:51.474431 | node1 | ok: [node2] 2026-08-18 13:44:51.474591 | node1 | ok: [node3] 2026-08-18 13:44:51.474735 | node1 | 2026-08-18 13:44:51.474898 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:44:51.475095 | node1 | Tuesday 18 August 2026 13:44:51 +0000 (0:00:00.593) 0:13:46.425 ******** 2026-08-18 13:44:51.553488 | node1 | skipping: [node1] 2026-08-18 13:44:51.553627 | node1 | skipping: [node2] 2026-08-18 13:44:51.553805 | node1 | skipping: [node3] 2026-08-18 13:44:51.554046 | node1 | 2026-08-18 13:44:51.554236 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:44:51.554406 | node1 | Tuesday 18 August 2026 13:44:51 +0000 (0:00:00.078) 0:13:46.504 ******** 2026-08-18 13:44:52.025700 | node1 | ok: [node2] 2026-08-18 13:44:52.025905 | node1 | ok: [node1] 2026-08-18 13:44:52.026187 | node1 | ok: [node3] 2026-08-18 13:44:52.026414 | node1 | 2026-08-18 13:44:52.026776 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:44:52.027110 | node1 | Tuesday 18 August 2026 13:44:52 +0000 (0:00:00.472) 0:13:46.976 ******** 2026-08-18 13:44:52.612093 | node1 | ok: [node1] 2026-08-18 13:44:52.612149 | node1 | ok: [node2] 2026-08-18 13:44:52.612168 | node1 | ok: [node3] 2026-08-18 13:44:52.612925 | node1 | 2026-08-18 13:44:52.612994 | node1 | TASK [vexxhost.ceph.monitoring : Locate the cluster admin host] **************** 2026-08-18 13:44:52.613008 | node1 | Tuesday 18 August 2026 13:44:52 +0000 (0:00:00.585) 0:13:47.562 ******** 2026-08-18 13:44:53.378271 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:44:53.378324 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:44:53.378335 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:44:53.378345 | node1 | 2026-08-18 13:44:53.378355 | node1 | TASK [vexxhost.ceph.monitoring : Select the admin host] ************************ 2026-08-18 13:44:53.378365 | node1 | Tuesday 18 August 2026 13:44:53 +0000 (0:00:00.765) 0:13:48.327 ******** 2026-08-18 13:44:53.473848 | node1 | ok: [node1] 2026-08-18 13:44:53.473880 | node1 | ok: [node2] 2026-08-18 13:44:53.473890 | node1 | ok: [node3] 2026-08-18 13:44:53.473899 | node1 | 2026-08-18 13:44:53.473908 | node1 | TASK [vexxhost.ceph.monitoring : Set the admin mon config path] **************** 2026-08-18 13:44:53.473917 | node1 | Tuesday 18 August 2026 13:44:53 +0000 (0:00:00.096) 0:13:48.423 ******** 2026-08-18 13:44:53.563506 | node1 | ok: [node1] 2026-08-18 13:44:53.563570 | node1 | ok: [node2] 2026-08-18 13:44:53.563673 | node1 | ok: [node3] 2026-08-18 13:44:53.563779 | node1 | 2026-08-18 13:44:53.563891 | node1 | TASK [vexxhost.ceph.monitoring : Pin monitoring container images] ************** 2026-08-18 13:44:53.564030 | node1 | Tuesday 18 August 2026 13:44:53 +0000 (0:00:00.089) 0:13:48.513 ******** 2026-08-18 13:44:53.599063 | node1 | skipping: [node1] 2026-08-18 13:44:53.599136 | node1 | 2026-08-18 13:44:53.599625 | node1 | TASK [vexxhost.ceph.monitoring : Apply the monitoring service specs] *********** 2026-08-18 13:44:53.599666 | node1 | Tuesday 18 August 2026 13:44:53 +0000 (0:00:00.035) 0:13:48.548 ******** 2026-08-18 13:44:56.989141 | node1 | changed: [node1] => (item=ceph-exporter) 2026-08-18 13:44:56.989220 | node1 | 2026-08-18 13:44:56.991002 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 13:44:56.991065 | node1 | node1 : ok=246 changed=47 unreachable=0 failed=0 skipped=54 rescued=0 ignored=0 2026-08-18 13:44:56.991078 | node1 | node2 : ok=206 changed=27 unreachable=0 failed=0 skipped=45 rescued=0 ignored=0 2026-08-18 13:44:56.991103 | node1 | node3 : ok=206 changed=27 unreachable=0 failed=0 skipped=45 rescued=0 ignored=0 2026-08-18 13:44:56.991117 | node1 | 2026-08-18 13:44:56.991128 | node1 | 2026-08-18 13:44:56.991140 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 13:44:56.991429 | node1 | Tuesday 18 August 2026 13:44:56 +0000 (0:00:03.391) 0:13:51.939 ******** 2026-08-18 13:44:56.991736 | node1 | =============================================================================== 2026-08-18 13:44:56.992084 | node1 | vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices ------ 364.18s 2026-08-18 13:44:56.992401 | node1 | vexxhost.ceph.mon : Bootstrap the cluster ------------------------------ 47.90s 2026-08-18 13:44:56.992984 | node1 | vexxhost.ceph.cephadm : Pre-pull the Ceph image ------------------------ 42.27s 2026-08-18 13:44:56.993340 | node1 | vexxhost.ceph.configure : Configure pools ------------------------------ 36.97s 2026-08-18 13:44:56.993682 | node1 | vexxhost.containers.containerd : Install AppArmor packages ------------- 34.39s 2026-08-18 13:44:56.994251 | node1 | vexxhost.ceph.cephadm_host : Register the host (labels) ---------------- 22.47s 2026-08-18 13:44:56.994634 | node1 | vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host -- 21.92s 2026-08-18 13:44:56.995232 | node1 | vexxhost.ceph.configure : Apply central Ceph configuration ------------- 19.56s 2026-08-18 13:44:56.996048 | node1 | vexxhost.ceph.mgr : Configure ceph-mgr modules ------------------------- 12.56s 2026-08-18 13:44:56.996485 | node1 | vexxhost.ceph.configure : Configure CRUSH rules ------------------------ 11.65s 2026-08-18 13:44:56.996853 | node1 | vexxhost.ceph.configure : Configure dashboard users -------------------- 10.69s 2026-08-18 13:44:56.997245 | node1 | vexxhost.ceph.mon : Wait for this monitor's daemon to be present -------- 8.09s 2026-08-18 13:44:56.997614 | node1 | vexxhost.ceph.mgr : Apply the manager service spec ---------------------- 7.96s 2026-08-18 13:44:56.998092 | node1 | vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate --- 7.19s 2026-08-18 13:44:56.998433 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 4.83s 2026-08-18 13:44:56.998576 | node1 | vexxhost.ceph.rgw : Configure RGW realms -------------------------------- 4.72s 2026-08-18 13:44:56.998713 | node1 | vexxhost.ceph.mon : Apply the monitor service spec ---------------------- 4.02s 2026-08-18 13:44:56.998861 | node1 | vexxhost.ceph.configure : Configure erasure-code profiles --------------- 3.97s 2026-08-18 13:44:56.999038 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.42s 2026-08-18 13:44:56.999189 | node1 | vexxhost.ceph.rgw : Apply the RGW service specs ------------------------- 3.40s 2026-08-18 13:44:57.217353 | node1 | INFO [default > converge] Executed: Successful 2026-08-18 13:44:57.228615 | node1 | INFO [default > idempotence] Executing 2026-08-18 13:44:58.478826 | node1 | 2026-08-18 13:44:58.479321 | node1 | PLAY [Deploy Ceph monitors & managers] ***************************************** 2026-08-18 13:44:58.479644 | node1 | 2026-08-18 13:44:58.480376 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:44:58.480403 | node1 | Tuesday 18 August 2026 13:44:58 +0000 (0:00:00.060) 0:00:00.060 ******** 2026-08-18 13:44:59.814010 | node1 | [WARNING]: Platform linux on host node3 is using the discovered Python 2026-08-18 13:44:59.814061 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:44:59.814067 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:44:59.814072 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:44:59.814077 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:44:59.889796 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 13:44:59.889844 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:44:59.889850 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:44:59.889855 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:44:59.889860 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:44:59.896110 | node1 | [WARNING]: Platform linux on host node2 is using the discovered Python 2026-08-18 13:44:59.896196 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:44:59.896208 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:44:59.896225 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:44:59.896372 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:44:59.957834 | node1 | ok: [node3] 2026-08-18 13:44:59.958013 | node1 | ok: [node1] 2026-08-18 13:44:59.958195 | node1 | ok: [node2] 2026-08-18 13:44:59.958411 | node1 | 2026-08-18 13:44:59.958563 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:44:59.958743 | node1 | Tuesday 18 August 2026 13:44:59 +0000 (0:00:01.478) 0:00:01.539 ******** 2026-08-18 13:45:00.349988 | node1 | ok: [node3] 2026-08-18 13:45:00.350025 | node1 | ok: [node1] 2026-08-18 13:45:00.350029 | node1 | ok: [node2] 2026-08-18 13:45:00.350167 | node1 | 2026-08-18 13:45:00.350415 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:45:00.350655 | node1 | Tuesday 18 August 2026 13:45:00 +0000 (0:00:00.391) 0:00:01.931 ******** 2026-08-18 13:45:00.448045 | node1 | skipping: [node1] 2026-08-18 13:45:00.448283 | node1 | skipping: [node2] 2026-08-18 13:45:00.448523 | node1 | skipping: [node3] 2026-08-18 13:45:00.448743 | node1 | 2026-08-18 13:45:00.449033 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:45:00.449283 | node1 | Tuesday 18 August 2026 13:45:00 +0000 (0:00:00.098) 0:00:02.030 ******** 2026-08-18 13:45:00.841061 | node1 | ok: [node1] 2026-08-18 13:45:00.841248 | node1 | ok: [node3] 2026-08-18 13:45:00.841521 | node1 | ok: [node2] 2026-08-18 13:45:00.841740 | node1 | 2026-08-18 13:45:00.842021 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:45:00.842268 | node1 | Tuesday 18 August 2026 13:45:00 +0000 (0:00:00.392) 0:00:02.422 ******** 2026-08-18 13:45:00.962285 | node1 | ok: [node1] => { 2026-08-18 13:45:00.962458 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:45:00.962718 | node1 | } 2026-08-18 13:45:00.963170 | node1 | ok: [node2] => { 2026-08-18 13:45:00.963412 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:45:00.963637 | node1 | } 2026-08-18 13:45:00.963873 | node1 | ok: [node3] => { 2026-08-18 13:45:00.964184 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:45:00.964405 | node1 | } 2026-08-18 13:45:00.964626 | node1 | 2026-08-18 13:45:00.964870 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:45:00.965159 | node1 | Tuesday 18 August 2026 13:45:00 +0000 (0:00:00.121) 0:00:02.543 ******** 2026-08-18 13:45:01.450646 | node1 | ok: [node3] 2026-08-18 13:45:01.450838 | node1 | ok: [node1] 2026-08-18 13:45:01.451121 | node1 | ok: [node2] 2026-08-18 13:45:01.451329 | node1 | 2026-08-18 13:45:01.451577 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:45:01.451820 | node1 | Tuesday 18 August 2026 13:45:01 +0000 (0:00:00.488) 0:00:03.032 ******** 2026-08-18 13:45:01.574442 | node1 | skipping: [node1] 2026-08-18 13:45:01.574671 | node1 | skipping: [node2] 2026-08-18 13:45:01.574959 | node1 | skipping: [node3] 2026-08-18 13:45:01.575292 | node1 | 2026-08-18 13:45:01.575557 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:45:01.575807 | node1 | Tuesday 18 August 2026 13:45:01 +0000 (0:00:00.123) 0:00:03.156 ******** 2026-08-18 13:45:01.614185 | node1 | skipping: [node1] 2026-08-18 13:45:01.704437 | node1 | skipping: [node2] 2026-08-18 13:45:01.704743 | node1 | skipping: [node3] 2026-08-18 13:45:01.705205 | node1 | 2026-08-18 13:45:01.705518 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:45:01.705832 | node1 | Tuesday 18 August 2026 13:45:01 +0000 (0:00:00.129) 0:00:03.286 ******** 2026-08-18 13:45:02.034096 | node1 | ok: [node1] 2026-08-18 13:45:02.034159 | node1 | ok: [node2] 2026-08-18 13:45:02.034169 | node1 | ok: [node3] 2026-08-18 13:45:02.034179 | node1 | 2026-08-18 13:45:02.034194 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:45:02.034427 | node1 | Tuesday 18 August 2026 13:45:02 +0000 (0:00:00.329) 0:00:03.615 ******** 2026-08-18 13:45:03.343125 | node1 | ok: [node2] 2026-08-18 13:45:03.343196 | node1 | ok: [node3] 2026-08-18 13:45:03.343209 | node1 | ok: [node1] 2026-08-18 13:45:03.343259 | node1 | 2026-08-18 13:45:03.343706 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:45:03.343759 | node1 | Tuesday 18 August 2026 13:45:03 +0000 (0:00:01.309) 0:00:04.924 ******** 2026-08-18 13:45:03.604549 | node1 | ok: [node1] => { 2026-08-18 13:45:03.604626 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:45:03.604634 | node1 | } 2026-08-18 13:45:03.604641 | node1 | ok: [node2] => { 2026-08-18 13:45:03.604649 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:45:03.604656 | node1 | } 2026-08-18 13:45:03.604663 | node1 | ok: [node3] => { 2026-08-18 13:45:03.604671 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:45:03.604678 | node1 | } 2026-08-18 13:45:03.604685 | node1 | 2026-08-18 13:45:03.604708 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:45:03.604717 | node1 | Tuesday 18 August 2026 13:45:03 +0000 (0:00:00.260) 0:00:05.184 ******** 2026-08-18 13:45:04.230035 | node1 | ok: [node1] 2026-08-18 13:45:04.230095 | node1 | ok: [node2] 2026-08-18 13:45:04.230109 | node1 | ok: [node3] 2026-08-18 13:45:04.230169 | node1 | 2026-08-18 13:45:04.230323 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:45:04.230476 | node1 | Tuesday 18 August 2026 13:45:04 +0000 (0:00:00.627) 0:00:05.811 ******** 2026-08-18 13:45:06.792613 | node1 | ok: [node2] 2026-08-18 13:45:06.792802 | node1 | ok: [node1] 2026-08-18 13:45:06.793130 | node1 | ok: [node3] 2026-08-18 13:45:06.793389 | node1 | 2026-08-18 13:45:06.793690 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:45:06.793986 | node1 | Tuesday 18 August 2026 13:45:06 +0000 (0:00:02.562) 0:00:08.374 ******** 2026-08-18 13:45:06.869740 | node1 | skipping: [node1] 2026-08-18 13:45:06.869971 | node1 | skipping: [node2] 2026-08-18 13:45:06.870221 | node1 | skipping: [node3] 2026-08-18 13:45:06.870445 | node1 | 2026-08-18 13:45:06.870696 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:45:06.870973 | node1 | Tuesday 18 August 2026 13:45:06 +0000 (0:00:00.077) 0:00:08.451 ******** 2026-08-18 13:45:06.944080 | node1 | skipping: [node1] 2026-08-18 13:45:06.944308 | node1 | skipping: [node2] 2026-08-18 13:45:06.944569 | node1 | skipping: [node3] 2026-08-18 13:45:06.945072 | node1 | 2026-08-18 13:45:06.945426 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:45:06.945732 | node1 | Tuesday 18 August 2026 13:45:06 +0000 (0:00:00.074) 0:00:08.526 ******** 2026-08-18 13:45:07.024781 | node1 | skipping: [node1] 2026-08-18 13:45:07.025001 | node1 | skipping: [node2] 2026-08-18 13:45:07.025250 | node1 | skipping: [node3] 2026-08-18 13:45:07.025477 | node1 | 2026-08-18 13:45:07.025738 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:45:07.026066 | node1 | Tuesday 18 August 2026 13:45:07 +0000 (0:00:00.080) 0:00:08.606 ******** 2026-08-18 13:45:08.121149 | node1 | ok: [node2] 2026-08-18 13:45:08.121307 | node1 | ok: [node3] 2026-08-18 13:45:08.121536 | node1 | ok: [node1] 2026-08-18 13:45:08.122098 | node1 | 2026-08-18 13:45:08.122378 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:45:08.122628 | node1 | Tuesday 18 August 2026 13:45:08 +0000 (0:00:01.096) 0:00:09.702 ******** 2026-08-18 13:45:08.732965 | node1 | ok: [node1] 2026-08-18 13:45:08.733003 | node1 | ok: [node2] 2026-08-18 13:45:08.733007 | node1 | ok: [node3] 2026-08-18 13:45:08.733062 | node1 | 2026-08-18 13:45:08.733318 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:45:08.733566 | node1 | Tuesday 18 August 2026 13:45:08 +0000 (0:00:00.611) 0:00:10.314 ******** 2026-08-18 13:45:09.899223 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:45:09.899568 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:45:09.899854 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:45:09.900167 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:45:09.900444 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:45:09.900715 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:45:09.901014 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:45:09.901283 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:45:09.901545 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:45:09.901829 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:45:09.902138 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:45:09.902415 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:45:09.902693 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:45:09.903025 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:45:09.903327 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:45:09.903616 | node1 | 2026-08-18 13:45:09.903848 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:45:09.904029 | node1 | Tuesday 18 August 2026 13:45:09 +0000 (0:00:01.166) 0:00:11.481 ******** 2026-08-18 13:45:10.638123 | node1 | ok: [node1] 2026-08-18 13:45:10.638313 | node1 | ok: [node3] 2026-08-18 13:45:10.638547 | node1 | ok: [node2] 2026-08-18 13:45:10.638773 | node1 | 2026-08-18 13:45:10.639080 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:45:10.639337 | node1 | Tuesday 18 August 2026 13:45:10 +0000 (0:00:00.713) 0:00:12.194 ******** 2026-08-18 13:45:10.639554 | node1 | 2026-08-18 13:45:10.639802 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:45:10.640096 | node1 | Tuesday 18 August 2026 13:45:10 +0000 (0:00:00.008) 0:00:12.203 ******** 2026-08-18 13:45:10.640305 | node1 | 2026-08-18 13:45:10.640548 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:45:10.640792 | node1 | Tuesday 18 August 2026 13:45:10 +0000 (0:00:00.008) 0:00:12.211 ******** 2026-08-18 13:45:10.641054 | node1 | 2026-08-18 13:45:10.641290 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:45:10.641534 | node1 | Tuesday 18 August 2026 13:45:10 +0000 (0:00:00.008) 0:00:12.219 ******** 2026-08-18 13:45:11.368699 | node1 | ok: [node3] 2026-08-18 13:45:11.368916 | node1 | ok: [node2] 2026-08-18 13:45:11.369204 | node1 | ok: [node1] 2026-08-18 13:45:11.369445 | node1 | 2026-08-18 13:45:11.369714 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:45:11.370022 | node1 | Tuesday 18 August 2026 13:45:11 +0000 (0:00:00.730) 0:00:12.950 ******** 2026-08-18 13:45:11.683429 | node1 | ok: [node1] 2026-08-18 13:45:11.683624 | node1 | ok: [node2] 2026-08-18 13:45:11.683857 | node1 | ok: [node3] 2026-08-18 13:45:11.684115 | node1 | 2026-08-18 13:45:11.684368 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:45:11.684639 | node1 | Tuesday 18 August 2026 13:45:11 +0000 (0:00:00.314) 0:00:13.265 ******** 2026-08-18 13:45:11.836624 | node1 | ok: [node1] => { 2026-08-18 13:45:11.836841 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:45:11.837136 | node1 | } 2026-08-18 13:45:11.837375 | node1 | ok: [node2] => { 2026-08-18 13:45:11.837625 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:45:11.837864 | node1 | } 2026-08-18 13:45:11.838154 | node1 | ok: [node3] => { 2026-08-18 13:45:11.838399 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:45:11.838685 | node1 | } 2026-08-18 13:45:11.838925 | node1 | 2026-08-18 13:45:11.839208 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:45:11.839452 | node1 | Tuesday 18 August 2026 13:45:11 +0000 (0:00:00.153) 0:00:13.418 ******** 2026-08-18 13:45:12.375023 | node1 | ok: [node1] 2026-08-18 13:45:12.375088 | node1 | ok: [node2] 2026-08-18 13:45:12.376044 | node1 | ok: [node3] 2026-08-18 13:45:12.376104 | node1 | 2026-08-18 13:45:12.376114 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:45:12.376124 | node1 | Tuesday 18 August 2026 13:45:12 +0000 (0:00:00.537) 0:00:13.956 ******** 2026-08-18 13:45:15.897966 | node1 | ok: [node1] 2026-08-18 13:45:15.898000 | node1 | ok: [node2] 2026-08-18 13:45:15.898004 | node1 | ok: [node3] 2026-08-18 13:45:15.898008 | node1 | 2026-08-18 13:45:15.898012 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:45:15.898017 | node1 | Tuesday 18 August 2026 13:45:15 +0000 (0:00:03.522) 0:00:17.478 ******** 2026-08-18 13:45:17.032099 | node1 | ok: [node2] 2026-08-18 13:45:17.032167 | node1 | ok: [node3] 2026-08-18 13:45:17.033147 | node1 | ok: [node1] 2026-08-18 13:45:17.033206 | node1 | 2026-08-18 13:45:17.033217 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:45:17.033227 | node1 | Tuesday 18 August 2026 13:45:17 +0000 (0:00:01.134) 0:00:18.613 ******** 2026-08-18 13:45:17.459999 | node1 | ok: [node1] 2026-08-18 13:45:17.460053 | node1 | ok: [node3] 2026-08-18 13:45:17.460062 | node1 | ok: [node2] 2026-08-18 13:45:17.460072 | node1 | 2026-08-18 13:45:17.460081 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:45:17.460110 | node1 | Tuesday 18 August 2026 13:45:17 +0000 (0:00:00.426) 0:00:19.040 ******** 2026-08-18 13:45:18.103149 | node1 | ok: [node1] 2026-08-18 13:45:18.103200 | node1 | ok: [node2] 2026-08-18 13:45:18.103210 | node1 | ok: [node3] 2026-08-18 13:45:18.103219 | node1 | 2026-08-18 13:45:18.103228 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:45:18.103388 | node1 | Tuesday 18 August 2026 13:45:18 +0000 (0:00:00.643) 0:00:19.684 ******** 2026-08-18 13:45:18.848607 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:45:18.848861 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:45:18.849235 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:45:18.849491 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:45:18.849747 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:45:18.850070 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:45:18.850332 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:45:18.850594 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:45:18.850825 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:45:18.851148 | node1 | 2026-08-18 13:45:18.851385 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:45:18.851632 | node1 | Tuesday 18 August 2026 13:45:18 +0000 (0:00:00.746) 0:00:20.430 ******** 2026-08-18 13:45:19.452021 | node1 | ok: [node2] 2026-08-18 13:45:19.452073 | node1 | ok: [node1] 2026-08-18 13:45:19.452084 | node1 | ok: [node3] 2026-08-18 13:45:19.452093 | node1 | 2026-08-18 13:45:19.452420 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:45:19.452460 | node1 | Tuesday 18 August 2026 13:45:19 +0000 (0:00:00.602) 0:00:21.032 ******** 2026-08-18 13:45:20.069988 | node1 | ok: [node2] 2026-08-18 13:45:20.070036 | node1 | ok: [node1] 2026-08-18 13:45:20.070045 | node1 | ok: [node3] 2026-08-18 13:45:20.070055 | node1 | 2026-08-18 13:45:20.070064 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:45:20.070073 | node1 | Tuesday 18 August 2026 13:45:20 +0000 (0:00:00.591) 0:00:21.624 ******** 2026-08-18 13:45:20.070081 | node1 | 2026-08-18 13:45:20.070090 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:45:20.070137 | node1 | Tuesday 18 August 2026 13:45:20 +0000 (0:00:00.008) 0:00:21.632 ******** 2026-08-18 13:45:20.071922 | node1 | 2026-08-18 13:45:20.072128 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:45:20.072140 | node1 | Tuesday 18 August 2026 13:45:20 +0000 (0:00:00.008) 0:00:21.640 ******** 2026-08-18 13:45:20.072149 | node1 | 2026-08-18 13:45:20.072158 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:45:20.072166 | node1 | Tuesday 18 August 2026 13:45:20 +0000 (0:00:00.008) 0:00:21.648 ******** 2026-08-18 13:45:20.573204 | node1 | ok: [node1] 2026-08-18 13:45:20.573266 | node1 | ok: [node3] 2026-08-18 13:45:20.573275 | node1 | ok: [node2] 2026-08-18 13:45:20.573284 | node1 | 2026-08-18 13:45:20.573293 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:45:20.573302 | node1 | Tuesday 18 August 2026 13:45:20 +0000 (0:00:00.504) 0:00:22.153 ******** 2026-08-18 13:45:20.655125 | node1 | ok: [node1] => { 2026-08-18 13:45:20.655190 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:45:20.655201 | node1 | } 2026-08-18 13:45:20.655210 | node1 | ok: [node2] => { 2026-08-18 13:45:20.655218 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:45:20.655226 | node1 | } 2026-08-18 13:45:20.655241 | node1 | ok: [node3] => { 2026-08-18 13:45:20.655372 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:45:20.655600 | node1 | } 2026-08-18 13:45:20.655820 | node1 | 2026-08-18 13:45:20.656120 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:45:20.656360 | node1 | Tuesday 18 August 2026 13:45:20 +0000 (0:00:00.081) 0:00:22.235 ******** 2026-08-18 13:45:21.036552 | node1 | ok: [node1] 2026-08-18 13:45:21.037075 | node1 | ok: [node3] 2026-08-18 13:45:21.037651 | node1 | ok: [node2] 2026-08-18 13:45:21.037806 | node1 | 2026-08-18 13:45:21.038208 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:45:21.038452 | node1 | Tuesday 18 August 2026 13:45:21 +0000 (0:00:00.383) 0:00:22.618 ******** 2026-08-18 13:45:21.077224 | node1 | skipping: [node1] 2026-08-18 13:45:21.139902 | node1 | skipping: [node2] 2026-08-18 13:45:21.140180 | node1 | skipping: [node3] 2026-08-18 13:45:21.140421 | node1 | 2026-08-18 13:45:21.140677 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:45:21.140989 | node1 | Tuesday 18 August 2026 13:45:21 +0000 (0:00:00.103) 0:00:22.722 ******** 2026-08-18 13:45:21.237351 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:45:21.237577 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:45:21.237835 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:45:21.238092 | node1 | 2026-08-18 13:45:21.238339 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:45:21.238580 | node1 | Tuesday 18 August 2026 13:45:21 +0000 (0:00:00.097) 0:00:22.819 ******** 2026-08-18 13:45:22.524279 | node1 | ok: [node3] 2026-08-18 13:45:22.524494 | node1 | ok: [node1] 2026-08-18 13:45:22.524729 | node1 | ok: [node2] 2026-08-18 13:45:22.525032 | node1 | 2026-08-18 13:45:22.525289 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:45:22.525593 | node1 | Tuesday 18 August 2026 13:45:22 +0000 (0:00:01.287) 0:00:24.106 ******** 2026-08-18 13:45:23.368139 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:45:23.368209 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:45:23.368219 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:45:23.368227 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:45:23.368313 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:45:23.368560 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:45:23.368824 | node1 | 2026-08-18 13:45:23.369194 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:45:23.369376 | node1 | Tuesday 18 August 2026 13:45:23 +0000 (0:00:00.842) 0:00:24.948 ******** 2026-08-18 13:45:23.637101 | node1 | ok: [node1] 2026-08-18 13:45:23.637244 | node1 | ok: [node2] 2026-08-18 13:45:23.637387 | node1 | ok: [node3] 2026-08-18 13:45:23.637531 | node1 | 2026-08-18 13:45:23.637700 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:45:23.637865 | node1 | Tuesday 18 August 2026 13:45:23 +0000 (0:00:00.270) 0:00:25.219 ******** 2026-08-18 13:45:24.106898 | node1 | ok: [node2] 2026-08-18 13:45:24.107136 | node1 | ok: [node3] 2026-08-18 13:45:24.107373 | node1 | ok: [node1] 2026-08-18 13:45:24.107614 | node1 | 2026-08-18 13:45:24.108018 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:45:24.108324 | node1 | Tuesday 18 August 2026 13:45:24 +0000 (0:00:00.469) 0:00:25.688 ******** 2026-08-18 13:45:24.690625 | node1 | ok: [node1] 2026-08-18 13:45:24.690824 | node1 | ok: [node2] 2026-08-18 13:45:24.691111 | node1 | ok: [node3] 2026-08-18 13:45:24.691322 | node1 | 2026-08-18 13:45:24.691571 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:45:24.691817 | node1 | Tuesday 18 August 2026 13:45:24 +0000 (0:00:00.583) 0:00:26.272 ******** 2026-08-18 13:45:24.769822 | node1 | skipping: [node1] 2026-08-18 13:45:24.770063 | node1 | skipping: [node2] 2026-08-18 13:45:24.770300 | node1 | skipping: [node3] 2026-08-18 13:45:24.770546 | node1 | 2026-08-18 13:45:24.770777 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:45:24.771045 | node1 | Tuesday 18 August 2026 13:45:24 +0000 (0:00:00.079) 0:00:26.352 ******** 2026-08-18 13:45:25.429100 | node1 | ok: [node1] 2026-08-18 13:45:25.429401 | node1 | ok: [node3] 2026-08-18 13:45:25.429715 | node1 | ok: [node2] 2026-08-18 13:45:25.430053 | node1 | 2026-08-18 13:45:25.430425 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:45:25.430745 | node1 | Tuesday 18 August 2026 13:45:25 +0000 (0:00:00.658) 0:00:27.010 ******** 2026-08-18 13:45:26.038146 | node1 | ok: [node1] 2026-08-18 13:45:26.038206 | node1 | ok: [node2] 2026-08-18 13:45:26.039069 | node1 | ok: [node3] 2026-08-18 13:45:26.039123 | node1 | 2026-08-18 13:45:26.039134 | node1 | TASK [vexxhost.ceph.mon : Find this monitor's addresses on the public network] *** 2026-08-18 13:45:26.039143 | node1 | Tuesday 18 August 2026 13:45:26 +0000 (0:00:00.608) 0:00:27.619 ******** 2026-08-18 13:45:26.166644 | node1 | ok: [node1] 2026-08-18 13:45:26.166674 | node1 | ok: [node2] 2026-08-18 13:45:26.166683 | node1 | ok: [node3] 2026-08-18 13:45:26.166692 | node1 | 2026-08-18 13:45:26.166701 | node1 | TASK [vexxhost.ceph.mon : Assert exactly one address matches the public network] *** 2026-08-18 13:45:26.166710 | node1 | Tuesday 18 August 2026 13:45:26 +0000 (0:00:00.127) 0:00:27.746 ******** 2026-08-18 13:45:26.251004 | node1 | ok: [node1] => { 2026-08-18 13:45:26.252606 | node1 | "changed": false, 2026-08-18 13:45:26.252641 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:26.252651 | node1 | } 2026-08-18 13:45:26.252659 | node1 | ok: [node2] => { 2026-08-18 13:45:26.252667 | node1 | "changed": false, 2026-08-18 13:45:26.252680 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:26.254102 | node1 | } 2026-08-18 13:45:26.254770 | node1 | ok: [node3] => { 2026-08-18 13:45:26.254781 | node1 | "changed": false, 2026-08-18 13:45:26.254795 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:26.254803 | node1 | } 2026-08-18 13:45:26.254812 | node1 | 2026-08-18 13:45:26.255278 | node1 | TASK [vexxhost.ceph.mon : Set this monitor's Ceph public IP] ******************* 2026-08-18 13:45:26.255328 | node1 | Tuesday 18 August 2026 13:45:26 +0000 (0:00:00.085) 0:00:27.832 ******** 2026-08-18 13:45:26.330153 | node1 | ok: [node1] 2026-08-18 13:45:26.330181 | node1 | ok: [node2] 2026-08-18 13:45:26.330187 | node1 | ok: [node3] 2026-08-18 13:45:26.330193 | node1 | 2026-08-18 13:45:26.330199 | node1 | TASK [vexxhost.ceph.mon : Check whether any monitor is already bootstrapped] *** 2026-08-18 13:45:26.330204 | node1 | Tuesday 18 August 2026 13:45:26 +0000 (0:00:00.078) 0:00:27.910 ******** 2026-08-18 13:45:27.091119 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:45:27.091171 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:45:27.091180 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:45:27.091189 | node1 | 2026-08-18 13:45:27.091199 | node1 | TASK [vexxhost.ceph.mon : Select the bootstrap node (existing one, else the first monitor)] *** 2026-08-18 13:45:27.091208 | node1 | Tuesday 18 August 2026 13:45:27 +0000 (0:00:00.760) 0:00:28.670 ******** 2026-08-18 13:45:27.185222 | node1 | ok: [node1] 2026-08-18 13:45:27.185288 | node1 | ok: [node2] 2026-08-18 13:45:27.186202 | node1 | ok: [node3] 2026-08-18 13:45:27.186260 | node1 | 2026-08-18 13:45:27.186270 | node1 | TASK [vexxhost.ceph.mon : Bootstrap the cluster] ******************************* 2026-08-18 13:45:27.186280 | node1 | Tuesday 18 August 2026 13:45:27 +0000 (0:00:00.094) 0:00:28.765 ******** 2026-08-18 13:45:27.274972 | node1 | skipping: [node1] 2026-08-18 13:45:27.275024 | node1 | skipping: [node2] 2026-08-18 13:45:27.275034 | node1 | skipping: [node3] 2026-08-18 13:45:27.275043 | node1 | 2026-08-18 13:45:27.275052 | node1 | TASK [Register this monitor host] ********************************************** 2026-08-18 13:45:27.275067 | node1 | Tuesday 18 August 2026 13:45:27 +0000 (0:00:00.089) 0:00:28.855 ******** 2026-08-18 13:45:27.417709 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:45:27.417757 | node1 | 2026-08-18 13:45:27.417767 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:45:27.417777 | node1 | Tuesday 18 August 2026 13:45:27 +0000 (0:00:00.142) 0:00:28.997 ******** 2026-08-18 13:45:29.185203 | node1 | ok: [node1] 2026-08-18 13:45:29.185325 | node1 | 2026-08-18 13:45:29.186071 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:45:29.186143 | node1 | Tuesday 18 August 2026 13:45:29 +0000 (0:00:01.768) 0:00:30.765 ******** 2026-08-18 13:45:29.275112 | node1 | ok: [node1] 2026-08-18 13:45:29.275171 | node1 | ok: [node2] 2026-08-18 13:45:29.276071 | node1 | ok: [node3] 2026-08-18 13:45:29.276142 | node1 | 2026-08-18 13:45:29.276189 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:45:29.276509 | node1 | Tuesday 18 August 2026 13:45:29 +0000 (0:00:00.090) 0:00:30.856 ******** 2026-08-18 13:45:29.356116 | node1 | ok: [node1] => { 2026-08-18 13:45:29.356224 | node1 | "changed": false, 2026-08-18 13:45:29.356374 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:29.356526 | node1 | } 2026-08-18 13:45:29.356678 | node1 | ok: [node2] => { 2026-08-18 13:45:29.356827 | node1 | "changed": false, 2026-08-18 13:45:29.356992 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:29.357170 | node1 | } 2026-08-18 13:45:29.357313 | node1 | ok: [node3] => { 2026-08-18 13:45:29.357463 | node1 | "changed": false, 2026-08-18 13:45:29.357614 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:29.357756 | node1 | } 2026-08-18 13:45:29.357902 | node1 | 2026-08-18 13:45:29.358132 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:45:29.358259 | node1 | Tuesday 18 August 2026 13:45:29 +0000 (0:00:00.081) 0:00:30.938 ******** 2026-08-18 13:45:29.814020 | node1 | ok: [node2] 2026-08-18 13:45:29.814074 | node1 | ok: [node1] 2026-08-18 13:45:29.814084 | node1 | ok: [node3] 2026-08-18 13:45:29.814093 | node1 | 2026-08-18 13:45:29.814103 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:45:29.814112 | node1 | Tuesday 18 August 2026 13:45:29 +0000 (0:00:00.454) 0:00:31.392 ******** 2026-08-18 13:45:29.903089 | node1 | ok: [node1] 2026-08-18 13:45:29.903151 | node1 | ok: [node2] 2026-08-18 13:45:29.903354 | node1 | ok: [node3] 2026-08-18 13:45:29.903589 | node1 | 2026-08-18 13:45:29.903873 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:45:29.904184 | node1 | Tuesday 18 August 2026 13:45:29 +0000 (0:00:00.092) 0:00:31.484 ******** 2026-08-18 13:45:30.029402 | node1 | ok: [node1] 2026-08-18 13:45:30.029600 | node1 | ok: [node2] 2026-08-18 13:45:30.029847 | node1 | ok: [node3] 2026-08-18 13:45:30.030156 | node1 | 2026-08-18 13:45:30.030443 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:45:30.030727 | node1 | Tuesday 18 August 2026 13:45:30 +0000 (0:00:00.126) 0:00:31.611 ******** 2026-08-18 13:45:30.122321 | node1 | ok: [node1] => { 2026-08-18 13:45:30.122374 | node1 | "changed": false, 2026-08-18 13:45:30.122519 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:30.122644 | node1 | } 2026-08-18 13:45:30.122779 | node1 | ok: [node2] => { 2026-08-18 13:45:30.122889 | node1 | "changed": false, 2026-08-18 13:45:30.123031 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:30.123132 | node1 | } 2026-08-18 13:45:30.123245 | node1 | ok: [node3] => { 2026-08-18 13:45:30.123345 | node1 | "changed": false, 2026-08-18 13:45:30.123452 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:30.123590 | node1 | } 2026-08-18 13:45:30.123717 | node1 | 2026-08-18 13:45:30.123858 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:45:30.124017 | node1 | Tuesday 18 August 2026 13:45:30 +0000 (0:00:00.093) 0:00:31.704 ******** 2026-08-18 13:45:32.081442 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:32.081776 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:32.082079 | node1 | ok: [node1] 2026-08-18 13:45:32.082332 | node1 | 2026-08-18 13:45:32.082587 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:45:32.082834 | node1 | Tuesday 18 August 2026 13:45:32 +0000 (0:00:01.958) 0:00:33.663 ******** 2026-08-18 13:45:32.196754 | node1 | skipping: [node1] 2026-08-18 13:45:32.196963 | node1 | skipping: [node2] 2026-08-18 13:45:32.197238 | node1 | skipping: [node3] 2026-08-18 13:45:32.197562 | node1 | 2026-08-18 13:45:32.198028 | node1 | TASK [vexxhost.ceph.mon : Apply the monitor service spec] ********************** 2026-08-18 13:45:32.198301 | node1 | Tuesday 18 August 2026 13:45:32 +0000 (0:00:00.115) 0:00:33.779 ******** 2026-08-18 13:45:34.039024 | node1 | ok: [node1] 2026-08-18 13:45:34.039083 | node1 | 2026-08-18 13:45:34.039108 | node1 | TASK [vexxhost.ceph.mon : Wait for this monitor's daemon to be present] ******** 2026-08-18 13:45:34.039116 | node1 | Tuesday 18 August 2026 13:45:34 +0000 (0:00:01.840) 0:00:35.620 ******** 2026-08-18 13:45:35.971469 | node1 | ok: [node1] 2026-08-18 13:45:35.971541 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:35.971554 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:35.971567 | node1 | 2026-08-18 13:45:35.971579 | node1 | TASK [vexxhost.ceph.mgr : Locate the cluster admin host] *********************** 2026-08-18 13:45:35.971591 | node1 | Tuesday 18 August 2026 13:45:35 +0000 (0:00:01.931) 0:00:37.552 ******** 2026-08-18 13:45:36.717015 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:45:36.717203 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:45:36.717528 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:45:36.717822 | node1 | 2026-08-18 13:45:36.718247 | node1 | TASK [vexxhost.ceph.mgr : Select the admin host] ******************************* 2026-08-18 13:45:36.718603 | node1 | Tuesday 18 August 2026 13:45:36 +0000 (0:00:00.745) 0:00:38.297 ******** 2026-08-18 13:45:36.815172 | node1 | ok: [node1] 2026-08-18 13:45:36.815255 | node1 | ok: [node2] 2026-08-18 13:45:36.815268 | node1 | ok: [node3] 2026-08-18 13:45:36.815280 | node1 | 2026-08-18 13:45:36.815293 | node1 | TASK [Register this manager host] ********************************************** 2026-08-18 13:45:36.815305 | node1 | Tuesday 18 August 2026 13:45:36 +0000 (0:00:00.098) 0:00:38.396 ******** 2026-08-18 13:45:36.956008 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:45:36.956128 | node1 | 2026-08-18 13:45:36.956388 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:45:36.956657 | node1 | Tuesday 18 August 2026 13:45:36 +0000 (0:00:00.138) 0:00:38.534 ******** 2026-08-18 13:45:38.714012 | node1 | ok: [node1] 2026-08-18 13:45:38.714217 | node1 | 2026-08-18 13:45:38.714565 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:45:38.714784 | node1 | Tuesday 18 August 2026 13:45:38 +0000 (0:00:01.750) 0:00:40.285 ******** 2026-08-18 13:45:38.790058 | node1 | ok: [node1] 2026-08-18 13:45:38.790130 | node1 | ok: [node2] 2026-08-18 13:45:38.790196 | node1 | ok: [node3] 2026-08-18 13:45:38.790294 | node1 | 2026-08-18 13:45:38.790406 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:45:38.790526 | node1 | Tuesday 18 August 2026 13:45:38 +0000 (0:00:00.087) 0:00:40.372 ******** 2026-08-18 13:45:38.877900 | node1 | ok: [node1] => { 2026-08-18 13:45:38.878106 | node1 | "changed": false, 2026-08-18 13:45:38.878347 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:38.878618 | node1 | } 2026-08-18 13:45:38.878856 | node1 | ok: [node2] => { 2026-08-18 13:45:38.879133 | node1 | "changed": false, 2026-08-18 13:45:38.879359 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:38.879582 | node1 | } 2026-08-18 13:45:38.879820 | node1 | ok: [node3] => { 2026-08-18 13:45:38.880098 | node1 | "changed": false, 2026-08-18 13:45:38.880351 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:38.880560 | node1 | } 2026-08-18 13:45:38.880771 | node1 | 2026-08-18 13:45:38.881066 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:45:38.881308 | node1 | Tuesday 18 August 2026 13:45:38 +0000 (0:00:00.087) 0:00:40.460 ******** 2026-08-18 13:45:39.283304 | node1 | ok: [node2] 2026-08-18 13:45:39.283388 | node1 | ok: [node1] 2026-08-18 13:45:39.283501 | node1 | ok: [node3] 2026-08-18 13:45:39.283612 | node1 | 2026-08-18 13:45:39.283748 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:45:39.283877 | node1 | Tuesday 18 August 2026 13:45:39 +0000 (0:00:00.405) 0:00:40.865 ******** 2026-08-18 13:45:39.390433 | node1 | ok: [node1] 2026-08-18 13:45:39.390658 | node1 | ok: [node2] 2026-08-18 13:45:39.390898 | node1 | ok: [node3] 2026-08-18 13:45:39.391149 | node1 | 2026-08-18 13:45:39.391412 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:45:39.391657 | node1 | Tuesday 18 August 2026 13:45:39 +0000 (0:00:00.107) 0:00:40.972 ******** 2026-08-18 13:45:39.521476 | node1 | ok: [node1] 2026-08-18 13:45:39.521700 | node1 | ok: [node2] 2026-08-18 13:45:39.521980 | node1 | ok: [node3] 2026-08-18 13:45:39.522227 | node1 | 2026-08-18 13:45:39.522507 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:45:39.522839 | node1 | Tuesday 18 August 2026 13:45:39 +0000 (0:00:00.130) 0:00:41.103 ******** 2026-08-18 13:45:39.604384 | node1 | ok: [node1] => { 2026-08-18 13:45:39.604587 | node1 | "changed": false, 2026-08-18 13:45:39.604822 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:39.605087 | node1 | } 2026-08-18 13:45:39.605326 | node1 | ok: [node2] => { 2026-08-18 13:45:39.605555 | node1 | "changed": false, 2026-08-18 13:45:39.605793 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:39.606047 | node1 | } 2026-08-18 13:45:39.606282 | node1 | ok: [node3] => { 2026-08-18 13:45:39.606513 | node1 | "changed": false, 2026-08-18 13:45:39.606779 | node1 | "msg": "All assertions passed" 2026-08-18 13:45:39.607047 | node1 | } 2026-08-18 13:45:39.607250 | node1 | 2026-08-18 13:45:39.607497 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:45:39.607735 | node1 | Tuesday 18 August 2026 13:45:39 +0000 (0:00:00.083) 0:00:41.186 ******** 2026-08-18 13:45:41.417728 | node1 | ok: [node1] 2026-08-18 13:45:41.418030 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:41.418356 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:41.418526 | node1 | 2026-08-18 13:45:41.418783 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:45:41.419136 | node1 | Tuesday 18 August 2026 13:45:41 +0000 (0:00:01.813) 0:00:42.999 ******** 2026-08-18 13:45:41.542926 | node1 | skipping: [node1] 2026-08-18 13:45:41.543068 | node1 | skipping: [node2] 2026-08-18 13:45:41.543177 | node1 | skipping: [node3] 2026-08-18 13:45:41.543278 | node1 | 2026-08-18 13:45:41.543389 | node1 | TASK [vexxhost.ceph.mgr : Apply the manager service spec] ********************** 2026-08-18 13:45:41.543508 | node1 | Tuesday 18 August 2026 13:45:41 +0000 (0:00:00.125) 0:00:43.125 ******** 2026-08-18 13:45:43.342369 | node1 | ok: [node1] 2026-08-18 13:45:43.342440 | node1 | 2026-08-18 13:45:43.342544 | node1 | TASK [vexxhost.ceph.mgr : Wait for this manager's daemon to be present] ******** 2026-08-18 13:45:43.342652 | node1 | Tuesday 18 August 2026 13:45:43 +0000 (0:00:01.797) 0:00:44.922 ******** 2026-08-18 13:45:45.219013 | node1 | ok: [node1] 2026-08-18 13:45:45.219828 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:45.219873 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:45:45.219881 | node1 | 2026-08-18 13:45:45.219890 | node1 | TASK [vexxhost.ceph.mgr : Configure ceph-mgr modules] ************************** 2026-08-18 13:45:45.219899 | node1 | Tuesday 18 August 2026 13:45:45 +0000 (0:00:01.878) 0:00:46.801 ******** 2026-08-18 13:45:56.449020 | node1 | ok: [node1] => (item={'name': 'diskprediction_local'}) 2026-08-18 13:45:56.449101 | node1 | ok: [node1] => (item={'name': 'dashboard'}) 2026-08-18 13:45:56.449116 | node1 | ok: [node1] => (item={'name': 'iostat'}) 2026-08-18 13:45:56.449127 | node1 | ok: [node1] => (item={'name': 'prometheus'}) 2026-08-18 13:45:56.449140 | node1 | ok: [node1] => (item={'name': 'rgw'}) 2026-08-18 13:45:56.449151 | node1 | 2026-08-18 13:45:56.449179 | node1 | PLAY [Deploy Ceph OSDs] ******************************************************** 2026-08-18 13:45:56.449191 | node1 | 2026-08-18 13:45:56.449438 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:45:56.449784 | node1 | Tuesday 18 August 2026 13:45:56 +0000 (0:00:11.226) 0:00:58.028 ******** 2026-08-18 13:45:57.498034 | node1 | ok: [node3] 2026-08-18 13:45:57.498464 | node1 | ok: [node1] 2026-08-18 13:45:57.498989 | node1 | ok: [node2] 2026-08-18 13:45:57.499331 | node1 | 2026-08-18 13:45:57.499668 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:45:57.500029 | node1 | Tuesday 18 August 2026 13:45:57 +0000 (0:00:01.050) 0:00:59.078 ******** 2026-08-18 13:45:57.798877 | node1 | ok: [node1] 2026-08-18 13:45:57.799309 | node1 | ok: [node2] 2026-08-18 13:45:57.799663 | node1 | ok: [node3] 2026-08-18 13:45:57.799995 | node1 | 2026-08-18 13:45:57.800359 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:45:57.800711 | node1 | Tuesday 18 August 2026 13:45:57 +0000 (0:00:00.301) 0:00:59.380 ******** 2026-08-18 13:45:57.892429 | node1 | skipping: [node1] 2026-08-18 13:45:57.892507 | node1 | skipping: [node2] 2026-08-18 13:45:57.892995 | node1 | skipping: [node3] 2026-08-18 13:45:57.893030 | node1 | 2026-08-18 13:45:57.893034 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:45:57.893039 | node1 | Tuesday 18 August 2026 13:45:57 +0000 (0:00:00.093) 0:00:59.473 ******** 2026-08-18 13:45:58.204799 | node1 | ok: [node1] 2026-08-18 13:45:58.204868 | node1 | ok: [node2] 2026-08-18 13:45:58.204883 | node1 | ok: [node3] 2026-08-18 13:45:58.204896 | node1 | 2026-08-18 13:45:58.204916 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:45:58.205235 | node1 | Tuesday 18 August 2026 13:45:58 +0000 (0:00:00.310) 0:00:59.784 ******** 2026-08-18 13:45:58.308798 | node1 | ok: [node1] => { 2026-08-18 13:45:58.309055 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:45:58.309287 | node1 | } 2026-08-18 13:45:58.309526 | node1 | ok: [node2] => { 2026-08-18 13:45:58.309774 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:45:58.310030 | node1 | } 2026-08-18 13:45:58.310266 | node1 | ok: [node3] => { 2026-08-18 13:45:58.310520 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:45:58.310744 | node1 | } 2026-08-18 13:45:58.311049 | node1 | 2026-08-18 13:45:58.311292 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:45:58.311533 | node1 | Tuesday 18 August 2026 13:45:58 +0000 (0:00:00.106) 0:00:59.890 ******** 2026-08-18 13:45:58.730420 | node1 | ok: [node1] 2026-08-18 13:45:58.730497 | node1 | ok: [node2] 2026-08-18 13:45:58.730578 | node1 | ok: [node3] 2026-08-18 13:45:58.730680 | node1 | 2026-08-18 13:45:58.730793 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:45:58.730905 | node1 | Tuesday 18 August 2026 13:45:58 +0000 (0:00:00.420) 0:01:00.311 ******** 2026-08-18 13:45:58.835019 | node1 | skipping: [node1] 2026-08-18 13:45:58.835131 | node1 | skipping: [node2] 2026-08-18 13:45:58.835195 | node1 | skipping: [node3] 2026-08-18 13:45:58.835986 | node1 | 2026-08-18 13:45:58.836023 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:45:58.836028 | node1 | Tuesday 18 August 2026 13:45:58 +0000 (0:00:00.106) 0:01:00.417 ******** 2026-08-18 13:45:58.955491 | node1 | skipping: [node1] 2026-08-18 13:45:58.955521 | node1 | skipping: [node2] 2026-08-18 13:45:58.955530 | node1 | skipping: [node3] 2026-08-18 13:45:58.955539 | node1 | 2026-08-18 13:45:58.955548 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:45:58.955556 | node1 | Tuesday 18 August 2026 13:45:58 +0000 (0:00:00.119) 0:01:00.536 ******** 2026-08-18 13:45:59.254123 | node1 | ok: [node1] 2026-08-18 13:45:59.254208 | node1 | ok: [node2] 2026-08-18 13:45:59.254219 | node1 | ok: [node3] 2026-08-18 13:45:59.254228 | node1 | 2026-08-18 13:45:59.254238 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:45:59.254256 | node1 | Tuesday 18 August 2026 13:45:59 +0000 (0:00:00.298) 0:01:00.835 ******** 2026-08-18 13:46:00.320043 | node1 | ok: [node2] 2026-08-18 13:46:00.320117 | node1 | ok: [node1] 2026-08-18 13:46:00.320128 | node1 | ok: [node3] 2026-08-18 13:46:00.320138 | node1 | 2026-08-18 13:46:00.320156 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:46:00.320388 | node1 | Tuesday 18 August 2026 13:46:00 +0000 (0:00:01.065) 0:01:01.900 ******** 2026-08-18 13:46:00.577327 | node1 | ok: [node1] => { 2026-08-18 13:46:00.577603 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:46:00.577855 | node1 | } 2026-08-18 13:46:00.578198 | node1 | ok: [node2] => { 2026-08-18 13:46:00.578488 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:46:00.578743 | node1 | } 2026-08-18 13:46:00.579074 | node1 | ok: [node3] => { 2026-08-18 13:46:00.579368 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:46:00.579615 | node1 | } 2026-08-18 13:46:00.579860 | node1 | 2026-08-18 13:46:00.580167 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:46:00.580407 | node1 | Tuesday 18 August 2026 13:46:00 +0000 (0:00:00.258) 0:01:02.159 ******** 2026-08-18 13:46:01.180185 | node1 | ok: [node1] 2026-08-18 13:46:01.180257 | node1 | ok: [node2] 2026-08-18 13:46:01.180328 | node1 | ok: [node3] 2026-08-18 13:46:01.180554 | node1 | 2026-08-18 13:46:01.180819 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:46:01.181173 | node1 | Tuesday 18 August 2026 13:46:01 +0000 (0:00:00.602) 0:01:02.761 ******** 2026-08-18 13:46:03.465382 | node1 | ok: [node1] 2026-08-18 13:46:03.465617 | node1 | ok: [node3] 2026-08-18 13:46:03.465863 | node1 | ok: [node2] 2026-08-18 13:46:03.466260 | node1 | 2026-08-18 13:46:03.466546 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:46:03.466840 | node1 | Tuesday 18 August 2026 13:46:03 +0000 (0:00:02.285) 0:01:05.047 ******** 2026-08-18 13:46:03.533707 | node1 | skipping: [node1] 2026-08-18 13:46:03.533970 | node1 | skipping: [node2] 2026-08-18 13:46:03.534219 | node1 | skipping: [node3] 2026-08-18 13:46:03.534445 | node1 | 2026-08-18 13:46:03.534695 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:46:03.534971 | node1 | Tuesday 18 August 2026 13:46:03 +0000 (0:00:00.068) 0:01:05.115 ******** 2026-08-18 13:46:03.598877 | node1 | skipping: [node1] 2026-08-18 13:46:03.599170 | node1 | skipping: [node2] 2026-08-18 13:46:03.599396 | node1 | skipping: [node3] 2026-08-18 13:46:03.599616 | node1 | 2026-08-18 13:46:03.599867 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:46:03.600159 | node1 | Tuesday 18 August 2026 13:46:03 +0000 (0:00:00.065) 0:01:05.181 ******** 2026-08-18 13:46:03.667356 | node1 | skipping: [node1] 2026-08-18 13:46:03.667574 | node1 | skipping: [node2] 2026-08-18 13:46:03.667806 | node1 | skipping: [node3] 2026-08-18 13:46:03.668085 | node1 | 2026-08-18 13:46:03.668390 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:46:03.668596 | node1 | Tuesday 18 August 2026 13:46:03 +0000 (0:00:00.068) 0:01:05.249 ******** 2026-08-18 13:46:04.804984 | node1 | ok: [node1] 2026-08-18 13:46:04.805069 | node1 | ok: [node2] 2026-08-18 13:46:04.805167 | node1 | ok: [node3] 2026-08-18 13:46:04.805268 | node1 | 2026-08-18 13:46:04.805382 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:46:04.805496 | node1 | Tuesday 18 August 2026 13:46:04 +0000 (0:00:01.137) 0:01:06.386 ******** 2026-08-18 13:46:05.384390 | node1 | ok: [node2] 2026-08-18 13:46:05.384466 | node1 | ok: [node1] 2026-08-18 13:46:05.384541 | node1 | ok: [node3] 2026-08-18 13:46:05.384641 | node1 | 2026-08-18 13:46:05.384755 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:46:05.384866 | node1 | Tuesday 18 August 2026 13:46:05 +0000 (0:00:00.579) 0:01:06.966 ******** 2026-08-18 13:46:06.556093 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:46:06.556140 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:46:06.556149 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:46:06.558009 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:46:06.558068 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:46:06.558081 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:46:06.558092 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:46:06.558104 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:46:06.558115 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:46:06.558150 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:46:06.558477 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:46:06.558803 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:46:06.559168 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:46:06.559501 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:46:06.559826 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:46:06.560150 | node1 | 2026-08-18 13:46:06.560471 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:46:06.560782 | node1 | Tuesday 18 August 2026 13:46:06 +0000 (0:00:01.170) 0:01:08.137 ******** 2026-08-18 13:46:07.302693 | node1 | ok: [node1] 2026-08-18 13:46:07.303167 | node1 | ok: [node2] 2026-08-18 13:46:07.303513 | node1 | ok: [node3] 2026-08-18 13:46:07.303822 | node1 | 2026-08-18 13:46:07.304241 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:46:07.304607 | node1 | Tuesday 18 August 2026 13:46:07 +0000 (0:00:00.729) 0:01:08.866 ******** 2026-08-18 13:46:07.304914 | node1 | 2026-08-18 13:46:07.305401 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:46:07.305792 | node1 | Tuesday 18 August 2026 13:46:07 +0000 (0:00:00.006) 0:01:08.872 ******** 2026-08-18 13:46:07.306160 | node1 | 2026-08-18 13:46:07.306524 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:46:07.306874 | node1 | Tuesday 18 August 2026 13:46:07 +0000 (0:00:00.005) 0:01:08.878 ******** 2026-08-18 13:46:07.307244 | node1 | 2026-08-18 13:46:07.307606 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:46:07.308009 | node1 | Tuesday 18 August 2026 13:46:07 +0000 (0:00:00.005) 0:01:08.884 ******** 2026-08-18 13:46:07.786928 | node1 | ok: [node1] 2026-08-18 13:46:07.787354 | node1 | ok: [node2] 2026-08-18 13:46:07.787659 | node1 | ok: [node3] 2026-08-18 13:46:07.788493 | node1 | 2026-08-18 13:46:07.788520 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:46:07.788530 | node1 | Tuesday 18 August 2026 13:46:07 +0000 (0:00:00.484) 0:01:09.368 ******** 2026-08-18 13:46:08.122773 | node1 | ok: [node1] 2026-08-18 13:46:08.122836 | node1 | ok: [node3] 2026-08-18 13:46:08.123000 | node1 | ok: [node2] 2026-08-18 13:46:08.123224 | node1 | 2026-08-18 13:46:08.123460 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:46:08.123719 | node1 | Tuesday 18 August 2026 13:46:08 +0000 (0:00:00.335) 0:01:09.704 ******** 2026-08-18 13:46:08.238503 | node1 | ok: [node1] => { 2026-08-18 13:46:08.238757 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:46:08.239083 | node1 | } 2026-08-18 13:46:08.239310 | node1 | ok: [node2] => { 2026-08-18 13:46:08.239608 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:46:08.239847 | node1 | } 2026-08-18 13:46:08.240141 | node1 | ok: [node3] => { 2026-08-18 13:46:08.240391 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:46:08.240622 | node1 | } 2026-08-18 13:46:08.240841 | node1 | 2026-08-18 13:46:08.241145 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:46:08.241380 | node1 | Tuesday 18 August 2026 13:46:08 +0000 (0:00:00.116) 0:01:09.820 ******** 2026-08-18 13:46:08.731107 | node1 | ok: [node1] 2026-08-18 13:46:08.731160 | node1 | ok: [node2] 2026-08-18 13:46:08.731170 | node1 | ok: [node3] 2026-08-18 13:46:08.731179 | node1 | 2026-08-18 13:46:08.731188 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:46:08.731197 | node1 | Tuesday 18 August 2026 13:46:08 +0000 (0:00:00.490) 0:01:10.311 ******** 2026-08-18 13:46:14.064146 | node1 | ok: [node2] 2026-08-18 13:46:14.065027 | node1 | ok: [node3] 2026-08-18 13:46:14.065041 | node1 | ok: [node1] 2026-08-18 13:46:14.065053 | node1 | 2026-08-18 13:46:14.065066 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:46:14.065086 | node1 | Tuesday 18 August 2026 13:46:14 +0000 (0:00:05.333) 0:01:15.644 ******** 2026-08-18 13:46:15.124152 | node1 | ok: [node1] 2026-08-18 13:46:15.124372 | node1 | ok: [node3] 2026-08-18 13:46:15.124621 | node1 | ok: [node2] 2026-08-18 13:46:15.124826 | node1 | 2026-08-18 13:46:15.125126 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:46:15.125370 | node1 | Tuesday 18 August 2026 13:46:15 +0000 (0:00:01.061) 0:01:16.706 ******** 2026-08-18 13:46:15.399014 | node1 | ok: [node1] 2026-08-18 13:46:15.399285 | node1 | ok: [node2] 2026-08-18 13:46:15.399299 | node1 | ok: [node3] 2026-08-18 13:46:15.399311 | node1 | 2026-08-18 13:46:15.399330 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:46:15.399353 | node1 | Tuesday 18 August 2026 13:46:15 +0000 (0:00:00.271) 0:01:16.977 ******** 2026-08-18 13:46:15.990379 | node1 | ok: [node1] 2026-08-18 13:46:15.990495 | node1 | ok: [node2] 2026-08-18 13:46:15.990644 | node1 | ok: [node3] 2026-08-18 13:46:15.990785 | node1 | 2026-08-18 13:46:15.990974 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:46:15.991130 | node1 | Tuesday 18 August 2026 13:46:15 +0000 (0:00:00.594) 0:01:17.572 ******** 2026-08-18 13:46:16.723084 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:46:16.723222 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:46:16.723237 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:46:16.723255 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:46:16.724035 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:46:16.724087 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:46:16.724100 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:46:16.725018 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:46:16.725084 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:46:16.725200 | node1 | 2026-08-18 13:46:16.725525 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:46:16.725841 | node1 | Tuesday 18 August 2026 13:46:16 +0000 (0:00:00.732) 0:01:18.304 ******** 2026-08-18 13:46:17.300353 | node1 | ok: [node1] 2026-08-18 13:46:17.300412 | node1 | ok: [node2] 2026-08-18 13:46:17.300522 | node1 | ok: [node3] 2026-08-18 13:46:17.300627 | node1 | 2026-08-18 13:46:17.300746 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:46:17.300863 | node1 | Tuesday 18 August 2026 13:46:17 +0000 (0:00:00.577) 0:01:18.882 ******** 2026-08-18 13:46:17.890063 | node1 | ok: [node3] 2026-08-18 13:46:17.890125 | node1 | ok: [node1] 2026-08-18 13:46:17.890139 | node1 | ok: [node2] 2026-08-18 13:46:17.890151 | node1 | 2026-08-18 13:46:17.890163 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:46:17.890175 | node1 | Tuesday 18 August 2026 13:46:17 +0000 (0:00:00.569) 0:01:19.451 ******** 2026-08-18 13:46:17.890193 | node1 | 2026-08-18 13:46:17.891006 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:46:17.891067 | node1 | Tuesday 18 August 2026 13:46:17 +0000 (0:00:00.006) 0:01:19.457 ******** 2026-08-18 13:46:17.891080 | node1 | 2026-08-18 13:46:17.891190 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:46:17.891587 | node1 | Tuesday 18 August 2026 13:46:17 +0000 (0:00:00.006) 0:01:19.463 ******** 2026-08-18 13:46:17.891903 | node1 | 2026-08-18 13:46:17.892267 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:46:17.892584 | node1 | Tuesday 18 August 2026 13:46:17 +0000 (0:00:00.006) 0:01:19.470 ******** 2026-08-18 13:46:18.376647 | node1 | ok: [node1] 2026-08-18 13:46:18.377093 | node1 | ok: [node2] 2026-08-18 13:46:18.377445 | node1 | ok: [node3] 2026-08-18 13:46:18.377732 | node1 | 2026-08-18 13:46:18.378360 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:46:18.378386 | node1 | Tuesday 18 August 2026 13:46:18 +0000 (0:00:00.488) 0:01:19.958 ******** 2026-08-18 13:46:18.454021 | node1 | ok: [node1] => { 2026-08-18 13:46:18.454070 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:46:18.454075 | node1 | } 2026-08-18 13:46:18.454079 | node1 | ok: [node2] => { 2026-08-18 13:46:18.454083 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:46:18.454087 | node1 | } 2026-08-18 13:46:18.454090 | node1 | ok: [node3] => { 2026-08-18 13:46:18.454981 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:46:18.455040 | node1 | } 2026-08-18 13:46:18.455051 | node1 | 2026-08-18 13:46:18.455061 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:46:18.455350 | node1 | Tuesday 18 August 2026 13:46:18 +0000 (0:00:00.075) 0:01:20.033 ******** 2026-08-18 13:46:18.816488 | node1 | ok: [node2] 2026-08-18 13:46:18.816866 | node1 | ok: [node1] 2026-08-18 13:46:18.817218 | node1 | ok: [node3] 2026-08-18 13:46:18.817518 | node1 | 2026-08-18 13:46:18.817839 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:46:18.818228 | node1 | Tuesday 18 August 2026 13:46:18 +0000 (0:00:00.364) 0:01:20.398 ******** 2026-08-18 13:46:18.905435 | node1 | skipping: [node1] 2026-08-18 13:46:18.905832 | node1 | skipping: [node2] 2026-08-18 13:46:18.906234 | node1 | skipping: [node3] 2026-08-18 13:46:18.906650 | node1 | 2026-08-18 13:46:18.907044 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:46:18.907442 | node1 | Tuesday 18 August 2026 13:46:18 +0000 (0:00:00.088) 0:01:20.487 ******** 2026-08-18 13:46:18.998043 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:46:18.998133 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:46:18.998197 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:46:18.998214 | node1 | 2026-08-18 13:46:18.998502 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:46:18.998789 | node1 | Tuesday 18 August 2026 13:46:18 +0000 (0:00:00.092) 0:01:20.579 ******** 2026-08-18 13:46:20.154303 | node1 | ok: [node1] 2026-08-18 13:46:20.154365 | node1 | ok: [node3] 2026-08-18 13:46:20.154378 | node1 | ok: [node2] 2026-08-18 13:46:20.154390 | node1 | 2026-08-18 13:46:20.154402 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:46:20.154414 | node1 | Tuesday 18 August 2026 13:46:20 +0000 (0:00:01.155) 0:01:21.734 ******** 2026-08-18 13:46:21.016058 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:46:21.016097 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:46:21.017880 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:46:21.017974 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:46:21.017992 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:46:21.018004 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:46:21.018015 | node1 | 2026-08-18 13:46:21.018028 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:46:21.018040 | node1 | Tuesday 18 August 2026 13:46:21 +0000 (0:00:00.862) 0:01:22.596 ******** 2026-08-18 13:46:21.296003 | node1 | ok: [node1] 2026-08-18 13:46:21.296058 | node1 | ok: [node2] 2026-08-18 13:46:21.296071 | node1 | ok: [node3] 2026-08-18 13:46:21.296083 | node1 | 2026-08-18 13:46:21.296096 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:46:21.296120 | node1 | Tuesday 18 August 2026 13:46:21 +0000 (0:00:00.279) 0:01:22.876 ******** 2026-08-18 13:46:21.637580 | node1 | ok: [node1] 2026-08-18 13:46:21.637620 | node1 | ok: [node2] 2026-08-18 13:46:21.637633 | node1 | ok: [node3] 2026-08-18 13:46:21.637646 | node1 | 2026-08-18 13:46:21.637658 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:46:21.637693 | node1 | Tuesday 18 August 2026 13:46:21 +0000 (0:00:00.341) 0:01:23.218 ******** 2026-08-18 13:46:22.211125 | node1 | ok: [node1] 2026-08-18 13:46:22.211176 | node1 | ok: [node2] 2026-08-18 13:46:22.211193 | node1 | ok: [node3] 2026-08-18 13:46:22.211378 | node1 | 2026-08-18 13:46:22.211647 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:46:22.211920 | node1 | Tuesday 18 August 2026 13:46:22 +0000 (0:00:00.574) 0:01:23.792 ******** 2026-08-18 13:46:22.300226 | node1 | skipping: [node1] 2026-08-18 13:46:22.339830 | node1 | skipping: [node2] 2026-08-18 13:46:22.340129 | node1 | skipping: [node3] 2026-08-18 13:46:22.340345 | node1 | 2026-08-18 13:46:22.340595 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:46:22.340838 | node1 | Tuesday 18 August 2026 13:46:22 +0000 (0:00:00.129) 0:01:23.922 ******** 2026-08-18 13:46:22.790426 | node1 | ok: [node2] 2026-08-18 13:46:22.791616 | node1 | ok: [node3] 2026-08-18 13:46:22.791661 | node1 | ok: [node1] 2026-08-18 13:46:22.791671 | node1 | 2026-08-18 13:46:22.791680 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:46:22.791689 | node1 | Tuesday 18 August 2026 13:46:22 +0000 (0:00:00.450) 0:01:24.372 ******** 2026-08-18 13:46:23.384093 | node1 | ok: [node3] 2026-08-18 13:46:23.384149 | node1 | ok: [node2] 2026-08-18 13:46:23.384159 | node1 | ok: [node1] 2026-08-18 13:46:23.384217 | node1 | 2026-08-18 13:46:23.384380 | node1 | TASK [vexxhost.ceph.osd : Locate the cluster admin host] *********************** 2026-08-18 13:46:23.384542 | node1 | Tuesday 18 August 2026 13:46:23 +0000 (0:00:00.593) 0:01:24.965 ******** 2026-08-18 13:46:24.117434 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:46:24.117497 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:46:24.117777 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:46:24.117801 | node1 | 2026-08-18 13:46:24.117827 | node1 | TASK [vexxhost.ceph.osd : Select the admin host and its mon config path] ******* 2026-08-18 13:46:24.117979 | node1 | Tuesday 18 August 2026 13:46:24 +0000 (0:00:00.731) 0:01:25.697 ******** 2026-08-18 13:46:24.222218 | node1 | ok: [node1] 2026-08-18 13:46:24.222462 | node1 | ok: [node2] 2026-08-18 13:46:24.222712 | node1 | ok: [node3] 2026-08-18 13:46:24.222992 | node1 | 2026-08-18 13:46:24.223261 | node1 | TASK [vexxhost.ceph.osd : Set the admin mon config path] *********************** 2026-08-18 13:46:24.223538 | node1 | Tuesday 18 August 2026 13:46:24 +0000 (0:00:00.106) 0:01:25.804 ******** 2026-08-18 13:46:24.309587 | node1 | ok: [node1] 2026-08-18 13:46:24.309829 | node1 | ok: [node2] 2026-08-18 13:46:24.310114 | node1 | ok: [node3] 2026-08-18 13:46:24.310354 | node1 | 2026-08-18 13:46:24.310619 | node1 | TASK [Register this OSD host] ************************************************** 2026-08-18 13:46:24.310884 | node1 | Tuesday 18 August 2026 13:46:24 +0000 (0:00:00.087) 0:01:25.891 ******** 2026-08-18 13:46:24.441159 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:46:24.441399 | node1 | 2026-08-18 13:46:24.441700 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:46:24.441976 | node1 | Tuesday 18 August 2026 13:46:24 +0000 (0:00:00.131) 0:01:26.023 ******** 2026-08-18 13:46:26.158305 | node1 | ok: [node1] 2026-08-18 13:46:26.158358 | node1 | 2026-08-18 13:46:26.158372 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:46:26.158384 | node1 | Tuesday 18 August 2026 13:46:26 +0000 (0:00:01.716) 0:01:27.739 ******** 2026-08-18 13:46:26.233056 | node1 | ok: [node1] 2026-08-18 13:46:26.233096 | node1 | ok: [node2] 2026-08-18 13:46:26.234019 | node1 | ok: [node3] 2026-08-18 13:46:26.234074 | node1 | 2026-08-18 13:46:26.234088 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:46:26.234101 | node1 | Tuesday 18 August 2026 13:46:26 +0000 (0:00:00.074) 0:01:27.814 ******** 2026-08-18 13:46:26.309588 | node1 | ok: [node1] => { 2026-08-18 13:46:26.309678 | node1 | "changed": false, 2026-08-18 13:46:26.309790 | node1 | "msg": "All assertions passed" 2026-08-18 13:46:26.309901 | node1 | } 2026-08-18 13:46:26.311045 | node1 | ok: [node2] => { 2026-08-18 13:46:26.311083 | node1 | "changed": false, 2026-08-18 13:46:26.311094 | node1 | "msg": "All assertions passed" 2026-08-18 13:46:26.311104 | node1 | } 2026-08-18 13:46:26.311113 | node1 | ok: [node3] => { 2026-08-18 13:46:26.311122 | node1 | "changed": false, 2026-08-18 13:46:26.311131 | node1 | "msg": "All assertions passed" 2026-08-18 13:46:26.311140 | node1 | } 2026-08-18 13:46:26.311150 | node1 | 2026-08-18 13:46:26.311160 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:46:26.311176 | node1 | Tuesday 18 August 2026 13:46:26 +0000 (0:00:00.075) 0:01:27.889 ******** 2026-08-18 13:46:26.676363 | node1 | ok: [node1] 2026-08-18 13:46:26.676419 | node1 | ok: [node2] 2026-08-18 13:46:26.677995 | node1 | ok: [node3] 2026-08-18 13:46:26.679026 | node1 | 2026-08-18 13:46:26.679086 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:46:26.679116 | node1 | Tuesday 18 August 2026 13:46:26 +0000 (0:00:00.368) 0:01:28.258 ******** 2026-08-18 13:46:26.755397 | node1 | ok: [node1] 2026-08-18 13:46:26.755461 | node1 | ok: [node2] 2026-08-18 13:46:26.755566 | node1 | ok: [node3] 2026-08-18 13:46:26.755650 | node1 | 2026-08-18 13:46:26.755770 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:46:26.755883 | node1 | Tuesday 18 August 2026 13:46:26 +0000 (0:00:00.078) 0:01:28.336 ******** 2026-08-18 13:46:26.868641 | node1 | ok: [node1] 2026-08-18 13:46:26.868700 | node1 | ok: [node2] 2026-08-18 13:46:26.868787 | node1 | ok: [node3] 2026-08-18 13:46:26.868920 | node1 | 2026-08-18 13:46:26.869060 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:46:26.869171 | node1 | Tuesday 18 August 2026 13:46:26 +0000 (0:00:00.114) 0:01:28.451 ******** 2026-08-18 13:46:26.945134 | node1 | ok: [node1] => { 2026-08-18 13:46:26.945200 | node1 | "changed": false, 2026-08-18 13:46:26.945210 | node1 | "msg": "All assertions passed" 2026-08-18 13:46:26.945221 | node1 | } 2026-08-18 13:46:26.945232 | node1 | ok: [node2] => { 2026-08-18 13:46:26.945242 | node1 | "changed": false, 2026-08-18 13:46:26.945388 | node1 | "msg": "All assertions passed" 2026-08-18 13:46:26.945613 | node1 | } 2026-08-18 13:46:26.945847 | node1 | ok: [node3] => { 2026-08-18 13:46:26.946139 | node1 | "changed": false, 2026-08-18 13:46:26.946365 | node1 | "msg": "All assertions passed" 2026-08-18 13:46:26.946575 | node1 | } 2026-08-18 13:46:26.946792 | node1 | 2026-08-18 13:46:26.947088 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:46:26.947323 | node1 | Tuesday 18 August 2026 13:46:26 +0000 (0:00:00.074) 0:01:28.525 ******** 2026-08-18 13:46:28.763170 | node1 | ok: [node1] 2026-08-18 13:46:28.763233 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:46:28.763248 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:46:28.763265 | node1 | 2026-08-18 13:46:28.763474 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:46:28.763722 | node1 | Tuesday 18 August 2026 13:46:28 +0000 (0:00:01.819) 0:01:30.344 ******** 2026-08-18 13:46:28.872131 | node1 | skipping: [node1] 2026-08-18 13:46:28.872335 | node1 | skipping: [node2] 2026-08-18 13:46:28.872597 | node1 | skipping: [node3] 2026-08-18 13:46:28.872790 | node1 | 2026-08-18 13:46:28.873066 | node1 | TASK [vexxhost.ceph.osd : Apply OSD service specs (drive groups)] ************** 2026-08-18 13:46:28.873318 | node1 | Tuesday 18 August 2026 13:46:28 +0000 (0:00:00.109) 0:01:30.454 ******** 2026-08-18 13:46:28.903961 | node1 | skipping: [node1] 2026-08-18 13:46:28.904210 | node1 | 2026-08-18 13:46:28.904457 | node1 | TASK [vexxhost.ceph.osd : List existing ceph-volume OSDs on this host] ********* 2026-08-18 13:46:28.904702 | node1 | Tuesday 18 August 2026 13:46:28 +0000 (0:00:00.032) 0:01:30.486 ******** 2026-08-18 13:46:30.461236 | node1 | ok: [node1] 2026-08-18 13:46:30.461299 | node1 | ok: [node3] 2026-08-18 13:46:30.461392 | node1 | ok: [node2] 2026-08-18 13:46:30.461492 | node1 | 2026-08-18 13:46:30.461602 | node1 | TASK [vexxhost.ceph.osd : Deploy OSDs on the explicitly listed devices] ******** 2026-08-18 13:46:30.461711 | node1 | Tuesday 18 August 2026 13:46:30 +0000 (0:00:01.557) 0:01:32.043 ******** 2026-08-18 13:46:30.501474 | node1 | skipping: [node1] => (item=/dev/ceph-node1-osd0/data) 2026-08-18 13:46:30.592422 | node1 | skipping: [node1] => (item=/dev/ceph-node1-osd1/data) 2026-08-18 13:46:30.592652 | node1 | skipping: [node2] => (item=/dev/ceph-node2-osd0/data) 2026-08-18 13:46:30.592902 | node1 | skipping: [node1] => (item=/dev/ceph-node1-osd2/data) 2026-08-18 13:46:30.593253 | node1 | skipping: [node1] 2026-08-18 13:46:30.593495 | node1 | skipping: [node2] => (item=/dev/ceph-node2-osd1/data) 2026-08-18 13:46:30.593739 | node1 | skipping: [node3] => (item=/dev/ceph-node3-osd0/data) 2026-08-18 13:46:30.594029 | node1 | skipping: [node2] => (item=/dev/ceph-node2-osd2/data) 2026-08-18 13:46:30.594262 | node1 | skipping: [node2] 2026-08-18 13:46:30.594509 | node1 | skipping: [node3] => (item=/dev/ceph-node3-osd1/data) 2026-08-18 13:46:30.594752 | node1 | skipping: [node3] => (item=/dev/ceph-node3-osd2/data) 2026-08-18 13:46:30.595036 | node1 | skipping: [node3] 2026-08-18 13:46:30.595258 | node1 | 2026-08-18 13:46:30.595525 | node1 | TASK [vexxhost.ceph.osd : Wait for explicitly requested OSD daemons to be running on this host] *** 2026-08-18 13:46:30.595773 | node1 | Tuesday 18 August 2026 13:46:30 +0000 (0:00:00.130) 0:01:32.174 ******** 2026-08-18 13:46:32.406050 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:46:32.406625 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:46:32.406887 | node1 | ok: [node1] 2026-08-18 13:46:32.407168 | node1 | 2026-08-18 13:46:32.407447 | node1 | TASK [vexxhost.ceph.osd : Wait for OSD drive-group services to deploy at least one daemon] *** 2026-08-18 13:46:32.407761 | node1 | Tuesday 18 August 2026 13:46:32 +0000 (0:00:01.813) 0:01:33.987 ******** 2026-08-18 13:46:32.484459 | node1 | skipping: [node1] 2026-08-18 13:46:32.484638 | node1 | 2026-08-18 13:46:32.484888 | node1 | PLAY [Apply Ceph configuration] ************************************************ 2026-08-18 13:46:32.485192 | node1 | 2026-08-18 13:46:32.485425 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:46:32.485674 | node1 | Tuesday 18 August 2026 13:46:32 +0000 (0:00:00.078) 0:01:34.066 ******** 2026-08-18 13:46:33.524914 | node1 | ok: [node1] 2026-08-18 13:46:33.525057 | node1 | ok: [node3] 2026-08-18 13:46:33.525309 | node1 | ok: [node2] 2026-08-18 13:46:33.525525 | node1 | 2026-08-18 13:46:33.525777 | node1 | TASK [vexxhost.ceph.configure : Locate the cluster admin host] ***************** 2026-08-18 13:46:33.526076 | node1 | Tuesday 18 August 2026 13:46:33 +0000 (0:00:01.039) 0:01:35.106 ******** 2026-08-18 13:46:34.241986 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:46:34.242206 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:46:34.242472 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:46:34.242696 | node1 | 2026-08-18 13:46:34.242984 | node1 | TASK [vexxhost.ceph.configure : Select the admin host] ************************* 2026-08-18 13:46:34.243244 | node1 | Tuesday 18 August 2026 13:46:34 +0000 (0:00:00.717) 0:01:35.823 ******** 2026-08-18 13:46:34.313209 | node1 | ok: [node1] 2026-08-18 13:46:34.313427 | node1 | ok: [node2] 2026-08-18 13:46:34.313661 | node1 | ok: [node3] 2026-08-18 13:46:34.313883 | node1 | 2026-08-18 13:46:34.314206 | node1 | TASK [vexxhost.ceph.configure : Set the admin mon config path] ***************** 2026-08-18 13:46:34.314458 | node1 | Tuesday 18 August 2026 13:46:34 +0000 (0:00:00.071) 0:01:35.895 ******** 2026-08-18 13:46:34.385683 | node1 | ok: [node1] 2026-08-18 13:46:34.385897 | node1 | ok: [node2] 2026-08-18 13:46:34.386206 | node1 | ok: [node3] 2026-08-18 13:46:34.386413 | node1 | 2026-08-18 13:46:34.386665 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 13:46:34.386929 | node1 | Tuesday 18 August 2026 13:46:34 +0000 (0:00:00.072) 0:01:35.967 ******** 2026-08-18 13:46:34.473444 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/config.yml for node1, node2, node3 2026-08-18 13:46:34.473681 | node1 | 2026-08-18 13:46:34.473959 | node1 | TASK [vexxhost.ceph.configure : Apply central Ceph configuration] ************** 2026-08-18 13:46:34.474214 | node1 | Tuesday 18 August 2026 13:46:34 +0000 (0:00:00.087) 0:01:36.055 ******** 2026-08-18 13:46:44.742076 | node1 | ok: [node1] => (item=mgr/mgr/dashboard/standby_behaviour) 2026-08-18 13:46:44.742136 | node1 | ok: [node1] => (item=mgr/mgr/dashboard/RGW_API_SSL_VERIFY) 2026-08-18 13:46:44.742149 | node1 | ok: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_CEPHFS) 2026-08-18 13:46:44.742203 | node1 | ok: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_ISCSI) 2026-08-18 13:46:44.742451 | node1 | ok: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_MIRRORING) 2026-08-18 13:46:44.742697 | node1 | ok: [node1] => (item=mgr/mgr/dashboard/FEATURE_TOGGLE_NFS) 2026-08-18 13:46:44.742918 | node1 | 2026-08-18 13:46:44.743209 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 13:46:44.743457 | node1 | Tuesday 18 August 2026 13:46:44 +0000 (0:00:10.267) 0:01:46.323 ******** 2026-08-18 13:46:44.821803 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/ec_profiles.yml for node1, node2, node3 2026-08-18 13:46:44.822047 | node1 | 2026-08-18 13:46:44.822369 | node1 | TASK [vexxhost.ceph.configure : Configure erasure-code profiles] *************** 2026-08-18 13:46:44.823035 | node1 | Tuesday 18 August 2026 13:46:44 +0000 (0:00:00.081) 0:01:46.404 ******** 2026-08-18 13:46:46.626640 | node1 | ok: [node1] => (item=ec_hdd_s3) 2026-08-18 13:46:46.626866 | node1 | 2026-08-18 13:46:46.627253 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 13:46:46.627534 | node1 | Tuesday 18 August 2026 13:46:46 +0000 (0:00:01.804) 0:01:48.208 ******** 2026-08-18 13:46:46.706204 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/crush_rules.yml for node1, node2, node3 2026-08-18 13:46:46.706346 | node1 | 2026-08-18 13:46:46.706602 | node1 | TASK [vexxhost.ceph.configure : Configure CRUSH rules] ************************* 2026-08-18 13:46:46.706844 | node1 | Tuesday 18 August 2026 13:46:46 +0000 (0:00:00.079) 0:01:48.288 ******** 2026-08-18 13:46:51.888135 | node1 | ok: [node1] => (item=rep_hdd) 2026-08-18 13:46:51.888370 | node1 | ok: [node1] => (item=rep_lab) 2026-08-18 13:46:51.888625 | node1 | ok: [node1] => (item=ec_hdd_s3) 2026-08-18 13:46:51.888917 | node1 | 2026-08-18 13:46:51.889319 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 13:46:51.889626 | node1 | Tuesday 18 August 2026 13:46:51 +0000 (0:00:05.181) 0:01:53.469 ******** 2026-08-18 13:46:51.974270 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/pools.yml for node1, node2, node3 2026-08-18 13:46:51.974491 | node1 | 2026-08-18 13:46:51.974750 | node1 | TASK [vexxhost.ceph.configure : Configure pools] ******************************* 2026-08-18 13:46:51.975044 | node1 | Tuesday 18 August 2026 13:46:51 +0000 (0:00:00.086) 0:01:53.556 ******** 2026-08-18 13:47:29.009075 | node1 | ok: [node1] => (item=molecule-1.rgw.buckets.data) 2026-08-18 13:47:29.009163 | node1 | ok: [node1] => (item=molecule-1.rgw.buckets.index) 2026-08-18 13:47:29.009411 | node1 | ok: [node1] => (item=molecule-1.rgw.buckets.non-ec) 2026-08-18 13:47:29.009676 | node1 | ok: [node1] => (item=molecule-1.rgw.control) 2026-08-18 13:47:29.009979 | node1 | ok: [node1] => (item=molecule-1.rgw.log) 2026-08-18 13:47:29.010297 | node1 | ok: [node1] => (item=molecule-1.rgw.meta) 2026-08-18 13:47:29.010538 | node1 | 2026-08-18 13:47:29.010796 | node1 | TASK [vexxhost.ceph.configure : Configure CephX keys] ************************** 2026-08-18 13:47:29.011106 | node1 | Tuesday 18 August 2026 13:47:29 +0000 (0:00:37.032) 0:02:30.588 ******** 2026-08-18 13:47:29.071165 | node1 | skipping: [node1] 2026-08-18 13:47:29.071391 | node1 | skipping: [node2] 2026-08-18 13:47:29.071629 | node1 | skipping: [node3] 2026-08-18 13:47:29.071874 | node1 | 2026-08-18 13:47:29.072219 | node1 | TASK [vexxhost.ceph.configure : Enroll the dashboard TLS certificate] ********** 2026-08-18 13:47:29.072450 | node1 | Tuesday 18 August 2026 13:47:29 +0000 (0:00:00.064) 0:02:30.653 ******** 2026-08-18 13:47:29.213614 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard.yml for node1, node2, node3 2026-08-18 13:47:29.213771 | node1 | 2026-08-18 13:47:29.214094 | node1 | TASK [vexxhost.ceph.configure : Check whether the dashboard mgr module is enabled] *** 2026-08-18 13:47:29.214331 | node1 | Tuesday 18 August 2026 13:47:29 +0000 (0:00:00.142) 0:02:30.795 ******** 2026-08-18 13:47:31.045127 | node1 | ok: [node1] 2026-08-18 13:47:31.045328 | node1 | 2026-08-18 13:47:31.045613 | node1 | TASK [vexxhost.ceph.configure : Get the current dashboard certificate] ********* 2026-08-18 13:47:31.045859 | node1 | Tuesday 18 August 2026 13:47:31 +0000 (0:00:01.830) 0:02:32.626 ******** 2026-08-18 13:47:32.960992 | node1 | ok: [node1] 2026-08-18 13:47:32.961265 | node1 | 2026-08-18 13:47:32.961534 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate] ***************** 2026-08-18 13:47:32.961837 | node1 | Tuesday 18 August 2026 13:47:32 +0000 (0:00:01.915) 0:02:34.542 ******** 2026-08-18 13:47:33.003256 | node1 | skipping: [node1] 2026-08-18 13:47:33.003482 | node1 | 2026-08-18 13:47:33.003757 | node1 | TASK [vexxhost.ceph.configure : Set the dashboard certificate key] ************* 2026-08-18 13:47:33.004068 | node1 | Tuesday 18 August 2026 13:47:33 +0000 (0:00:00.042) 0:02:34.585 ******** 2026-08-18 13:47:33.048457 | node1 | skipping: [node1] 2026-08-18 13:47:33.048616 | node1 | 2026-08-18 13:47:33.049146 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 13:47:33.049382 | node1 | Tuesday 18 August 2026 13:47:33 +0000 (0:00:00.044) 0:02:34.630 ******** 2026-08-18 13:47:33.093263 | node1 | skipping: [node1] 2026-08-18 13:47:33.093333 | node1 | 2026-08-18 13:47:33.093543 | node1 | TASK [vexxhost.ceph.configure : Create a self-signed certificate] ************** 2026-08-18 13:47:33.093708 | node1 | Tuesday 18 August 2026 13:47:33 +0000 (0:00:00.044) 0:02:34.674 ******** 2026-08-18 13:47:33.143420 | node1 | skipping: [node1] 2026-08-18 13:47:33.143620 | node1 | 2026-08-18 13:47:33.143974 | node1 | TASK [vexxhost.ceph.configure : Flag the dashboard for reload] ***************** 2026-08-18 13:47:33.144251 | node1 | Tuesday 18 August 2026 13:47:33 +0000 (0:00:00.049) 0:02:34.724 ******** 2026-08-18 13:47:33.188163 | node1 | skipping: [node1] 2026-08-18 13:47:33.188232 | node1 | 2026-08-18 13:47:33.188374 | node1 | TASK [vexxhost.ceph.configure : Reload the dashboard so it picks up the certificate] *** 2026-08-18 13:47:33.188492 | node1 | Tuesday 18 August 2026 13:47:33 +0000 (0:00:00.045) 0:02:34.770 ******** 2026-08-18 13:47:33.259431 | node1 | skipping: [node1] => (item=ceph mgr module disable dashboard) 2026-08-18 13:47:33.259752 | node1 | skipping: [node1] => (item=ceph mgr module enable dashboard) 2026-08-18 13:47:33.260146 | node1 | skipping: [node1] => (item=ceph orch reconfig mgr) 2026-08-18 13:47:33.260430 | node1 | skipping: [node1] 2026-08-18 13:47:33.260692 | node1 | 2026-08-18 13:47:33.261057 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 13:47:33.261411 | node1 | Tuesday 18 August 2026 13:47:33 +0000 (0:00:00.071) 0:02:34.841 ******** 2026-08-18 13:47:33.350066 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/configure/tasks/dashboard_users.yml for node1, node2, node3 2026-08-18 13:47:33.350167 | node1 | 2026-08-18 13:47:33.350304 | node1 | TASK [vexxhost.ceph.configure : Configure dashboard users] ********************* 2026-08-18 13:47:33.350435 | node1 | Tuesday 18 August 2026 13:47:33 +0000 (0:00:00.090) 0:02:34.932 ******** 2026-08-18 13:47:40.807995 | node1 | ok: [node1] => (item=None) 2026-08-18 13:47:40.808072 | node1 | ok: [node1] => (item=None) 2026-08-18 13:47:40.808126 | node1 | ok: [node1 -> {{ ceph_configure_admin_host }}] 2026-08-18 13:47:40.808234 | node1 | 2026-08-18 13:47:40.808348 | node1 | PLAY [Deploy Ceph RADOS Gateways] ********************************************** 2026-08-18 13:47:40.808450 | node1 | 2026-08-18 13:47:40.808562 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:47:40.808663 | node1 | Tuesday 18 August 2026 13:47:40 +0000 (0:00:07.446) 0:02:42.378 ******** 2026-08-18 13:47:43.007478 | node1 | ok: [node1] 2026-08-18 13:47:43.007550 | node1 | ok: [node2] 2026-08-18 13:47:43.007623 | node1 | ok: [node3] 2026-08-18 13:47:43.007725 | node1 | 2026-08-18 13:47:43.007865 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:47:43.008039 | node1 | Tuesday 18 August 2026 13:47:43 +0000 (0:00:02.210) 0:02:44.588 ******** 2026-08-18 13:47:43.311462 | node1 | ok: [node1] 2026-08-18 13:47:43.311529 | node1 | ok: [node2] 2026-08-18 13:47:43.311616 | node1 | ok: [node3] 2026-08-18 13:47:43.311719 | node1 | 2026-08-18 13:47:43.311836 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:47:43.313646 | node1 | Tuesday 18 August 2026 13:47:43 +0000 (0:00:00.304) 0:02:44.892 ******** 2026-08-18 13:47:43.355394 | node1 | skipping: [node1] 2026-08-18 13:47:43.413744 | node1 | skipping: [node2] 2026-08-18 13:47:43.414013 | node1 | skipping: [node3] 2026-08-18 13:47:43.414324 | node1 | 2026-08-18 13:47:43.414556 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:47:43.414798 | node1 | Tuesday 18 August 2026 13:47:43 +0000 (0:00:00.103) 0:02:44.996 ******** 2026-08-18 13:47:43.729648 | node1 | ok: [node1] 2026-08-18 13:47:43.729885 | node1 | ok: [node2] 2026-08-18 13:47:43.730161 | node1 | ok: [node3] 2026-08-18 13:47:43.730382 | node1 | 2026-08-18 13:47:43.730632 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:47:43.730880 | node1 | Tuesday 18 August 2026 13:47:43 +0000 (0:00:00.315) 0:02:45.311 ******** 2026-08-18 13:47:43.845569 | node1 | ok: [node1] => { 2026-08-18 13:47:43.845810 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:47:43.846085 | node1 | } 2026-08-18 13:47:43.846329 | node1 | ok: [node2] => { 2026-08-18 13:47:43.846581 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:47:43.846808 | node1 | } 2026-08-18 13:47:43.847079 | node1 | ok: [node3] => { 2026-08-18 13:47:43.847334 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:47:43.847557 | node1 | } 2026-08-18 13:47:43.847775 | node1 | 2026-08-18 13:47:43.848058 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:47:43.848306 | node1 | Tuesday 18 August 2026 13:47:43 +0000 (0:00:00.116) 0:02:45.427 ******** 2026-08-18 13:47:44.280589 | node1 | ok: [node1] 2026-08-18 13:47:44.280814 | node1 | ok: [node3] 2026-08-18 13:47:44.281205 | node1 | ok: [node2] 2026-08-18 13:47:44.281380 | node1 | 2026-08-18 13:47:44.281675 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:47:44.282128 | node1 | Tuesday 18 August 2026 13:47:44 +0000 (0:00:00.434) 0:02:45.862 ******** 2026-08-18 13:47:44.314576 | node1 | skipping: [node1] 2026-08-18 13:47:44.393447 | node1 | skipping: [node2] 2026-08-18 13:47:44.393737 | node1 | skipping: [node3] 2026-08-18 13:47:44.394057 | node1 | 2026-08-18 13:47:44.394371 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:47:44.394656 | node1 | Tuesday 18 August 2026 13:47:44 +0000 (0:00:00.112) 0:02:45.975 ******** 2026-08-18 13:47:44.502815 | node1 | skipping: [node1] 2026-08-18 13:47:44.503095 | node1 | skipping: [node2] 2026-08-18 13:47:44.503327 | node1 | skipping: [node3] 2026-08-18 13:47:44.503557 | node1 | 2026-08-18 13:47:44.503837 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:47:44.504164 | node1 | Tuesday 18 August 2026 13:47:44 +0000 (0:00:00.109) 0:02:46.084 ******** 2026-08-18 13:47:44.796166 | node1 | ok: [node1] 2026-08-18 13:47:44.796385 | node1 | ok: [node2] 2026-08-18 13:47:44.796624 | node1 | ok: [node3] 2026-08-18 13:47:44.796850 | node1 | 2026-08-18 13:47:44.797157 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:47:44.797397 | node1 | Tuesday 18 August 2026 13:47:44 +0000 (0:00:00.293) 0:02:46.378 ******** 2026-08-18 13:47:45.885883 | node1 | ok: [node3] 2026-08-18 13:47:45.886091 | node1 | ok: [node2] 2026-08-18 13:47:45.886335 | node1 | ok: [node1] 2026-08-18 13:47:45.886575 | node1 | 2026-08-18 13:47:45.886840 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:47:45.887299 | node1 | Tuesday 18 August 2026 13:47:45 +0000 (0:00:01.089) 0:02:47.467 ******** 2026-08-18 13:47:46.134281 | node1 | ok: [node1] => { 2026-08-18 13:47:46.134491 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:47:46.134748 | node1 | } 2026-08-18 13:47:46.135059 | node1 | ok: [node2] => { 2026-08-18 13:47:46.135337 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:47:46.135585 | node1 | } 2026-08-18 13:47:46.135864 | node1 | ok: [node3] => { 2026-08-18 13:47:46.136264 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:47:46.136445 | node1 | } 2026-08-18 13:47:46.136678 | node1 | 2026-08-18 13:47:46.136971 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:47:46.137247 | node1 | Tuesday 18 August 2026 13:47:46 +0000 (0:00:00.248) 0:02:47.715 ******** 2026-08-18 13:47:46.730676 | node1 | ok: [node1] 2026-08-18 13:47:46.730892 | node1 | ok: [node2] 2026-08-18 13:47:46.731163 | node1 | ok: [node3] 2026-08-18 13:47:46.731379 | node1 | 2026-08-18 13:47:46.731624 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:47:46.731905 | node1 | Tuesday 18 August 2026 13:47:46 +0000 (0:00:00.596) 0:02:48.312 ******** 2026-08-18 13:47:49.073986 | node1 | ok: [node2] 2026-08-18 13:47:49.074214 | node1 | ok: [node3] 2026-08-18 13:47:49.074450 | node1 | ok: [node1] 2026-08-18 13:47:49.074673 | node1 | 2026-08-18 13:47:49.074922 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:47:49.075210 | node1 | Tuesday 18 August 2026 13:47:49 +0000 (0:00:02.342) 0:02:50.655 ******** 2026-08-18 13:47:49.113854 | node1 | skipping: [node1] 2026-08-18 13:47:49.147616 | node1 | skipping: [node2] 2026-08-18 13:47:49.147839 | node1 | skipping: [node3] 2026-08-18 13:47:49.148124 | node1 | 2026-08-18 13:47:49.148365 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:47:49.148606 | node1 | Tuesday 18 August 2026 13:47:49 +0000 (0:00:00.074) 0:02:50.730 ******** 2026-08-18 13:47:49.215077 | node1 | skipping: [node1] 2026-08-18 13:47:49.215312 | node1 | skipping: [node2] 2026-08-18 13:47:49.215546 | node1 | skipping: [node3] 2026-08-18 13:47:49.215766 | node1 | 2026-08-18 13:47:49.216045 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:47:49.216353 | node1 | Tuesday 18 August 2026 13:47:49 +0000 (0:00:00.067) 0:02:50.797 ******** 2026-08-18 13:47:49.279839 | node1 | skipping: [node1] 2026-08-18 13:47:49.280119 | node1 | skipping: [node2] 2026-08-18 13:47:49.280355 | node1 | skipping: [node3] 2026-08-18 13:47:49.280573 | node1 | 2026-08-18 13:47:49.280821 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:47:49.281106 | node1 | Tuesday 18 August 2026 13:47:49 +0000 (0:00:00.064) 0:02:50.862 ******** 2026-08-18 13:47:51.333164 | node1 | ok: [node2] 2026-08-18 13:47:51.333231 | node1 | ok: [node1] 2026-08-18 13:47:51.333241 | node1 | ok: [node3] 2026-08-18 13:47:51.333251 | node1 | 2026-08-18 13:47:51.333260 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:47:51.333379 | node1 | Tuesday 18 August 2026 13:47:51 +0000 (0:00:02.051) 0:02:52.913 ******** 2026-08-18 13:47:51.930059 | node1 | ok: [node2] 2026-08-18 13:47:51.930120 | node1 | ok: [node1] 2026-08-18 13:47:51.930143 | node1 | ok: [node3] 2026-08-18 13:47:51.930153 | node1 | 2026-08-18 13:47:51.930162 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:47:51.930171 | node1 | Tuesday 18 August 2026 13:47:51 +0000 (0:00:00.596) 0:02:53.510 ******** 2026-08-18 13:47:53.100047 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:47:53.100097 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:47:53.100108 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:47:53.100117 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:47:53.100127 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:47:53.100142 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:47:53.100191 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:47:53.100206 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:47:53.100382 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:47:53.100541 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:47:53.100699 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:47:53.100858 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:47:53.101170 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:47:53.101437 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:47:53.101710 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:47:53.101978 | node1 | 2026-08-18 13:47:53.102254 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:47:53.102517 | node1 | Tuesday 18 August 2026 13:47:53 +0000 (0:00:01.171) 0:02:54.681 ******** 2026-08-18 13:47:53.981502 | node1 | ok: [node3] 2026-08-18 13:47:53.981732 | node1 | ok: [node1] 2026-08-18 13:47:53.982011 | node1 | ok: [node2] 2026-08-18 13:47:53.982254 | node1 | 2026-08-18 13:47:53.982527 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:47:53.982794 | node1 | Tuesday 18 August 2026 13:47:53 +0000 (0:00:00.783) 0:02:55.464 ******** 2026-08-18 13:47:53.983070 | node1 | 2026-08-18 13:47:53.983328 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:47:53.983593 | node1 | Tuesday 18 August 2026 13:47:53 +0000 (0:00:00.006) 0:02:55.471 ******** 2026-08-18 13:47:53.983852 | node1 | 2026-08-18 13:47:53.984180 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:47:53.984429 | node1 | Tuesday 18 August 2026 13:47:53 +0000 (0:00:00.009) 0:02:55.480 ******** 2026-08-18 13:47:53.984660 | node1 | 2026-08-18 13:47:53.984926 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:47:53.985229 | node1 | Tuesday 18 August 2026 13:47:53 +0000 (0:00:00.082) 0:02:55.562 ******** 2026-08-18 13:47:54.508118 | node1 | ok: [node1] 2026-08-18 13:47:54.508186 | node1 | ok: [node3] 2026-08-18 13:47:54.509111 | node1 | ok: [node2] 2026-08-18 13:47:54.509189 | node1 | 2026-08-18 13:47:54.509200 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:47:54.509210 | node1 | Tuesday 18 August 2026 13:47:54 +0000 (0:00:00.526) 0:02:56.089 ******** 2026-08-18 13:47:54.826031 | node1 | ok: [node1] 2026-08-18 13:47:54.826103 | node1 | ok: [node3] 2026-08-18 13:47:54.826113 | node1 | ok: [node2] 2026-08-18 13:47:54.826122 | node1 | 2026-08-18 13:47:54.826131 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:47:54.826141 | node1 | Tuesday 18 August 2026 13:47:54 +0000 (0:00:00.316) 0:02:56.405 ******** 2026-08-18 13:47:54.948131 | node1 | ok: [node1] => { 2026-08-18 13:47:54.948254 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:47:54.950015 | node1 | } 2026-08-18 13:47:54.950076 | node1 | ok: [node2] => { 2026-08-18 13:47:54.950086 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:47:54.950095 | node1 | } 2026-08-18 13:47:54.950104 | node1 | ok: [node3] => { 2026-08-18 13:47:54.950113 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:47:54.950371 | node1 | } 2026-08-18 13:47:54.950663 | node1 | 2026-08-18 13:47:54.951051 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:47:54.951397 | node1 | Tuesday 18 August 2026 13:47:54 +0000 (0:00:00.123) 0:02:56.529 ******** 2026-08-18 13:47:55.427753 | node1 | ok: [node1] 2026-08-18 13:47:55.428162 | node1 | ok: [node3] 2026-08-18 13:47:55.428467 | node1 | ok: [node2] 2026-08-18 13:47:55.428743 | node1 | 2026-08-18 13:47:55.429088 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:47:55.429429 | node1 | Tuesday 18 August 2026 13:47:55 +0000 (0:00:00.480) 0:02:57.009 ******** 2026-08-18 13:47:58.811914 | node1 | ok: [node3] 2026-08-18 13:47:58.812420 | node1 | ok: [node2] 2026-08-18 13:47:58.812773 | node1 | ok: [node1] 2026-08-18 13:47:58.813139 | node1 | 2026-08-18 13:47:58.813498 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:47:58.813833 | node1 | Tuesday 18 August 2026 13:47:58 +0000 (0:00:03.383) 0:03:00.393 ******** 2026-08-18 13:48:00.925437 | node1 | ok: [node1] 2026-08-18 13:48:00.926761 | node1 | ok: [node3] 2026-08-18 13:48:00.927063 | node1 | ok: [node2] 2026-08-18 13:48:00.927295 | node1 | 2026-08-18 13:48:00.927546 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:48:00.927791 | node1 | Tuesday 18 August 2026 13:48:00 +0000 (0:00:02.113) 0:03:02.507 ******** 2026-08-18 13:48:01.195814 | node1 | ok: [node1] 2026-08-18 13:48:01.196108 | node1 | ok: [node2] 2026-08-18 13:48:01.196364 | node1 | ok: [node3] 2026-08-18 13:48:01.196603 | node1 | 2026-08-18 13:48:01.196982 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:48:01.197253 | node1 | Tuesday 18 August 2026 13:48:01 +0000 (0:00:00.270) 0:03:02.777 ******** 2026-08-18 13:48:01.782867 | node1 | ok: [node1] 2026-08-18 13:48:01.783127 | node1 | ok: [node2] 2026-08-18 13:48:01.783372 | node1 | ok: [node3] 2026-08-18 13:48:01.783591 | node1 | 2026-08-18 13:48:01.783838 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:48:01.784113 | node1 | Tuesday 18 August 2026 13:48:01 +0000 (0:00:00.586) 0:03:03.364 ******** 2026-08-18 13:48:02.485205 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:48:02.485457 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:48:02.485692 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:48:02.485954 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:48:02.486227 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:48:02.486474 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:48:02.486735 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:48:02.487041 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:48:02.487296 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:48:02.487519 | node1 | 2026-08-18 13:48:02.487762 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:48:02.488036 | node1 | Tuesday 18 August 2026 13:48:02 +0000 (0:00:00.701) 0:03:04.066 ******** 2026-08-18 13:48:03.053949 | node1 | ok: [node2] 2026-08-18 13:48:03.054047 | node1 | ok: [node1] 2026-08-18 13:48:03.054161 | node1 | ok: [node3] 2026-08-18 13:48:03.054266 | node1 | 2026-08-18 13:48:03.054384 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:48:03.054500 | node1 | Tuesday 18 August 2026 13:48:03 +0000 (0:00:00.569) 0:03:04.635 ******** 2026-08-18 13:48:03.653389 | node1 | ok: [node1] 2026-08-18 13:48:03.653643 | node1 | ok: [node2] 2026-08-18 13:48:03.653891 | node1 | ok: [node3] 2026-08-18 13:48:03.654193 | node1 | 2026-08-18 13:48:03.654464 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:48:03.654733 | node1 | Tuesday 18 August 2026 13:48:03 +0000 (0:00:00.577) 0:03:05.213 ******** 2026-08-18 13:48:03.655002 | node1 | 2026-08-18 13:48:03.655278 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:48:03.655528 | node1 | Tuesday 18 August 2026 13:48:03 +0000 (0:00:00.006) 0:03:05.220 ******** 2026-08-18 13:48:03.655765 | node1 | 2026-08-18 13:48:03.656063 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:48:03.656323 | node1 | Tuesday 18 August 2026 13:48:03 +0000 (0:00:00.008) 0:03:05.228 ******** 2026-08-18 13:48:03.656546 | node1 | 2026-08-18 13:48:03.656812 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:48:03.657202 | node1 | Tuesday 18 August 2026 13:48:03 +0000 (0:00:00.006) 0:03:05.235 ******** 2026-08-18 13:48:04.152979 | node1 | ok: [node2] 2026-08-18 13:48:04.153194 | node1 | ok: [node1] 2026-08-18 13:48:04.153432 | node1 | ok: [node3] 2026-08-18 13:48:04.153703 | node1 | 2026-08-18 13:48:04.153973 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:48:04.154217 | node1 | Tuesday 18 August 2026 13:48:04 +0000 (0:00:00.499) 0:03:05.734 ******** 2026-08-18 13:48:04.232174 | node1 | ok: [node1] => { 2026-08-18 13:48:04.232409 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:48:04.232639 | node1 | } 2026-08-18 13:48:04.232873 | node1 | ok: [node2] => { 2026-08-18 13:48:04.233154 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:48:04.233378 | node1 | } 2026-08-18 13:48:04.233611 | node1 | ok: [node3] => { 2026-08-18 13:48:04.233882 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:48:04.234137 | node1 | } 2026-08-18 13:48:04.234342 | node1 | 2026-08-18 13:48:04.234602 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:48:04.234848 | node1 | Tuesday 18 August 2026 13:48:04 +0000 (0:00:00.079) 0:03:05.814 ******** 2026-08-18 13:48:04.607193 | node1 | ok: [node1] 2026-08-18 13:48:04.607366 | node1 | ok: [node2] 2026-08-18 13:48:04.607601 | node1 | ok: [node3] 2026-08-18 13:48:04.607821 | node1 | 2026-08-18 13:48:04.608123 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:48:04.608358 | node1 | Tuesday 18 August 2026 13:48:04 +0000 (0:00:00.374) 0:03:06.188 ******** 2026-08-18 13:48:04.697554 | node1 | skipping: [node1] 2026-08-18 13:48:04.697610 | node1 | skipping: [node2] 2026-08-18 13:48:04.697619 | node1 | skipping: [node3] 2026-08-18 13:48:04.697628 | node1 | 2026-08-18 13:48:04.697638 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:48:04.697647 | node1 | Tuesday 18 August 2026 13:48:04 +0000 (0:00:00.089) 0:03:06.278 ******** 2026-08-18 13:48:04.788117 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:48:04.788184 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:48:04.788594 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:48:04.788826 | node1 | 2026-08-18 13:48:04.789129 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:48:04.789366 | node1 | Tuesday 18 August 2026 13:48:04 +0000 (0:00:00.088) 0:03:06.367 ******** 2026-08-18 13:48:05.749922 | node1 | ok: [node2] 2026-08-18 13:48:05.750961 | node1 | ok: [node3] 2026-08-18 13:48:05.750987 | node1 | ok: [node1] 2026-08-18 13:48:05.750996 | node1 | 2026-08-18 13:48:05.751006 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:48:05.751190 | node1 | Tuesday 18 August 2026 13:48:05 +0000 (0:00:00.964) 0:03:07.331 ******** 2026-08-18 13:48:07.616094 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:48:07.617964 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:48:07.618027 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:48:07.618064 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:48:07.618073 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:48:07.618081 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:48:07.618090 | node1 | 2026-08-18 13:48:07.618100 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:48:07.618109 | node1 | Tuesday 18 August 2026 13:48:07 +0000 (0:00:01.865) 0:03:09.197 ******** 2026-08-18 13:48:07.902024 | node1 | ok: [node1] 2026-08-18 13:48:07.902087 | node1 | ok: [node2] 2026-08-18 13:48:07.903092 | node1 | ok: [node3] 2026-08-18 13:48:07.903141 | node1 | 2026-08-18 13:48:07.903152 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:48:07.903168 | node1 | Tuesday 18 August 2026 13:48:07 +0000 (0:00:00.286) 0:03:09.483 ******** 2026-08-18 13:48:08.247746 | node1 | ok: [node1] 2026-08-18 13:48:08.247854 | node1 | ok: [node2] 2026-08-18 13:48:08.248034 | node1 | ok: [node3] 2026-08-18 13:48:08.248575 | node1 | 2026-08-18 13:48:08.248585 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:48:08.248590 | node1 | Tuesday 18 August 2026 13:48:08 +0000 (0:00:00.345) 0:03:09.828 ******** 2026-08-18 13:48:08.812869 | node1 | ok: [node1] 2026-08-18 13:48:08.812918 | node1 | ok: [node3] 2026-08-18 13:48:08.812963 | node1 | ok: [node2] 2026-08-18 13:48:08.812977 | node1 | 2026-08-18 13:48:08.813001 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:48:08.813011 | node1 | Tuesday 18 August 2026 13:48:08 +0000 (0:00:00.564) 0:03:10.393 ******** 2026-08-18 13:48:08.900152 | node1 | skipping: [node1] 2026-08-18 13:48:08.900217 | node1 | skipping: [node2] 2026-08-18 13:48:08.900434 | node1 | skipping: [node3] 2026-08-18 13:48:08.900648 | node1 | 2026-08-18 13:48:08.900920 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:48:08.901448 | node1 | Tuesday 18 August 2026 13:48:08 +0000 (0:00:00.088) 0:03:10.481 ******** 2026-08-18 13:48:09.372738 | node1 | ok: [node3] 2026-08-18 13:48:09.373002 | node1 | ok: [node1] 2026-08-18 13:48:09.373233 | node1 | ok: [node2] 2026-08-18 13:48:09.373457 | node1 | 2026-08-18 13:48:09.373727 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:48:09.374025 | node1 | Tuesday 18 August 2026 13:48:09 +0000 (0:00:00.472) 0:03:10.954 ******** 2026-08-18 13:48:10.017126 | node1 | ok: [node2] 2026-08-18 13:48:10.017197 | node1 | ok: [node1] 2026-08-18 13:48:10.017207 | node1 | ok: [node3] 2026-08-18 13:48:10.017223 | node1 | 2026-08-18 13:48:10.017535 | node1 | TASK [vexxhost.ceph.rgw : Locate the cluster admin host] *********************** 2026-08-18 13:48:10.017881 | node1 | Tuesday 18 August 2026 13:48:10 +0000 (0:00:00.644) 0:03:11.598 ******** 2026-08-18 13:48:10.796964 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:48:10.797002 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:48:10.797010 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:48:10.797234 | node1 | 2026-08-18 13:48:10.797479 | node1 | TASK [vexxhost.ceph.rgw : Select the admin host] ******************************* 2026-08-18 13:48:10.797717 | node1 | Tuesday 18 August 2026 13:48:10 +0000 (0:00:00.779) 0:03:12.378 ******** 2026-08-18 13:48:10.889368 | node1 | ok: [node1] 2026-08-18 13:48:10.889602 | node1 | ok: [node2] 2026-08-18 13:48:10.889831 | node1 | ok: [node3] 2026-08-18 13:48:10.890090 | node1 | 2026-08-18 13:48:10.890336 | node1 | TASK [vexxhost.ceph.rgw : Set the admin mon config path] *********************** 2026-08-18 13:48:10.890576 | node1 | Tuesday 18 August 2026 13:48:10 +0000 (0:00:00.092) 0:03:12.471 ******** 2026-08-18 13:48:10.978353 | node1 | ok: [node1] 2026-08-18 13:48:10.978664 | node1 | ok: [node2] 2026-08-18 13:48:10.978981 | node1 | ok: [node3] 2026-08-18 13:48:10.979212 | node1 | 2026-08-18 13:48:10.979598 | node1 | TASK [Register this RGW host] ************************************************** 2026-08-18 13:48:10.979894 | node1 | Tuesday 18 August 2026 13:48:10 +0000 (0:00:00.088) 0:03:12.560 ******** 2026-08-18 13:48:11.112142 | node1 | included: vexxhost.ceph.cephadm_host for node1, node2, node3 2026-08-18 13:48:11.112391 | node1 | 2026-08-18 13:48:11.113190 | node1 | TASK [vexxhost.ceph.cephadm_host : Get the cephadm public key] ***************** 2026-08-18 13:48:11.113297 | node1 | Tuesday 18 August 2026 13:48:11 +0000 (0:00:00.133) 0:03:12.693 ******** 2026-08-18 13:48:12.857491 | node1 | ok: [node1] 2026-08-18 13:48:12.857665 | node1 | 2026-08-18 13:48:12.858020 | node1 | TASK [vexxhost.ceph.cephadm_host : Resolve the cluster SSH public key to authorise] *** 2026-08-18 13:48:12.858276 | node1 | Tuesday 18 August 2026 13:48:12 +0000 (0:00:01.744) 0:03:14.438 ******** 2026-08-18 13:48:12.939710 | node1 | ok: [node1] 2026-08-18 13:48:12.940032 | node1 | ok: [node2] 2026-08-18 13:48:12.940265 | node1 | ok: [node3] 2026-08-18 13:48:12.940532 | node1 | 2026-08-18 13:48:12.940882 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert a cluster SSH key was resolved] ****** 2026-08-18 13:48:12.941198 | node1 | Tuesday 18 August 2026 13:48:12 +0000 (0:00:00.082) 0:03:14.521 ******** 2026-08-18 13:48:13.027005 | node1 | ok: [node1] => { 2026-08-18 13:48:13.027224 | node1 | "changed": false, 2026-08-18 13:48:13.027463 | node1 | "msg": "All assertions passed" 2026-08-18 13:48:13.027741 | node1 | } 2026-08-18 13:48:13.028013 | node1 | ok: [node2] => { 2026-08-18 13:48:13.028258 | node1 | "changed": false, 2026-08-18 13:48:13.028535 | node1 | "msg": "All assertions passed" 2026-08-18 13:48:13.028779 | node1 | } 2026-08-18 13:48:13.029067 | node1 | ok: [node3] => { 2026-08-18 13:48:13.029299 | node1 | "changed": false, 2026-08-18 13:48:13.029584 | node1 | "msg": "All assertions passed" 2026-08-18 13:48:13.029818 | node1 | } 2026-08-18 13:48:13.030137 | node1 | 2026-08-18 13:48:13.030339 | node1 | TASK [vexxhost.ceph.cephadm_host : Authorise the cluster SSH key for the cephadm user] *** 2026-08-18 13:48:13.030578 | node1 | Tuesday 18 August 2026 13:48:13 +0000 (0:00:00.087) 0:03:14.609 ******** 2026-08-18 13:48:13.407584 | node1 | ok: [node2] 2026-08-18 13:48:13.407753 | node1 | ok: [node1] 2026-08-18 13:48:13.408011 | node1 | ok: [node3] 2026-08-18 13:48:13.408240 | node1 | 2026-08-18 13:48:13.408524 | node1 | TASK [vexxhost.ceph.cephadm_host : Derive this host's cephadm labels from inventory membership] *** 2026-08-18 13:48:13.408774 | node1 | Tuesday 18 August 2026 13:48:13 +0000 (0:00:00.379) 0:03:14.989 ******** 2026-08-18 13:48:13.559994 | node1 | ok: [node1] 2026-08-18 13:48:13.560235 | node1 | ok: [node2] 2026-08-18 13:48:13.560486 | node1 | ok: [node3] 2026-08-18 13:48:13.560754 | node1 | 2026-08-18 13:48:13.561172 | node1 | TASK [vexxhost.ceph.cephadm_host : Find this host's addresses on the public network] *** 2026-08-18 13:48:13.561376 | node1 | Tuesday 18 August 2026 13:48:13 +0000 (0:00:00.152) 0:03:15.141 ******** 2026-08-18 13:48:13.679673 | node1 | ok: [node1] 2026-08-18 13:48:13.679866 | node1 | ok: [node2] 2026-08-18 13:48:13.680228 | node1 | ok: [node3] 2026-08-18 13:48:13.680383 | node1 | 2026-08-18 13:48:13.680647 | node1 | TASK [vexxhost.ceph.cephadm_host : Assert exactly one address matches the public network] *** 2026-08-18 13:48:13.680900 | node1 | Tuesday 18 August 2026 13:48:13 +0000 (0:00:00.119) 0:03:15.261 ******** 2026-08-18 13:48:13.762434 | node1 | ok: [node1] => { 2026-08-18 13:48:13.762487 | node1 | "changed": false, 2026-08-18 13:48:13.762674 | node1 | "msg": "All assertions passed" 2026-08-18 13:48:13.762923 | node1 | } 2026-08-18 13:48:13.763283 | node1 | ok: [node2] => { 2026-08-18 13:48:13.763542 | node1 | "changed": false, 2026-08-18 13:48:13.763790 | node1 | "msg": "All assertions passed" 2026-08-18 13:48:13.764067 | node1 | } 2026-08-18 13:48:13.764308 | node1 | ok: [node3] => { 2026-08-18 13:48:13.764553 | node1 | "changed": false, 2026-08-18 13:48:13.764799 | node1 | "msg": "All assertions passed" 2026-08-18 13:48:13.765138 | node1 | } 2026-08-18 13:48:13.765305 | node1 | 2026-08-18 13:48:13.765559 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host (labels)] ***************** 2026-08-18 13:48:13.765803 | node1 | Tuesday 18 August 2026 13:48:13 +0000 (0:00:00.082) 0:03:15.344 ******** 2026-08-18 13:48:15.651991 | node1 | ok: [node1] 2026-08-18 13:48:15.652059 | node1 | ok: [node2 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:48:15.652067 | node1 | ok: [node3 -> node1({{ nodepool.private_ipv4 }})] 2026-08-18 13:48:15.652074 | node1 | 2026-08-18 13:48:15.652081 | node1 | TASK [vexxhost.ceph.cephadm_host : Register the host with a CRUSH location] **** 2026-08-18 13:48:15.652093 | node1 | Tuesday 18 August 2026 13:48:15 +0000 (0:00:01.888) 0:03:17.232 ******** 2026-08-18 13:48:15.781006 | node1 | skipping: [node1] 2026-08-18 13:48:15.781292 | node1 | skipping: [node2] 2026-08-18 13:48:15.781562 | node1 | skipping: [node3] 2026-08-18 13:48:15.781802 | node1 | 2026-08-18 13:48:15.782129 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW multisite (realm, zonegroup, zone)] **** 2026-08-18 13:48:15.782409 | node1 | Tuesday 18 August 2026 13:48:15 +0000 (0:00:00.130) 0:03:17.362 ******** 2026-08-18 13:48:15.854367 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/multisite.yml for node1 2026-08-18 13:48:15.854575 | node1 | 2026-08-18 13:48:15.854840 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW realms] ******************************** 2026-08-18 13:48:15.855118 | node1 | Tuesday 18 August 2026 13:48:15 +0000 (0:00:00.073) 0:03:17.436 ******** 2026-08-18 13:48:17.373653 | node1 | ok: [node1] => (item=molecule) 2026-08-18 13:48:17.373760 | node1 | 2026-08-18 13:48:17.373981 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zonegroups] **************************** 2026-08-18 13:48:17.374145 | node1 | Tuesday 18 August 2026 13:48:17 +0000 (0:00:01.519) 0:03:18.955 ******** 2026-08-18 13:48:19.880318 | node1 | ok: [node1] => (item=molecule) 2026-08-18 13:48:19.880500 | node1 | 2026-08-18 13:48:19.880785 | node1 | TASK [vexxhost.ceph.rgw : Configure RGW zones] ********************************* 2026-08-18 13:48:19.881112 | node1 | Tuesday 18 August 2026 13:48:19 +0000 (0:00:02.506) 0:03:21.461 ******** 2026-08-18 13:48:23.520724 | node1 | ok: [node1] => (item=None) 2026-08-18 13:48:23.520988 | node1 | ok: [node1 -> {{ ceph_rgw_admin_host }}] 2026-08-18 13:48:23.521251 | node1 | 2026-08-18 13:48:23.521511 | node1 | TASK [vexxhost.ceph.rgw : Commit the RGW period] ******************************* 2026-08-18 13:48:23.521766 | node1 | Tuesday 18 August 2026 13:48:23 +0000 (0:00:03.640) 0:03:25.102 ******** 2026-08-18 13:48:23.597977 | node1 | skipping: [node1] 2026-08-18 13:48:23.598187 | node1 | 2026-08-18 13:48:23.598439 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 13:48:23.598685 | node1 | Tuesday 18 August 2026 13:48:23 +0000 (0:00:00.077) 0:03:25.180 ******** 2026-08-18 13:48:23.664855 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/services.yml for node1 2026-08-18 13:48:23.665130 | node1 | 2026-08-18 13:48:23.665382 | node1 | TASK [vexxhost.ceph.rgw : Snapshot current RGW service specs] ****************** 2026-08-18 13:48:23.665636 | node1 | Tuesday 18 August 2026 13:48:23 +0000 (0:00:00.066) 0:03:25.246 ******** 2026-08-18 13:48:25.488290 | node1 | ok: [node1] 2026-08-18 13:48:25.488389 | node1 | 2026-08-18 13:48:25.488403 | node1 | TASK [vexxhost.ceph.rgw : Index current RGW specs by service name] ************* 2026-08-18 13:48:25.488417 | node1 | Tuesday 18 August 2026 13:48:25 +0000 (0:00:01.822) 0:03:27.069 ******** 2026-08-18 13:48:25.548049 | node1 | ok: [node1] 2026-08-18 13:48:25.548142 | node1 | 2026-08-18 13:48:25.548720 | node1 | TASK [vexxhost.ceph.rgw : Apply the RGW service specs] ************************* 2026-08-18 13:48:25.548778 | node1 | Tuesday 18 August 2026 13:48:25 +0000 (0:00:00.059) 0:03:27.129 ******** 2026-08-18 13:48:27.397992 | node1 | ok: [node1] => (item=molecule) 2026-08-18 13:48:27.398076 | node1 | 2026-08-18 13:48:27.398091 | node1 | TASK [vexxhost.ceph.rgw : Redeploy RGW services whose redeploy-triggering keys changed] *** 2026-08-18 13:48:27.398120 | node1 | Tuesday 18 August 2026 13:48:27 +0000 (0:00:01.848) 0:03:28.977 ******** 2026-08-18 13:48:27.474047 | node1 | skipping: [node1] => (item=rgw.molecule) 2026-08-18 13:48:27.474268 | node1 | skipping: [node1] 2026-08-18 13:48:27.474488 | node1 | 2026-08-18 13:48:27.474742 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone / zonegroup / pools] ********* 2026-08-18 13:48:27.475046 | node1 | Tuesday 18 August 2026 13:48:27 +0000 (0:00:00.077) 0:03:29.055 ******** 2026-08-18 13:48:27.538834 | node1 | included: /home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/rgw/tasks/cleanup.yml for node1 2026-08-18 13:48:27.539137 | node1 | 2026-08-18 13:48:27.539371 | node1 | TASK [vexxhost.ceph.rgw : List RGW zones] ************************************** 2026-08-18 13:48:27.539694 | node1 | Tuesday 18 August 2026 13:48:27 +0000 (0:00:00.066) 0:03:29.121 ******** 2026-08-18 13:48:28.908054 | node1 | ok: [node1] 2026-08-18 13:48:28.908141 | node1 | 2026-08-18 13:48:28.908156 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zonegroup] ************************ 2026-08-18 13:48:28.908169 | node1 | Tuesday 18 August 2026 13:48:28 +0000 (0:00:01.367) 0:03:30.489 ******** 2026-08-18 13:48:28.960243 | node1 | skipping: [node1] 2026-08-18 13:48:28.960437 | node1 | 2026-08-18 13:48:28.960688 | node1 | TASK [vexxhost.ceph.rgw : Remove the default zone] ***************************** 2026-08-18 13:48:28.960967 | node1 | Tuesday 18 August 2026 13:48:28 +0000 (0:00:00.052) 0:03:30.541 ******** 2026-08-18 13:48:29.015604 | node1 | skipping: [node1] 2026-08-18 13:48:29.015797 | node1 | 2026-08-18 13:48:29.016132 | node1 | TASK [vexxhost.ceph.rgw : Get the current pool deletion setting] *************** 2026-08-18 13:48:29.016373 | node1 | Tuesday 18 August 2026 13:48:29 +0000 (0:00:00.056) 0:03:30.597 ******** 2026-08-18 13:48:29.077411 | node1 | skipping: [node1] 2026-08-18 13:48:29.077635 | node1 | 2026-08-18 13:48:29.077958 | node1 | TASK [vexxhost.ceph.rgw : Allow pool deletion] ********************************* 2026-08-18 13:48:29.078353 | node1 | Tuesday 18 August 2026 13:48:29 +0000 (0:00:00.060) 0:03:30.658 ******** 2026-08-18 13:48:29.134566 | node1 | skipping: [node1] 2026-08-18 13:48:29.134761 | node1 | 2026-08-18 13:48:29.135066 | node1 | TASK [vexxhost.ceph.rgw : Remove the default RGW pools] ************************ 2026-08-18 13:48:29.135310 | node1 | Tuesday 18 August 2026 13:48:29 +0000 (0:00:00.057) 0:03:30.716 ******** 2026-08-18 13:48:29.211383 | node1 | skipping: [node1] => (item=default.rgw.control) 2026-08-18 13:48:29.211572 | node1 | skipping: [node1] => (item=default.rgw.log) 2026-08-18 13:48:29.211819 | node1 | skipping: [node1] => (item=default.rgw.meta) 2026-08-18 13:48:29.212153 | node1 | skipping: [node1] 2026-08-18 13:48:29.212346 | node1 | 2026-08-18 13:48:29.212594 | node1 | TASK [vexxhost.ceph.rgw : Restore the configured pool deletion setting] ******** 2026-08-18 13:48:29.212839 | node1 | Tuesday 18 August 2026 13:48:29 +0000 (0:00:00.077) 0:03:30.793 ******** 2026-08-18 13:48:29.256010 | node1 | skipping: [node1] 2026-08-18 13:48:29.256208 | node1 | 2026-08-18 13:48:29.256461 | node1 | TASK [vexxhost.ceph.rgw : Remove the temporary pool deletion setting] ********** 2026-08-18 13:48:29.256708 | node1 | Tuesday 18 August 2026 13:48:29 +0000 (0:00:00.044) 0:03:30.838 ******** 2026-08-18 13:48:29.361882 | node1 | skipping: [node1] 2026-08-18 13:48:29.362124 | node1 | 2026-08-18 13:48:29.362434 | node1 | PLAY [Deploy Ceph monitoring stack] ******************************************** 2026-08-18 13:48:29.362675 | node1 | 2026-08-18 13:48:29.362962 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:48:29.363245 | node1 | Tuesday 18 August 2026 13:48:29 +0000 (0:00:00.105) 0:03:30.944 ******** 2026-08-18 13:48:30.443994 | node1 | ok: [node1] 2026-08-18 13:48:30.444052 | node1 | ok: [node2] 2026-08-18 13:48:30.444177 | node1 | ok: [node3] 2026-08-18 13:48:30.444279 | node1 | 2026-08-18 13:48:30.444397 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:48:30.444523 | node1 | Tuesday 18 August 2026 13:48:30 +0000 (0:00:01.081) 0:03:32.025 ******** 2026-08-18 13:48:30.760599 | node1 | ok: [node1] 2026-08-18 13:48:30.760871 | node1 | ok: [node2] 2026-08-18 13:48:30.761198 | node1 | ok: [node3] 2026-08-18 13:48:30.761469 | node1 | 2026-08-18 13:48:30.761777 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:48:30.762154 | node1 | Tuesday 18 August 2026 13:48:30 +0000 (0:00:00.316) 0:03:32.342 ******** 2026-08-18 13:48:30.851015 | node1 | skipping: [node1] 2026-08-18 13:48:30.851273 | node1 | skipping: [node2] 2026-08-18 13:48:30.851604 | node1 | skipping: [node3] 2026-08-18 13:48:30.851835 | node1 | 2026-08-18 13:48:30.852165 | node1 | TASK [vexxhost.containers.directory : Create directory (/var/lib/downloads)] *** 2026-08-18 13:48:30.852454 | node1 | Tuesday 18 August 2026 13:48:30 +0000 (0:00:00.090) 0:03:32.433 ******** 2026-08-18 13:48:31.163550 | node1 | ok: [node1] 2026-08-18 13:48:31.163619 | node1 | ok: [node2] 2026-08-18 13:48:31.164094 | node1 | ok: [node3] 2026-08-18 13:48:31.164167 | node1 | 2026-08-18 13:48:31.164179 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:48:31.164188 | node1 | Tuesday 18 August 2026 13:48:31 +0000 (0:00:00.311) 0:03:32.744 ******** 2026-08-18 13:48:31.286221 | node1 | ok: [node1] => { 2026-08-18 13:48:31.286545 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:48:31.286894 | node1 | } 2026-08-18 13:48:31.287301 | node1 | ok: [node2] => { 2026-08-18 13:48:31.287683 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:48:31.287969 | node1 | } 2026-08-18 13:48:31.288239 | node1 | ok: [node3] => { 2026-08-18 13:48:31.288517 | node1 | "msg": "https://github.com/opencontainers/runc/releases/download/v1.4.2/runc.amd64" 2026-08-18 13:48:31.288758 | node1 | } 2026-08-18 13:48:31.289044 | node1 | 2026-08-18 13:48:31.289345 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:48:31.289680 | node1 | Tuesday 18 August 2026 13:48:31 +0000 (0:00:00.123) 0:03:32.867 ******** 2026-08-18 13:48:31.733495 | node1 | ok: [node1] 2026-08-18 13:48:31.733669 | node1 | ok: [node3] 2026-08-18 13:48:31.733855 | node1 | ok: [node2] 2026-08-18 13:48:31.734066 | node1 | 2026-08-18 13:48:31.734318 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:48:31.734466 | node1 | Tuesday 18 August 2026 13:48:31 +0000 (0:00:00.447) 0:03:33.315 ******** 2026-08-18 13:48:31.775782 | node1 | skipping: [node1] 2026-08-18 13:48:31.848429 | node1 | skipping: [node2] 2026-08-18 13:48:31.848627 | node1 | skipping: [node3] 2026-08-18 13:48:31.848872 | node1 | 2026-08-18 13:48:31.849355 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:48:31.849568 | node1 | Tuesday 18 August 2026 13:48:31 +0000 (0:00:00.115) 0:03:33.430 ******** 2026-08-18 13:48:31.956438 | node1 | skipping: [node1] 2026-08-18 13:48:31.956621 | node1 | skipping: [node2] 2026-08-18 13:48:31.956858 | node1 | skipping: [node3] 2026-08-18 13:48:31.957160 | node1 | 2026-08-18 13:48:31.957420 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:48:31.957663 | node1 | Tuesday 18 August 2026 13:48:31 +0000 (0:00:00.107) 0:03:33.538 ******** 2026-08-18 13:48:32.256221 | node1 | ok: [node1] 2026-08-18 13:48:32.256429 | node1 | ok: [node2] 2026-08-18 13:48:32.256726 | node1 | ok: [node3] 2026-08-18 13:48:32.256976 | node1 | 2026-08-18 13:48:32.257242 | node1 | TASK [vexxhost.containers.package : Update state for tar] ********************** 2026-08-18 13:48:32.257484 | node1 | Tuesday 18 August 2026 13:48:32 +0000 (0:00:00.299) 0:03:33.838 ******** 2026-08-18 13:48:33.468113 | node1 | ok: [node2] 2026-08-18 13:48:33.468199 | node1 | ok: [node3] 2026-08-18 13:48:33.468209 | node1 | ok: [node1] 2026-08-18 13:48:33.468876 | node1 | 2026-08-18 13:48:33.468919 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:48:33.468967 | node1 | Tuesday 18 August 2026 13:48:33 +0000 (0:00:01.211) 0:03:35.049 ******** 2026-08-18 13:48:33.816074 | node1 | ok: [node1] => { 2026-08-18 13:48:33.818025 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:48:33.818095 | node1 | } 2026-08-18 13:48:33.818106 | node1 | ok: [node2] => { 2026-08-18 13:48:33.818116 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:48:33.818125 | node1 | } 2026-08-18 13:48:33.818134 | node1 | ok: [node3] => { 2026-08-18 13:48:33.818143 | node1 | "msg": "https://github.com/containerd/containerd/releases/download/v2.3.2/containerd-2.3.2-linux-amd64.tar.gz" 2026-08-18 13:48:33.818485 | node1 | } 2026-08-18 13:48:33.818792 | node1 | 2026-08-18 13:48:33.819269 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:48:33.819606 | node1 | Tuesday 18 August 2026 13:48:33 +0000 (0:00:00.347) 0:03:35.397 ******** 2026-08-18 13:48:34.517808 | node1 | ok: [node2] 2026-08-18 13:48:34.518232 | node1 | ok: [node3] 2026-08-18 13:48:34.518553 | node1 | ok: [node1] 2026-08-18 13:48:34.518835 | node1 | 2026-08-18 13:48:34.519187 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:48:34.519504 | node1 | Tuesday 18 August 2026 13:48:34 +0000 (0:00:00.701) 0:03:36.098 ******** 2026-08-18 13:48:37.971530 | node1 | ok: [node1] 2026-08-18 13:48:37.971591 | node1 | ok: [node3] 2026-08-18 13:48:37.971709 | node1 | ok: [node2] 2026-08-18 13:48:37.971832 | node1 | 2026-08-18 13:48:37.972007 | node1 | TASK [vexxhost.containers.containerd : Install SELinux packages] *************** 2026-08-18 13:48:37.972151 | node1 | Tuesday 18 August 2026 13:48:37 +0000 (0:00:03.453) 0:03:39.551 ******** 2026-08-18 13:48:38.036248 | node1 | skipping: [node2] 2026-08-18 13:48:38.036368 | node1 | skipping: [node1] 2026-08-18 13:48:38.036498 | node1 | skipping: [node3] 2026-08-18 13:48:38.036624 | node1 | 2026-08-18 13:48:38.036778 | node1 | TASK [vexxhost.containers.containerd : Set SELinux to permissive at runtime] *** 2026-08-18 13:48:38.036944 | node1 | Tuesday 18 August 2026 13:48:38 +0000 (0:00:00.066) 0:03:39.618 ******** 2026-08-18 13:48:38.075439 | node1 | skipping: [node1] 2026-08-18 13:48:38.103171 | node1 | skipping: [node2] 2026-08-18 13:48:38.103270 | node1 | skipping: [node3] 2026-08-18 13:48:38.103281 | node1 | 2026-08-18 13:48:38.103299 | node1 | TASK [vexxhost.containers.containerd : Persist SELinux permissive mode] ******** 2026-08-18 13:48:38.103309 | node1 | Tuesday 18 August 2026 13:48:38 +0000 (0:00:00.066) 0:03:39.685 ******** 2026-08-18 13:48:38.145182 | node1 | skipping: [node1] 2026-08-18 13:48:38.183609 | node1 | skipping: [node2] 2026-08-18 13:48:38.183752 | node1 | skipping: [node3] 2026-08-18 13:48:38.183898 | node1 | 2026-08-18 13:48:38.184094 | node1 | TASK [vexxhost.containers.containerd : Install AppArmor packages] ************** 2026-08-18 13:48:38.184247 | node1 | Tuesday 18 August 2026 13:48:38 +0000 (0:00:00.079) 0:03:39.764 ******** 2026-08-18 13:48:39.484495 | node1 | ok: [node1] 2026-08-18 13:48:39.484567 | node1 | ok: [node2] 2026-08-18 13:48:39.484680 | node1 | ok: [node3] 2026-08-18 13:48:39.484783 | node1 | 2026-08-18 13:48:39.484904 | node1 | TASK [vexxhost.containers.containerd : Create systemd service file for containerd] *** 2026-08-18 13:48:39.485053 | node1 | Tuesday 18 August 2026 13:48:39 +0000 (0:00:01.301) 0:03:41.065 ******** 2026-08-18 13:48:40.084188 | node1 | ok: [node1] 2026-08-18 13:48:40.084241 | node1 | ok: [node3] 2026-08-18 13:48:40.084250 | node1 | ok: [node2] 2026-08-18 13:48:40.084260 | node1 | 2026-08-18 13:48:40.084269 | node1 | TASK [vexxhost.containers.containerd : Create folders for configuration] ******* 2026-08-18 13:48:40.084291 | node1 | Tuesday 18 August 2026 13:48:40 +0000 (0:00:00.598) 0:03:41.664 ******** 2026-08-18 13:48:41.271107 | node1 | ok: [node1] => (item={'path': '/etc/containerd'}) 2026-08-18 13:48:41.271182 | node1 | ok: [node2] => (item={'path': '/etc/containerd'}) 2026-08-18 13:48:41.271193 | node1 | ok: [node3] => (item={'path': '/etc/containerd'}) 2026-08-18 13:48:41.271202 | node1 | ok: [node1] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:48:41.271212 | node1 | ok: [node2] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:48:41.271221 | node1 | ok: [node3] => (item={'path': '/var/lib/containerd', 'mode': '0o700'}) 2026-08-18 13:48:41.271230 | node1 | ok: [node1] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:48:41.271239 | node1 | ok: [node3] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:48:41.271248 | node1 | ok: [node2] => (item={'path': '/run/containerd', 'mode': '0o711'}) 2026-08-18 13:48:41.271264 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:48:41.272010 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:48:41.272056 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.grpc.v1.cri', 'mode': '0o700'}) 2026-08-18 13:48:41.272063 | node1 | ok: [node1] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:48:41.272130 | node1 | ok: [node3] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:48:41.272348 | node1 | ok: [node2] => (item={'path': '/run/containerd/io.containerd.sandbox.controller.v1.shim', 'mode': '0o700'}) 2026-08-18 13:48:41.272540 | node1 | 2026-08-18 13:48:41.272750 | node1 | TASK [vexxhost.containers.containerd : Create containerd config file] ********** 2026-08-18 13:48:41.273007 | node1 | Tuesday 18 August 2026 13:48:41 +0000 (0:00:01.187) 0:03:42.851 ******** 2026-08-18 13:48:41.990832 | node1 | ok: [node1] 2026-08-18 13:48:41.991256 | node1 | ok: [node3] 2026-08-18 13:48:41.991574 | node1 | ok: [node2] 2026-08-18 13:48:41.991864 | node1 | 2026-08-18 13:48:41.992256 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:48:41.992583 | node1 | Tuesday 18 August 2026 13:48:41 +0000 (0:00:00.700) 0:03:43.551 ******** 2026-08-18 13:48:41.992867 | node1 | 2026-08-18 13:48:41.994092 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:48:41.994164 | node1 | Tuesday 18 August 2026 13:48:41 +0000 (0:00:00.008) 0:03:43.560 ******** 2026-08-18 13:48:41.994175 | node1 | 2026-08-18 13:48:41.994193 | node1 | TASK [vexxhost.containers.containerd : Force any restarts if necessary] ******** 2026-08-18 13:48:41.995018 | node1 | Tuesday 18 August 2026 13:48:41 +0000 (0:00:00.005) 0:03:43.566 ******** 2026-08-18 13:48:41.995035 | node1 | 2026-08-18 13:48:41.995044 | node1 | TASK [vexxhost.containers.containerd : Enable and start service] *************** 2026-08-18 13:48:41.995086 | node1 | Tuesday 18 August 2026 13:48:41 +0000 (0:00:00.006) 0:03:43.572 ******** 2026-08-18 13:48:42.492033 | node1 | ok: [node1] 2026-08-18 13:48:42.492085 | node1 | ok: [node2] 2026-08-18 13:48:42.492091 | node1 | ok: [node3] 2026-08-18 13:48:42.492097 | node1 | 2026-08-18 13:48:42.492104 | node1 | TASK [vexxhost.containers.forget_package : Forget package] ********************* 2026-08-18 13:48:42.492252 | node1 | Tuesday 18 August 2026 13:48:42 +0000 (0:00:00.499) 0:03:44.071 ******** 2026-08-18 13:48:42.801542 | node1 | ok: [node1] 2026-08-18 13:48:42.801630 | node1 | ok: [node2] 2026-08-18 13:48:42.801693 | node1 | ok: [node3] 2026-08-18 13:48:42.801855 | node1 | 2026-08-18 13:48:42.801918 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:48:42.802081 | node1 | Tuesday 18 August 2026 13:48:42 +0000 (0:00:00.309) 0:03:44.381 ******** 2026-08-18 13:48:42.919750 | node1 | ok: [node1] => { 2026-08-18 13:48:42.919850 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:48:42.919972 | node1 | } 2026-08-18 13:48:42.921034 | node1 | ok: [node2] => { 2026-08-18 13:48:42.921091 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:48:42.921097 | node1 | } 2026-08-18 13:48:42.921102 | node1 | ok: [node3] => { 2026-08-18 13:48:42.921106 | node1 | "msg": "https://download.docker.com/linux/static/stable/x86_64/docker-24.0.9.tgz" 2026-08-18 13:48:42.921110 | node1 | } 2026-08-18 13:48:42.921114 | node1 | 2026-08-18 13:48:42.921118 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:48:42.921123 | node1 | Tuesday 18 August 2026 13:48:42 +0000 (0:00:00.119) 0:03:44.501 ******** 2026-08-18 13:48:43.417355 | node1 | ok: [node1] 2026-08-18 13:48:43.417416 | node1 | ok: [node2] 2026-08-18 13:48:43.417427 | node1 | ok: [node3] 2026-08-18 13:48:43.417437 | node1 | 2026-08-18 13:48:43.417447 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:48:43.417457 | node1 | Tuesday 18 August 2026 13:48:43 +0000 (0:00:00.494) 0:03:44.996 ******** 2026-08-18 13:48:46.782083 | node1 | ok: [node3] 2026-08-18 13:48:46.782172 | node1 | ok: [node1] 2026-08-18 13:48:46.783110 | node1 | ok: [node2] 2026-08-18 13:48:46.783189 | node1 | 2026-08-18 13:48:46.783201 | node1 | TASK [vexxhost.containers.docker : Install AppArmor packages] ****************** 2026-08-18 13:48:46.783326 | node1 | Tuesday 18 August 2026 13:48:46 +0000 (0:00:03.366) 0:03:48.363 ******** 2026-08-18 13:48:47.879137 | node1 | ok: [node2] 2026-08-18 13:48:47.879484 | node1 | ok: [node3] 2026-08-18 13:48:47.879756 | node1 | ok: [node1] 2026-08-18 13:48:47.880031 | node1 | 2026-08-18 13:48:47.880299 | node1 | TASK [vexxhost.containers.docker : Ensure group "docker" exists] *************** 2026-08-18 13:48:47.880582 | node1 | Tuesday 18 August 2026 13:48:47 +0000 (0:00:01.097) 0:03:49.460 ******** 2026-08-18 13:48:48.163802 | node1 | ok: [node1] 2026-08-18 13:48:48.164056 | node1 | ok: [node2] 2026-08-18 13:48:48.164304 | node1 | ok: [node3] 2026-08-18 13:48:48.164564 | node1 | 2026-08-18 13:48:48.164845 | node1 | TASK [vexxhost.containers.docker : Create systemd service file for docker] ***** 2026-08-18 13:48:48.165211 | node1 | Tuesday 18 August 2026 13:48:48 +0000 (0:00:00.284) 0:03:49.745 ******** 2026-08-18 13:48:48.785092 | node1 | ok: [node3] 2026-08-18 13:48:48.785302 | node1 | ok: [node1] 2026-08-18 13:48:48.785542 | node1 | ok: [node2] 2026-08-18 13:48:48.785770 | node1 | 2026-08-18 13:48:48.786069 | node1 | TASK [vexxhost.containers.docker : Create folders for configuration] *********** 2026-08-18 13:48:48.786326 | node1 | Tuesday 18 August 2026 13:48:48 +0000 (0:00:00.621) 0:03:50.366 ******** 2026-08-18 13:48:49.513093 | node1 | ok: [node1] => (item={'path': '/etc/docker'}) 2026-08-18 13:48:49.513134 | node1 | ok: [node2] => (item={'path': '/etc/docker'}) 2026-08-18 13:48:49.513327 | node1 | ok: [node3] => (item={'path': '/etc/docker'}) 2026-08-18 13:48:49.513617 | node1 | ok: [node1] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:48:49.513874 | node1 | ok: [node2] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:48:49.514199 | node1 | ok: [node3] => (item={'path': '/var/lib/docker', 'mode': '0o710'}) 2026-08-18 13:48:49.514461 | node1 | ok: [node1] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:48:49.514738 | node1 | ok: [node2] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:48:49.515057 | node1 | ok: [node3] => (item={'path': '/run/docker', 'mode': '0o711'}) 2026-08-18 13:48:49.515292 | node1 | 2026-08-18 13:48:49.515565 | node1 | TASK [vexxhost.containers.docker : Create systemd socket file for docker] ****** 2026-08-18 13:48:49.515831 | node1 | Tuesday 18 August 2026 13:48:49 +0000 (0:00:00.727) 0:03:51.094 ******** 2026-08-18 13:48:50.087983 | node1 | ok: [node2] 2026-08-18 13:48:50.088213 | node1 | ok: [node1] 2026-08-18 13:48:50.088453 | node1 | ok: [node3] 2026-08-18 13:48:50.088676 | node1 | 2026-08-18 13:48:50.088937 | node1 | TASK [vexxhost.containers.docker : Create docker daemon config file] *********** 2026-08-18 13:48:50.089226 | node1 | Tuesday 18 August 2026 13:48:50 +0000 (0:00:00.575) 0:03:51.669 ******** 2026-08-18 13:48:50.681458 | node1 | ok: [node1] 2026-08-18 13:48:50.681664 | node1 | ok: [node2] 2026-08-18 13:48:50.681901 | node1 | ok: [node3] 2026-08-18 13:48:50.682274 | node1 | 2026-08-18 13:48:50.682551 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:48:50.682773 | node1 | Tuesday 18 August 2026 13:48:50 +0000 (0:00:00.570) 0:03:52.240 ******** 2026-08-18 13:48:50.683042 | node1 | 2026-08-18 13:48:50.683317 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:48:50.683584 | node1 | Tuesday 18 August 2026 13:48:50 +0000 (0:00:00.009) 0:03:52.250 ******** 2026-08-18 13:48:50.683842 | node1 | 2026-08-18 13:48:50.684149 | node1 | TASK [vexxhost.containers.docker : Force any restarts if necessary] ************ 2026-08-18 13:48:50.684429 | node1 | Tuesday 18 August 2026 13:48:50 +0000 (0:00:00.006) 0:03:52.256 ******** 2026-08-18 13:48:50.684650 | node1 | 2026-08-18 13:48:50.684915 | node1 | TASK [vexxhost.containers.docker : Enable and start service] ******************* 2026-08-18 13:48:50.685215 | node1 | Tuesday 18 August 2026 13:48:50 +0000 (0:00:00.006) 0:03:52.263 ******** 2026-08-18 13:48:51.182471 | node1 | ok: [node1] 2026-08-18 13:48:51.182611 | node1 | ok: [node2] 2026-08-18 13:48:51.182869 | node1 | ok: [node3] 2026-08-18 13:48:51.183139 | node1 | 2026-08-18 13:48:51.183400 | node1 | TASK [vexxhost.containers.download_artifact : Starting download of file] ******* 2026-08-18 13:48:51.183648 | node1 | Tuesday 18 August 2026 13:48:51 +0000 (0:00:00.500) 0:03:52.764 ******** 2026-08-18 13:48:51.270102 | node1 | ok: [node1] => { 2026-08-18 13:48:51.270303 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:48:51.270533 | node1 | } 2026-08-18 13:48:51.270771 | node1 | ok: [node2] => { 2026-08-18 13:48:51.271074 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:48:51.271306 | node1 | } 2026-08-18 13:48:51.271559 | node1 | ok: [node3] => { 2026-08-18 13:48:51.271787 | node1 | "msg": "https://download.ceph.com/rpm-20.2.3/el9/noarch/cephadm" 2026-08-18 13:48:51.272112 | node1 | } 2026-08-18 13:48:51.272326 | node1 | 2026-08-18 13:48:51.272574 | node1 | TASK [vexxhost.containers.download_artifact : Download item] ******************* 2026-08-18 13:48:51.272818 | node1 | Tuesday 18 August 2026 13:48:51 +0000 (0:00:00.087) 0:03:52.852 ******** 2026-08-18 13:48:51.645977 | node1 | ok: [node2] 2026-08-18 13:48:51.646194 | node1 | ok: [node1] 2026-08-18 13:48:51.646429 | node1 | ok: [node3] 2026-08-18 13:48:51.646648 | node1 | 2026-08-18 13:48:51.646896 | node1 | TASK [vexxhost.containers.download_artifact : Extract archive] ***************** 2026-08-18 13:48:51.647178 | node1 | Tuesday 18 August 2026 13:48:51 +0000 (0:00:00.375) 0:03:53.227 ******** 2026-08-18 13:48:51.740539 | node1 | skipping: [node1] 2026-08-18 13:48:51.740748 | node1 | skipping: [node2] 2026-08-18 13:48:51.741016 | node1 | skipping: [node3] 2026-08-18 13:48:51.741234 | node1 | 2026-08-18 13:48:51.741481 | node1 | TASK [vexxhost.ceph.cephadm : Gather variables for each operating system] ****** 2026-08-18 13:48:51.741721 | node1 | Tuesday 18 August 2026 13:48:51 +0000 (0:00:00.094) 0:03:53.322 ******** 2026-08-18 13:48:51.837555 | node1 | ok: [node1] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:48:51.837786 | node1 | ok: [node2] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:48:51.838108 | node1 | ok: [node3] => (item=/home/zuul/.ansible/collections/ansible_collections/vexxhost/ceph/roles/cephadm/vars/debian.yml) 2026-08-18 13:48:51.838320 | node1 | 2026-08-18 13:48:51.838568 | node1 | TASK [vexxhost.ceph.cephadm : Install packages] ******************************** 2026-08-18 13:48:51.838815 | node1 | Tuesday 18 August 2026 13:48:51 +0000 (0:00:00.097) 0:03:53.419 ******** 2026-08-18 13:48:53.030794 | node1 | ok: [node2] 2026-08-18 13:48:53.031044 | node1 | ok: [node1] 2026-08-18 13:48:53.031372 | node1 | ok: [node3] 2026-08-18 13:48:53.031615 | node1 | 2026-08-18 13:48:53.031887 | node1 | TASK [vexxhost.ceph.cephadm : Ensure services are started] ********************* 2026-08-18 13:48:53.032215 | node1 | Tuesday 18 August 2026 13:48:53 +0000 (0:00:01.193) 0:03:54.612 ******** 2026-08-18 13:48:53.894503 | node1 | ok: [node1] => (item=systemd-timesyncd) 2026-08-18 13:48:53.894573 | node1 | ok: [node2] => (item=systemd-timesyncd) 2026-08-18 13:48:53.894677 | node1 | ok: [node3] => (item=systemd-timesyncd) 2026-08-18 13:48:53.894783 | node1 | ok: [node1] => (item=ssh) 2026-08-18 13:48:53.895096 | node1 | ok: [node2] => (item=ssh) 2026-08-18 13:48:53.895116 | node1 | ok: [node3] => (item=ssh) 2026-08-18 13:48:53.895129 | node1 | 2026-08-18 13:48:53.895226 | node1 | TASK [vexxhost.ceph.cephadm : Remove cephadm from old path] ******************** 2026-08-18 13:48:53.895345 | node1 | Tuesday 18 August 2026 13:48:53 +0000 (0:00:00.863) 0:03:55.475 ******** 2026-08-18 13:48:54.190077 | node1 | ok: [node1] 2026-08-18 13:48:54.190138 | node1 | ok: [node3] 2026-08-18 13:48:54.190147 | node1 | ok: [node2] 2026-08-18 13:48:54.190157 | node1 | 2026-08-18 13:48:54.190166 | node1 | TASK [vexxhost.ceph.cephadm : Ensure the cephadm SSH user exists] ************** 2026-08-18 13:48:54.190175 | node1 | Tuesday 18 August 2026 13:48:54 +0000 (0:00:00.295) 0:03:55.770 ******** 2026-08-18 13:48:54.533478 | node1 | ok: [node1] 2026-08-18 13:48:54.533601 | node1 | ok: [node3] 2026-08-18 13:48:54.533612 | node1 | ok: [node2] 2026-08-18 13:48:54.533621 | node1 | 2026-08-18 13:48:54.533630 | node1 | TASK [vexxhost.ceph.cephadm : Grant passwordless sudo to the cephadm SSH user] *** 2026-08-18 13:48:54.533640 | node1 | Tuesday 18 August 2026 13:48:54 +0000 (0:00:00.341) 0:03:56.112 ******** 2026-08-18 13:48:55.115183 | node1 | ok: [node3] 2026-08-18 13:48:55.115255 | node1 | ok: [node2] 2026-08-18 13:48:55.115265 | node1 | ok: [node1] 2026-08-18 13:48:55.115274 | node1 | 2026-08-18 13:48:55.115284 | node1 | TASK [vexxhost.ceph.cephadm : Log in to the container registry] **************** 2026-08-18 13:48:55.115308 | node1 | Tuesday 18 August 2026 13:48:55 +0000 (0:00:00.582) 0:03:56.694 ******** 2026-08-18 13:48:55.197207 | node1 | skipping: [node1] 2026-08-18 13:48:55.197267 | node1 | skipping: [node2] 2026-08-18 13:48:55.197276 | node1 | skipping: [node3] 2026-08-18 13:48:55.197286 | node1 | 2026-08-18 13:48:55.197295 | node1 | TASK [vexxhost.ceph.cephadm : Pre-pull the Ceph image] ************************* 2026-08-18 13:48:55.197304 | node1 | Tuesday 18 August 2026 13:48:55 +0000 (0:00:00.081) 0:03:56.776 ******** 2026-08-18 13:48:55.675078 | node1 | ok: [node1] 2026-08-18 13:48:55.675148 | node1 | ok: [node3] 2026-08-18 13:48:55.675998 | node1 | ok: [node2] 2026-08-18 13:48:55.676015 | node1 | 2026-08-18 13:48:55.676024 | node1 | TASK [vexxhost.ceph.cephadm : Pin CEPHADM_IMAGE for interactive login shells] *** 2026-08-18 13:48:55.676237 | node1 | Tuesday 18 August 2026 13:48:55 +0000 (0:00:00.478) 0:03:57.255 ******** 2026-08-18 13:48:56.299525 | node1 | ok: [node1] 2026-08-18 13:48:56.299622 | node1 | ok: [node3] 2026-08-18 13:48:56.299636 | node1 | ok: [node2] 2026-08-18 13:48:56.299795 | node1 | 2026-08-18 13:48:56.299870 | node1 | TASK [vexxhost.ceph.monitoring : Locate the cluster admin host] **************** 2026-08-18 13:48:56.300055 | node1 | Tuesday 18 August 2026 13:48:56 +0000 (0:00:00.625) 0:03:57.881 ******** 2026-08-18 13:48:57.148123 | node1 | ok: [node1] => (item=node1) 2026-08-18 13:48:57.148236 | node1 | ok: [node1 -> node2({{ nodepool.private_ipv4 }})] => (item=node2) 2026-08-18 13:48:57.148279 | node1 | ok: [node1 -> node3({{ nodepool.private_ipv4 }})] => (item=node3) 2026-08-18 13:48:57.148288 | node1 | 2026-08-18 13:48:57.148298 | node1 | TASK [vexxhost.ceph.monitoring : Select the admin host] ************************ 2026-08-18 13:48:57.148315 | node1 | Tuesday 18 August 2026 13:48:57 +0000 (0:00:00.846) 0:03:58.727 ******** 2026-08-18 13:48:57.294381 | node1 | ok: [node1] 2026-08-18 13:48:57.294463 | node1 | ok: [node2] 2026-08-18 13:48:57.294472 | node1 | ok: [node3] 2026-08-18 13:48:57.294482 | node1 | 2026-08-18 13:48:57.294491 | node1 | TASK [vexxhost.ceph.monitoring : Set the admin mon config path] **************** 2026-08-18 13:48:57.294501 | node1 | Tuesday 18 August 2026 13:48:57 +0000 (0:00:00.146) 0:03:58.874 ******** 2026-08-18 13:48:57.380155 | node1 | ok: [node1] 2026-08-18 13:48:57.380228 | node1 | ok: [node2] 2026-08-18 13:48:57.380239 | node1 | ok: [node3] 2026-08-18 13:48:57.380249 | node1 | 2026-08-18 13:48:57.380259 | node1 | TASK [vexxhost.ceph.monitoring : Pin monitoring container images] ************** 2026-08-18 13:48:57.380269 | node1 | Tuesday 18 August 2026 13:48:57 +0000 (0:00:00.085) 0:03:58.959 ******** 2026-08-18 13:48:57.414963 | node1 | skipping: [node1] 2026-08-18 13:48:57.415030 | node1 | 2026-08-18 13:48:57.415035 | node1 | TASK [vexxhost.ceph.monitoring : Apply the monitoring service specs] *********** 2026-08-18 13:48:57.415041 | node1 | Tuesday 18 August 2026 13:48:57 +0000 (0:00:00.036) 0:03:58.996 ******** 2026-08-18 13:48:59.239126 | node1 | ok: [node1] => (item=ceph-exporter) 2026-08-18 13:48:59.239202 | node1 | 2026-08-18 13:48:59.239218 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 13:48:59.239231 | node1 | node1 : ok=231 changed=0 unreachable=0 failed=0 skipped=57 rescued=0 ignored=0 2026-08-18 13:48:59.239249 | node1 | node2 : ok=200 changed=0 unreachable=0 failed=0 skipped=39 rescued=0 ignored=0 2026-08-18 13:48:59.239448 | node1 | node3 : ok=200 changed=0 unreachable=0 failed=0 skipped=39 rescued=0 ignored=0 2026-08-18 13:48:59.239671 | node1 | 2026-08-18 13:48:59.239890 | node1 | 2026-08-18 13:48:59.240178 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 13:48:59.240431 | node1 | Tuesday 18 August 2026 13:48:59 +0000 (0:00:01.823) 0:04:00.819 ******** 2026-08-18 13:48:59.240665 | node1 | =============================================================================== 2026-08-18 13:48:59.240960 | node1 | vexxhost.ceph.configure : Configure pools ------------------------------ 37.03s 2026-08-18 13:48:59.241236 | node1 | vexxhost.ceph.mgr : Configure ceph-mgr modules ------------------------- 11.23s 2026-08-18 13:48:59.241510 | node1 | vexxhost.ceph.configure : Apply central Ceph configuration ------------- 10.27s 2026-08-18 13:48:59.241760 | node1 | vexxhost.ceph.configure : Configure dashboard users --------------------- 7.45s 2026-08-18 13:48:59.242045 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 5.33s 2026-08-18 13:48:59.242283 | node1 | vexxhost.ceph.configure : Configure CRUSH rules ------------------------- 5.18s 2026-08-18 13:48:59.242580 | node1 | vexxhost.ceph.rgw : Configure RGW zones --------------------------------- 3.64s 2026-08-18 13:48:59.242826 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.52s 2026-08-18 13:48:59.243117 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.45s 2026-08-18 13:48:59.243351 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.38s 2026-08-18 13:48:59.243619 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 3.37s 2026-08-18 13:48:59.243866 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 2.56s 2026-08-18 13:48:59.244162 | node1 | vexxhost.ceph.rgw : Configure RGW zonegroups ---------------------------- 2.51s 2026-08-18 13:48:59.244395 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 2.34s 2026-08-18 13:48:59.244680 | node1 | vexxhost.containers.download_artifact : Extract archive ----------------- 2.29s 2026-08-18 13:48:59.244976 | node1 | Gathering Facts --------------------------------------------------------- 2.21s 2026-08-18 13:48:59.245249 | node1 | vexxhost.containers.docker : Install AppArmor packages ------------------ 2.11s 2026-08-18 13:48:59.245492 | node1 | vexxhost.containers.containerd : Install AppArmor packages -------------- 2.05s 2026-08-18 13:48:59.245758 | node1 | vexxhost.ceph.cephadm_host : Register the host (labels) ----------------- 1.96s 2026-08-18 13:48:59.246052 | node1 | vexxhost.ceph.mon : Wait for this monitor's daemon to be present -------- 1.93s 2026-08-18 13:48:59.435291 | node1 | INFO [default > idempotence] Executed: Successful 2026-08-18 13:48:59.443980 | node1 | INFO [default > side_effect] Executing 2026-08-18 13:48:59.444328 | node1 | WARNING [default > side_effect] Executed: Missing playbook (Remove from test_sequence to suppress) 2026-08-18 13:48:59.450086 | node1 | INFO [default > verify] Executing 2026-08-18 13:49:00.299060 | node1 | 2026-08-18 13:49:00.299273 | node1 | PLAY [Verify the deployed Ceph cluster] **************************************** 2026-08-18 13:49:00.299498 | node1 | 2026-08-18 13:49:00.299747 | node1 | TASK [Gathering Facts] ********************************************************* 2026-08-18 13:49:00.300028 | node1 | Tuesday 18 August 2026 13:49:00 +0000 (0:00:00.010) 0:00:00.010 ******** 2026-08-18 13:49:02.574193 | node1 | [WARNING]: Platform linux on host node1 is using the discovered Python 2026-08-18 13:49:02.574267 | node1 | interpreter at /usr/bin/python3.12, but future installation of another Python 2026-08-18 13:49:02.574286 | node1 | interpreter could change the meaning of that path. See 2026-08-18 13:49:02.574295 | node1 | https://docs.ansible.com/ansible- 2026-08-18 13:49:02.574409 | node1 | core/2.18/reference_appendices/interpreter_discovery.html for more information. 2026-08-18 13:49:02.585121 | node1 | ok: [node1] 2026-08-18 13:49:02.585244 | node1 | 2026-08-18 13:49:02.585257 | node1 | TASK [Get cluster health] ****************************************************** 2026-08-18 13:49:02.586086 | node1 | Tuesday 18 August 2026 13:49:02 +0000 (0:00:02.286) 0:00:02.296 ******** 2026-08-18 13:49:04.567333 | node1 | ok: [node1] 2026-08-18 13:49:04.567576 | node1 | 2026-08-18 13:49:04.568058 | node1 | TASK [Assert the cluster is HEALTH_OK] ***************************************** 2026-08-18 13:49:04.568380 | node1 | Tuesday 18 August 2026 13:49:04 +0000 (0:00:01.981) 0:00:04.278 ******** 2026-08-18 13:49:04.608705 | node1 | ok: [node1] => { 2026-08-18 13:49:04.608998 | node1 | "changed": false, 2026-08-18 13:49:04.609289 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:04.609565 | node1 | } 2026-08-18 13:49:04.609828 | node1 | 2026-08-18 13:49:04.610175 | node1 | TASK [Get monitor quorum] ****************************************************** 2026-08-18 13:49:04.610433 | node1 | Tuesday 18 August 2026 13:49:04 +0000 (0:00:00.041) 0:00:04.320 ******** 2026-08-18 13:49:06.473918 | node1 | ok: [node1] 2026-08-18 13:49:06.474061 | node1 | 2026-08-18 13:49:06.474080 | node1 | TASK [Assert every monitor is in quorum] *************************************** 2026-08-18 13:49:06.474096 | node1 | Tuesday 18 August 2026 13:49:06 +0000 (0:00:01.863) 0:00:06.183 ******** 2026-08-18 13:49:06.522655 | node1 | ok: [node1] => { 2026-08-18 13:49:06.522864 | node1 | "changed": false, 2026-08-18 13:49:06.523168 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:06.523385 | node1 | } 2026-08-18 13:49:06.523663 | node1 | 2026-08-18 13:49:06.523906 | node1 | TASK [Get manager status] ****************************************************** 2026-08-18 13:49:06.524208 | node1 | Tuesday 18 August 2026 13:49:06 +0000 (0:00:00.051) 0:00:06.234 ******** 2026-08-18 13:49:08.346495 | node1 | ok: [node1] 2026-08-18 13:49:08.346724 | node1 | 2026-08-18 13:49:08.347030 | node1 | TASK [Assert a manager is active] ********************************************** 2026-08-18 13:49:08.347296 | node1 | Tuesday 18 August 2026 13:49:08 +0000 (0:00:01.822) 0:00:08.057 ******** 2026-08-18 13:49:08.386564 | node1 | ok: [node1] => { 2026-08-18 13:49:08.386763 | node1 | "changed": false, 2026-08-18 13:49:08.387070 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:08.387283 | node1 | } 2026-08-18 13:49:08.387504 | node1 | 2026-08-18 13:49:08.387772 | node1 | TASK [Get OSD stat] ************************************************************ 2026-08-18 13:49:08.388030 | node1 | Tuesday 18 August 2026 13:49:08 +0000 (0:00:00.040) 0:00:08.098 ******** 2026-08-18 13:49:10.096057 | node1 | ok: [node1] 2026-08-18 13:49:10.096120 | node1 | 2026-08-18 13:49:10.096134 | node1 | TASK [Assert all OSDs are up and in] ******************************************* 2026-08-18 13:49:10.096147 | node1 | Tuesday 18 August 2026 13:49:10 +0000 (0:00:01.708) 0:00:09.806 ******** 2026-08-18 13:49:10.141572 | node1 | ok: [node1] => { 2026-08-18 13:49:10.141613 | node1 | "changed": false, 2026-08-18 13:49:10.141736 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:10.142169 | node1 | } 2026-08-18 13:49:10.142186 | node1 | 2026-08-18 13:49:10.142192 | node1 | TASK [List orchestrator services] ********************************************** 2026-08-18 13:49:10.142197 | node1 | Tuesday 18 August 2026 13:49:10 +0000 (0:00:00.045) 0:00:09.852 ******** 2026-08-18 13:49:11.908059 | node1 | ok: [node1] 2026-08-18 13:49:11.908142 | node1 | 2026-08-18 13:49:11.908213 | node1 | TASK [Assert every orchestrator service has all its daemons running] *********** 2026-08-18 13:49:11.908479 | node1 | Tuesday 18 August 2026 13:49:11 +0000 (0:00:01.766) 0:00:11.618 ******** 2026-08-18 13:49:11.997647 | node1 | failed: [node1] (item=rgw.molecule) => { 2026-08-18 13:49:11.997711 | node1 | "ansible_loop_var": "item", 2026-08-18 13:49:11.997817 | node1 | "assertion": "(item.status.running | default(0)) > 0", 2026-08-18 13:49:11.997947 | node1 | "changed": false, 2026-08-18 13:49:11.998053 | node1 | "evaluated_to": false, 2026-08-18 13:49:11.998158 | node1 | "item": { 2026-08-18 13:49:11.998256 | node1 | "events": [ 2026-08-18 13:49:11.998384 | node1 | "2026-08-18T13:44:29.826432Z service:rgw.molecule [INFO] \"service was created\"" 2026-08-18 13:49:11.998481 | node1 | ], 2026-08-18 13:49:11.998587 | node1 | "networks": [ 2026-08-18 13:49:11.998689 | node1 | "0.0.0.0/0" 2026-08-18 13:49:11.998802 | node1 | ], 2026-08-18 13:49:11.998909 | node1 | "placement": { 2026-08-18 13:49:11.999068 | node1 | "count_per_host": 1, 2026-08-18 13:49:11.999175 | node1 | "label": "rgw" 2026-08-18 13:49:11.999278 | node1 | }, 2026-08-18 13:49:11.999389 | node1 | "service_id": "molecule", 2026-08-18 13:49:11.999496 | node1 | "service_name": "rgw.molecule", 2026-08-18 13:49:11.999603 | node1 | "service_type": "rgw", 2026-08-18 13:49:11.999706 | node1 | "spec": { 2026-08-18 13:49:11.999812 | node1 | "generate_cert": true, 2026-08-18 13:49:11.999918 | node1 | "rgw_exit_timeout_secs": 120, 2026-08-18 13:49:12.000292 | node1 | "rgw_frontend_port": 8080, 2026-08-18 13:49:12.000397 | node1 | "rgw_realm": "molecule", 2026-08-18 13:49:12.000505 | node1 | "rgw_zone": "molecule-1", 2026-08-18 13:49:12.000613 | node1 | "rgw_zonegroup": "molecule", 2026-08-18 13:49:12.000720 | node1 | "ssl": true 2026-08-18 13:49:12.000840 | node1 | }, 2026-08-18 13:49:12.000956 | node1 | "status": { 2026-08-18 13:49:12.001069 | node1 | "created": "2026-08-18T13:44:29.795983Z", 2026-08-18 13:49:12.001171 | node1 | "ports": [ 2026-08-18 13:49:12.001277 | node1 | 8080 2026-08-18 13:49:12.001380 | node1 | ], 2026-08-18 13:49:12.001484 | node1 | "running": 0, 2026-08-18 13:49:12.001587 | node1 | "size": 3 2026-08-18 13:49:12.001690 | node1 | } 2026-08-18 13:49:12.001793 | node1 | }, 2026-08-18 13:49:12.001901 | node1 | "msg": "Service rgw.molecule running 0/3" 2026-08-18 13:49:12.002072 | node1 | } 2026-08-18 13:49:12.002226 | node1 | ok: [node1] => (item=ceph-exporter) => { 2026-08-18 13:49:12.002336 | node1 | "ansible_loop_var": "item", 2026-08-18 13:49:12.002444 | node1 | "changed": false, 2026-08-18 13:49:12.002549 | node1 | "item": { 2026-08-18 13:49:12.002655 | node1 | "events": [ 2026-08-18 13:49:12.002775 | node1 | "2026-08-18T13:45:08.511720Z service:ceph-exporter [INFO] \"service was created\"" 2026-08-18 13:49:12.002877 | node1 | ], 2026-08-18 13:49:12.003021 | node1 | "placement": { 2026-08-18 13:49:12.003123 | node1 | "host_pattern": "*" 2026-08-18 13:49:12.003229 | node1 | }, 2026-08-18 13:49:12.003339 | node1 | "service_name": "ceph-exporter", 2026-08-18 13:49:12.003447 | node1 | "service_type": "ceph-exporter", 2026-08-18 13:49:12.003552 | node1 | "spec": { 2026-08-18 13:49:12.003657 | node1 | "prio_limit": 5, 2026-08-18 13:49:12.003774 | node1 | "stats_period": 5 2026-08-18 13:49:12.003879 | node1 | }, 2026-08-18 13:49:12.004022 | node1 | "status": { 2026-08-18 13:49:12.004118 | node1 | "created": "2026-08-18T13:44:56.670608Z", 2026-08-18 13:49:12.004219 | node1 | "last_refresh": "2026-08-18T13:45:14.357817Z", 2026-08-18 13:49:12.004322 | node1 | "ports": [ 2026-08-18 13:49:12.004426 | node1 | 9926 2026-08-18 13:49:12.004528 | node1 | ], 2026-08-18 13:49:12.004632 | node1 | "running": 3, 2026-08-18 13:49:12.004738 | node1 | "size": 3 2026-08-18 13:49:12.004842 | node1 | } 2026-08-18 13:49:12.004966 | node1 | }, 2026-08-18 13:49:12.005071 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:12.005181 | node1 | } 2026-08-18 13:49:12.005292 | node1 | ok: [node1] => (item=crash) => { 2026-08-18 13:49:12.005399 | node1 | "ansible_loop_var": "item", 2026-08-18 13:49:12.005505 | node1 | "changed": false, 2026-08-18 13:49:12.005609 | node1 | "item": { 2026-08-18 13:49:12.005715 | node1 | "placement": { 2026-08-18 13:49:12.005821 | node1 | "host_pattern": "*" 2026-08-18 13:49:12.005940 | node1 | }, 2026-08-18 13:49:12.006049 | node1 | "service_name": "crash", 2026-08-18 13:49:12.006157 | node1 | "service_type": "crash", 2026-08-18 13:49:12.006259 | node1 | "status": { 2026-08-18 13:49:12.006367 | node1 | "created": "2026-08-18T13:33:45.721825Z", 2026-08-18 13:49:12.006477 | node1 | "last_refresh": "2026-08-18T13:45:14.357083Z", 2026-08-18 13:49:12.006583 | node1 | "running": 3, 2026-08-18 13:49:12.006687 | node1 | "size": 3 2026-08-18 13:49:12.006790 | node1 | } 2026-08-18 13:49:12.006893 | node1 | }, 2026-08-18 13:49:12.007093 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:12.007195 | node1 | } 2026-08-18 13:49:12.007299 | node1 | ok: [node1] => (item=mgr) => { 2026-08-18 13:49:12.007405 | node1 | "ansible_loop_var": "item", 2026-08-18 13:49:12.007511 | node1 | "changed": false, 2026-08-18 13:49:12.007628 | node1 | "item": { 2026-08-18 13:49:12.007719 | node1 | "placement": { 2026-08-18 13:49:12.007823 | node1 | "label": "mgr" 2026-08-18 13:49:12.007941 | node1 | }, 2026-08-18 13:49:12.008089 | node1 | "service_name": "mgr", 2026-08-18 13:49:12.008201 | node1 | "service_type": "mgr", 2026-08-18 13:49:12.008310 | node1 | "status": { 2026-08-18 13:49:12.008414 | node1 | "created": "2026-08-18T13:34:40.995126Z", 2026-08-18 13:49:12.008523 | node1 | "last_refresh": "2026-08-18T13:45:14.357477Z", 2026-08-18 13:49:12.008627 | node1 | "running": 3, 2026-08-18 13:49:12.008730 | node1 | "size": 3 2026-08-18 13:49:12.008832 | node1 | } 2026-08-18 13:49:12.008970 | node1 | }, 2026-08-18 13:49:12.009061 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:12.009162 | node1 | } 2026-08-18 13:49:12.009280 | node1 | ok: [node1] => (item=mon) => { 2026-08-18 13:49:12.009377 | node1 | "ansible_loop_var": "item", 2026-08-18 13:49:12.009482 | node1 | "changed": false, 2026-08-18 13:49:12.009586 | node1 | "item": { 2026-08-18 13:49:12.009692 | node1 | "placement": { 2026-08-18 13:49:12.009797 | node1 | "label": "mon" 2026-08-18 13:49:12.009902 | node1 | }, 2026-08-18 13:49:12.010032 | node1 | "service_name": "mon", 2026-08-18 13:49:12.010134 | node1 | "service_type": "mon", 2026-08-18 13:49:12.010237 | node1 | "status": { 2026-08-18 13:49:12.010395 | node1 | "created": "2026-08-18T13:34:18.987806Z", 2026-08-18 13:49:12.010508 | node1 | "last_refresh": "2026-08-18T13:45:14.357305Z", 2026-08-18 13:49:12.010614 | node1 | "running": 3, 2026-08-18 13:49:12.010718 | node1 | "size": 3 2026-08-18 13:49:12.010820 | node1 | } 2026-08-18 13:49:12.010958 | node1 | }, 2026-08-18 13:49:12.011071 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:12.011175 | node1 | } 2026-08-18 13:49:12.011288 | node1 | ok: [node1] => (item=osd.default) => { 2026-08-18 13:49:12.011391 | node1 | "ansible_loop_var": "item", 2026-08-18 13:49:12.011496 | node1 | "changed": false, 2026-08-18 13:49:12.011609 | node1 | "item": { 2026-08-18 13:49:12.011706 | node1 | "placement": { 2026-08-18 13:49:12.011811 | node1 | "host_pattern": "node3" 2026-08-18 13:49:12.011913 | node1 | }, 2026-08-18 13:49:12.012050 | node1 | "service_id": "default", 2026-08-18 13:49:12.012159 | node1 | "service_name": "osd.default", 2026-08-18 13:49:12.012266 | node1 | "service_type": "osd", 2026-08-18 13:49:12.012380 | node1 | "spec": { 2026-08-18 13:49:12.012476 | node1 | "data_devices": { 2026-08-18 13:49:12.012578 | node1 | "paths": [ 2026-08-18 13:49:12.012685 | node1 | "/dev/ceph-node3-osd0/data" 2026-08-18 13:49:12.012786 | node1 | ] 2026-08-18 13:49:12.012888 | node1 | }, 2026-08-18 13:49:12.013019 | node1 | "filter_logic": "AND", 2026-08-18 13:49:12.013117 | node1 | "objectstore": "bluestore" 2026-08-18 13:49:12.013219 | node1 | }, 2026-08-18 13:49:12.013327 | node1 | "status": { 2026-08-18 13:49:12.013433 | node1 | "created": "2026-08-18T13:35:31.565416Z", 2026-08-18 13:49:12.013542 | node1 | "last_refresh": "2026-08-18T13:45:14.357647Z", 2026-08-18 13:49:12.013647 | node1 | "running": 9, 2026-08-18 13:49:12.013750 | node1 | "size": 9 2026-08-18 13:49:12.013852 | node1 | } 2026-08-18 13:49:12.013984 | node1 | }, 2026-08-18 13:49:12.014087 | node1 | "msg": "All assertions passed" 2026-08-18 13:49:12.014189 | node1 | } 2026-08-18 13:49:12.014294 | node1 | 2026-08-18 13:49:12.014400 | node1 | PLAY RECAP ********************************************************************* 2026-08-18 13:49:12.014516 | node1 | node1 : ok=10 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 2026-08-18 13:49:12.014613 | node1 | 2026-08-18 13:49:12.014710 | node1 | 2026-08-18 13:49:12.014816 | node1 | TASKS RECAP ******************************************************************** 2026-08-18 13:49:12.014985 | node1 | Tuesday 18 August 2026 13:49:12 +0000 (0:00:00.094) 0:00:11.713 ******** 2026-08-18 13:49:12.015078 | node1 | =============================================================================== 2026-08-18 13:49:12.015181 | node1 | Gathering Facts --------------------------------------------------------- 2.29s 2026-08-18 13:49:12.015288 | node1 | Get cluster health ------------------------------------------------------ 1.98s 2026-08-18 13:49:12.015395 | node1 | Get monitor quorum ------------------------------------------------------ 1.86s 2026-08-18 13:49:12.015498 | node1 | Get manager status ------------------------------------------------------ 1.82s 2026-08-18 13:49:12.015605 | node1 | List orchestrator services ---------------------------------------------- 1.77s 2026-08-18 13:49:12.015709 | node1 | Get OSD stat ------------------------------------------------------------ 1.71s 2026-08-18 13:49:12.015817 | node1 | Assert every orchestrator service has all its daemons running ----------- 0.09s 2026-08-18 13:49:12.015924 | node1 | Assert every monitor is in quorum --------------------------------------- 0.05s 2026-08-18 13:49:12.016066 | node1 | Assert all OSDs are up and in ------------------------------------------- 0.05s 2026-08-18 13:49:12.016172 | node1 | Assert the cluster is HEALTH_OK ----------------------------------------- 0.04s 2026-08-18 13:49:12.016279 | node1 | Assert a manager is active ---------------------------------------------- 0.04s 2026-08-18 13:49:12.097212 | node1 | CRITICAL Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/verify.yml 2026-08-18 13:49:12.098026 | node1 | ERROR [default > verify] Executed: Failed 2026-08-18 13:49:12.098038 | node1 | ERROR Ansible return code was 2, command was: ansible-playbook --inventory /home/zuul/.ansible/tmp/molecule.kre9.default/inventory --skip-tags molecule-notest,notest --inventory=/home/zuul/src/github.com/vexxhost/ansible-collection-ceph/inventory.yaml /home/zuul/src/github.com/vexxhost/ansible-collection-ceph/extensions/molecule/default/verify.yml 2026-08-18 13:49:12.463561 | node1 | ERROR 2026-08-18 13:49:12.463835 | node1 | { 2026-08-18 13:49:12.463877 | node1 | "delta": "0:23:33.575960", 2026-08-18 13:49:12.463907 | node1 | "end": "2026-08-18 13:49:12.191986", 2026-08-18 13:49:12.463934 | node1 | "msg": "non-zero return code", 2026-08-18 13:49:12.463959 | node1 | "rc": 2, 2026-08-18 13:49:12.463984 | node1 | "start": "2026-08-18 13:25:38.616026" 2026-08-18 13:49:12.464009 | node1 | } failure 2026-08-18 13:49:12.512554 | 2026-08-18 13:49:12.512833 | PLAY RECAP 2026-08-18 13:49:12.512890 | node1 | ok: 3 changed: 2 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0 2026-08-18 13:49:12.512912 | 2026-08-18 13:49:12.845583 | RUN END RESULT_NORMAL: [untrusted : github.com/vexxhost/zuul-jobs/playbooks/molecule/run.yaml@main] 2026-08-18 13:49:12.860388 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 13:49:13.907255 | 2026-08-18 13:49:13.907396 | PLAY [all] 2026-08-18 13:49:13.920398 | 2026-08-18 13:49:13.920473 | TASK [fetch-output : Set log path for multiple nodes] 2026-08-18 13:49:13.973053 | node1 | ok 2026-08-18 13:49:13.978901 | node2 | ok 2026-08-18 13:49:13.984448 | node3 | ok 2026-08-18 13:49:13.989411 | 2026-08-18 13:49:13.989509 | TASK [fetch-output : Set log path for single node] 2026-08-18 13:49:14.027678 | node1 | skipping: Conditional result was False 2026-08-18 13:49:14.028694 | node2 | skipping: Conditional result was False 2026-08-18 13:49:14.039252 | node3 | skipping: Conditional result was False 2026-08-18 13:49:14.049839 | 2026-08-18 13:49:14.049917 | LOOP [fetch-output : Ensure local output dirs] 2026-08-18 13:49:14.519674 | node3 -> localhost | changed: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/logs/node3" 2026-08-18 13:49:14.521994 | node2 -> localhost | changed: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/logs/node2" 2026-08-18 13:49:14.523770 | node1 -> localhost | changed: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/logs/node1" 2026-08-18 13:49:14.726595 | node2 -> localhost | changed: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/artifacts" 2026-08-18 13:49:14.743863 | node3 -> localhost | ok: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/artifacts" 2026-08-18 13:49:14.754356 | node1 -> localhost | ok: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/artifacts" 2026-08-18 13:49:14.922398 | node2 -> localhost | changed: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/docs" 2026-08-18 13:49:14.961500 | node1 -> localhost | ok: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/docs" 2026-08-18 13:49:14.961646 | node1 -> localhost | changed: All items complete 2026-08-18 13:49:14.961682 | 2026-08-18 13:49:14.975192 | node3 -> localhost | ok: "/var/lib/zuul/builds/b7fff044e3b543c1a2af2431bf46dcc5/work/docs" 2026-08-18 13:49:14.999956 | 2026-08-18 13:49:15.000035 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-08-18 13:49:15.886026 | node1 | changed: .d..t...... ./ 2026-08-18 13:49:15.886446 | node1 | changed: All items complete 2026-08-18 13:49:15.886499 | 2026-08-18 13:49:15.894057 | node3 | changed: .d..t...... ./ 2026-08-18 13:49:15.899006 | node2 | changed: .d..t...... ./ 2026-08-18 13:49:16.395204 | node3 | ok 2026-08-18 13:49:16.401331 | node2 | changed: .d..t...... ./ 2026-08-18 13:49:16.414716 | node1 | ok 2026-08-18 13:49:16.880347 | node1 | changed: .d..t...... ./ 2026-08-18 13:49:16.901455 | node2 | ok 2026-08-18 13:49:16.910569 | node3 | ok 2026-08-18 13:49:16.940851 | 2026-08-18 13:49:16.941078 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-08-18 13:49:17.376714 | node1 -> localhost | ok: Item: artifacts Runtime: 0:00:00.007007 2026-08-18 13:49:17.376993 | node1 -> localhost | changed: All items complete 2026-08-18 13:49:17.377037 | 2026-08-18 13:49:17.594433 | node1 -> localhost | ok: Item: docs Runtime: 0:00:00.006284 2026-08-18 13:49:17.633711 | 2026-08-18 13:49:17.633873 | PLAY [all] 2026-08-18 13:49:17.649319 | 2026-08-18 13:49:17.649619 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes] 2026-08-18 13:49:18.180305 | node1 | changed 2026-08-18 13:49:18.359523 | node3 | changed 2026-08-18 13:49:18.381180 | node2 | changed 2026-08-18 13:49:18.409722 | 2026-08-18 13:49:18.409866 | PLAY RECAP 2026-08-18 13:49:18.409918 | node1 | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 13:49:18.409997 | node2 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 13:49:18.410027 | node3 | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-08-18 13:49:18.410046 | 2026-08-18 13:49:18.616584 | POST-RUN END RESULT_NORMAL: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post.yaml@main] 2026-08-18 13:49:18.630959 | POST-RUN START: [trusted : github.com/vexxhost/zuul-config/playbooks/base/post-logs.yaml@main] 2026-08-18 13:49:19.414985 | 2026-08-18 13:49:19.415127 | PLAY [localhost] 2026-08-18 13:49:19.425549 | 2026-08-18 13:49:19.425619 | TASK [Generate Zuul manifest] 2026-08-18 13:49:19.446095 | localhost | ok 2026-08-18 13:49:19.566458 | 2026-08-18 13:49:19.566561 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-08-18 13:49:20.024359 | localhost | changed 2026-08-18 13:49:20.036636 | 2026-08-18 13:49:20.036705 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-08-18 13:49:20.099900 | localhost | ok 2026-08-18 13:49:20.107018 | 2026-08-18 13:49:20.107082 | TASK [Upload logs] 2026-08-18 13:49:20.127924 | localhost | ok 2026-08-18 13:49:20.251694 | 2026-08-18 13:49:20.251845 | TASK [Set zuul-log-path fact] 2026-08-18 13:49:20.270818 | localhost | ok 2026-08-18 13:49:20.282034 | 2026-08-18 13:49:20.282101 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-08-18 13:49:20.310113 | localhost | ok 2026-08-18 13:49:20.316814 | 2026-08-18 13:49:20.316876 | TASK [upload-logs : Create log directories] 2026-08-18 13:49:20.663160 | localhost | changed 2026-08-18 13:49:20.670285 | 2026-08-18 13:49:20.670367 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-08-18 13:49:21.016252 | localhost -> localhost | ok: Runtime: 0:00:00.005190 2026-08-18 13:49:21.022151 | 2026-08-18 13:49:21.022218 | TASK [upload-logs : Upload logs to log server] 2026-08-18 13:49:21.491928 | localhost | Output suppressed because no_log was given 2026-08-18 13:49:21.498604 | 2026-08-18 13:49:21.498674 | LOOP [upload-logs : Compress console log and json output] 2026-08-18 13:49:21.539893 | localhost | skipping: Conditional result was False 2026-08-18 13:49:21.546994 | localhost | skipping: Conditional result was False 2026-08-18 13:49:21.563147 | 2026-08-18 13:49:21.563331 | LOOP [upload-logs : Upload compressed console log and json output] 2026-08-18 13:49:21.600843 | localhost | skipping: Conditional result was False 2026-08-18 13:49:21.601245 | 2026-08-18 13:49:21.605236 | localhost | skipping: Conditional result was False 2026-08-18 13:49:21.615751 | 2026-08-18 13:49:21.615926 | LOOP [upload-logs : Upload console log and json output]