May 21 23:20:46.194623 np0000176093 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service. May 21 23:20:47.706725 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.common.config [-] Logging enabled! May 21 23:20:47.707493 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev264 May 21 23:20:47.707493 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=104501) setup_logging /opt/stack/neutron/neutron/common/config.py:125}} May 21 23:20:47.707728 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting... May 21 23:20:47.707797 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=104501) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}} May 21 23:20:47.707918 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=104501) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}} May 21 23:20:47.708010 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=104501) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}} May 21 23:20:47.708010 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=104501) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}} May 21 23:20:47.708149 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=104501) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}} May 21 23:20:47.708207 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [-] Loading core plugin: ml2 May 21 23:20:48.000604 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.000754 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.000754 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.001387 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.001479 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.001572 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.130936 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.131088 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.131088 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.137562 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.137672 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.137672 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.189932 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.190150 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.190150 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.192635 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.192778 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.192778 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.243672 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.243813 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.243813 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.243813 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: network precommit_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.244090 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298118 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298229 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298295 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298354 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298429 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298499 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298557 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298698 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298698 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298782 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.298782 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network precommit_delete 0, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299070 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port provisioning_complete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299133 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299187 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299255 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299339 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299390 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete 0, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299480 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299526 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299613 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299682 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.299960 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.300014 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.300113 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.300414 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:48.300479 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:48.300520 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:48.300564 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:48.300608 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:48.300673 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:48.300888 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve'] May 21 23:20:48.301586 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public'] May 21 23:20:48.302250 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete May 21 23:20:48.302762 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]}) May 21 23:20:48.302837 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan'] May 21 23:20:48.302999 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan']) May 21 23:20:48.303102 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve'] May 21 23:20:48.303223 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos'] May 21 23:20:48.303731 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos'] May 21 23:20:48.303845 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos'] May 21 23:20:48.303845 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn'] May 21 23:20:48.341392 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn'] May 21 23:20:48.341514 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn'] May 21 23:20:48.341577 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling. May 21 23:20:48.342315 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_response 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.342401 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_response 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:48.367984 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' May 21 23:20:48.367984 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete May 21 23:20:48.368201 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' May 21 23:20:48.368201 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)] May 21 23:20:48.412118 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO dbcounter [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Registered counter for database neutron May 21 23:20:48.422288 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_db.sqlalchemy.engines [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104501) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}} May 21 23:20:48.832645 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG dbcounter [-] [104501] Writer thread running {{(pid=104501) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}} May 21 23:20:49.011859 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Initializing driver for type 'local' May 21 23:20:49.011859 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Initializing driver for type 'vlan' May 21 23:20:49.124395 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] VlanTypeDriver initialization complete May 21 23:20:49.124395 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Initializing extension driver 'port_security' May 21 23:20:49.124666 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.extensions.port_security [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] PortSecurityExtensionDriver initialization complete May 21 23:20:49.124666 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Initializing extension driver 'qos' May 21 23:20:49.124666 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.plugins.ml2.extensions.qos [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QosExtensionDriver initialization complete {{(pid=104501) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}} May 21 23:20:49.124666 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Initializing mechanism driver 'ovn' May 21 23:20:49.124876 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Starting OVNMechanismDriver May 21 23:20:49.126144 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > process before_spawn 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126259 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > process after_init 55550000, True {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126346 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > segment after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126409 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > segment after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126471 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > segment after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126558 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > allowed_address_pair before_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126623 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126718 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126799 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126926 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group precommit_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.126997 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127078 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group_rule precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127193 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group_rule after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127260 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group_rule before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127339 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > address_group after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127399 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > address_group after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127501 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > address_group after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127665 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > qos_plugin after_init 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127792 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > trunk_plugin after_init 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.127984 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > logging-plugin after_init 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.128391 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.services.logapi.drivers.ovn.driver [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] OVN logging driver registered May 21 23:20:49.129598 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.plugins.ml2.plugin [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=104501) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}} May 21 23:20:49.129843 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > port after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.129941 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > port after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130004 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > port after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130143 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > subnet after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130212 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > subnet after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130290 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > subnet after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130385 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > network after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130473 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > network after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130570 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > network after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130703 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130820 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.130951 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.131083 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group_rule after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.131190 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group_rule after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.131292 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > security_group_rule after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.131371 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > address_group after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.131487 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > address_group after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.131575 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > address_group after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.131646 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ML2 OVO RPC backend initialized. {{(pid=104501) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}} May 21 23:20:49.132389 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.quota [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded quota_driver: . May 21 23:20:49.132477 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.plugin [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Modular L2 Plugin initialization complete May 21 23:20:49.132537 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading services supported by the core plugin {{(pid=104501) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}} May 21 23:20:49.132650 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.plugins.ml2.managers [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Got port-security extension from driver 'port_security' May 21 23:20:49.132875 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=104501) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}} May 21 23:20:49.132939 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading Plugin: ovn-router May 21 23:20:49.140724 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.140847 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: port before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.140924 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: port after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141022 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > port precommit_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141093 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: port before_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141232 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: port before_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141319 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141405 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141486 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141573 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: subnet after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.141746 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Tracking information for resource: router configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:49.141813 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.quota.resource_registry [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Tracking information for resource: floatingip configured {{(pid=104501) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} May 21 23:20:49.141873 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.services.ovn_l3.plugin [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Starting OVNL3RouterPlugin May 21 23:20:49.142073 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > port_forwarding_plugin after_init 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.142246 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router before_create 55550100, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.142344 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_delete 55550100, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.142408 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_create 55550100, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.142495 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_update 55550100, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.144967 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.services.provider_configuration [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Service providers = [] {{(pid=104501) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}} May 21 23:20:49.145101 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.db.servicetype_db [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=104501) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:45}} May 21 23:20:49.146081 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_create 55549900, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146179 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router_interface after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146271 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > floatingip after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146335 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > floatingip precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146433 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > floatingip precommit_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146500 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > floatingip after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146609 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > floatingip after_status_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146687 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > floatingip after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146749 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > port_forwarding after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146828 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router_interface after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.146913 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router after_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.147010 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.147113 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router after_delete 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.147184 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > router after_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.147251 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.services.service_base [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=104501) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 21 23:20:49.147378 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: > process after_init 55550001, True {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.147467 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=104501) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} May 21 23:20:49.147528 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading Plugin: auto_allocate May 21 23:20:49.148275 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: network precommit_update 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.148344 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron_lib.callbacks.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Subscribe: network precommit_create 55550000, False {{(pid=104501) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.148534 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=104501) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} May 21 23:20:49.148613 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading Plugin: tag May 21 23:20:49.150720 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=104501) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} May 21 23:20:49.150800 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading Plugin: timestamp May 21 23:20:49.151827 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=104501) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} May 21 23:20:49.151892 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading Plugin: network_ip_availability May 21 23:20:49.153142 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=104501) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} May 21 23:20:49.153227 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading Plugin: flavors May 21 23:20:49.153974 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=104501) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} May 21 23:20:49.154053 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading Plugin: revisions May 21 23:20:49.154897 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.manager [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=104501) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} May 21 23:20:49.154997 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_concurrency.lockutils [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.447s {{(pid=104501) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}} May 21 23:20:49.155191 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=104501) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}} May 21 23:20:49.155265 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Initializing extension manager. May 21 23:20:49.155499 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: __init__.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.155582 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: __pycache__ {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.155650 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: aap_reject_multicast.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.155869 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Allowed address pairs reject multicast extension" alias="aap-reject-multicast" description="Reject multicast addresses in Allowed Address Pairs" updated="2026-04-01T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.156063 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension aap-reject-multicast not supported by any of loaded plugins May 21 23:20:49.156147 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: address_group.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.156405 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.156549 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: address-group May 21 23:20:49.156624 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: address_scope.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.156857 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.157002 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: address-scope May 21 23:20:49.157090 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: admin_state_down_before_update.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.157298 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.157442 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins May 21 23:20:49.157517 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: agent.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.157755 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.157895 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: agent May 21 23:20:49.157967 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: agent_resources_synced.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.158169 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.158306 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension agent-resources-synced not supported by any of loaded plugins May 21 23:20:49.158377 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: agent_sort_key.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.158562 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Agents Sort Key" alias="agent_sort_key" description="Enabling the keyword sort_key for sorting in agents." updated="2013-02-03T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.158694 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension agent_sort_key not supported by any of loaded plugins May 21 23:20:49.158765 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: allowedaddresspairs.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.159346 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.159495 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: allowed-address-pairs May 21 23:20:49.159555 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: auto_allocated_topology.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.159772 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.159933 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: auto-allocated-topology May 21 23:20:49.160003 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: availability_zone.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.160254 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.160385 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: availability_zone May 21 23:20:49.160457 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: availability_zone_filter.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.160655 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.160786 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension availability_zone_filter not supported by any of loaded plugins May 21 23:20:49.160868 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: data_plane_status.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.161056 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.161221 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension data-plane-status not supported by any of loaded plugins May 21 23:20:49.161295 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: default_subnetpools.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.161480 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.161613 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: default-subnetpools May 21 23:20:49.161680 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: dhcpagentscheduler.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.161980 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.162134 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: dhcp_agent_scheduler May 21 23:20:49.162188 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: dns.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.162389 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.162532 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: dns-integration May 21 23:20:49.162598 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: dns_domain_ports.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.162791 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.162966 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: dns-domain-ports May 21 23:20:49.163027 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.163228 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like , , and as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.163399 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: dns-integration-domain-keywords May 21 23:20:49.163471 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: dvr.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.163635 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.163783 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension dvr not supported by any of loaded plugins May 21 23:20:49.163861 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: ecmp_routes.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.164111 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.164263 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension ecmp_routes not supported by any of loaded plugins May 21 23:20:49.164337 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: empty_string_filtering.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.164502 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.164634 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: empty-string-filtering May 21 23:20:49.164705 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: evpn.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.164886 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="EVPN" alias="evpn" description="Router extension to support EVPN Type-5 route advertisement" updated="2026-04-28T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.165018 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension evpn not supported by any of loaded plugins May 21 23:20:49.165113 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.165310 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.165444 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins May 21 23:20:49.165500 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.165692 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.165825 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins May 21 23:20:49.165896 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: external_net.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.166089 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.166223 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: external-net May 21 23:20:49.166309 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: extra_dhcp_opt.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.166478 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.166608 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: extra_dhcp_opt May 21 23:20:49.166679 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: extraroute.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.166855 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.167016 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: extraroute May 21 23:20:49.167104 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: extraroute_atomic.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.167296 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.167430 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: extraroute-atomic May 21 23:20:49.167500 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: filter_validation.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.167693 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.167827 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: filter-validation May 21 23:20:49.167899 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: fip_pf_description.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.168090 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.168334 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins May 21 23:20:49.168403 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: fip_pf_detail.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.168587 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.168732 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: floating-ip-port-forwarding-detail May 21 23:20:49.168804 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: fip_pf_port_range.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.168981 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.169134 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins May 21 23:20:49.169214 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: fip_port_details.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.169400 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.169534 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: fip-port-details May 21 23:20:49.169605 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: flavors.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.169803 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.169936 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: flavors May 21 23:20:49.170006 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.170271 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.170407 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins May 21 23:20:49.170477 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: floatingip_pools.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.170686 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.170821 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: floatingip-pools May 21 23:20:49.170912 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: ip_allocation.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.171108 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.171240 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension ip_allocation not supported by any of loaded plugins May 21 23:20:49.171312 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.171499 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.171620 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension ip-substring-filtering not supported by any of loaded plugins May 21 23:20:49.171689 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l2_adjacency.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.171868 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.171996 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension l2_adjacency not supported by any of loaded plugins May 21 23:20:49.172092 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.172423 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.172578 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: router May 21 23:20:49.172647 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_conntrack_helper.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.173275 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.173420 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension l3-conntrack-helper not supported by any of loaded plugins May 21 23:20:49.173495 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.173703 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.173975 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: enable-default-route-bfd May 21 23:20:49.174035 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.174251 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.174375 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: enable-default-route-ecmp May 21 23:20:49.174445 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.174634 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.174764 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: ext-gw-mode May 21 23:20:49.174836 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.175061 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.175208 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: l3-ha May 21 23:20:49.175281 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.175461 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.175591 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins May 21 23:20:49.175661 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_extra_gws.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.175839 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.175966 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: external-gateway-multihoming May 21 23:20:49.176054 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_flavors.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.176233 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.176364 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: l3-flavors May 21 23:20:49.176438 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_ndp_proxy.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.176678 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.176803 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension l3-ndp-proxy not supported by any of loaded plugins May 21 23:20:49.176873 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.177061 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.177223 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins May 21 23:20:49.177295 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: l3agentscheduler.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.177713 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.177850 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension l3_agent_scheduler not supported by any of loaded plugins May 21 23:20:49.177923 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: local_ip.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.178193 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.178328 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension local_ip not supported by any of loaded plugins May 21 23:20:49.178400 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: logging.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.178642 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.178773 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension logging not supported by any of loaded plugins May 21 23:20:49.178840 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: metering.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.179253 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.179392 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension metering not supported by any of loaded plugins May 21 23:20:49.179466 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.179668 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.179805 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins May 21 23:20:49.179872 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: multiprovidernet.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.180066 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.180205 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: multi-provider May 21 23:20:49.180277 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: netmtu.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.180458 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.180608 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: net-mtu May 21 23:20:49.180681 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: netmtu_writable.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.180862 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.180996 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: net-mtu-writable May 21 23:20:49.181082 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: network_availability_zone.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.181304 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.181435 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: network_availability_zone May 21 23:20:49.181506 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: network_ha.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.181687 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.181818 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension network_ha not supported by any of loaded plugins May 21 23:20:49.181890 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: network_ip_availability.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.182098 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.182231 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: network-ip-availability May 21 23:20:49.182302 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: network_ip_availability_details.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.182496 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.182632 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: network-ip-availability-details May 21 23:20:49.182704 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: network_segment_range.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.182976 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.183130 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension network-segment-range not supported by any of loaded plugins May 21 23:20:49.183203 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: pagination.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.183401 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.183471 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: pagination May 21 23:20:49.183541 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_device_profile.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.183735 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.183865 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: port-device-profile May 21 23:20:49.183936 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_hardware_offload_type.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.184141 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.184267 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-hardware-offload-type not supported by any of loaded plugins May 21 23:20:49.184337 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.184515 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.184648 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins May 21 23:20:49.184718 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_hints.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.184893 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.185024 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-hints not supported by any of loaded plugins May 21 23:20:49.185125 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_mac_address_override.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.185315 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.185444 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-mac-override not supported by any of loaded plugins May 21 23:20:49.185530 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.185718 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.185851 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: port-mac-address-regenerate May 21 23:20:49.185918 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.186118 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.186251 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: port-numa-affinity-policy May 21 23:20:49.186321 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.186497 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.186631 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: port-numa-affinity-policy-socket May 21 23:20:49.186706 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_resource_request.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.186938 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.187096 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-resource-request not supported by any of loaded plugins May 21 23:20:49.187170 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_resource_request_groups.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.187361 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.187494 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-resource-request-groups not supported by any of loaded plugins May 21 23:20:49.187566 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: port_trusted_vif.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.187742 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.187873 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-trusted-vif not supported by any of loaded plugins May 21 23:20:49.187944 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: portbindings.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.188138 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.188271 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: binding May 21 23:20:49.188341 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: portbindings_extended.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.188547 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.188678 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: binding-extended May 21 23:20:49.188746 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: portsecurity.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.188921 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.189055 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: port-security May 21 23:20:49.189145 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: project_id.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.189339 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.189414 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: project-id May 21 23:20:49.189487 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: providernet.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.189666 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.189799 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: provider May 21 23:20:49.189871 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qinq.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.190083 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x88a8 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.190218 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: qinq May 21 23:20:49.190295 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.190994 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.191152 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos not supported by any of loaded plugins May 21 23:20:49.191207 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.191448 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.191574 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins May 21 23:20:49.191652 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.191875 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.192053 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins May 21 23:20:49.192158 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_default.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.192342 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.192474 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-default not supported by any of loaded plugins May 21 23:20:49.192545 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_fip.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.192729 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.192862 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-fip not supported by any of loaded plugins May 21 23:20:49.192932 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_fip_network_policy.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.193129 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.193271 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-fip-network-policy not supported by any of loaded plugins May 21 23:20:49.193342 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_gateway_ip.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.193523 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.193656 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-gateway-ip not supported by any of loaded plugins May 21 23:20:49.193727 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_port_network_policy.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.193905 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.194059 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-port-network-policy not supported by any of loaded plugins May 21 23:20:49.194145 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.194336 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.194473 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-pps-minimum not supported by any of loaded plugins May 21 23:20:49.194565 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.194769 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.194925 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins May 21 23:20:49.195016 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_pps_rule.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.195293 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.195463 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-pps not supported by any of loaded plugins May 21 23:20:49.195554 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_rule_type_details.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.195782 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.195951 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-rule-type-details not supported by any of loaded plugins May 21 23:20:49.196035 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_rule_type_filter.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.196243 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.196373 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-rule-type-filter not supported by any of loaded plugins May 21 23:20:49.196444 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: qos_rules_alias.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.196635 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.196765 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension qos-rules-alias not supported by any of loaded plugins May 21 23:20:49.196835 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: quota_check_limit.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.197010 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.197175 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: quota-check-limit May 21 23:20:49.197249 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: quota_check_limit_default.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.197428 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.197561 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: quota-check-limit-default May 21 23:20:49.197632 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: quotasv2.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.197991 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.198087 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: quotas May 21 23:20:49.198160 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: quotasv2_detail.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.198562 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.198645 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: quota_details May 21 23:20:49.198719 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: rbac.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.199166 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.199315 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: rbac-policies May 21 23:20:49.199388 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: rbac_address_group.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.199580 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.199722 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension rbac-address-group not supported by any of loaded plugins May 21 23:20:49.199781 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: rbac_address_scope.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.199968 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.200117 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: rbac-address-scope May 21 23:20:49.200191 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: rbac_security_groups.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.200373 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.200509 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: rbac-security-groups May 21 23:20:49.200580 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: rbac_subnetpool.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.200761 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.200893 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension rbac-subnetpool not supported by any of loaded plugins May 21 23:20:49.200964 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: revisionifmatch.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.201176 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.201314 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: revision-if-match May 21 23:20:49.201383 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: revisions.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.201579 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.201716 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: standard-attr-revisions May 21 23:20:49.201788 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: router_availability_zone.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.201986 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.202138 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: router_availability_zone May 21 23:20:49.202206 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: router_enable_snat.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.202384 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.202516 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: router-enable-snat May 21 23:20:49.202588 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: routerservicetype.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.202773 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.202932 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension router-service-type not supported by any of loaded plugins May 21 23:20:49.203001 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: security_groups_default_rules.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.203372 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.203512 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: security-groups-default-rules May 21 23:20:49.203566 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.203764 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.203902 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: security-groups-normalized-cidr May 21 23:20:49.203968 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: security_groups_port_filtering.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.204179 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.204304 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension port-security-groups-filtering not supported by any of loaded plugins May 21 23:20:49.204376 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.204566 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.204701 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: security-groups-remote-address-group May 21 23:20:49.204768 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.204947 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.205098 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: security-groups-rules-belongs-to-default-sg May 21 23:20:49.205183 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.205368 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.205501 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: security-groups-shared-filtering May 21 23:20:49.205572 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: securitygroup.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.206891 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.207056 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: security-group May 21 23:20:49.207121 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: segment.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.207376 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.207519 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension segment not supported by any of loaded plugins May 21 23:20:49.207590 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.207786 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.207916 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins May 21 23:20:49.207983 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: servicetype.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.208213 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.208332 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: service-type May 21 23:20:49.208404 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: sorting.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.208589 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.208669 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: sorting May 21 23:20:49.208739 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: standard_attr_segment.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.208916 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.209060 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension standard-attr-segment not supported by any of loaded plugins May 21 23:20:49.209142 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: standardattrdescription.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.209349 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.209484 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: standard-attr-description May 21 23:20:49.209556 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: stateful_security_group.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.209755 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.209890 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: stateful-security-group May 21 23:20:49.209961 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: stdattrs_common.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.210146 np0000176093 neutron-ovn-maintenance-worker[104501]: WARNING neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py May 21 23:20:49.210221 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.210396 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.210526 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: subnet-dns-publish-fixed-ip May 21 23:20:49.210596 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: subnet_external_network.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.210770 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.210950 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: subnet-external-network May 21 23:20:49.211023 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: subnet_onboard.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.211232 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.211371 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension subnet_onboard not supported by any of loaded plugins May 21 23:20:49.211430 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.211615 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.211748 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins May 21 23:20:49.211820 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: subnet_service_types.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.211996 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.212147 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: subnet-service-types May 21 23:20:49.212208 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: subnetallocation.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.212531 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.212679 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: subnet_allocation May 21 23:20:49.212739 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.212951 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.213111 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: subnetpool-prefix-ops May 21 23:20:49.213195 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: tag_creation.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.213393 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.213524 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: tag-creation May 21 23:20:49.213598 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.213789 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.213918 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins May 21 23:20:49.213991 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: tagging.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.214570 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.214721 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: standard-attr-tag May 21 23:20:49.214788 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: timestamp.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.215035 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.215197 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: standard-attr-timestamp May 21 23:20:49.215259 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: trunk.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.215462 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.215591 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension trunk not supported by any of loaded plugins May 21 23:20:49.215664 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: trunk_details.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.215851 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.215977 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension trunk-details not supported by any of loaded plugins May 21 23:20:49.216060 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: uplink_status_propagation.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.216247 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.216379 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension uplink-status-propagation not supported by any of loaded plugins May 21 23:20:49.216449 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.216628 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.216759 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins May 21 23:20:49.216830 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loading extension file: vlantransparent.py {{(pid=104501) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} May 21 23:20:49.217021 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=104501) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} May 21 23:20:49.217189 np0000176093 neutron-ovn-maintenance-worker[104501]: INFO neutron.api.extensions [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Loaded extension: vlan-transparent May 21 23:20:49.219402 np0000176093 neutron-ovn-maintenance-worker[104501]: WARNING oslo_service.backend._threading.service [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback. May 21 23:20:49.219722 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Full set of CONF: {{(pid=104501) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}} May 21 23:20:49.219806 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ******************************************************************************** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3229}} May 21 23:20:49.219887 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Configuration options gathered from: {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3230}} May 21 23:20:49.219972 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3231}} May 21 23:20:49.220107 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3232}} May 21 23:20:49.220167 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ================================================================================ {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3237}} May 21 23:20:49.220279 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] agent_down_time = 75 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.220370 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] allow_automatic_dhcp_failover = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.220458 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] allow_bulk = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.220637 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.220727 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] api_extensions_path = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.220816 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] api_paste_config = api-paste.ini {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.220912 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] api_workers = 4 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221001 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] auth_strategy = keystone {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221184 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] base_mac = fa:16:3e:00:00:00 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221246 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] bind_host = 0.0.0.0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221343 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] bind_port = 9696 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221430 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] client_socket_timeout = 900 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221522 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] config_dir = [] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221589 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221677 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] config_source = [] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221769 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] control_exchange = neutron {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221861 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] core_plugin = ml2 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.221950 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] debug = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222038 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] default_availability_zones = [] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222208 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] default_log_levels = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222300 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] dhcp_agent_notification = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222387 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] dhcp_agents_per_network = 1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222473 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] dhcp_lease_duration = 86400 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222566 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] dhcp_load_type = networks {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222654 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] dns_domain = openstacklocal {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222744 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.222847 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] enable_default_route_bfd = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223007 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] enable_default_route_ecmp = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223115 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] enable_dvr = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223199 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] enable_new_agents = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223283 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223367 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] enable_snat_by_default = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223447 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] enable_traditional_dhcp = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223528 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] external_dns_driver = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223610 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] filter_validation = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223701 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] global_physnet_mtu = 1430 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223763 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] graceful_shutdown_timeout = 60 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.223919 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] host = np0000176093 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224031 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] host_dvr_for_dhcp = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224146 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] http_retries = 3 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224246 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] instance_format = [instance: %(uuid)s] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224348 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224437 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ipam_driver = internal {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224522 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ipv6_pd_enabled = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224608 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] l3_ha = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224698 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224786 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] l3_ha_network_physical_name = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224875 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] l3_ha_network_type = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.224964 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_color = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225070 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_config_append = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225163 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225248 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_dir = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225333 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_file = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225396 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_options = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225481 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_rotate_interval = 1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225572 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_rotate_interval_type = days {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225660 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] log_rotation_type = none {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225759 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225790 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225883 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.225969 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226032 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226133 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_allowed_address_pair = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226230 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_dns_nameservers = 5 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226317 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_header_line = 16384 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226402 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_l3_agents_per_router = 3 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226487 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_logfile_count = 30 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226570 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_logfile_size_mb = 200 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226655 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_routes = 30 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226739 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] max_subnet_host_routes = 20 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226853 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] my_ip = 199.19.213.138 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.226939 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] my_ipv6 = 2604:e100:1:0:f816:3eff:fe24:5c07 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227023 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] network_auto_schedule = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227123 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] network_link_prefix = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227213 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227302 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] notify_nova_on_port_data_changes = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227392 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] notify_nova_on_port_status_changes = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227481 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] pagination_max_limit = -1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227563 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] periodic_fuzzy_delay = 5 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227654 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] periodic_interval = 40 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227737 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] publish_errors = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227819 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rate_limit_burst = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227907 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rate_limit_except_level = CRITICAL {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.227989 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rate_limit_interval = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228121 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] router_distributed = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228210 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rpc_resources_processing_step = 20 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228382 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rpc_response_max_timeout = 600 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228382 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rpc_response_timeout = 60 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228451 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rpc_state_report_workers = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228538 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] rpc_workers = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228619 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] send_events_interval = 2 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228711 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] service_plugins = ['ovn-router'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228800 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] setproctitle = on {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228880 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] shell_completion = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.228966 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] state_path = /opt/stack/data/neutron {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229076 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] syslog_log_facility = LOG_USER {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229163 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] tcp_keepidle = 600 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229251 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] transport_url = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229333 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] use_journal = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229414 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] use_json = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229495 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] use_stderr = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229580 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] use_syslog = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229660 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] vlan_qinq = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229738 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] vlan_transparent = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229819 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] wsgi_default_pool_size = 100 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229900 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] wsgi_keep_alive = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.229986 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.230083 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] wsgi_server_debug = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.230183 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_concurrency.disable_process_locking = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230276 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230373 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.connection_string = messaging:// {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230461 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.enabled = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230552 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.es_doc_type = notification {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230759 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.es_scroll_size = 10000 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230759 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.es_scroll_time = 2m {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230845 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.filter_error_trace = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.230983 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.hmac_keys = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231114 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.sentinel_service_name = mymaster {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231213 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.socket_timeout = 0.1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231304 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.trace_requests = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231392 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler.trace_sqlalchemy = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231489 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler_jaeger.process_tags = {} {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231575 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler_jaeger.service_name_prefix = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231685 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] profiler_otlp.service_name_prefix = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231813 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.231935 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.232088 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.232222 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.232348 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.232488 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.enforce_new_defaults = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.232749 np0000176093 neutron-ovn-maintenance-worker[104501]: WARNING oslo_config.cfg [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.). Its value may be silently ignored in the future. May 21 23:20:49.232882 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.enforce_scope = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233016 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.policy_default_rule = default {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233177 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233309 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233443 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233567 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233687 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233805 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.233926 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.234056 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_policy.remote_timeout = 60.0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.234219 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.234346 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] service_providers.service_provider = [] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.234478 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.auth_section = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.234608 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.auth_type = password {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.234734 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.auth_url = https://199.19.213.138/identity {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.234891 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.cafile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235012 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.certfile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235158 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.collect_timing = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235277 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.default_domain_id = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235396 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.default_domain_name = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235514 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.domain_id = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235627 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.domain_name = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235755 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.endpoint_type = public {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235876 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.insecure = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.235994 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.keyfile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.236142 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.password = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.236262 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.project_domain_id = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.236396 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.project_domain_name = Default {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.236511 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.project_id = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.236639 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.project_name = service {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.236764 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.region_name = RegionOne {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.236884 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.split_loggers = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237000 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.system_scope = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237139 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.timeout = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237258 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.trust_id = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237375 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.user_domain_id = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237498 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.user_domain_name = Default {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237616 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.user_id = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237748 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] nova.username = nova {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.237871 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.auth_section = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238006 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.auth_type = password {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238138 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.cafile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238257 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.certfile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238378 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.collect_timing = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238505 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.endpoint_type = public {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238625 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.insecure = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238741 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.keyfile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.238882 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.region_name = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.239012 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.split_loggers = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.239150 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] placement.timeout = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.239277 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.auth_section = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.239396 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.auth_type = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.239508 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.cafile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.239630 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.certfile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.239827 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.collect_timing = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.240024 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.connect_retries = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.240254 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.connect_retry_delay = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.240455 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.enable_notifications = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.240655 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.endpoint_override = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.240854 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.insecure = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.241058 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.interface = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.241276 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.keyfile = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.241477 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.max_version = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.241670 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.min_version = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.241868 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.region_name = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.242099 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.retriable_status_codes = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.242303 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.service_name = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.242501 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.service_type = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.242704 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.split_loggers = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.242982 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.status_code_retries = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.243227 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.status_code_retry_delay = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.243423 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.timeout = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.243619 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.valid_interfaces = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.243708 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ironic.version = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.243795 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] cli_script.dry_run = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.243892 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.243979 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244090 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244179 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244266 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244351 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244440 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244532 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244621 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.hostname = np0000176093 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244706 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244799 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244889 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.244979 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245087 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245183 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245270 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245358 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245451 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245539 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245921 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245921 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245921 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.245921 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246024 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246024 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246133 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246332 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246332 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246392 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246466 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.ssl = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246555 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246647 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246733 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246861 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.ssl_enforce_hostname_verification = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.246933 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247025 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.ssl_version = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247128 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247226 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_notifications.driver = [] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247311 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_notifications.retry = -1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247409 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247491 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_messaging_notifications.transport_url = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247580 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_reports.file_event_handler = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247666 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247750 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_reports.log_dir = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247841 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.247933 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.default_quota = -1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248028 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_driver = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248131 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_floatingip = 50 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248216 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_network = 100 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248302 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_port = 500 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248392 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_rbac_policy = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248474 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_router = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248562 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_security_group = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248677 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248788 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.quota_subnet = 100 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248873 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] QUOTAS.track_quota_usage = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.248975 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249073 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep.group = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249177 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep.helper_command = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249272 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep.log_daemon_traceback = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249364 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep.logger_name = oslo_privsep.daemon {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249449 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep.thread_pool_size = 16 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249533 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep.user = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249626 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249712 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_dhcp_release.group = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249794 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_dhcp_release.helper_command = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249877 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.249967 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250073 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250156 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_dhcp_release.user = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250248 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250333 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_ovs_vsctl.group = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250420 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_ovs_vsctl.helper_command = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250506 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250596 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_ovs_vsctl.logger_name = oslo_privsep.daemon {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250683 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250768 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_ovs_vsctl.user = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250899 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_namespace.capabilities = [21] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.250977 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_namespace.group = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251080 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_namespace.helper_command = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251165 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_namespace.log_daemon_traceback = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251255 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_namespace.logger_name = oslo_privsep.daemon {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251338 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_namespace.thread_pool_size = 16 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251420 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_namespace.user = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251509 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_conntrack.capabilities = [12] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251592 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_conntrack.group = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251675 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_conntrack.helper_command = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251759 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251846 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_conntrack.logger_name = oslo_privsep.daemon {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.251930 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_conntrack.thread_pool_size = 16 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252013 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_conntrack.user = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252133 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_link.capabilities = [12, 21] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252227 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_link.group = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252310 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_link.helper_command = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252395 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_link.log_daemon_traceback = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252482 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_link.logger_name = oslo_privsep.daemon {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252567 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_link.thread_pool_size = 16 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252655 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] privsep_link.user = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252755 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] AGENT.root_helper = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252850 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] AGENT.root_helper_daemon = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.252935 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] AGENT.use_helper_for_ns_read = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253025 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] SECURITYGROUP.enable_ipset = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253143 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] SECURITYGROUP.enable_security_group = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253233 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] SECURITYGROUP.firewall_driver = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253323 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253414 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] experimental.ipv6_pd_enabled = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253517 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.extension_drivers = ['port_security', 'qos'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253602 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.external_network_type = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253697 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.mechanism_drivers = ['ovn'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253791 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.overlay_ip_version = 4 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253878 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.path_mtu = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.253966 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.physical_network_mtus = {} {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254081 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.project_network_types = ['geneve'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254174 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.tunnelled_network_rp_name = rp_tunnelled {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254270 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2.type_drivers = ['local', 'flat', 'vlan', 'geneve'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254372 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2_type_flat.flat_networks = ['public'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254468 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2_type_geneve.max_header_size = 38 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254563 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2_type_geneve.vni_ranges = ['1:65536'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254661 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254755 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] network_log.burst_limit = 25 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.254914 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] network_log.local_output_log_base = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255021 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] network_log.rate_limit = 100 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255140 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.asyncio_connection = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255224 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.asyncio_slave_connection = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255316 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.backend = sqlalchemy {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255407 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.connection = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255493 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.connection_debug = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255582 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.connection_parameters = {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255668 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.connection_recycle_time = 3600 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255753 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.connection_trace = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255836 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.db_inc_retry_interval = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.255920 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.db_max_retries = 20 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256005 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.db_max_retry_interval = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256107 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.db_retry_interval = 1 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256191 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.max_overflow = 50 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256274 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.max_pool_size = 5 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256357 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.max_retries = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256446 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256529 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.mysql_wsrep_sync_wait = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256613 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.pool_timeout = None {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256697 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.retry_interval = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256779 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.slave_connection = **** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256864 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.sqlite_synchronous = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.256947 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.synchronous_reader = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257030 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] database.use_db_reconnect = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257144 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.bfd_min_rx = 1000 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257227 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.bfd_min_tx = 100 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257311 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.bfd_mult = 3 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257395 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257480 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.dhcp_default_lease_time = 43200 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257564 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257649 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.dns_records_ovn_owned = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257733 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.dns_servers = [] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257818 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.enable_distributed_floating_ip = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257902 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.fdb_age_threshold = 300 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.257994 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ha_failover_strategy = normal {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258099 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.live_migration_activation_strategy = rarp {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258185 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.localnet_learn_fdb = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258269 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.mac_binding_age_threshold = 300 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258359 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.neutron_sync_mode = log {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258445 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_dhcp4_global_options = {} {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258531 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_dhcp6_global_options = {} {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258620 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_l3_scheduler = leastloaded {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258709 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_metadata_enabled = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258798 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_nb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.258999 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_nb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259162 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_nb_connection = ['ssl:199.19.213.138:6641'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259291 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_nb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259419 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_router_indirect_snat = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259551 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_sb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259679 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_sb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259798 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_sb_connection = ['ssl:199.19.213.138:6642'] {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259889 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovn_sb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.259976 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovs_create_tap = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260084 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovsdb_connection_timeout = 180 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260178 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovsdb_log_level = INFO {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260265 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovsdb_probe_interval = 60000 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260350 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.ovsdb_retry_max_interval = 180 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260436 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.stateless_nat_enabled = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260526 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn.vhost_sock_dir = /var/run/openvswitch {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260619 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] OVS.bridge_mac_table_size = 50000 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260703 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] OVS.igmp_flood = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260788 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] OVS.igmp_flood_reports = True {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260875 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] OVS.igmp_flood_unregistered = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.260965 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] OVS.igmp_snooping_enable = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.261061 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] OVS.ovsdb_timeout = 10 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.261168 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.261253 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn_nb_global.ignore_lsp_down = False {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.261340 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.261406 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG cotyledon.oslo_config_glue [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] ******************************************************************************** {{(pid=104501) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3263}} May 21 23:20:49.261496 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG oslo_service.backend._threading.service [None req-ed1e0bdc-0e8a-42c9-9cdb-ddd5940e7fd7 None None] Selected the multiprocessing context: and updated it in Cotyledon ServiceManager: {{(pid=104501) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:405}} May 21 23:20:49.290988 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=105248) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}} May 21 23:20:49.291605 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3229}} May 21 23:20:49.291605 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3230}} May 21 23:20:49.291775 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3231}} May 21 23:20:49.291931 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3232}} May 21 23:20:49.292019 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3237}} May 21 23:20:49.292313 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time = 75 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.292483 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.292666 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.293066 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.293270 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.293453 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config = api-paste.ini {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.293659 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] api_workers = 4 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.293832 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy = keystone {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.294006 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] base_mac = fa:16:3e:00:00:00 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.294372 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] bind_host = 0.0.0.0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.294544 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] bind_port = 9696 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.294697 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout = 900 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.294934 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] config_dir = [] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.295007 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.295226 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] config_source = [] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.295409 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] control_exchange = neutron {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.295572 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] core_plugin = ml2 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.295740 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] debug = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.295921 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones = [] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.296265 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.296426 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.296584 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network = 1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.296734 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration = 86400 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.296916 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type = networks {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.297096 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] dns_domain = openstacklocal {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.297270 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.297427 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.297558 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.297716 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.297865 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.298013 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.298184 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.298340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.298487 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.298568 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.298714 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] filter_validation = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.298927 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu = 1430 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.299076 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout = 60 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.299441 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] host = np0000176093 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.300516 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.300611 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] http_retries = 3 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.300717 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] instance_format = [instance: %(uuid)s] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301030 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301142 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver = internal {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301243 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301358 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] l3_ha = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301478 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301582 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301681 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301793 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_color = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.301916 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_config_append = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302014 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302133 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_dir = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302240 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_file = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302298 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_options = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302393 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval = 1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302499 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type = days {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302595 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type = none {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302685 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.302750 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303087 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303161 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303233 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303343 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303439 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers = 5 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303528 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_header_line = 16384 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303617 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router = 3 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303706 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count = 30 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.303795 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb = 200 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304110 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_routes = 30 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304213 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes = 20 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304325 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] my_ip = 199.19.213.138 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304433 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6 = 2604:e100:1:0:f816:3eff:fe24:5c07 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304532 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304619 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304736 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304843 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.304965 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.305104 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit = -1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.305201 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay = 5 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.305300 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval = 40 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.305398 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] publish_errors = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.305599 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.305973 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level = CRITICAL {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.306152 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.306474 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] router_distributed = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.306846 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step = 20 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307088 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout = 600 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307212 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout = 60 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307310 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307402 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307493 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval = 2 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307626 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] service_plugins = ['ovn-router'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307715 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] setproctitle = on {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307819 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] shell_completion = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.307923 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] state_path = /opt/stack/data/neutron {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308036 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility = LOG_USER {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308146 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle = 600 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308261 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] transport_url = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308354 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] use_journal = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308443 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] use_json = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308542 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] use_stderr = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308642 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] use_syslog = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308730 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308817 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.308918 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size = 100 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.309009 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.309134 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.309226 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} May 21 23:20:49.309389 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.309510 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.309647 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string = messaging:// {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.309778 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.309876 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type = notification {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.309986 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size = 10000 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310108 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time = 2m {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310214 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310325 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310430 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310540 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout = 0.1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310631 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310743 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.310917 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags = {} {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311004 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311125 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311252 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311354 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311467 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311582 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311687 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311818 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311895 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.311976 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312079 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312141 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312214 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312286 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312345 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312410 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312463 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312535 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout = 60.0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312627 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312705 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312811 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312886 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type = password {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.312958 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url = https://199.19.213.138/identity {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313013 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313103 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313176 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313241 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313301 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313363 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313434 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313480 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type = public {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313557 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313640 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313687 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.password = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313754 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313814 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name = Default {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313878 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.313939 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name = service {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314010 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name = RegionOne {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314078 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314142 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314201 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314263 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314323 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314376 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name = Default {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314446 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314508 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] nova.username = nova {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314576 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314643 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type = password {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314710 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314762 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314866 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314911 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type = public {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.314958 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315029 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315094 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315169 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315217 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315299 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315458 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315458 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315458 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315568 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315614 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315660 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315706 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315785 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315842 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315906 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.315971 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316018 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316109 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316168 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316242 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316313 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316358 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316427 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316480 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316561 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316609 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316667 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316729 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ironic.version = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316790 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316902 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.316962 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317032 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317122 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317181 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317244 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317307 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317377 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317431 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000176093 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317518 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317573 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317647 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317709 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317782 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317853 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317907 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.317975 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318038 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318125 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318182 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318237 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318290 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318359 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318430 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318475 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318544 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318613 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318657 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318738 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318821 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318876 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.318949 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319020 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319105 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_hostname_verification = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319178 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319223 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319290 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319361 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319432 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319491 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319549 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319638 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319710 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319763 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319854 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.319950 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota = -1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320020 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320190 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip = 50 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320256 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network = 100 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320329 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port = 500 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320405 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320479 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320694 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320694 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320694 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet = 100 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320813 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320813 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320907 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep.group = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.320947 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321023 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321136 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name = oslo_privsep.daemon {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321202 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size = 16 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321291 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep.user = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321357 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321417 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321530 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321571 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321611 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321773 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321773 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321773 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321888 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321888 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.321976 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322016 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name = oslo_privsep.daemon {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322104 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322152 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322287 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322287 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322357 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322406 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322467 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name = oslo_privsep.daemon {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322526 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322589 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322657 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322728 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322774 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322907 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.322949 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name = oslo_privsep.daemon {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323001 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323159 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323245 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities = [12, 21] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323357 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323423 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323531 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323531 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name = oslo_privsep.daemon {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323611 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size = 16 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323663 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323746 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323810 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.323880 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324010 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324079 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324206 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324206 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324297 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324338 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers = ['port_security', 'qos'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324418 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324497 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers = ['ovn'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324551 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version = 4 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324618 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324680 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus = {} {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324745 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types = ['geneve'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324803 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name = rp_tunnelled {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324880 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.324969 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks = ['public'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325069 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325143 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges = ['1:65536'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325232 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325327 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit = 25 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325394 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325466 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit = 100 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325565 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325615 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325689 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.backend = sqlalchemy {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325762 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.connection = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325817 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325900 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters = {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.325971 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326035 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326102 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326173 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries = 20 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326233 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326305 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval = 1 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326350 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow = 50 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326418 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size = 5 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326470 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326540 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326610 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326656 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout = None {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326723 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326894 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection = **** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.326978 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327030 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327115 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327217 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx = 1000 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327276 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx = 100 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327333 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult = 3 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327398 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327454 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time = 43200 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327520 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327592 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327638 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers = [] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327706 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.327830 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold = 300 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328006 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy = normal {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328087 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328142 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328190 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold = 300 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328275 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode = log {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328345 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options = {} {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328419 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options = {} {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328492 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler = leastloaded {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328553 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328615 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328699 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328770 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection = ['ssl:199.19.213.138:6641'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328842 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328888 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.328959 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329006 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329133 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection = ['ssl:199.19.213.138:6642'] {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329184 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329244 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329374 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout = 180 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329374 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level = INFO {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329452 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval = 60000 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329601 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval = 180 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329701 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329812 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir = /var/run/openvswitch {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.329964 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size = 50000 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330099 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330208 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports = True {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330341 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330454 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330554 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout = 10 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330668 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330833 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down = False {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330833 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} May 21 23:20:49.330935 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105248) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3263}} May 21 23:20:49.331278 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [105248] {{(pid=105248) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}} May 21 23:20:49.332796 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-521564', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-25062'] for process (None), after_init {{(pid=105248) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} May 21 23:20:49.335090 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry May 21 23:20:49.393454 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} May 21 23:20:49.393611 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} May 21 23:20:49.393789 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} May 21 23:20:49.393944 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394107 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394223 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394342 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394467 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394615 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394695 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394870 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.394990 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.395244 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.395375 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.395491 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.401490 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.138:6641: connecting... May 21 23:20:49.407242 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.138:6641: connected May 21 23:20:49.432797 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry May 21 23:20:49.463618 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} May 21 23:20:49.463618 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} May 21 23:20:49.463851 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.463851 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=105248) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} May 21 23:20:49.466343 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.138:6642: connecting... May 21 23:20:49.472456 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.138:6642: connected May 21 23:20:49.514857 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip precommit_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515041 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip precommit_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515195 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515264 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515349 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515485 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port precommit_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515726 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-14645'] for port_forwarding_plugin (None), after_init {{(pid=105248) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} May 21 23:20:49.515828 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port_forwarding after_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515892 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > router_interface after_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.515953 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port_forwarding after_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.516014 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port_forwarding after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.516166 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > router_interface after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.516270 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension May 21 23:20:49.516365 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension May 21 23:20:49.516842 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment before_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.516949 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete_associations 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517265 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517372 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517372 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517531 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517531 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517531 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517699 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517699 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.517809 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.580541 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.580541 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.580541 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet before_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.580817 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet before_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.583165 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.583291 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.583357 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1333836'] for logging-plugin (None), after_init {{(pid=105248) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} May 21 23:20:49.583518 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group_rule after_create 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.583614 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group_rule after_update 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:49.583614 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group_rule after_delete 55550000, False {{(pid=105248) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} May 21 23:20:59.017565 np0000176093 neutron-ovn-maintenance-worker[104501]: DEBUG dbcounter [-] [104501] Writing DB stats neutron:SELECT=4 {{(pid=104501) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}} May 21 23:20:59.587548 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-05-21 23:20:59.587042 {{(pid=105248) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}} May 21 23:20:59.589635 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync port groups started @ 2026-05-21 23:20:59.589221 {{(pid=105248) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}} May 21 23:20:59.610809 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG dbcounter [-] [105248] Writer thread running {{(pid=105248) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}} May 21 23:20:59.627965 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync port groups completed @ 2026-05-21 23:20:59.627585 {{(pid=105248) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}} May 21 23:20:59.628135 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-05-21 23:20:59.627860 {{(pid=105248) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1152}} May 21 23:20:59.746298 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.services.segments.db [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['04670050-f514-47b5-ba66-c826bfb5281f'], 'is_dynamic': False}} {{(pid=105248) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}} May 21 23:20:59.751689 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync metadata ports started {{(pid=105248) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1097}} May 21 23:21:00.002898 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync metadata ports completed {{(pid=105248) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1149}} May 21 23:21:00.003157 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=105248) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:949}} May 21 23:21:00.054145 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-05-21 23:21:00.053911 {{(pid=105248) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1022}} May 21 23:21:00.055002 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} May 21 23:21:00.055895 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=105248) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1027}} May 21 23:21:00.055998 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=105248) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1086}} May 21 23:21:00.056081 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-05-21 23:21:00.055958 {{(pid=105248) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1321}} May 21 23:21:00.056213 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync ACLs started @ 2026-05-21 23:21:00.056103 {{(pid=105248) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}} May 21 23:21:00.091320 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync ACLs completed @ 2026-05-21 23:21:00.090981 {{(pid=105248) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:431}} May 21 23:21:00.091540 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync routers and router ports started @ 2026-05-21 23:21:00.091292 {{(pid=105248) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:736}} May 21 23:21:00.145527 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} May 21 23:21:00.145527 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync routers and router ports completed @ 2026-05-21 23:21:00.145227 {{(pid=105248) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:944}} May 21 23:21:00.145527 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync port QoS policies started @ 2026-05-21 23:21:00.145437 {{(pid=105248) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1369}} May 21 23:21:00.146026 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] Starting OVNClientQosExtension May 21 23:21:00.166867 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync port QoS policies completed @ 2026-05-21 23:21:00.166589 {{(pid=105248) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1392}} May 21 23:21:00.166972 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync Floating IP QoS policies started @ 2026-05-21 23:21:00.166840 {{(pid=105248) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1397}} May 21 23:21:00.167117 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] Starting OVNClientQosExtension May 21 23:21:00.190507 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-05-21 23:21:00.190325 {{(pid=105248) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1420}} May 21 23:21:00.190601 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync Floating IP NAT rules started @ 2026-05-21 23:21:00.190476 {{(pid=105248) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1425}} May 21 23:21:00.190810 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-05-21 23:21:00.190703 {{(pid=105248) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1451}} May 21 23:21:00.190874 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-ac034a04-568e-490e-a305-fac7e16190a0 None None] OVN-Northbound DB sync process completed @ 2026-05-21 23:21:00.190797 {{(pid=105248) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}} May 21 23:21:09.602954 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-05-21 23:21:09.602350 {{(pid=105248) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1480}} May 21 23:21:09.607495 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] OVN-SB Sync hostname and physical networks started @ 2026-05-21 23:21:09.604170 {{(pid=105248) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1511}} May 21 23:21:09.614039 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension May 21 23:21:09.621436 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621546 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621546 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621703 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621703 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621944 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621944 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621944 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621944 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.621944 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622704 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622704 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622704 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622704 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622704 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.622704 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.623191 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.623191 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.623191 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.623191 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=105248) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} May 21 23:21:09.624624 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.628470 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] Agents populated in the agent cache: {{(pid=105248) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:309}} May 21 23:21:09.628812 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started May 21 23:21:09.629394 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] - Agent type: OVN Controller Gateway agent, hostname: np0000176093, name: 6ce28520-0caa-4a86-bd1f-a321e4713fc7 {{(pid=105248) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:311}} May 21 23:21:09.629826 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization May 21 23:21:09.630280 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.632120 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] - Agent type: OVN Neutron agent, hostname: np0000176093, name: 6ce28520-0caa-4a86-bd1f-a321e4713fc7 {{(pid=105248) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:311}} May 21 23:21:09.633536 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.636449 np0000176093 neutron-ovn-maintenance-worker[105248]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] New host np0000176093 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron May 21 23:21:09.636827 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.636827 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.636967 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.636967 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637193 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637279 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637337 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637403 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637545 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637631 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637738 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637845 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.637927 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.638433 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.638585 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.639925 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.640517 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.640922 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.641210 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.641415 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.641557 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.641704 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.641852 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=105248) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} May 21 23:21:09.648618 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:09.648828 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.648938 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.692804 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG oslo_concurrency.lockutils [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=105248) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}} May 21 23:21:09.693220 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG oslo_concurrency.lockutils [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.001s {{(pid=105248) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}} May 21 23:21:09.703631 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.054 seconds May 21 23:21:09.703783 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:09.703924 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.703998 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.723965 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron_lib.callbacks.manager [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-866925', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-1599425'] for segment_host_mapping (None), after_create {{(pid=105248) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} May 21 23:21:09.750328 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.services.segments.db [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] Segments {'2f6bea11-307b-4d4b-81f5-3a358af1f629'} mapped to the host np0000176093 {{(pid=105248) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}} May 21 23:21:09.756134 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG oslo_concurrency.lockutils [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.063s {{(pid=105248) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}} May 21 23:21:09.756297 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] OVN-SB Sync hostname and physical networks completed @ 2026-05-21 23:21:09.756128 {{(pid=105248) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1538}} May 21 23:21:09.771879 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_dhcp_options_consistency finished in 0.068 seconds May 21 23:21:09.772105 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:09.772369 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.772545 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.772947 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds May 21 23:21:09.773195 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:09.773398 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.773563 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.779266 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:09.853444 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:09.853889 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:09.862625 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.089 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:09.870598 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:09.871915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:09.872778 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:09.872778 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:09.873503 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:09.874483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.000 seconds May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:09.875457 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.876006 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.908566 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=105248) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}} May 21 23:21:09.908827 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_for_inconsistencies finished in 0.034 seconds May 21 23:21:09.908965 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.909130 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.909324 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds May 21 23:21:09.909447 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:09.909592 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.909676 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:09.910795 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} May 21 23:21:09.911400 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.001 seconds May 21 23:21:09.911764 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:09.912438 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:09.912729 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.014536 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.101 seconds May 21 23:21:10.015948 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.016212 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.016379 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.116168 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_provider_distributed_ports finished in 0.099 seconds May 21 23:21:10.120556 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.121744 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.121896 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.228637 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} May 21 23:21:10.229037 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7541497a-3c23-4ded-be6c-36316b2c538b None None] OVN-Southbound DB sync process completed @ 2026-05-21 23:21:10.228754 {{(pid=105248) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1488}} May 21 23:21:10.238937 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.117 seconds May 21 23:21:10.239121 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.239324 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.239448 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.240979 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:10.241208 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.241856 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:10.242971 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.243363 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:10.243700 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.243700 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:10.244134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:10.244567 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.245121 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:10.245121 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.245121 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:10.245121 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.245121 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.250299 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.db.ovn_hash_ring_db [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Cleaned up Hash Ring nodes older than 5 days May 21 23:21:10.250494 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.006 seconds May 21 23:21:10.250652 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: configure_nb_global {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.250803 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.251064 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:10.251581 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:10.251878 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.252545 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:10.253655 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.254263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task configure_nb_global failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.254959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:10.255313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:10.255704 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.255704 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:10.255704 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.255704 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:10.256211 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.256661 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:10.257184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:10.257184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.257184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:10.257184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.257184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:10.257184 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} May 21 23:21:10.257344 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.257449 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.257610 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.258024 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:10.258421 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:10.258629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.259178 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:10.259953 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.260358 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.260958 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:10.261323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:10.261664 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.261664 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:10.261664 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.261664 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:10.261859 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.262295 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.262770 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.268783 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_network_type_and_physnet finished in 0.006 seconds May 21 23:21:10.268992 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.269147 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.269235 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.269426 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds May 21 23:21:10.269518 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.269620 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.269696 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.269872 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:10.270341 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:10.270534 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.270982 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:10.271586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.271981 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.272333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:10.272626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:10.273001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.273001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:10.273001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.273001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.273190 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:10.273652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:10.274147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.274147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:10.274147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.274147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:10.274147 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.274147 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.274147 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.000 seconds May 21 23:21:10.274381 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.274381 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.274447 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.274573 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:10.275059 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:10.275243 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:10.275349 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.275802 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:10.276491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.276897 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:10.277237 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.277237 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:10.277302 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:10.277715 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:10.278189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:10.278189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:10.278189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:10.278189 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.278189 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.281300 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_router_distributed_flag finished in 0.003 seconds May 21 23:21:10.281420 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.281588 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_router_static_routes {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.282350 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.282731 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_router_static_routes finished in 0.001 seconds May 21 23:21:10.282918 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.283141 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:10.283293 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:10.294773 np0000176093 neutron-ovn-maintenance-worker[105248]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_security_group_with_address_group finished in 0.011 seconds May 21 23:21:10.295005 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:10.295735 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105248) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} May 21 23:21:14.773641 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:14.774248 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:14.774419 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:14.775759 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:14.808811 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:14.809676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:14.810318 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:14.810692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:14.811062 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:14.811447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:14.812267 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:14.813146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:14.813146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:14.813146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:14.813146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:14.813146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:14.813146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:15.239600 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:15.240244 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:15.240483 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:15.244881 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:15.245391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.246547 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:15.247658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.248319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:15.248959 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:15.250371 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.251688 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:15.251688 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:15.251688 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.251688 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:15.251688 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.251688 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:15.257696 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:15.258109 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:15.258221 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:15.258376 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:15.258832 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:15.259134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.259800 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:15.260621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.261032 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:15.261856 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:15.262240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.262240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:15.262240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.262240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:15.262739 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.263191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:15.263884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.263884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:15.263884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:15.263884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.263884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:15.263884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.263884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:15.269991 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:15.270408 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:15.270511 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:15.270689 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.272472 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:15.273875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.274928 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.275626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:15.276378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:15.277123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.277123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:15.277123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.277123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:15.277123 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.277787 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:15.279221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:15.279893 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.280239 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:15.281262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.281629 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:15.281960 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.281960 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:15.282258 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:15.282689 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:15.283182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:15.283182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:15.283182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:19.774756 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:19.775649 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:19.775649 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:19.776217 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:19.814769 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:19.815070 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:19.815886 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:19.817604 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:19.818407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:19.818945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:19.819585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:19.820421 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:19.820421 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:19.820421 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:19.820421 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:19.820421 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:19.820421 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:20.240327 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:20.241231 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:20.241428 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:20.242462 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:20.242692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.243417 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:20.244476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.244836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:20.245226 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.245226 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:20.245646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:20.246123 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.246607 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:20.246607 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.246607 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:20.258362 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:20.258865 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:20.259187 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:20.259567 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:20.260298 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:20.260676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.261556 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:20.262908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.263214 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.264692 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:20.265079 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:20.265472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.265472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:20.265472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.265472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:20.266202 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.266648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:20.267146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.267146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:20.267146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:20.267146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.267146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:20.267146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.267146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:20.270499 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:20.270825 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:20.270961 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:20.271129 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:20.271605 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:20.271860 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.272428 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:20.273169 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.273725 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.274225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:20.275397 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:20.276227 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:20.276719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.277408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:20.278637 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:20.278637 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:20.278637 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:20.278637 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:20.278997 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:20.279246 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:20.279353 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.279994 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:20.280865 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.281560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:20.282321 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:20.283189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:20.284527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:20.284527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:20.284527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:20.284527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:20.284527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:20.284527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:24.775733 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:24.776665 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:24.776665 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:24.777149 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:24.806573 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:24.806841 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:24.807561 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:24.808373 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:24.808988 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:24.809631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:24.810593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:24.811686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:24.811686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:24.811686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:24.811686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:24.811686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:24.811686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:25.241161 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:25.241758 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:25.241949 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:25.243017 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:25.243533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.244460 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:25.246017 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.246647 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:25.247406 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.247406 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:25.247880 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:25.248783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.250658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:25.250658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.250658 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:25.258749 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:25.259354 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:25.259538 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:25.259726 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:25.260316 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:25.260700 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.261723 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:25.263072 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.263584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.264529 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:25.265249 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:25.266006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.266006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:25.266006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.266006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:25.266309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.266924 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:25.267763 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.267763 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:25.267763 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:25.267763 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.267763 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:25.267763 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.267763 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:25.270884 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:25.271070 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:25.271150 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:25.271282 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:25.271587 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:25.271858 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.272395 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:25.272911 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.273348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.273648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:25.273973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:25.274478 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:25.274921 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.275583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:25.278518 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:25.278693 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:25.278753 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:25.278877 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:25.279333 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:25.279615 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:25.279735 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.280270 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:25.280851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.281221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:25.281663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:25.282104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:25.282605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:25.282605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:25.282605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:25.282605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:25.282605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:25.282605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:29.776360 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:29.777233 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:29.777233 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:29.778594 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:29.814009 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:29.814429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:29.815353 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:29.816410 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:29.817562 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:29.818111 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:29.818754 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:29.819631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:29.819631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:29.819631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:29.819631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:29.819631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:29.819631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:30.243480 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:30.244374 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:30.244574 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:30.245976 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:30.246447 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.247532 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:30.249575 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.250429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:30.251231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.251231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:30.251476 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:30.252290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.253257 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:30.253257 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.253257 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:30.259288 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:30.259562 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:30.259672 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:30.259801 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:30.260407 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:30.260759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.261484 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:30.262585 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.263434 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.263987 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:30.264734 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:30.265681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.265681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:30.265681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.265681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:30.265902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.266696 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:30.267601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.267601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:30.267601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:30.267601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.267601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:30.267601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.267601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:30.271185 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:30.271569 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:30.271750 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:30.271884 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:30.272361 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:30.272651 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.273316 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:30.274037 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.274505 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.274903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:30.275269 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:30.275693 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.275693 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:30.275693 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.275693 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.275965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:30.276469 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:30.277134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.277134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:30.277134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.277134 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:30.278777 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:30.279020 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:30.279165 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:30.279316 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:30.279845 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:30.280253 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:30.280452 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.281119 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:30.281954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.282355 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:30.282717 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.282717 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:30.282985 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:30.283576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:30.284146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:30.284146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:30.284146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:34.776903 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:34.777680 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:34.777865 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:34.778847 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:34.815798 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:34.816041 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:34.816740 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:34.817910 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:34.818532 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:34.819107 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:34.819718 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:34.820537 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:34.820537 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:34.820537 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:34.820537 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:34.820537 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:34.820537 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:35.244432 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:35.245075 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:35.245259 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:35.246220 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:35.246551 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.250076 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:35.250570 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.251001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:35.251494 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.251494 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:35.252184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:35.253430 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.255801 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:35.255801 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.255801 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:35.259778 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:35.260300 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:35.260300 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:35.260494 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:35.260876 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:35.261023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.261752 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:35.262515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.263186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.263589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:35.264261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:35.264898 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.265605 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.266549 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:35.271758 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:35.272139 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:35.272248 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:35.272389 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:35.272780 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:35.273058 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.273559 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:35.274368 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.274825 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.275482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:35.275830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:35.276251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.276251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:35.276251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.276251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.276646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:35.277210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:35.277771 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.277771 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:35.277771 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.277771 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:35.279124 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:35.279380 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:35.279480 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:35.279650 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:35.279884 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:35.280136 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:35.280255 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.280720 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:35.281576 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.281963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:35.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:35.282644 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:35.283140 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:35.283680 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:35.283680 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:35.283680 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:39.778216 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:39.779384 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:39.779384 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:39.780430 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:39.815772 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:39.815932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:39.816489 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:39.817439 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:39.818104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:39.819096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:39.820002 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:39.820952 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:39.820952 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:39.820952 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:39.820952 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:39.820952 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:39.820952 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:40.245281 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:40.245882 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:40.246206 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:40.247079 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:40.247353 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.248159 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:40.249527 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.250104 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:40.250902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.250902 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:40.251342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:40.251876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.252408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:40.252408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.252408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:40.260193 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:40.260446 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:40.260561 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:40.260677 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:40.260939 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:40.261132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.261574 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:40.263521 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:40.264174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.264864 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.265370 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:40.265741 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.266231 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:40.266785 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.266785 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:40.266785 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:40.266785 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.266785 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:40.266785 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.266785 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:40.272198 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:40.272490 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:40.272612 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:40.272722 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:40.272971 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:40.273222 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.273634 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:40.274348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.274788 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.275404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:40.275759 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:40.276189 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:40.276599 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.277236 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:40.279478 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:40.279747 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:40.279851 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:40.279995 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:40.280252 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:40.280456 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:40.280563 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.281003 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:40.281820 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:40.282601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.282601 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:40.282793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:40.283288 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:40.283868 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:40.283868 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:40.283868 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:44.778812 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:44.779585 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:44.779585 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:44.780452 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:44.811859 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:44.812080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:44.812766 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:44.813533 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:44.814271 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:44.814905 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:44.815624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:44.816652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:44.816652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:44.816652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:44.816652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:44.816652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:44.816652 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:45.245932 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:45.246587 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:45.246789 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:45.247738 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:45.248041 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.248875 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:45.250631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.251241 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:45.251823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.251823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:45.252356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:45.253103 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.253931 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:45.253931 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.253931 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:45.260683 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:45.261303 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:45.261521 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:45.261726 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:45.262312 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:45.262603 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.263498 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:45.265342 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.265813 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.266631 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:45.267011 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:45.267429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.267429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:45.267429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.267429 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:45.267990 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.268587 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:45.269132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.269132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:45.269132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:45.269132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.269132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:45.269132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.269132 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:45.272929 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:45.273027 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:45.273219 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:45.273363 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:45.273908 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:45.274423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.275490 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:45.276838 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:45.277366 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:45.277797 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.278260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.278577 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:45.279145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:45.279714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.279714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:45.279714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.279714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:45.279835 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:45.280084 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:45.280195 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:45.280488 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:45.280894 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:45.280990 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:45.281112 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.281653 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:45.282628 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.283006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:45.283396 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.283396 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:45.283755 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:45.284246 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:45.284610 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:45.284610 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:45.284610 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:49.779771 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:49.782985 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:49.782985 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:49.782985 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:49.814448 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:49.814648 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:49.817262 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:49.817719 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:49.818091 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:49.818473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:49.818998 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:50.246734 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:50.247841 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:50.247841 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:50.248698 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:50.249033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:50.253081 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:50.253815 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.254467 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:50.254467 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:50.254467 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.254467 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:50.254467 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.254467 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:50.254976 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:50.255728 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.256569 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:50.256569 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.256569 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:50.261132 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:50.261606 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:50.261606 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:50.261606 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:50.261913 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:50.262153 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.264674 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:50.265116 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.265423 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.265675 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:50.266009 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:50.266319 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.266830 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:50.273215 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:50.273658 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:50.273961 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:50.274220 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:50.274862 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:50.275176 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.278836 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:50.279542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:50.280159 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.280645 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:50.281064 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:50.281545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:50.282243 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:50.282739 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:50.282967 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:50.283238 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:50.283860 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:50.284176 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:50.284306 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:50.286235 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:50.286586 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.286872 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:50.287174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:50.287572 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:54.781130 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:54.781950 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:54.781950 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:54.782508 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:54.820076 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:54.820626 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:54.821542 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:54.822545 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:54.823263 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:54.823884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:54.824641 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:54.825723 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:54.825723 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:54.825723 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:54.825723 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:54.825723 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:54.825723 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:55.247584 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:55.248396 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:55.248547 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:55.249608 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:55.249925 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.250895 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:55.252417 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.253172 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:55.253908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.253908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:55.254135 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:55.255221 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.256323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:55.256323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.256323 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:55.261783 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:55.262320 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:55.262486 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:55.262668 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:21:55.263382 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:55.263693 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.264491 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:55.265686 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.266154 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.266965 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:55.267409 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:55.267793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.267793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:55.267793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.267793 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:55.268590 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.269381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:55.270361 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.270361 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:55.270361 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:55.270361 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.270361 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:55.270361 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.270361 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:55.273719 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:55.273995 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:55.274128 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:55.274262 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:55.274955 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:55.275360 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.275946 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:21:55.276714 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.277491 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.277765 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:55.278205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:55.278589 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:55.279015 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.279588 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:55.283015 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:55.283404 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:55.283522 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:55.283687 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:55.284851 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:55.285748 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:55.285748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.286660 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:55.287663 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.288033 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:55.288372 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.288372 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:55.289515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:55.289989 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:55.290555 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:55.290555 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:55.290555 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:21:59.781833 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:21:59.782654 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:21:59.782654 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:21:59.783791 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:21:59.785621 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG dbcounter [-] [105248] Writing DB stats neutron:SELECT=520,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=105248) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}} May 21 23:21:59.814073 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:21:59.814205 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:59.814670 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:21:59.815348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:59.815879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:21:59.816472 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:21:59.817181 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:21:59.818003 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:21:59.818003 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:21:59.818003 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:21:59.818003 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:21:59.818003 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:21:59.818003 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:00.248431 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:00.248921 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:00.248998 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:00.249546 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:00.249860 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.250243 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:00.251018 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.251442 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:00.251948 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:00.252657 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.253584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:00.253584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:00.253584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.253584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:00.253584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.253584 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:00.262345 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:00.262864 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:00.262941 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:00.263060 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:22:00.264104 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:00.264565 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.265184 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:00.265876 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:00.266750 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.267441 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.267996 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:00.268538 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.269244 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:00.270085 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.270085 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:00.270085 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:00.270085 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.270085 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:00.270085 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.270085 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:00.274112 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:00.274407 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:00.274523 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:00.274668 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:00.275012 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:00.275311 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.275854 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:00.276627 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.277000 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.277612 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:00.277932 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:00.278287 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.278287 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:00.278287 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.278287 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.278837 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:00.279313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:00.279778 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.279778 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:00.279778 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.279778 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:00.283427 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:00.283638 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:00.283754 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:00.283918 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:00.284215 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:00.284447 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:00.284583 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.285109 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:00.285969 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.286304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:00.286646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.286646 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:00.287060 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:00.287457 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:00.287931 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:00.287931 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:00.287931 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:04.782587 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:04.783572 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:04.783572 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:04.785283 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:04.823526 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:04.823852 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:04.824210 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:04.825086 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:04.825660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:04.826240 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:04.826908 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:04.827800 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:04.827800 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:04.827800 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:04.827800 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:04.827800 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:04.827800 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:05.249251 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:05.251748 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:05.251921 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:05.252741 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:05.252903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.253530 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:05.255023 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.256290 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:05.257530 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:05.258950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.259918 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:05.259918 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:05.259918 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.259918 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:05.259918 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.259918 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:05.262795 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:05.263019 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:05.263129 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:05.263266 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:22:05.263661 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:05.263912 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.264621 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:05.265378 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:05.266350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.266973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.267333 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:05.267635 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.268110 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:05.268669 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.268669 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:05.268669 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:05.268669 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.268669 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:05.268669 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.268669 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:05.274513 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:05.274807 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:05.274894 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:05.275003 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:05.275430 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:05.275666 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.276234 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:05.276995 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.277440 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.277884 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:05.278230 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:05.278624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.278624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:05.278624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.278624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.279282 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:05.279790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:05.280731 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.280731 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:05.280731 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.280731 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:05.284285 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:05.284612 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:05.284721 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:05.284877 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:05.285195 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:05.285368 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:05.285461 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.285903 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:05.286702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.287074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:05.287408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.287408 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:05.287656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:05.288653 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:05.289194 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:05.289194 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:05.289194 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:09.783337 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:09.784129 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:09.784250 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:09.785509 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:09.824538 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:09.825006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:09.826004 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:09.827155 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:09.827774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:09.828436 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:09.829080 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:09.829681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:09.829681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:09.829681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:09.829681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:09.829681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:09.829681 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:10.252227 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:10.252931 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:10.253104 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:10.254077 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:10.254385 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.255192 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:10.256356 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.257286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:10.258096 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:10.259100 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.260404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:10.260404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:10.260404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.260404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:10.260404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.260404 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:10.263439 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:10.264008 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:10.264130 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:10.264238 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:22:10.264965 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:10.265375 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.266186 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:10.266839 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:10.267560 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.268228 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.268808 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:10.269341 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.270066 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:10.270846 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.270846 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:10.270846 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:10.270846 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.270846 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:10.270846 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.270846 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:10.275286 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:10.275789 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:10.275999 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:10.276258 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:10.276930 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:10.277277 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.277960 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:10.279201 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.279823 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.280608 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:10.281199 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:10.282082 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.282082 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:10.282082 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.282082 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.282348 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:10.283448 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:10.284649 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.284649 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:10.284649 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.284649 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:10.284949 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:10.285312 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:10.285427 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:10.285599 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:10.286251 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:10.286692 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:10.286920 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.287733 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:10.288757 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.289339 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:10.289854 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:10.290509 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:10.291276 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:10.291276 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:10.291276 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:10.291276 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:10.291276 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:10.291276 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:14.784105 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:14.785172 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:14.785172 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:14.786028 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:14.812671 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:14.813174 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:14.814074 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:14.814916 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:14.815670 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:14.816327 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:14.817193 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:14.817730 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:14.817730 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:14.817730 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:14.817730 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:14.817730 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:14.817730 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:15.252980 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:15.253617 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:15.253796 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:15.254686 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:15.254915 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.255581 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:15.257261 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.258602 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:15.259973 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:15.260963 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.261799 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:15.261799 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:15.261799 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.261799 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:15.261799 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.261799 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:15.263975 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:15.264339 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:15.264516 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:15.264726 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:22:15.265151 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:15.265334 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.265822 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:15.266662 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:15.267515 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.268286 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.268774 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:15.269101 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.269488 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:15.269935 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.269935 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:15.269935 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:15.269935 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.269935 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:15.269935 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.269935 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:15.275952 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:15.276346 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:15.276462 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:15.276620 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:15.277009 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:15.277216 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.277783 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:15.278954 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.279729 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.280407 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:15.281403 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:15.282736 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:15.283415 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.284182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:15.285436 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:15.285798 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:15.285967 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:15.286214 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:15.286531 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:15.286738 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:15.286851 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.287367 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:15.288215 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.289029 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:15.289791 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:15.290260 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:15.290716 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:15.290716 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:15.290716 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:15.290716 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:15.290716 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:15.290716 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:19.785207 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:19.785998 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:19.786252 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:19.787663 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:19.825087 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:19.825309 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:19.826068 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:19.827293 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:19.828114 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:19.829001 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:19.829709 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:19.830769 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:19.830769 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:19.830769 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:19.830769 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:19.830769 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:19.830769 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:20.254087 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:20.254936 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:20.255142 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:20.256259 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:20.256534 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.257394 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:20.259006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.259781 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:20.260593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.260593 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:20.261251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:20.262702 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.263964 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:20.263964 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.263964 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:20.264562 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:20.264897 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:20.265033 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:20.265167 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:22:20.265957 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:20.266401 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.267664 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:20.268500 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.268946 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.269393 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:20.269917 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:20.270726 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.271362 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.272113 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:20.276587 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:20.276948 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:20.277081 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:20.277237 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:20.278073 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:20.278624 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.279493 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:20.280379 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.281212 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.281748 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:20.282206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:20.282960 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.282960 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:20.282960 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.282960 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.283480 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:20.284217 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:20.285278 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.285278 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:20.285278 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.285278 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:20.286138 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:20.286634 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:20.286826 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:20.287090 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:20.287811 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:20.288257 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:20.288413 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.289337 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:20.290795 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.291615 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:20.291945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.291945 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:20.292391 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:20.293006 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:20.293483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:20.293483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:20.293483 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:24.785915 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:24.786871 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:24.786871 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:24.787272 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:24.822646 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:24.822841 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:24.823334 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:24.824145 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:24.824790 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:24.825399 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:24.826195 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:24.827350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:24.827350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:24.827350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:24.827350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:24.827350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:24.827350 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:25.254892 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:25.255362 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:25.255480 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:25.257698 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-c99052bf-083c-4a36-b8dc-df9d60bd564a, routes=[], if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:25.257931 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.258536 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:25.259304 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.260146 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:25.260950 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:25.261944 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.263180 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:25.263180 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:25.263180 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.263180 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:25.263180 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.263180 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:25.264995 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:25.265299 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:25.265499 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:25.265499 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting fip-distributed flag in NB_Global to False {{(pid=105248) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}} May 21 23:22:25.265924 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:25.266251 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.266899 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:25.267445 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.267893 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.268182 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:25.268520 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:25.268879 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^ May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.269275 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.269886 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:25.277171 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:25.277753 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_ha_failover {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:25.278008 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:25.278297 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:25.278911 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:25.279326 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.280226 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() May 21 23:22:25.281473 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.282344 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.command May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.282829 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:25.283482 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:25.284225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.284225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:25.284225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.284225 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.284656 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:25.285147 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:25.285633 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.285633 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:25.285633 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.285633 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics May 21 23:22:25.286772 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105248) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} May 21 23:22:25.287167 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} May 21 23:22:25.287340 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105248) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} May 21 23:22:25.287514 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105248) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}} May 21 23:22:25.288083 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:25.288475 np0000176093 neutron-ovn-maintenance-worker[105248]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105248) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} May 21 23:22:25.288660 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Traceback (most recent call last): May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: txn.results.put(txn.do_commit()) May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: ^^^^^^^^^^^^^^^ May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: raise RuntimeError(msg) May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.289381 np0000176093 neutron-ovn-maintenance-worker[105248]: May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: May 21 23:22:25.290206 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.290553 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) May 21 23:22:25.290885 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.290885 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics [None req-9de5012a-2697-4cbe-a356-ae910c739240 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics Traceback (most recent call last): May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics work() May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(*args, **kwargs) May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics return f(self, *args, **kwargs) May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics result = f(*args, **kwargs) May 21 23:22:25.291191 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics next(self.gen) May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics with self.create_transaction( May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics self.result = self.commit() May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^ May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise result.ex May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics txn.results.put(txn.do_commit()) May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ May 21 23:22:25.291676 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit May 21 23:22:25.292313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics raise RuntimeError(msg) May 21 23:22:25.292313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it May 21 23:22:25.292313 np0000176093 neutron-ovn-maintenance-worker[105248]: ERROR futurist.periodics